Microsoft 365 Alert : Service Degradation – Microsoft 365 suite – Some users may experience degraded performance with multiple Microsoft 365 services and features – RESOLVED

Announcements Sourav Nandi

07/09/2022 14:38:00 PM (BST)

NHSmail Reference : INC31498119

Microsoft Reference : MO427469

Issue Status : RESOLVED

Issue Description : Users may have experienced degraded performance with multiple Microsoft 365 services and features.

More info : During the impact period affected services and features included, but were not limited to:

– Users experienced delays with calling and message notifications and may have been unable to manage/create channels, call queues and auto attendant within Microsoft Teams. Sending or receiving messages may have failed to complete, and Teams Meeting Rooms were also experiencing impact. Third party Session Initiation Protocol (SIP) devices that were provisioned in Europe, Middle East, and Africa (EMEA) may also have been unable to sign in to the service or make and receive calls.
– Yammer feeds were not loading.
– Users were unable to access the Microsoft Defender for Identity (MDI) Portal or load MDI pages within the Security and Compliance Center (SCC). Additionally, some MDI related alerts may not have been delivered.
– Delays were experienced in automated and custom investigations within Microsoft Defender for Endpoint (MDE).
– Users were unable to check-in devices within Microsoft Intune.
– Power App operations and app management may not have performed as expected.
– Failures when a user attempted to run approvals, universal search scenarios, and/or connection wizard within Power Automate.
– Intermittently unable to create Roadmaps within Project for the web service.
– Unable to access Usage Reports and Adoption scores within the Admin Center, Graph APIs and Power BI services.
– OneNote Section Groups and Lists experienced failures to sync approximately 20 percent of hierarchy change operations, such as creating, moving, or modifying a section’s order, color, or title. Opening a Notebook also failed in some circumstances. This affected Universal Windows Platform, iOS, Mac, and Android platform OneNote apps.
– Admins experience timeouts within Microsoft Purview eDiscovery (Premium) when opening existing or creating new site collections.

Scope of impact : Impact is specific to requests which are routed through impacted components within Northern Europe.

Final Update : 08/09/2022 08:35:00 AM – Normal service operation has been restored for the affected Azure Cosmos DB and subsequently Microsoft 365 services. Microsoft has determined that the confluence of several internal service operations caused an unexpected increase in traffic for the affected Azure Cosmos DB. The affected DB was recovered as of Wednesday, September 7, 2022, at 5:21 PM UTC. While the issue was ongoing, they performed fail over actions for core services which alleviated the load on the impacted Cosmos DB, and also performed load balancing to further stabilize the environment. Microsoft is closely monitoring service availability and will investigate the factors that led to the unexpected high utilization.

Root cause : The confluence of multiple internal service operations caused an unexpected increase in traffic load for the affected Azure Cosmos DB. This pushed the cluster above the designed threshold, causing service issues for Microsoft 365 services hosted on the infrastructure.