Skip to content

The news behind the software that runs companies, explained for everyone.

Security

Microsoft says a July patch bug took over 80% of its 365 gateway servers offline. Its rollback could not quickly undo it.

The explanation came on 16 November from Charles Lamanna, a Microsoft executive, in posts on X that The Stack reported.

ERP LEADERS news desk · · First reported by The Stack · Edition no. 70
Photo: Wikimedia Commons/Wikimedia, Public domain · original

Three things combined, he says. A July patch for the gateway servers that route traffic to many Microsoft 365 services carried a hidden defect. A surge in load set it off. And nearly 14% of gateway capacity was already out for an unrelated reason.

By Microsoft's account, its load testing focused on weaker servers with fewer processor cores and missed the defect. In less than two hours the defect was triggered across the fleet. Because the bug was months old, its usual tools for undoing a change could not reverse it fast.

SiliconANGLE reported that the outage disrupted businesses and government agencies in more than 70 countries. Maersk and Euronext were among those that warned customers.

Our reading for CIOs: no customer could test, delay or undo that server patch. The hours without email and Teams still landed on their own continuity plans. Microsoft has promised its final review this week.

Sources

Spotted an error? Tell us. Every correction is logged on Standards.

Your reading room

Companies in this story

Our file on Microsoft

  1. 19 NovFiled later

    Microsoft's finance saves over 250,000 hours yearly with AI, it reports. It runs on SAP, though Microsoft sells ERP.

  2. 19 NovFiled later

    Microsoft's CEO said a lot of its Dynamics business is now usage, not seats. CFOs still budget software per seat.

  3. 19 NovFiled later

    Microsoft's cloud replaces a Basque hospital's Navision ERP in a €194,240 contract, a notice shows. Navision's last release loses support in 2028.

  4. 18 NovFiled later

    Microsoft is how 98% to 99% of UK civil servants communicate, a minister said. No date was given for moving away.

  5. 17 NovFiled later

    Microsoft data gives Claude under a tenth of Copilot's weekly Office users, The Information reported. Microsoft stock dipped after Claude's new tools.

All 92 Microsoft stories

For your next steering committee

Questions for your own programme. They are not findings about any company in this story.

  1. 1Ask when systems of this kind in your landscape were last patched, and who confirms it.
  2. 2Ask which finance and ERP accounts can be reached from outside, and how each sign-in is protected.
  3. 3Ask how long a restore took the last time it was tested.

Issue 01 of the ERP LEADERS brief puts a timeline, the three-number review and six questions on one page. Read issue 01

Founding reader view is on in this browser. It is a reading view, not secure access. Print the working sheet

Founding reader

Welcome back. · Issue 01

3 comments under this story on LinkedIn. Read the discussion

Keep reading

The ERP LEADERS brief Get the brief