Why did GitHub go dark for nearly eight hours on August 17?

GitHub was unreachable for 7 hours and 47 minutes on August 17. The outage broke github.com, authentication, Actions, APIs, pull requests, issues, and Copilot for developers worldwide.

GitHub is a platform where developers store code, run automated tests, and collaborate on software. On August 17 it stopped working for almost a full workday.

The outage began when traffic hit a new peak. A critical infrastructure component in the Central US data center failed to scale with that peak. Capacity pressure then spread through the system, triggering authentication failures and taking down multiple services at once.

Recovery required several coordinated moves. Teams rerouted traffic, isolated the affected infrastructure, and brought services back in stages. Most services returned earlier that day. Copilot services took longer because errors in those services triggered a client-side retry loop that added more traffic during recovery. That behavior had to be mitigated before traffic could be safely restored.

Neither incident came from a code change

Both the August 6 Actions failure and the August 17 outage share a root cause. Both were capacity failures. No code deployment or configuration change triggered either one. GitHub simply failed to scale critical components before demand exceeded their capacity.

Since April, monthly commits have grown from 1.4 billion to 2.9 billion. That growth explains the pressure. It does not excuse the outages.

Hardware additions and a cloud migration

As part of reliability commitments made earlier this year, GitHub focused on three priorities: adding capacity, improving efficiency, and removing architectural bottlenecks. The company has since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity.

Teams installed as much hardware as available power allowed in existing data centers while accelerating migration to Azure. Today Azure serves roughly 58% of GitHub's platform load and half of all Git operations. In May that figure was 12% of platform load.

The expanded footprint has also supported growth in GitHub Actions job runs. Azure's infrastructure has accelerated work to scale the largest monorepos.

Reading at scale

The next milestone is an architecture that scales read capacity linearly with the number of readers. That would enable unlimited read operations. Rollout will be gradual, beginning with the largest monorepos.

Scale is not the only challenge. As the pace and complexity of change increased, existing operational practices did not keep up. GitHub has redirected teams and resources toward availability. Investments include stronger testing and safer rollouts. Better observability and more effective alerting are also underway. Progress has been made but the work is not complete.

Decoupling core services

Work is also underway to isolate critical systems and remove shared dependencies between them. This effort is designed to reduce the likelihood of an outage and limit its impact when one occurs.

The August 6 and August 17 incidents led to two immediate changes. First, GitHub is applying consistent retry limits and retry budgets across service-to-service interactions. Variable timeouts will also be enforced to prevent retry storms and cascading load. Second, the company is reviewing lower-priority CPU and memory alerts to identify components that could fail during sudden traffic spikes.

What the post does not say

The announcement does not quantify how many developers or organizations were affected. It does not disclose whether any data was lost. It does not state whether the August 6 incident lasted a similar duration. It does not provide a timeline for when the linear-read architecture will reach general availability.

The sources this article was written from

  • "GitHub Blog
    https://github.blog/news-insights/company-news/the-august-17-outage-and-the-work-ahead/"

Read next