case studyresilienceoperations

Case Study: How a Small Clinic Survived a Major Cloud Provider Outage

ssimplymed

2026-02-05

10 min read

A 2026 case study showing how a small clinic survived a Cloudflare/AWS outage—practical DR steps, tools, outcomes, and lessons learned.

How a small clinic survived a major Cloudflare/AWS outage: a realistic 2026 case study

Hook: If you run a clinic, one of your top fears is losing access to electronic health records, telehealth, or patient portals during a cloud outage — especially when HIPAA-protected health information (PHI) is at stake. In early 2026 large-scale outages affecting Cloudflare and parts of the cloud ecosystem reminded healthcare operators that downtime isn't theoretical. This case study walks through how a small clinic prepared for, responded to, and recovered from such an outage — the decisions made, tools used, patient outcomes, and the precise lessons learned you can apply this quarter.

Executive summary — the incident at a glance

On January 16, 2026, widespread reports showed disruptions tied to Cloudflare services that cascaded into DNS/CDN failures and degraded connectivity for several internet-dependent services. Our hypothetical but realistic clinic — Riverside Family Clinic, a 7-provider primary care practice with an onsite nurse triage line — experienced partial outage conditions: the patient portal and telehealth landing pages were unavailable, inbound DNS resolution was flaky, and a subset of cloud-hosted EHR APIs responded slowly.

Because Riverside had invested in a layered disaster recovery (DR) and operational resilience approach during 2024–2025, the clinic avoided canceled urgent visits, retained access to local patient records, and maintained HIPAA-compliant communication channels. Key results from the event:

Over 90% of scheduled visits were completed (in-person or via backup telehealth) despite degraded web access.
Zero PHI exposures; all fallbacks used pre-approved Business Associate Agreement (BAA) vendors or local encrypted storage.
Average clinic check-in time increased by only 12 minutes for affected patients due to manual intake procedures.

Pre-outage preparation: what Riverside did right

1. Multi-layer redundancy, not just multi-cloud buzzwords

Riverside avoided the common pitfall of treating “multi-cloud” as a checkbox. Instead they built redundancy across layers: DNS, content delivery, application endpoints, and data availability. Their contracts and architecture included:

DNS failover using an authoritative secondary provider with automatic health checks and low TTL settings for rapid switchover.
CDN/edge fallback — a basic static copy of the patient-facing portal hosted at an alternate provider and refreshed hourly for static content (scheduling, phone numbers, triage instructions).
Local read replicas of the EHR database for read-only access (synchronized every 5 minutes) enabling clinicians to view recent notes and meds even if cloud API calls failed.
Cross-region and cross-account backups for critical data (S3-style object storage with cross-region replication) to meet strict RPO and RTO targets.

2. Clear, practiced disaster runbooks and tabletop exercises

Every quarter Riverside ran tabletop drills that simulated DNS/CDN outages and telehealth provider interruptions. The runbook spelled out who would do what in the first 30 minutes, 2 hours, and 6 hours:

Activate Incident Lead and notify staff via PagerDuty and SMS.
Switch DNS to secondary provider if health checks fail for 2 consecutive minutes.
Fail over telehealth links and enable phone-based triage workflows.
Record all actions in the incident log for post-mortem and compliance.

3. Vendor governance and BAAs

Before relying on any third-party, Riverside required a signed BAA and confirmation of disaster recovery capabilities. Crucially, their vendor agreements included explicit clauses about fallback communications and the ability to use a backup vendor temporarily. This meant when the primary telehealth portal was unreachable, a pre-approved backup (Zoom for Healthcare / Doxy.me — both under BAA) could be enabled without legal delays.

4. Staff training and manual workflows

Technology wasn't the only preparation. Riverside invested in practical training: front-desk staff practiced paper intake with secure scanning, nurses practiced phone-based SOAP notes, and billing staff simulated offline claims queues. Role rotation meant someone always knew how to operate the local read replica and manual scheduling board.

The outage timeline — decisions, actions, and turning points

0–30 minutes: detection and containment

At 08:05 local time, synthetic monitoring (Datadog and internal probes) flagged elevated DNS resolution times and Cloudflare status alerts trending upward. Riverside’s on-call SRE received a PagerDuty alert, confirmed the issue from multiple vantage points, and initiated the incident runbook.

Immediate actions:

Switch DNS to secondary provider (automatic via health checks after 2 failed probes).
Enable static portal mirror with emergency contact info, hosted at the secondary CDN.
Notified staff via Statuspage and SMS templates to expect manual check-in.

30–120 minutes: tactical failovers and patient triage

As the Cloudflare-related anomalies persisted, some cloud-hosted APIs slowed. Riverside moved to its pre-approved fallbacks:

Clinicians used the local read-only EHR replica to access problem lists, meds, and recent notes. For new notes, they used encrypted local templates to later batch-submit updates when the connection returned.
Telehealth appointments were routed to the backup telehealth provider using pre-signed URLs and phone-first meeting joins. Staff sent secure SMS instructions with a HIPAA-compliant SMS vendor already under BAA.
Urgent patients who could not be served remotely were prioritized for in-person slots; the clinic extended hours by one hour to accommodate rescheduling.

2–6 hours: stabilization and communication

By hour three, DNS had stabilized via the secondary provider and the static portal mirror served essential instructions. Riverside maintained a single incident channel for all communication and kept a running incident log for OCR/HIPAA compliance.

Billing workflows continued offline; claims were queued with timestamps to preserve order and ensure accurate later reconciliation.
Phone triage handled an urgent behavioral health referral successfully; all notes entered into the local encrypted laptop and later reconciled.

Tools and technical controls that mattered

Below are the categories of tools and specific controls Riverside used. You can adapt this list to your clinic's tech stack.

Observability and detection

Synthetic monitoring: proactive probes for DNS, CDN, EHR API endpoints.
Paginated alerting: PagerDuty for escalation, integrated with Statuspage for public communication.
Real user monitoring: to detect client-side failures not visible to synthetic checks.

Network and DNS

Secondary authoritative DNS with health checks and low TTL to enable rapid switchover.
CDN static mirrors for patient-facing content and emergency instructions.
IP allowlists and temporary VPN for secure clinician access when normal routes are down.

Data resilience

Local read replicas with write-forwarding scripts to batch updates when connectivity returns.
Encrypted offline capture: secure laptops that store offline notes and then sync to the EHR once connectivity is restored.
Cross-region backups and immutable snapshots to protect against data loss.

Communications

Pre-approved backup telehealth providers under BAAs.
HIPAA-compliant SMS/email for patient instructions and appointment changes.
Automated voice trees for common instructions (e.g., “If your telehealth link fails, press 3 to speak with triage”).

Patient outcomes and operational metrics

Riverside tracked several KPIs during and after the outage to measure impact and improvement opportunities. Key outcomes:

Visit completion: 92% of scheduled appointments proceeded (either in-person or via fallback telehealth).
Wait times: Average patient check-in time rose from 9 minutes to 21 minutes; actionable changes reduced this to 15 minutes in subsequent incidents.
Billing impact: Claims submission was delayed by 8–36 hours due to manual batching but no denials resulted from data quality issues.
Patient satisfaction: Net Promoter Score (NPS) dip of 4 points on the day, recovered within two weeks after targeted communications.

Most importantly, no privacy incidents occurred. All fallbacks were pre-approved and signed under BAAs, and all offline PHI storage was encrypted and logged for audit.

Lessons learned — tactical and strategic takeaways

From Riverside's experience, here are the practical lessons every small clinic should apply.

1. Design for degraded mode, not just full recovery

Systems should have a known degraded mode: what do clinicians absolutely need (meds, allergies, last notes) and how will they get it if APIs fail? Build small, secure read-only caches and document the exact steps to switch to them.

2. Keep legal and operational approvals ready for fallbacks

Pre-sign BAAs and have a short list of trusted, pre-vetted backup vendors. When outages occur, you want to enable a replacement telehealth or SMS provider in minutes, not days.

3. Practice human workflows as often as you exercise technical failovers

Regular tabletop exercises that include front-desk, nursing, billing, and clinicians reduce friction. Riverside’s drills exposed small UI gaps in the offline intake forms that were fixed before a real emergency.

4. Instrument for early detection — and practice your runbook triggers

Health checks need to be actionable. Riverside reduced false positives by using multi-vantage probes and defining a 2-minute confirmation window before automatic DNS failover to avoid flapping during transient network issues.

5. Maintain clear patient communication templates

Prepared templates for SMS, voicemail, and portal notices cut confusion and reduced call center volume by 40% during the incident.

Actionable checklist: 30–90 day plan for clinics

Use this prioritized checklist to harden your operations within the next quarter.

Inventory all internet-dependent services and identify the minimum data set clinicians need in degraded mode.
Implement a secondary authoritative DNS and configure health checks with a low TTL.
Deploy a read-only EHR replica or cached viewer for essential PHI with strict encryption and audit logging.
Pre-contract at least one BAA-covered backup telehealth and communication vendor.
Run a tabletop drill covering DNS/CDN outages; update runbook within 48 hours of lessons identified.
Set up synthetic monitoring for critical endpoints and tie alerts to a staffed escalation process (PagerDuty or equivalent).
Create patient communication templates and test automated channels (SMS, voicemail, portal notices).
Confirm cross-region backups and test restore quarterly; maintain immutable snapshots for 30–90 days.

2026 trends and the future of clinic resilience

Late 2025 and early 2026 have accelerated several trends that clinics must watch and adopt:

Edge + hybrid architectures: Clinics are placing critical read-only services at the edge to reduce dependency on centralized CDNs and avoid single-vendor cascades. See edge auditability and decision-plane guidance for operational playbooks.
Chaos engineering for healthcare: More small practices are running low-risk chaos tests (DNS latency injection, API throttling) to validate DR runbooks without disrupting patients.
AI-driven failover orchestration: Emerging tools can recommend and automate the safest failover path, reducing human error during incidents.
Regulatory focus on vendor resilience: Expect greater scrutiny and recommended language in BAAs about business continuity and notification timelines following large cloud provider outages.

These trends mean clinics should balance investment: some resilience can be achieved with low-cost architectural changes and staff training rather than expensive full-scale replication.

Post-incident review: Riverside's measurable changes

After the event, Riverside implemented specific improvements informed by metrics and staff feedback:

Lowered DNS TTL from 300s to 60s and tuned health-check thresholds to reduce failover latency without increasing flapping.
Increased local EHR replication frequency from 5 minutes to 1 minute for critical tables (meds, allergies, problem list), keeping write queuing intact.
Expanded tabletop exercises to include vendor-side contacts and legal counsel for rapid BAA invocation if a fallback vendor was outside the original contract scope.
Published a patient-facing continuity-of-care webpage and automated voicemail script used by front-desk staff during future incidents.

Final recommendations — the pragmatic path to operational resilience

Building resilience is not about eliminating risk — that’s impossible — but about reducing blast radius, maintaining patient care, and protecting PHI when providers or CDNs fail. Start by mapping your critical services, then apply layered redundancy: DNS, CDN/edge fallbacks, and local read-only access to PHI.

“Resilience is the product of planning, simple fallbacks, and practiced human workflows — not just the size of your cloud bill.”

Prioritize drills, vendor governance, and simple manual workflows that clinicians can use without training wheels. If you can afford only three investments this quarter, make them:

Secondary DNS with automatic health checks.
Pre-approved backup telehealth and communication vendors under BAAs.
Quarterly tabletop exercises that include full front-line staff participation.

Call to action

Want a ready-made clinic disaster recovery runbook and a 90-day resilience checklist tailored to small practices? Contact simplymed.cloud for a free operational resilience assessment and sample runbook. We'll help you implement low-cost, high-impact controls so your clinic stays open and HIPAA-compliant when the next cloud outage happens.

simplymed

Contributor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.