How dual-channel isolation took model availability from 97% to 99.8%

The problem: a single channel’s ceiling

In late 2024, our core models (Claude Sonnet, GPT-4o) reached the upstream providers through a single channel. Monthly availability bounced between 96.5% and 97.2%. Sounds fine?

Do the math and it isn’t. 97% availability means about 21 hours of downtime per month. For teams that treat us as infrastructure, that number is unacceptable.

The root cause wasn’t our code — it was the upstream providers’ own volatility. One channel goes down and all traffic is hit. A single point of dependency is an architecture problem, not an ops problem.

The design: dual-channel isolation

The core idea is simple: maintain two independent supply channels per model (Channel A / Channel B), each a fallback for the other.

Routing logic

When a request comes in, the router does three things:

1. Query both channels’ real-time health scores (based on the last 5 minutes of error rate and latency)
2. Prefer the channel with the higher health score
3. If the primary returns a 5xx or times out, retry on the backup immediately (transparent to the user in the vast majority of cases)

async function route(model: string, req: Request) {
  const [chA, chB] = await getChannels(model)
  const primary = chA.healthScore >= chB.healthScore ? chA : chB
  const fallback = primary === chA ? chB : chA

  try {
    return await primary.forward(req, { timeout: 30_000 })
  } catch (e) {
    // primary failed, switch immediately — transparent to the user
    metrics.record('failover', { model, from: primary.id })
    return await fallback.forward(req, { timeout: 30_000 })
  }
}

Computing the health score

Each channel’s health score is a 0–100 sliding-window value:

• Error rate, weight 70% — 5xx / total requests over the last 5 minutes
• Latency, weight 30% — how far P95 latency deviates from baseline

A channel below 50 is automatically de-weighted; one below 30 for 3 consecutive minutes is temporarily pulled from the routing pool, periodically probed by an independent heartbeat, and re-added once recovered.

The result: 18 months of production data

After dual-channel isolation shipped, availability rose to 99.81%, and recovery time dropped from p95 4 minutes to p95 8 seconds — the vast majority of users never noticed a provider-side failure.

Metric	Single channel (2024 Q4)	Dual channel (2025 Q3)
Availability	97.0%	99.81%
Avg monthly downtime	21h	1.4h
failover p95	—	8s

Lessons learned

• No isolated timeout budget early on: the backup channel’s timeout was also 30s, doubling the wait during failures. Changing it to primary 30s / backup 20s cut perceived latency by 35%.
• Health-score lag: a 5-minute window reacts slowly to sudden failures, so we layered on a 30-second “recent failure” fast signal.
• Failover storms: a transient blip on the primary made both sides poll and overwhelm the backup. Client-side rate limiting + backoff eased it.

What’s next

Dual-channel already solves single-vendor failure. But we still depend on a single network egress, a single billing account, a single region. The next roadmap item is extending isolation to those layers — we’ll write that up once it’s implemented.