Adopting AV1 for Real-Time Communication at Scale
How Meta solved the encoding complexity, device eligibility, rate control, and error resilience problems that separate AV1 on-demand streaming from AV1 video calls — and why the RTC constraints make this a substantially harder problem.
The assumption is that once AV1 works for streaming, it works for video calls — you’re sending the same bitstream either way. But streaming and RTC operate under fundamentally different constraints, and those constraints are in direct tension with every technique that makes AV1 efficient. Multi-pass encoding improves quality but adds latency. Buffering smooths playback but breaks real-time feel. Keyframes recover from packet loss but spike the bitrate and cause the congestion they’re meant to survive.
Source
Adopting AV1 for Real-Time Communication (RTC) at Scale
By: Meta
Published: Jun 22 2026
Type: Engineering article
A detailed account of a multi-year deployment effort with unusual specificity on the failure modes — power regressions on Pixel 8, audio/video sync breakdowns on octa-core 2023 phones, binary size budgets that constrain an entire year of development. Worth reading because the RTC constraints are underrepresented in codec coverage.
The gap between AV1 for streaming and AV1 for calls
AV1 has been in production at YouTube and Netflix for years. What Meta documents here is a substantially different problem. For streaming, end-to-end latency is measured in seconds, bitrate can be managed across a large buffer, and the viewer never touches the encoder. For RTC, latency must stay below 300 milliseconds, bitrate must respond to network changes within the duration of a single conversation, and the same device is simultaneously encoding outbound and decoding inbound video. Every advantage AV1 provides over H.264/AVC — its advanced coding tools, palette mode for screen content, intra-block copy — comes with a compute cost that hits both sides of that equation.
The core motivation is straightforward: at least 20% bitrate reduction compared to H.264/AVC at the same quality in offline tests, with greater savings on devices that can afford higher encoder complexity. For real-world RTC, where bitrates in emerging markets typically run between 10 and 400 kbps, the difference between “blurry but recognizable” and “clearly readable” compresses into a narrow window around 100 kbps. AV1’s efficiency advantage is most visible exactly there.
The first practical obstacle was power. An open-source AV1 encoder on a Pixel 8 used 14% more power than H.264/AVC during a call — a meaningful regression on a device that people expect to last a full day. Meta’s response was to adopt an internal low-complexity encoder with an ultra-low-complexity preset designed to match H.264/AVC’s power profile. This isn’t simply a performance tuning story. Modern codecs have larger tool sets than older ones, which means a well-designed encoder has more room to find quality/complexity tradeoffs that previous-generation codecs couldn’t access. The ultra-low preset unlocks AV1 on mid-range and low-end devices that would fail a simple power test with any off-the-shelf encoder.
Binary size is a constraint that rarely appears in codec discussions but dominated this one. The AV1 encoder and decoder together add 1.7 MB to the application binary (600 kB compressed). That sounds manageable until you’re at Meta’s scale — 600 kB can consume an entire year’s binary size budget for a large team, because binary size affects update success rates, startup time, memory, crash rates, and how many users are running the latest version at any given moment. Meta explored dynamic download of the codec as a separate component, but download failures in poor network conditions degraded the same users the codec was supposed to help. They ended up contributing binary size reductions directly to the dav1d decoder project and removing unused encoder tools like the quantization matrix to claw back 60 kB per optimization pass.
Device eligibility at Android scale is intractable without ML. iOS has a small, predictable device matrix. Android has thousands of device models with wildly varying SoC configurations. Memory, OS version, and release year all proved insufficient as eligibility signals on their own. Meta’s solution was an ML-based framework trained on real-world performance metrics collected at call time — encoding and decoding latency measured in production, not lab benchmarks. The model outputs an rtc_score per device, which feeds both the eligibility decision and ongoing quality optimization as more AV1 traffic generates more training data. Model V2 introduced a two-tier approach distinguishing high-end from low-end devices, reflecting the real difference in AV1 encoding capability between a flagship and an entry-level phone.
Even device eligibility wasn’t sufficient. During A/B tests, a 2023 octa-core Android phone with a positive eligibility signal couldn’t encode 320×180 at 15fps in real time. The suspected cause is CPU frequency throttling during calls — a device that passes a synthetic test can fail under sustained thermal load. The fix is adaptive codec control during the call itself: continuous monitoring of encoding and decoding latency, with three distinct mechanisms to respond. Adaptive encoder preset adjustment reduces complexity within AV1 if encoding latency grows. Encoding latency-aware codec switching drops to H.264/AVC if preset reduction isn’t enough. Decoding latency-aware codec switching acts on feedback from the peer — if the receiver reports it can’t decode AV1 in real time, the sender switches, because a high-end sender calling a low-end receiver still needs a workable call.
This last mechanism enabled asymmetric codec design: mid-range devices that can’t encode AV1 in real time can still decode it. A high-end sender uses AV1; the mid-range receiver handles the decode. This asymmetry significantly expands AV1 coverage across the Android ecosystem without requiring full bidirectional AV1 capability.
The error resilience work addresses the fundamental conflict between AV1’s inter-frame compression and RTC’s packet loss reality. Temporal layers help by creating a base layer that decodes independently at a lower frame rate — enhancement layers add frames but aren’t required for continuity. Meta applies FEC to protect base-layer data and treats enhancement-layer retransmissions conservatively based on round-trip time. Temporal layers are enabled adaptively, turned on when loss rises and off when conditions recover, because the compression efficiency cost of always-on temporal layers is real. Long-Term Reference (LTR) frames add a different kind of resilience: the encoder periodically marks a high-quality frame as an LTR, pins it in a reference buffer, and can respond to loss events by producing an LTRP frame predicted from that pinned reference. Recovery from a broken dependency chain takes one round trip rather than a full keyframe cycle — which itself would spike the bitrate and potentially cause the congestion it’s meant to fix.
My take
The ML-based device eligibility framework is the part that doesn’t get enough attention in the summary. The insight that lab performance data is an unreliable predictor of real-world RTC performance — because devices throttle under sustained load, because network conditions interact with encode complexity, because call-time conditions differ from benchmark conditions — means that any eligibility system built on static device specs will have systematic blind spots. Training on real call data closes that gap iteratively. The fact that more AV1 traffic generates a better training set creates a positive feedback loop that’s self-correcting in a way that static rules aren’t.
The asymmetric codec design is worth noting as a general pattern for any feature rollout gated on bidirectional capability. Separating encode eligibility from decode eligibility nearly always expands the addressable population, because decoding is consistently less expensive than encoding across device tiers.
Where this is useful
- Deploying any computationally expensive codec to a heterogeneous mobile device fleet
- Building device eligibility systems where lab benchmarks don’t predict production behavior
- Designing RTC systems that need to degrade gracefully under packet loss without keyframe storms
- When NOT applicable: offline transcoding pipelines where latency, call duration, and peer device state are irrelevant
Apply this: Separate decode eligibility from encode eligibility in your codec rollout — many devices that can’t encode a newer codec in real time can still decode it, and asymmetric coverage is better than no coverage.
What I want to remember
Remember: RTC rate control must prevent undershoot as much as overshoot — consistent undershooting misleads bandwidth estimation, slows bitrate ramp-up, and degrades quality just as surely as congestion does.