4 Augmented Reality Live Streaming SDKs Compared on the Frame Budget

TL;DR

• For augmented reality live streaming, the AR SDK shares the device with the video encoder, so treat the frame rate a vendor publishes for an idle phone as a ceiling.
• DeepAR publishes the most usable numbers of the four I screened: per-device frame rates on five named devices, head angle and distance, plus a public MAU price table. Its own performance page also contradicts itself twice, and I show where.
• BytePlus Effects documents the widest tracking envelope, up to 10 portraits per frame and yaw and pitch within ±90°, plus the longest platform list. It publishes no price and no web SDK.
• Snap Camera Kit gives you Snapchat's Lens pipeline on iOS, Android and Web, with Snap branding and an app review step attached.
• MediaPipe Face Landmarker is Apache 2.0 and gives you landmarks, blendshapes and transforms. The renderer and the effects are yours to build, and its frame-dropping stream mode is not available on Web.
• No measurements of mine in this one. It is the screening pass that decides which two go onto real devices next.

When I add AR effects to a live stream, the frame budget breaks first. The camera, the AR pipeline and the encoder all want the same GPU and the same 33 ms per frame at 30 FPS, and the stream drops frames long before the filter looks wrong.

This post is step zero of that work. I read each vendor's own documentation in November 2026 and wrote down what they publish and what they leave out. Vendor pages change, so anything that decides a build should come from the vendor in writing.

What does a live stream need from an AR SDK?

I screen augmented reality live streaming candidates on five criteria. Each one comes from something that went wrong in a production stream.

1. Frame rate on a phone that is also encoding. I look for per-device numbers, not a single "real-time" claim, and I want at least one mid-range or older device in the list.
2. Tracking envelope. Streamers turn their heads, lean back and get up. Head angle and distance limits tell me how often the effect will fall off.
3. Faces per frame. Co-streams, duets and interviews put two or more people in front of one camera. I ran into the same limit on video calls in my face tracking tests for conferencing apps.
4. Delivery path. Does the vendor document an integration with the streaming SDK I already run, or do I write the frame hand-off myself?
5. Access and pricing. Open sign-up, a contact form or an approval step. A team can finish the technical evaluation and find there is nothing it is allowed to buy yet.

1. DeepAR

DeepAR documents the SDK on four platforms in its feature table: iOS, macOS, Android and Web. Background replacement, background blur and multiple face tracking are marked available on all four. Emotion detection is not available on Web, hair segmentation is iOS only, and body tracking is in closed beta on every platform.

What the docs publish

The performance page is the most useful page any of the four vendors has for this criteria list. DeepAR names its test devices: iPhone 13 Pro, Galaxy S23, Pixel 8, Galaxy S21+ and a 13-inch 2020 MacBook Pro.

Face tracking on the Galaxy S21+ under Android runs at 43 FPS, with tracking lost past 60 degrees and 180 cm. Maximum angle is 60 degrees in every row of that table. Maximum distance moves between 150 and 180 cm depending on the device.

Two places where the page argues with itself

DeepAR's prose says the frame rate is capped at 30 FPS on iOS and Web for battery efficiency. Its own tables then show 60 FPS marked "capped" for the iPhone 13 Pro in Safari, 60 for the Galaxy S23 in Chrome, and 50 for the iPhone 13 Pro under beautification. So the 30 FPS sentence is not a cap you can plan against. Read the cell, not the sentence.

The second one is smaller and tells you how the page was assembled. The background replacement table reports two rows for an iPhone Xs Max, and the iPhone Xs Max is not in the page's own list of test devices.

The number I would plan around

Background replacement drops to 22 FPS on the Galaxy S21+ in Chrome and 21 FPS on that unlisted iPhone Xs Max in Safari, against 35 FPS for the same Galaxy S21+ in the native Android SDK. DeepAR states that values on the page are for reference only, and lists room temperature, battery level, device load, session length, OS version and rendering resolution as things that move them. For a browser broadcast with a virtual background, that is the line I would re-measure first.

Delivery path

This is where DeepAR is ahead of the other three, but the support is not uniform. It publishes a full integration guide for Amazon IVS on iOS and Android, and names itself the official partner of Amazon IVS. Azure Communication Services is covered by Microsoft's own tutorial rather than a DeepAR page. Agora and Vonage appear as sample repositories on the video-calling index, not as written guides.

That difference matters more than the list length. A guide is a supported path, a sample repo is a starting point you maintain yourself, and there is no web guide even for Amazon IVS.

Access and pricing

Sign-up is open. Pricing is a public MAU table: free up to 10 MAU, then $25 per month up to 1,000, $100 up to 5,000, $500 up to 30,000 and $1,000 up to 100,000, with contact sales above that. The free license carries a watermark.

One detail matters for streaming audiences: a user who switches devices or browsers may be counted separately for each combination in the same month. The Beauty API is in beta, with a watermarked preview that shuts off after 3 minutes.

Where it wins

Published per-device numbers, one real streaming integration guide, and a price you can put in a spreadsheet before the first call.

2. BytePlus Effects

BytePlus Effects is ByteDance's effects SDK for third-party apps. The technical requirements page lists Android, iOS, Windows, macOS and Linux, with armeabi-v7 and arm64-v8a on mobile and x86 on desktop. The documented interfaces elsewhere in the same docs are C and Java.

What the docs publish

The tracking envelope is the widest of the four: yaw within ±90 degrees, pitch within ±90 degrees, and up to 10 portraits per frame. Face detection speed is given as under 2 ms on an iPhone 7. Android 6 and iOS 10 are the floors.

There is also a condition that is easy to miss. In video mode a face has to account for more than one eighth of the short side of the screen, and one sixteenth in photo mode. A streamer who steps back from a laptop camera can fall outside that.

What is missing for this use case

The 2 ms figure is face detection on one device, not the cost of a rendered effect while the encoder runs. I found no per-device frame rate for a full effect. The technical requirements list no web or browser SDK at all, and neither Flutter nor React Native appears on it.

Access and pricing

The product page claims 80,000+ effects and filters and says you can integrate effects into your app in two weeks. Every free trial button on it goes to a contact form, and the page states no price.

Where it wins

Faces per frame, head angle range and desktop coverage including Linux.

3. Snap Camera Kit

Snap Camera Kit puts Snap's Lens pipeline inside your own app on iOS, Android and Web. You build Lenses in Lens Studio, and the same Lenses can also be published on Snapchat.

What the docs publish

The overview is about the Lens workflow rather than performance. I found no frame-rate, angle or face-count figures on the overview page. For a live stream that means the encoder budget question stays open until you measure it yourself.

Access and pricing

Snap branding is required when a Lens is instantiated, and the documentation has a "Submit for App Review" section. The overview page I read states no price.

For a white-label streaming product, the branding requirement is the first thing to clear with whoever owns the brand, and it is a contractual question rather than a technical one.

Where it wins

The content pipeline. If your creators already build Lenses in Lens Studio, nothing else on this list lets them reuse that work.

4. MediaPipe Face Landmarker

MediaPipe is the open-source option, released under Apache 2.0 according to the LICENSE file in the repository. The Face Landmarker guide publishes setup guides for Android, iOS, Web and Python.

What the docs publish

The task predicts 478 3D facial surface landmarks per detected face, 52 blendshape scores, and a 4x4 facial transformation matrix that the guide describes as there "to perform the transformations required for effects rendering". In other words, it gives you geometry. The effect library, the renderer and the per-device tuning are your code.

The detail that matters for streaming, and the one that limits it

The LIVE_STREAM running mode is asynchronous and, in the guide's words, "automatically drops incoming frames when busy so results stay current with low latency". That is the right behavior for a stream, because a late landmark is worse than a skipped one. Your effect can then update less often than your video does, and you decide how the renderer fills the gap.

The limit is where that mode is available. The Web configuration table has no LIVE_STREAM option. On the browser, the guide pairs VIDEO mode with requestAnimationFrame so each paint handles the latest camera frame, and the Video mode is documented as processing every frame in order without dropping any. If your broadcast is a browser tab, you are building that back-pressure yourself.

One default to know before you plan a co-stream: num_faces is 1, and temporal smoothing is tied to it being 1.

Access and pricing

No sign-up, no license fee.

Where it wins

Cost, license and control over every frame. It loses on everything a team without CV engineers would need to ship. If you want the wider field of commercial face AR engines first, I went through six Face AR SDKs and their published specs earlier.

Augmented reality live streaming SDKs side by side

Every figure here is the vendor's own, read from its documentation in November 2026. None of them is my measurement.

Why does the stream drop frames when the AR filter turns on?

Because the vendor's number was measured without your encoder. Every frame rate above was taken with the AR pipeline as the main workload. In a live stream, the hardware encoder, the network stack and often a second camera preview compete for the same frame time.

This is why I do not trust a single FPS figure for this use case. I trust a vendor that names the device, and then I re-run that device with the encoder on.

DeepAR's own gap between 35 FPS native and 22 FPS in Chrome on one phone is a reminder of how much the runtime alone moves the result, before anything else is running.

Which documented constraint rules an option out?

I use the docs to cut the list. Picking between the survivors needs a device run.

• You broadcast from a browser: BytePlus Effects drops out, because its technical requirements list no web SDK. MediaPipe stays, but without its frame-dropping stream mode.
• You need a white-label product: Snap Camera Kit needs a conversation first, because Snap branding is required.
• You have no CV engineers: MediaPipe drops out, because you would be writing the renderer and the effects.
• You need a price before procurement: BytePlus and Snap need a sales or account step first. DeepAR does not.
• You co-stream with several people in frame: BytePlus is the only one here that publishes a face count, so the others need that number from the vendor in writing.
• You already run a specific streaming SDK: check whether the vendor publishes a guide or only a sample. Of the four, only DeepAR publishes either, and only Amazon IVS gets a guide.

The four documents disagree on what "real-time" means

DeepAR means a frame rate on a named phone. BytePlus means a detection time on an iPhone 7. MediaPipe means it will drop frames to stay current, except on the web. Snap does not put a number on it at all.

None of these is wrong. They answer different questions, and only the first one answers the question a live stream asks.

My next post in this series takes the two that survive my own constraints onto a mid-range Android and an older iPhone, with the encoder running, and reports every run including the ones I throw away.

FAQ

Which AR SDK has published frame rates for live streaming?

Of the four I screened here, DeepAR is the only one with per-device frame rates in its docs, on five named devices. BytePlus publishes a face detection time on an iPhone 7, which is a different measurement. Other vendors outside this four publish per-device or per-tier figures too, so treat this as a result about these four rather than about the whole market.

Does MediaPipe Face Landmarker work for live streaming?

Yes on Android, iOS and Python, where the LIVE_STREAM running mode drops frames when busy to keep results current. On the web there is no LIVE_STREAM option, and the guide pairs VIDEO mode with requestAnimationFrame instead. It outputs landmarks, blendshapes and transforms, so you build the effect rendering yourself.

How many faces can an AR SDK track in one stream?

BytePlus Effects documents up to 10 portraits per frame. DeepAR lists multiple face tracking on all four platforms but publishes no maximum. MediaPipe defaults to 1 and is configurable. Ask any vendor for the number in writing, on a device you actually ship to.

Is there an AR SDK for live streaming with public pricing?

DeepAR publishes an MAU table that runs from free at 10 MAU to $1,000 per month at 100,000 MAU. BytePlus Effects routes every trial to a contact form, and I found no price on the Snap Camera Kit overview page.

Does DeepAR cap frame rate at 30 FPS on iOS and the web?

Its prose says so, for battery efficiency, and its own tables disagree: the iPhone 13 Pro in Safari and the Galaxy S23 in Chrome both show 60 FPS marked as capped. Take the per-device cell rather than the sentence above it.

Why does my stream lag when I add an AR filter?

The AR pipeline and the video encoder share the same frame time on the device. A vendor frame rate measured without an encoder running is a ceiling. Re-measure on your lowest supported device with the stream live.