Building AI Products for Users on Bad Connections

Streaming responses, hopeful reconnects and generous payloads all assume a connection that a great many users do not have.

Mobile UI comparison contrasting a broken streaming prompt with a low-bandwidth, queue-based AI input state.
WebSockets and continuous streams break on high-jitter connections. Optimising for poor connectivity means local state preservation and compressed payloads.

The default architecture for a modern AI product assumes broadband: a token-by-token stream over a held-open connection, a heavy client, and a retry when something fails. On an intermittent mobile connection, each of those assumptions produces a specific failure.

Streaming is fragile by design

Streamed responses look excellent when the connection holds. When it drops mid-response the user has half an answer, the client often has no way to resume, and the request is billed in full. A queued job with a result the client can fetch when it reconnects is less impressive to demo and far more likely to deliver an answer.

Make the expensive work resumable

Anything that costs money to compute should be recoverable. Give the request an identifier, store the result, and let the client ask again for the same identifier. Users on unreliable connections will retry; without idempotency they are charged twice and may receive two different answers to the same question.

Weigh what you send

Users paying for a bundle notice a heavy application. So does a mid-range phone with limited memory. Server-rendered pages, small payloads and doing the work server-side rather than shipping a large client are unfashionable and correct for this audience.

Voice and language are not extras here

In markets with meaningful variation in literacy and many languages in daily use, voice input and local-language support are the difference between a product being usable and being a demonstration. This is where the shortage of training data in African languages has a direct commercial cost, and where a team with access to that data has an advantage no international competitor can quickly copy.

Get the next one by email

Research Digest

IBM Research Introduces Consistency Diagnostic for AI Agents

IBM Research has addressed the unreliability issue in artificial intelligence agents that pass tests on initial attempts but fail upon repetition. By evaluating decision uncertainty step by step, their new method identifies potential failure points and generates targeted guidelines to ensure repeatable execution.

3 min read

AI News

Google Launches Gemini 3.8 Live and 3.8 Live Extended Thinking

Google has introduced Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking to advance real time voice dialogue. These additions bring parallel background reasoning, mid conversation multilingual switching, and instant tool calling to conversational AI agents.

3 min read