The API was perfect. The user got nothing.

Most teams shipping a chat assistant test it at the API layer. Eval suites score the model's answers. Contract tests check that the endpoint returns the right shape. A staging script posts a message and asserts that a stream comes back. When all of that is green, the feature is considered working.

Below are seven bugs, all from the public issue trackers of widely deployed chat and assistant widgets, where all of that was green. In each one the backend behaved correctly, in several cases verifiably byte-for-byte correctly, and a person sitting in front of the product got nothing: no answer, no way to type, in two cases no usable page at all. They come from a wider sweep that found 18 such failures; these seven have enough public detail to check every claim, and each one links to its issue.

1. The answer arrived. Nobody saw it.

LibreChat is one of the most widely deployed open-source chat front ends. In issue #15044, a user sends a message and the reply streams back over the network in full. Every delta arrives on the wire, correctly keyed to the right conversation. The chat window stays empty. The reply only appears after a full page reload, and the wedged submission state blocks the next message too. The reporter's own summary: "the transport is perfect, the bug is purely client-side rendering." A test that asserts on the stream, which is what an API test is, passes on every single run of this bug.

2. The API said online. The widget said away.

Chatwoot's live-chat widget asks the backend whether agents are available. In issue #12868, the network trace shows the widget receiving availability_status: online. The widget then tells the visitor that the team is away. The reporter measured the failure at 100% on fresh page loads, and the cause was a front-end reactivity bug: the correct data was demonstrably sitting in the client's store the whole time. From the business's side nothing is wrong. Agents are online and the API confirms it. From the visitor's side, the site just said nobody is home, and a visitor who believes an away message closes the tab.

3. The widget failed to load and took the page with it.

Chatwoot issue #7974: when the widget cannot reach its server (the report is from mobile, with the websocket connection failing), it still injects its full-screen iframe. The iframe is transparent and empty, and it sits on top of everything. The chat is invisible, and the entire host page underneath stops responding to clicks. Notice what failed here. Not a request, not a response. There is no API surface involved at all. The bug is that an iframe exists with the wrong lifecycle, and no test that observes traffic can even express that, because the failure is a rectangle.

4. Blank in Safari.

Chainlit's embeddable copilot authenticates through a cookie set by a /jwt call. Safari blocks third-party cookies by default, and so do private windows and any browser with tracking protection turned up. Issue #2013 documents the result: the embed renders a blank screen. No error, no fallback, nothing to act on. The backend is fine; test it from curl, or from Chrome with default settings, and everything passes. The users who see nothing are not an edge case. They are every default-configuration Safari user, which includes essentially everyone opening the page on an iPhone's built-in browser.

5. The bot's own answer crashed the page.

Sendbird's chat-ai-widget, issue #314: a bot reply includes a markdown link whose URL carries an encoded token. The widget's renderer runs the URL through decodeURIComponent, which throws URIError: URI malformed, and the whole host page goes blank. The model produced a correct answer containing a working link. The rendering layer destroyed the answer and the page around it. It was a regression between two widget versions, with no backend change anywhere in the story. An eval that scores the model's output would score this conversation as a success.

6. The escalation number that dialed nothing.

Same widget, issue #333: markdown phone links in bot messages, the tel: kind, were rewritten by the widget to href="#". On a phone, tapping the number the agent just offered does not place a call. It reloads the page. Consider when a bot hands out a phone number: it is the moment the bot has decided a human is needed. The handoff the model correctly offered was dead on arrival, and no log says so, because a click on # produces no request worth noticing.

7. The users who could never type at all.

CopilotKit issue #3318: the chat input's layout code briefly clears the textarea on every input event to measure line height. For alphabetic scripts this is invisible. Chinese, Japanese, and Korean are entered through an IME that composes a character across several keystrokes, and clearing the field cancels the composition every time. On mobile, CJK users could not enter a message at all. This is the purest case on the list. The conversation never starts, so no request is ever sent, so there is nothing at the API layer to be wrong. The API's perfect record is the bug's camouflage.

Why this is a class, not a coincidence

Line the seven up and the pattern is plain. The failures live in rendering pipelines, front-end state timing, markdown sanitizers, iframe lifecycles, browser cookie policy, and input methods. Every one sits between the byte stream and the human. Every one is invisible from the request/response boundary, which is exactly where almost all agent testing stands.

That is not bad luck in four codebases. All four projects are widely used and actively maintained, and the bugs share nothing except their location. Some things only exist in a real browser meeting a real user: a viewport, a cookie jar with a policy, an IME mid-composition, a finger that needs a live click target, a DOM that a sanitizer has already rewritten. A test that does not have those things cannot in principle see these bugs. This is not fixable with more assertions at the API layer, because no assertion at that layer can express "the transparent iframe is on top of the checkout button."

Invisible twice

There is a second absence worth noticing. The public AI incident databases, the ones that catalog what goes wrong when AI systems meet the world, classify incidents by harm: bias, misinformation, physical safety, malicious use. None of their taxonomies has a category for a widget that renders nothing, and none of the 18 failures behind this article appears in any of them. So the class is invisible twice. Once to the tests, because they watch the wrong boundary, and once to the collective record, because a blank chat window is nobody's definition of an AI incident. Each team rediscovers the class alone, in its own tracker, after its users have already met it.

Simulated users will not save you either

The current answer to "API tests are too narrow" is user simulation: have an LLM play the customer and run whole conversations. That helps with conversation quality, and it does not help here, for two reasons. First, most simulated users are wired to the API too. A synthetic customer that speaks HTTP has no Safari cookie policy and no IME to interrupt. Second, a recent evaluation framework for user simulation (arXiv:2605.02624) finds that simulated users fail to reproduce the communication friction real users bring to a conversation, and concludes that evaluations built on them skew optimistic. So the standard stack is optimistic twice over: the simulated user is cleaner than a real one, and the vantage point hides the layer where all seven of these bugs live.

What would have caught them

Nothing exotic. A test that opens a real browser on the real host page and behaves like a person: a Safari profile with default cookie settings, a phone-sized viewport, an IME composing Chinese text, a finger that actually taps the phone link the bot offered. From that vantage, every one of the seven is obvious in seconds. The screen is blank. The status is wrong. The tap does nothing. The page will not click. None of this takes judgment calls or model-quality debate. It takes standing where the user stands.

That is the layer JinuGen tests. It watches your product through a real browser, the way a person meets it, and reports what a person would actually have gotten. If you ship a chat widget, you can run a free scan at jinugen.com.