Streaming, loading & errors
AI responses take seconds, not milliseconds — streaming the draft in as it generates, honest loading states, and the error UX that keeps trust when the magic hiccups.
Every API call so far answered in milliseconds; a model call takes 2–10 seconds, and those seconds are a UX problem you must design, not an inconvenience to apologize for. Three tools:
- Streaming — the model can send its answer word-by-word as it generates, and your route can pass that stream straight through to the browser, so the draft types itself into the reply box. Same total time, transformed feel: progress is visible from the first half-second. The assistant wires this with the provider's SDK (its ready-made code kit) in streaming mode + a standard pattern on the frontend; your review job is the seams, checked behaviorally, not by reading async code: click Draft, navigate away mid-stream, and look in the Network tab — the request should show as cancelled (don't pay for orphans); and a half-arrived draft should stay in the box, editable — half a good draft still beats a blank one.
- Loading states that tell the truth — the Draft button disables and shows progress while working ('Drafting…'), because double-clicks during silence are how you pay for three drafts and get a race condition (two requests racing to finish, landing in the wrong order). Skeletons and spinners aren't decoration; they're the app saying 'I heard you' during the only slow thing it does.
- Error UX in three registers: transient (timeout, overloaded — 'Drafting hit a snag — try again', with the retry preserving their message context), config (bad key, no credit — logged loudly for you, shown gently to the user as 'AI drafting is unavailable right now'), and content (the model returned something malformed or empty — fall back to the empty reply box without fanfare). In all three: the send-by-hand path remains fully usable. Write the three messages as product copy; error strings are the UX you're judged by on your worst day.
Two AI features, one pattern
Replyable ships two AI touches, and they share every pattern above: Draft reply (the star: message + tone setting + prompt → streamed draft into an editable box, was_ai_drafted recorded when sent) and Summarize (for long messages: a one-line TL;DR above the body, generated once and cached in the database — no reason to pay twice for the same summary; a summary column and a check-before-calling is your first taste of caching, the single most useful cost lever in AI apps). Notice the shape you now own: context in → prompt → model → validated output → human approves. That shape is this entire company's product line in miniature, and you built it from a spec.
The moment you send customer messages to a model API, you've made a data-flow decision: someone else's servers process your users' words. For v1 (your own test data) this is trivially fine; before real users, it's a disclosure line in your privacy note (Module 7) and a check of the provider's data-handling terms (no training on API inputs — the mainstream providers' standard commercial stance, but verify, in writing). Small app, real principle: you know exactly where your users' data goes, or you don't ship.