A feature that works for one user can fall over the moment it does real work

We shipped grounded probing: instead of asking AI engines from memory, the probe makes them actually search the live web, across seven engines (ChatGPT, Gemini, Claude, Perplexity, Grok, DeepSeek, and Google AI Overviews). It worked beautifully in testing. Then a full run hit a wall and returned an error that read, oddly, "Unexpected token, not valid JSON." That error is a good story about how real web work breaks serverless apps, and what we did about it.

The clue in a strange error

"Not valid JSON" was a red herring. The truth underneath was a 504, a gateway timeout (what that is). Our app runs on serverless functions capped at about 26 seconds. A grounded probe does real web searches across seven engines, repeated for reliability, plus a competitor comparison. Each live search takes several seconds. Stack them up and the work runs well past 26 seconds, so the platform kills the request and returns an error page. The browser then tries to read that HTML error page as data and complains about the "less-than" sign it found. The scary message was just a timeout wearing a costume.

Why you cannot just "make it faster"

The obvious instinct is to trim the work until it fits. We did some of that (the side-by-side comparison now uses a lighter pass). But trimming is fragile: a single slow search from one engine can still blow the budget, and we are not willing to drop engines or accuracy to win a race against a stopwatch. The real fix was to stop racing.

The fix: do the slow work in the background

We moved the heavy probe to a background job. Now when you click run, the request returns immediately with a ticket, a separate background process (allowed to run for up to fifteen minutes) does the actual searching, and the dashboard quietly checks back every couple of seconds until the results are ready. You see a "running in the background" note, then the full result appears. No timeout, because nothing is waiting on a 26-second clock anymore.

This is a standard pattern for any genuinely slow job: hand back a ticket, work in the background, let the client poll. It is less glamorous than a single instant response, but it is the difference between a feature that demos well and one that survives real use.

The honest footnote

We were straight with ourselves about the trade-offs. Two enrichment features (a semantic-space logging step and the optional meaning-based judge) are not in the background path yet, because they depend on parts of the app that do not bundle cleanly into a background process. They still work in the fast path. Moving them over is a follow-up, and we would rather ship the working core than block it on the extras.

What to do next

If you run a grounded or "both" probe, expect it to think for a minute or two and fill in when ready. That pause is the system doing real web searches properly rather than cutting corners. For what those grounded results actually mean once they arrive, read Parametric vs Grounded.

Key Takeaways

• Grounded probing makes seven AI engines search the live web, which is genuinely slow.
• A "not valid JSON" error was really a 504 timeout: serverless functions cap at about 26 seconds.
• Trimming work to fit the clock is fragile, because one slow search can still blow the budget.
• The fix is a background job plus polling: instant ticket, work runs up to fifteen minutes, dashboard checks back.
• Two enrichment steps are not in the background path yet; that is an honest, scoped follow-up.