You optimized for a reader that wasn't the problem
Here's a pattern I keep seeing on performance-obsessed teams. They spend a full quarter chasing Core Web Vitals. Lazy-loaded images, deferred scripts, a font-loading strategy someone argued about for two sprints. LCP drops under 1.5 seconds. CLS is basically zero. The Lighthouse score is a wall of green and everyone screenshots it for the all-hands.
Then someone asks why ChatGPT still describes the product wrong, or why Perplexity cites a competitor for a query you clearly answer better. And the performance work you just shipped did nothing to fix it. Not because it was bad work. Because it solved a problem the AI crawler doesn't have.
Core Web Vitals measure human-perceived speed. Layout stability, paint timing, input responsiveness. Those are visual, interaction-heavy signals that only matter to a person looking at a screen. A text-focused AI crawler isn't looking at anything. It wants your content as text, ideally in the raw HTML, and it wants it fast enough to be worth fetching. Those are two different scorecards, and green on one tells you almost nothing about the other.
Where the two scorecards actually diverge
Raw response time still matters for any crawler. A server that takes four seconds to return bytes burns crawl budget and gets fetched less often, human or bot. That overlap is real and worth keeping. But the specific metrics performance teams chase mostly live above that layer, in the render phase that many AI crawlers skip entirely.
The sharpest failure is client-side rendering. If your key content only appears after JavaScript runs, a crawler that doesn't execute JS sees an empty shell. Your LCP is gorgeous because the browser painted the hydrated page in 1.4 seconds. In this illustrative render audit, the bot pulled 18% of your actual copy and moved on. Layout shift, interaction delay, image optimization: none of it changes what a text crawler extracts. Meanwhile the things that do matter for AI, like whether your answer sits in server-rendered HTML and whether it's structured cleanly, never show up in a Web Vitals report.
Two scorecards, two audiences
| Comparison category | Human-speed optimization (Core Web Vitals) | AI-crawler optimization (AEO) |
|---|---|---|
| Primary audience | A person watching the page paint | A text parser that may not run JS |
| Metrics that move it | LCP, CLS, INP, image weight | Raw HTML content coverage, response time, structure |
| Does client-side rendering hurt? | No — browser hydrates fine | Yes — content can be invisible |
| What a green score proves | Page feels fast to humans | Nothing about what the bot extracted |
| Typical tooling | Lighthouse, CrUX, PageSpeed Insights | Crawler-view diffing, answer coverage checks |
Separate the concerns before you spend the next quarter
The fix isn't to abandon performance work. Fast servers help everyone and human speed still affects human conversion. The fix is to stop treating one green dashboard as proof the other problem is solved. They're different jobs for different readers.
This is exactly the split Crescive draws. It fetches your pages the way a text-focused AI crawler does, with JavaScript off, and shows you what actually survived: how much of your content is in the raw HTML, whether the answer to each target question is present, and how that maps to how assistants describe and cite you. When the gap is a rendering problem, it flags the pages where content vanishes without JS and drafts the server-side fix behind an approval gate. When the gap is genuinely response time, it says so instead of sending your team to re-optimize LCP for the tenth time. You end the quarter knowing which scorecard needed the work, and you can prove, under Crescive's automated criteria, the AI-visible content changed, not just the Lighthouse number.
Key takeaways
- Core Web Vitals measure human-perceived speed. They tell you almost nothing about what a text-focused AI crawler extracted from the same page.
- Raw response time overlaps for both audiences, but render-phase metrics like LCP and CLS don't map onto crawlers that skip JavaScript.
- Client-side rendering is the most common trap: a fast, hydrated page can still hand an AI crawler an empty shell.
- Audit the crawler's view separately before committing another quarter to performance work that won't move your AEO scorecard.
FAQ
Do Core Web Vitals affect how AI crawlers rank or cite my pages?
Not directly. Core Web Vitals measure human-perceived speed and visual stability, such as LCP, CLS, and INP, which matter to people viewing a page in a browser. Most text-focused AI crawlers don't render the page visually, so those metrics don't map onto what they extract. Raw server response time still matters because slow responses waste crawl budget, but the specific human-speed metrics performance teams optimize don't determine AI visibility.
Why does my page load fast but still show up empty to AI crawlers?
This usually means the page relies on client-side rendering. The browser runs JavaScript to build the content, so a human sees a fast, complete page and your LCP looks great. But an AI crawler that doesn't execute JavaScript receives only the raw HTML, which may contain a small fraction of your actual content. The fix is to serve key content in server-rendered HTML so it's present before any JavaScript runs.