We Checked 368 Small Business Sites. One in Four Hid Important Content From Non-Rendering Crawlers.

A website can look complete in a browser and return almost nothing useful in its initial HTML.

That gap matters because not every crawler executes the JavaScript that builds a modern page. Google queues successful pages for rendering and runs them through Chromium. Independent testing by Vercel found that several major AI crawlers fetch a page without executing the client-side JavaScript needed to construct its content.

It is also worth being precise about which crawlers we are talking about, because the names get collapsed. OpenAI runs several: GPTBot for potential training data, OAI-SearchBot for ChatGPT search, and ChatGPT-User for certain user-triggered visits. Perplexity separates PerplexityBot, which supports its search results, from Perplexity-User. "The crawler behind ChatGPT" is not one thing.

We wanted to know how often the rendering gap hides something that actually matters on ordinary small-business websites, so we measured it.

What we tested

We attempted dual captures of 405 North American small-business websites and obtained usable results for 368, a 90.9 percent response rate. Each site was fetched twice on the same pass.

The first capture loaded the page in headless Chromium and let its JavaScript execute. The second made a direct HTTP request with a GPTBot user agent and did not execute JavaScript. We then compared the rendered DOM against the initial HTML response and flagged any answer-critical element that appeared in one and was missing or substantially reduced in the other: body copy, headings, internal links, business name, address, phone, email, structured data, and contact or booking forms.

One framing note, because this subject is easy to overclaim. This does not reproduce Google's infrastructure, and it is not a recording of what happens inside ChatGPT, Claude, or Perplexity. It measures the difference between a browser-rendered page and what a non-rendering direct fetch receives. It is not a claim that "ChatGPT cannot know you exist." Those assistants also draw on search indexes, licensed feeds, cached material, and third-party citations. The direct-crawl path exists, and this study isolates one failure mode within it.

The sample was purposive, not random. The full methodology and the open dataset are here.

One in four sites lost something important

Of the 368 analyzable sites, 92 (25.0 percent) contained at least one answer-critical element that appeared in the rendered page but was absent or substantially reduced in the initial HTML.

Body copy was the most frequently affected: on 57 sites (15.5 percent), the words explaining what the business does appeared only after JavaScript ran. Internal links were affected on 51 sites (13.9 percent), which matters because crawlable links are one of the main ways any system discovers the rest of a site and understands how it is organized.

At least one name, address, or phone element was missing from the raw response on 54 sites (14.7 percent). That does not mean the direct fetch received a blank page. It may have received the business name without the address, or the service description without the phone number. For a local business, a partial loss is enough to matter.

The platform result was not what we expected

The usual advice is that hosted builders are bad for SEO and a custom build gives you control. That is not what the platform breakdown shows, but the honest version is more careful than a reversal.

Read the counts, not just the percentages. Wix and Webflow had the lowest observed rates, but each had exactly one affected site. The difference between 2.9 percent and 6.7 percent is mostly the size of the denominator, and neither cell is large enough to crown a platform. Six further sites sat in cells too small to report at all.

The Other / Custom category needs the same caution in reverse. It is a catch-all holding everything from hand-coded static HTML to heavily client-rendered single-page applications. Its 31.8 percent is not evidence that custom development is the problem, because a well-built server-rendered custom site can expose far more useful initial HTML than any hosted builder.

Squarespace is a builder and still sits near the top of the risk list, which is the tell. The platform label was not the mechanism. The mechanism was simply whether important content appeared in the HTML before JavaScript ran. Wix and Webflow tended to do that in this sample. Many affected WordPress, Squarespace, and custom implementations did not.

How to check your own site

You do not need a tool for the first pass. Open your homepage, right click, and choose View Page Source. Do not use Inspect for this test: Inspect shows the live DOM after the browser has run the scripts, while View Source shows the HTML that arrived before they ran. That difference is the whole point.

Search the source for your main heading, a sentence of your body copy, your phone number, your address, and your navigation links.

If you can see that information on the page but cannot find it in the source, client-side rendering is the most likely explanation. Treat that as a reason to inspect the network response and the rendering path, not as a final diagnosis: text can also be encoded differently, serialized into a data blob, or loaded in an iframe in ways a simple search will miss.

Google attempts to render successful pages and can index JavaScript-generated content when that process completes. A crawler that never executes the scripts only ever receives the initial response.

Put the important information in the initial HTML

The safest principle is to server-render or statically generate the content a person or a crawler needs to understand the business: the main copy, headings, navigation, contact details, canonical metadata, and structured data.

That usually does not mean a rebuild. JavaScript can still power forms, calculators, filters, and chat. It should enhance the page rather than be the only way its core information ever exists. Google can render JavaScript and still recommends server-side rendering, static rendering, or hydration, precisely because not every crawler has Google's rendering infrastructure.

Do not let a client-side widget be the only path to reach you.

Compare both versions of your own page

DeepAudit loads the page in Chromium, captures the rendered DOM, and compares it with the initial HTML response. It flags meaningful differences in body copy, headings, links, contact details, forms, and structured data.

A difference is not automatically a defect. It tells you which information depends on rendering, and then you decide whether that dependency is acceptable.

Run the free audit on your own site. It is free, there is no signup, and the report is yours to keep. If you would rather we read it with you, book a free 15-minute teardown and we will walk through what it found, live, no pitch.