Come, meet us at brightonSEO, UK on 25th and 26th April 2024 Click here
Scroll to top

Googlebot Renders and Indexes AJAX-Style Dynamic Content

Dynamic Content

Now it is possible to index your dynamic content through Google

There is a good news for those who use latest JavaScript to query online resource APIs and create dynamic content in pages. Google can now render and index client-side AJAX-style JavaScript POST requests and this development has come along with Evergreen Chromium.

HTTP Background

The HTTP standard explains a number of request types, out of which GET is most popular. Browsers, for instance, use GET to retrieve a URL when you either type it in the address bar or click links etc. The POST request type differs from GET because it comes with a payload definition that is meant to be opened on the server for use with an application program. HTML forms also use POST to send text from input fields for processing on the server itself.

POST problems   

After early experimentation, search engines usually avoid placing input fields for making POST requests on their own. If a site is created with rich database content which is accessible through a site search engine field without easy discovery of links to its results pages, it can’t be indexed—even by today’s Googlebot.

With the development of JavaScript, client-side POST requests have been programmatically available through browser’s XMLHttpRequest (XHR) object.

Some POST Requests Now Work with Google

Google’s new Evergreen Googlebot can crawl and even index XHR POST requests but the question is whether it does or doesn’t was prompted by technical SEO Valentin Pletzer, who follows the Evergreen Googlebot quite closely. One must be aware of the crawlers that still do not possess this capacity.

What you need to note

When Google renders dynamic content which is driven by XHR POST request method, each sub-request will count against your overall crawl budget. The content from the POST event isn’t cached, which actually decrease your crawl budget by number of XHR requests to assemble the page. Suppose if you had a crawl budget of 100 pages, and your template for them used one XHR POST request each for content on the fly, then only 50 of your pages would be cached for use with Google’s search index.

Author avatar
Public advisory against the common scams. Click to read more.