Google's Gary Illyes has explained in a blog post explaining how Googlebot's crawling systems operate. The post covers byte limits, partial fetching behavior, and how Google's crawling infrastructure is organized. Googlebot fetches up to 2 MB for any URL, excluding PDFs, and PDFs get a 64 MB limit. Each client sets its own configuration, including user agent string, robots.txt tokens, and byte limits. When a page exceeds 2 MB, Googlebot sends truncated content to Google's indexing systems and the Web Rendering Service (WRS), which processes JavaScript and executes client-side code to understand a page’s content and structure. The limit may change over time as web evolves and HTML pages grow in size. Best practices for staying under the limit include moving heavy CSS and JavaScript to external files.