HTTP headers are typically displayed in various cases, often in Pascal-Case like Content-Type. As per the HTTP specification, header names are case-insensitive, meaning content-type and Content-Type are identical.
However, different browsers handle this matter in diverse ways. For instance, under the HTTP1.1 protocol, Chrome and Firefox display the header name in the same case as the server sent it, or in Pascal-Case. This implies that when utilizing web scraping tools with HTTP1.1, it’s crucial to mimic the exact case of expected headers to avoid scraper blockage.
For HTTP2 and above, all headers are mandated to be in lowercase. Therefore, the scraper should consistently send headers in lowercase when scraping through HTTP2-compatible clients.
