Comprehensive Guide: How to Use Proxies PHP Guzzle Effectively

PHP’s Guzzle is a powerful HTTP client that is integral for developers who leverage web scraping to gather data across the internet. Utilizing Guzzle allows for sophisticated HTTP requests and handling responses in a streamlined manner, making it a preferred tool for many web scraping projects. However, a significant aspect of successful web scraping lies in the effective use of proxies. Proxies serve as intermediaries that can mask your IP address, manage request rates, and facilitate access to geo-restricted content, thereby ensuring the efficiency and anonymity of your web scraping efforts. This guide aims to shed light on the synergy between Guzzle and proxies, offering insights on how to seamlessly integrate web scraping APIs into your projects. By mastering the use of proxies with Guzzle, you can enhance your data collection techniques, ensuring they are both robust against countermeasures and optimized for performance.

<?php
require 'vendor/autoload.php';

use GuzzleHttpClient;

// Proxy pattern is:
// scheme://username:password@IP:PORT
// For example:
// no auth HTTP proxy:
$my_proxy = "http://160.11.12.13:1020";
// proxy with authentication
$my_proxy = "http://my_username:my_password@160.11.12.13:1020";
// Note: that username and password should be url encoded if they contain URL sensitive characters like "@":
$my_proxy = 'http://'.urlencode('foo@bar.com').':'.urlencode('password@123').'@160.11.12.13:1020';

$client = new Client([
    // Base URI is used with relative requests
    'base_uri' => 'https://httpbin.dev',
    // You can set any number of default request options.
    'timeout'  => 2.0,
    'proxy' => [
        'http'  => $my_proxy,      // This proxy will be applied to all 'http' URLs
        'https' => $my_proxy,      // This proxy will be applied to all 'https' URLs
        'https://httpbin.dev' => $my_proxy,  // This proxy will be applied only to 'https://httpbin.dev'
    ]
]);

$response = $client->request('GET', '/ip');
$body = $response->getBody();
print($body);

Unfortunately, Guzzle does not support SOCKS proxies. The only alternatives are PHP’s curl library or buzz.

It’s worth noting that Guzzle proxy can also be set through the standard *_PROXY environment variables:

$ export HTTP_PROXY="http://160.11.12.13:1020"
$ export HTTPS_PROXY="http://160.11.12.13:1020"
$ export ALL_PROXY="socks://160.11.12.13:1020"

When web scraping, it’s recommended to rotate proxies for each request. For more information on this, check out our article: How to Rotate Proxies in Web Scraping