Comprehensive Guide: How to Use Proxies NodeJS Axios Effectively

Axios, a prominent HTTP client for JavaScript, is particularly favored for web scraping tasks within the Node.js environment due to its ease of use and promise-based nature. This library facilitates making HTTP requests in Node.js applications, enabling developers to easily interact with APIs and fetch web content. When incorporating proxies into your Axios requests, it becomes possible to enhance privacy, circumvent geo-restrictions, and manage IP rate limiting effectively, making it an essential strategy for robust web scraping projects. To employ proxies with Axios, developers can utilize the proxy configuration in their request options, allowing requests to be routed through a specified proxy server. This setup is instrumental in executing web scraping tasks that require a high degree of anonymity and flexibility. For those aiming to further optimize their web scraping capabilities, integrating a proxy API for web scraping can offer advanced solutions that simplify complex data extraction challenges, providing a seamless and efficient way to access and collect data from the web.

To utilize proxies with axios, the proxy parameter can be employed in get and post methods:

const axios = require('axios');
axios.get('https://httpbin.dev/ip', {
  proxy: {
    host: "160.11.12.13",
    port: "8080",
    auth: {
      username: "username",
      password: "password",
    }
  }
})
.then((response) => {
  console.log(response.data);
})
.catch((error) => {
  console.error(error);
});

It’s important to note that axios does not support automatic proxy use through terminal environment variables HTTP_PROXY, HTTPS_PROXY or ALL_PROXY.

While axios does not directly support SOCKS proxies, this can be enabled through the socks-proxy-agent library:

const axios = require('axios');
const SocksProxyAgent = require('socks-proxy-agent');

const proxy = 'socks5://localhost:1080';

const httpAgent = new SocksProxyAgent(proxy);
const httpsAgent = new SocksProxyAgent(proxy);

axios.get('http://example.com', {
  httpAgent,
  httpsAgent,
})
.then((response) => {
  console.log(response.data);
})
.catch((error) => {
  console.error(error);
});

Lastly, when web scraping, it’s recommended to rotate proxies for each request. For more information on this, refer to our article: How to Rotate Proxies in Web Scraping