Service Documentation

Overview

BotProxy is infrastructure for sustained, high-volume scraping of public-data targets — government portals, property registries, court records and business directories. It provides three services on one platform: a rotating proxy for direct HTTP access, a Browser API for cloud-hosted Chrome automation, and an MCP server that connects AI agents to the web.

Rotating Proxy — Route requests through a continuously rotating pool of datacenter IPs via a single endpoint. Each request automatically rotates to a fresh IP, or pin a session to keep one. Anti-Detect Mode spoofs TLS fingerprints so your traffic looks like a real browser. All locations are included on every plan. Residential exits are available per request for the part of a workload that needs them. Set up in minutes with any HTTP client.

Browser API — Cloud-hosted headless Chrome workers you control via HTTP. Navigate pages, fill forms, solve CAPTCHAs, extract structured data, take screenshots, and generate PDFs — all routed through the proxy network. No browser infrastructure to manage. See the Browser API section or read the full API reference.

MCP Server — Connect AI agents (Claude, ChatGPT, Cursor, and others) to BotProxy via the Model Context Protocol. Your agent gets browser automation and web fetching tools out of the box — same API key, same billing. See the MCP Server section.

Quick Start

Select your programming language to see how to integrate BotProxy into your application. Each request should be authenticated by login and password or by whitelisting origin IP address (see Authentication section below). In the examples below Bot Anti-Detect Mode is enabled so your client should be configured to accept insecure certificates.

curl --insecure --proxy x.botproxy.net:8080 --proxy-user user-key:key-password "https://httpbin.org/ip"
              

curl --proxy x.botproxy.net:8080 --proxy-user user-key:key-password "https://httpbin.org/ip"
              

package example

import (
	"crypto/tls"
	"encoding/base64"
	"fmt"
	"io/ioutil"
	"net/http"
	"net/url"
)

func basicAuth(username, password string) string {
	auth := username + ":" + password
	return base64.StdEncoding.EncodeToString([]byte(auth))
}

func main() {
	// Configure the transport to accept insecure certificates
	tr := &http.Transport{
		Proxy: func(req *http.Request) (*url.URL, error) {
			return url.Parse("http://x.botproxy.net:8080") // For secure proxy: url.Parse("https://x.botproxy.net:8443")
		},
		TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // Skip SSL certificate verification
	}

	client := &http.Client{Transport: tr}
	req, _ := http.NewRequest("GET", "https://httpbin.org/ip", nil)
	req.Header.Add("Proxy-Authorization", "Basic "+basicAuth("pxu10000-0", "ProxyUser_password"))

	res, err := client.Do(req)
	if err != nil {
		fmt.Println("HTTP error: ", err)
		return
	}

	defer res.Body.Close()
	page, _ := ioutil.ReadAll(res.Body)
	fmt.Println(string(page))
}
              

#!/usr/bin/env node
const request = require('request-promise');

// Configure the request with the proxy and disable SSL verification
const options = {
    url: 'https://httpbin.org/ip',
    proxy: 'http://user-key:[email protected]:8080',
    strictSSL: false // Disable SSL certificate verification
};

// Send the request
request(options)
    .then(function (data) {
        console.log(data);
    })
    .catch(function (err) {
        console.error(err);
    });
              

package example;

import org.apache.http.HttpHost;
import org.apache.http.client.fluent.Executor;
import org.apache.http.client.fluent.Request;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.conn.DefaultProxyRoutePlanner;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.TrustAllStrategy;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.ssl.SSLContextBuilder;

public class Example {
    public static void main(String[] args) throws Exception {
        // Create an SSLContext that accepts all certificates
        CloseableHttpClient httpClient = HttpClientBuilder.create()
            .setSSLContext(new SSLContextBuilder().loadTrustMaterial(null, TrustAllStrategy.INSTANCE).build())
            .setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE)
            .build();

        HttpHost proxy = new HttpHost("x.botproxy.net", 8080);

        // Set up the proxy
        DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);

        // Configure the executor with the custom HttpClient
        Executor executor = Executor.newInstance(httpClient)
            .auth(proxy, "user-key", "key-password");

        // Execute the request via the proxy
        String res = executor.execute(
                Request.Get("https://httpbin.org/ip").viaProxy(proxy))
            .returnContent()
            .asString();

        System.out.println(res);
    }
}
              

package example;

import org.apache.http.HttpHost;
import org.apache.http.client.fluent.*;

public class Example {
  public static void main(String[] args) throws Exception {
    HttpHost proxy = new HttpHost("x.botproxy.net", 8080);
    String res = Executor.newInstance()
      .auth(proxy, "user-key", "key-password")
      .execute(Request.Get("https://httpbin.org/ip").viaProxy(proxy))
      .returnContent().asString();
    System.out.println(res);
  }
}
              

using System;
using System.Net;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;

class Example
{
    static void Main()
    {
        // Bypass SSL certificate validation
        ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(
            delegate { return true; });

        var client = new WebClient();
        client.Proxy = new WebProxy("x.botproxy.net:8080");
        client.Proxy.Credentials = new NetworkCredential("user-key", "key-password");

        try
        {
            string response = client.DownloadString("https://httpbin.org/ip");
            Console.WriteLine(response);
        }
        catch (Exception ex)
        {
            Console.WriteLine("Error: " + ex.Message);
        }
    }
}
              

using System;
using System.Net;

class Example
{
  static void Main()
  {
    var client = new WebClient();
    client.Proxy = new WebProxy("x.botproxy.net:8080");
    client.Proxy.Credentials = new NetworkCredential("user-key", "key-password");
    Console.WriteLine(client.DownloadString("https://httpbin.org/ip"));
  }
}
              

Imports System.Net
Imports System.Net.Security
Imports System.Security.Cryptography.X509Certificates

Module Example
    Sub Main()
        ' Bypass SSL certificate validation
        ServicePointManager.ServerCertificateValidationCallback = New RemoteCertificateValidationCallback(Function(sender, certificate, chain, sslPolicyErrors) True)

        Dim Client As New WebClient
        Client.Proxy = New WebProxy("http://x.botproxy.net:8080")
        Client.Proxy.Credentials = New NetworkCredential("user-key", "key-password")

        Try
            Dim response As String = Client.DownloadString("https://httpbin.org/ip")
            Console.WriteLine(response)
        Catch ex As Exception
            Console.WriteLine("Error: " & ex.Message)
        End Try
    End Sub
End Module
              

Imports System.Net

Module Example
Sub Main()
Dim Client As New WebClient
Client.Proxy = New WebProxy("http://x.botproxy.net:8080")
Client.Proxy.Credentials = New NetworkCredential("user-key", "key-password")
Console.WriteLine(Client.DownloadString("https://httpbin.org/ip"))
End Sub
End Module
              

<?php
$curl = curl_init('https://httpbin.org/ip');
curl_setopt($curl, CURLOPT_PROXY, 'http://x.botproxy.net:8080');
curl_setopt($curl, CURLOPT_PROXYUSERPWD, 'user-key:key-password');
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); // Disable SSL certificate verification
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); // Ignore host verification
curl_exec($curl);
curl_close($curl); // Close the cURL session
?>
              

#!/usr/bin/env python
import urllib.request
import ssl

# Create an SSL context that bypasses certificate verification
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE

# Create an opener with the proxy handler and SSL context
opener = urllib.request.build_opener(
    urllib.request.ProxyHandler(
        {
            'http': 'http://user-key:[email protected]:8080',
            'https': 'http://user-key:[email protected]:8080'
        }
    ),
    urllib.request.HTTPSHandler(context=ssl_context)  # Use the insecure SSL context
)

# Open the URL and print the response
response = opener.open('https://httpbin.org/ip').read()
print(response.decode('utf-8'))
              

import requests

res = requests.get(
    'https://httpbin.org/ip',  # Updated to use HTTPS
    proxies={
        'http': 'http://user-key:[email protected]:8080',
        'https': 'http://user-key:[email protected]:8080',
    },
    verify=False  # Disable SSL certificate verification
)

print(res.text)
              

#!/usr/bin/ruby

require 'uri'
require 'net/http'

uri = URI.parse('https://httpbin.org/ip')
proxy = Net::HTTP::Proxy('x.botproxy.net', 8080, 'user-key', 'key-password')

# Create an insecure SSL context
http = proxy.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE # Disable SSL verification

req = Net::HTTP::Get.new(uri)

result = http.start do |http|
  http.request(req)
end

puts result.body
              

#!/usr/bin/perl
use LWP::UserAgent;
use IO::Socket::SSL;

# Disable SSL verification
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
IO::Socket::SSL::set_defaults(SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE);

my $agent = LWP::UserAgent->new();
$agent->proxy(['http', 'https'], "http://user-key:key-password\@x.botproxy.net:8080");

my $response = $agent->get('https://httpbin.org/ip');

if ($response->is_success) {
    print $response->content();
} else {
    die "HTTP request failed: ", $response->status_line;
}
              

Authentication

Our proxies require authentication with each request. You need to provide proxy user login and password or whitelist your server IP addresses. These are not the login and password you used to sign up on our service. Proxy user login starts with pxu and looks like pxu1000-0.

You can find your proxy user credentials on your account page. Click on edit user button to access proxy user settings form where you can change password or whitelist IPs.

IP Rotation

Each request you make goes through our superproxy you connect to (x.botproxy.net) and one of the outgoing peer nodes. Superproxy selects outgoing peer randomly from the pool available to your account. Selected peer is stored in session and can be reused in next requests. Once the session expires (default is 60 seconds) a new session is started and a new IP address is selected. In addition to that all peers change their IPs at least every 24 hours or more often.

You can change maximum session age in proxy user details form to adjust for longer or shorter sessions. You can start a new session with each request to change IP every time, but we do not recommend doing so. Read more about session management in detail below.

Limitations

Our proxies are not designed to query a single website on a high volume or request rate. To ensure ethical use We limit request rate to a single website and block certain ports to prevent overloading of websites, sending spam or other illegal activity.

Please also check out Acceptable Use Policy. We ask that you not engage in prohibited activity on our network.

Demo Screencast

Watch the screencast to get the idea of using BotProxy username API to control your outgoing IP address. Please note that control headers are not supported anymore.

Browser Impersonation Mode

Overview

BotProxy’s Browser Impersonation Mode is a powerful feature designed to enhance anonymity and evade detection by anti-bot systems on target websites making web scraping easier. This mode works by spoofing TLS and HTTP/2 fingerprints, mimicking those of real browsers (Chrome, Firefox, Safari, or Edge), ensuring that bot detection mechanisms cannot differentiate your web scraping requests from legitimate ones. You can select which browser to impersonate in your proxy access settings.

TLS fingerprinting involves analyzing the unique characteristics of a device’s TLS handshake during secure connections. These characteristics, such as supported protocols, cipher suites, and extensions, create a unique signature. Anti-bot systems use these signatures to identify and block automated traffic.

How It Works

1. TLS Spoofing:
   • BotProxy’s superproxy intercepts outgoing TLS connections from your application.
   • It modifies the TLS fingerprint to match the signature of a Chrome browser running on an Android phone.
   • This makes your web scraping requests indistinguishable from those of legitimate mobile users.
2. MITM Proxy Mechanism:
   • Bot Anti-Detect Mode functions as a man-in-the-middle (MITM) proxy.
   • The superproxy establishes a secure TLS connection with the target website on behalf of the client and relays requests and responses between the two.
   • For this to work, clients must configure their software to operate in insecure mode, such as using the --insecure parameter in curl.
3. Default Enabled:
   • The Bot Anti-Detect Mode is enabled by default for all proxy users.
   • Users can toggle this feature in the proxy user form in the account console.
   • Disable it to use regular proxy behavior without TLS fingerprint spoofing.

Configuring Insecure Mode

To use Bot Anti-Detect Mode, the client application must allow insecure connections. Follow these steps to configure it:

1. Modify Client Settings:
   • Update your client software to accept insecure connections.
   • For example, when using curl, add the --insecure parameter to your request: curl --proxy x.botproxy.net:8080 --insecure "https://example.com"
2. Test the Connection:
   • Run a test request through the proxy to ensure the client is correctly configured to bypass certificate validation.

Limitations and Considerations

Compatibility: Some applications or libraries may have issues operating in insecure mode. Verify compatibility before enabling this mode.

Performance Impact: The MITM mechanism may introduce slight latency due to additional processing.

By leveraging the Bot Anti-Detect Mode, you can significantly enhance your web scraping scripts and reduce the likelihood of detection, allowing for seamless operation even in environments with strict anti-bot defenses.

Advanced Topics

Country and Location selection

By default we create a proxy user that has access to all locations available to your account except for OpenProxies. You can limit the locations using proxy user details form. Enter a comma separated list of location codes you want to use. For example if you want to make requests only from US locations enter us-ny,us-io,us-ca. To allow all locations except some add a minus sign in front of locations list. For example to exclude OpenProxies locations enter -xo.

You can have multiple proxy users on your account each with different available locations and other params.

There is also the possibility to adjust the country and location on each request. Read the Username API section for more details.

Username API

We provide username API to control outgoing locations using only proxy authentication credentials. To achieve this you should add special suffixes to your proxy user login as follows. Let assume your proxy user login is pxu1000-0

• Specify COUNTRY by adding +CN suffix to login. For example to limit outgoing locations to US only use pxu1000-0+US as your proxy user login
• Specify LOCATION by adding +loc suffix: use pxu1000-0+us-fl as your proxy username
• Add SESSION identifier to country or location by adding one more suffix as follows: pxu1000-0+US+SESSIONID or pxu1000-0+us-fl+SESSIONID. If you do not want to limit country or location but want to use session leave country/location blank and specify session as follows: pxu1000-0++SESSIONID
• Request a RESIDENTIAL exit with the RS prefix: pxu1000-0+RS for any region, or pxu1000-0+RS-US for a specific country. See Residential traffic below.

Note that RS always means residential, never Serbia. To reach a Serbian datacenter exit, address it by its location code (for example rs-bg) rather than by country.

The complete usage example to route outgoing traffic through US locations only and within session 123456 is:

$ curl --proxy x.botproxy.net:8080 --proxy-user "pxu1000-0+us+123456:password" "https://httpbin.org/ip"
      

Locations

Currently we have proxy peers in datacenters in the following locations:

You can find a list of locations available according with your current subscription in your account page as well as all current proxy IP addresses for each location.

Sessions: IP persistence

In many cases you may want to run several parallel sessions from the same client, use different outgoing IP per session, and be able to change them when needed. This can be done by utilizing sessions. Sessions provide you fine grained control over outgoing IP addresses.

Each request going through a proxy uses a session. Even when you do not provide session ID a so called default session is used. Session is a combination of proxy user login, country or location and a session ID. Only proxy user login is required, if other parameters are not provided they are added as blank strings to create internal session identifier in superproxy. Sessions are not shared between superproxies. Each session has an IP address. IP address is assigned to a session on session creation and does not change during session lifetime. Until you keep using the session and it is valid, the IP address stays the same.

Sessions has a maximum session age that can be set for each proxy user separately in user details form. When a session gets older it is invalidated and even if you provide the same location and session ID a new session with a new IP is created. You can set maximum session age to 0 to remove session age limit restriction. All sessions are invalidated after 5 minutes of inactivity.

Summary:

1. IP is persistent within one Session until session Age Limit is reached.
2. Session is login+country or location+session_id
3. Each time Session is created a new IP is assigned as follows:
   • We take all IP addresses available to your proxy user (considering Limit Locations user settings)
   • Filter IPs to match requested country or location
   • Randomly select IP address from the remaining IPs
4. When session age limit is reached IP is renewed
5. If session is not used for 5 minutes it is deleted

Generate the random number on thread startup, and change it when you want to change the proxy peer assigned for the thread connection. Session ID can be any random string/counter: requests with the same session will use the same proxy peer (as long as possible).

To force an IP change, just modify the session ID.

Residential traffic

Most public-data targets — government portals, property and court records, business directories — do not need residential exits. They have rate limits and session state rather than consumer-grade bot detection, and datacenter IPs with Anti-Detect Mode handle them well at a fraction of the cost. Residential is here for the part of a mixed workload that genuinely needs it.

Datacenter is always the default. Residential runs only when a request explicitly asks for it, by adding the RS prefix to your proxy user login:

# any residential region
$ curl --proxy x.botproxy.net:8080 --proxy-user "pxu1000-0+RS:password" "https://httpbin.org/ip"

# a specific country
$ curl --proxy x.botproxy.net:8080 --proxy-user "pxu1000-0+RS-US:password" "https://httpbin.org/ip"
      

Residential traffic is billed against your residential balance, not your plan's datacenter allowance. The balance has two parts, spent in this order:

1. The residential volume included with your plan, which resets every billing period.
2. Any prepaid packs you have bought. These do not expire and carry across billing periods.

Included volume is always consumed first, so a pack is never drawn on while free volume for the period is still unused.

The balance is a hard stop. At zero, residential requests fail with a clear error. There is no silent fallback to datacenter, no automatic charge and no negative balance, so a bug in a loop cannot produce a surprise invoice. Your datacenter traffic is unaffected and keeps running. Buy packs from your account dashboard; the per-GB rate falls as the pack size goes up.

Residential requires an active paid subscription and is not available during the free trial. If a subscription ends, any remaining prepaid balance is frozen rather than forfeited — resubscribe and it is still there.

Because the residential pool rotates its own exit addresses, session stickiness is best effort on residential and the X-PROXY-IP response header is not returned.

Daily traffic limits

Each proxy user carries its own daily traffic caps, so a runaway loop in one scraper cannot consume your whole plan or your whole residential balance. Datacenter and residential are limited independently.

Defaults apply to every proxy user, including the first one, so a new account is protected without any configuration. They are stored as a percentage rather than a fixed number, so they scale by themselves when you upgrade a plan or top up a pack. Set an explicit value in the proxy user form to pin it, or 0 for unlimited.

Limits reset at 00:00 UTC. When one is reached, requests through that credential are rejected with HTTP 429 and a message naming which limit was hit, on which proxy user, and when it resets. Traffic never falls back from one type to the other. Your account-level plan limits always apply on top: a per-user limit can never permit traffic beyond your plan, your overage settings, or your residential balance.

Today's usage against both limits is shown per proxy user in your proxy users list, and the account owner is emailed the first time a limit is hit each day.

Request filters

Not every request your scraper makes is one you need. A headless browser fetches webfonts, analytics beacons, ad tags and tracking pixels because a real browser would — and every one of them is traffic you pay for and then discard. Request filters let you refuse them before they are fetched.

Set Blocked URL patterns on a proxy user: one regular expression per line, blank lines and lines starting with # ignored. Matching is case-insensitive and unanchored, so a bare hostname is a complete rule. A request whose URL matches any line is refused with HTTP 556, and because it never reaches an outgoing proxy node, it uses no traffic and is not billed.

fonts\.gstatic\.com
\.(woff2?|ttf|otf|eot)($|\?)
google-analytics\.com
doubleclick\.net

What is matched depends on how the request reaches us. For plain HTTP, and for HTTPS with Anti-Detect Mode enabled, the full URL is matched including the path, so a rule such as \.woff2$ works. For HTTPS with Anti-Detect Mode disabled the connection is an encrypted tunnel that we do not open, so the only thing that exists to match against is https://host:port: hostname rules still work, path rules cannot fire. Enable Anti-Detect Mode if you need to filter by path or file extension over HTTPS.

Patterns use RE2 syntax. Lookahead, lookbehind and backreferences are not supported and are rejected when you save, rather than silently never matching.

Blocked requests appear in your request log marked blocked, and can be listed on their own with the Blocked by my filters status filter, so you can confirm a rule does what you meant. They are never counted against the success rate guard — blocking your own requests is not a failure.

Success rate guard

Daily traffic limits stop a runaway loop. The success rate guard stops the quieter failure: a scraper that is running normally but no longer getting anything usable back, because the target changed its markup, started challenging you, or began answering 403 to everything.

Set Minimum success rate on a proxy user to arm it. Leave it blank — the default — and nothing happens. When the rate over the configured window falls below the threshold, that proxy user is disabled and the account owner is emailed. Requests through it then fail with HTTP 403 and a message explaining why, rather than a bare authentication error. Other proxy users on the account are unaffected.

Only statuses returned by the target site are counted. Anything BotProxy itself returns — a daily traffic limit, a blocked host, an upstream failure — is excluded, so our own enforcement can never disable your credential. A proxy user also needs a reasonable number of requests inside the window before the guard will act, so a handful of failures on a quiet credential will not trip it.

With Anti-Detect Mode enabled, HTTPS responses are decrypted by the proxy and the real status code is recorded, which is what the guard works best from. With it off, an HTTPS request is an opaque tunnel and no status is visible, so failures are inferred from unusually small responses instead. That is a weaker signal: leave Count inferred failures on, or the guard can never fire for such a proxy user, and turn it off if you scrape targets that legitimately return very small responses.

Re-enable a disabled proxy user from the proxy users list once the scraper is fixed. It will not be disabled again for the same past failures — only traffic sent after you re-enable it is measured. Per-request status codes are shown in your request log, and the success rate is charted alongside traffic on your dashboard.

HTTP Headers

We return outgoing proxy IP address in HTTP response headers. For HTTPS requests response headers are added immediately after HTTP/1.1 200 Connection established response from proxy:

CONNECT httpbin.org:443 HTTP/1.1
Proxy-Authorization: Basic bG90...

HTTP/1.0 200 Connection established
X-RPOXY-IP: xx.xx.xx.xx
[encrypted data]
      

Errors

Proxy can respond only with the following error codes. All other codes are returned from the website you are connecting to:

Open Proxies

We maintain a list of known open proxy servers. If you choose "OpenProxies" location your request will be routed through one of these proxies. Your requests are forwarded through less reliable & higher latency servers. Open proxies IPs are located in many different countries around the world. These open proxy IPs typically do not stay online very long, and are not operated on reliable infrastructure. The tradeoff for this lack of reliability is a huge increase in quantity & variability of IP addresses.

The list is regularly updated and contains around 1000 working IP addresses around the world. If any request through an open proxy fails due to a proxy error, that error is recorded, and the request is re-tried up to 3 more times, using a different proxy for each retry. Failed proxy is immediately removed from the list.

By default Open Proxies location is disabled on a proxy user. You need to manually enable it in your account dashboard.

OpenProxies location code is xo. You can additionally specify the desired country by adding 2 letter ISO country code after a minus sign: xo-us (example location code for US).

Secure Proxy (HTTPS/SSL)

A secure web proxy is a web proxy that the browser (or your other software) communicates with via encrypted connection, as opposed to clear text. In insecure public networks, such as airports or cafes, browsing over HTTP may leave the user vulnerable to cookie stealing, session hijacking or worse. A secure web proxy can add a significant layer of defense in these cases. Diagrams below explain different scenarios in detail. Please note: not all clients (browsers) support secure proxies. Refer your client documentation and set HTTPS proxy as needed.

We provide a secure proxy at the following address: x.botproxy.net:8443 You can connect using the same proxy users.

To use secure proxy in your applications you need to first establish a TLS connection to our secure proxy and then initiate regular HTTP proxy connection over established encrypted connection. Some other programs and libraries have built in support for secure proxies such as curl or Go http client.

Find below an explanation of how requests go when using different kind of proxies:

• HTTP/HTTP
• HTTPS/HTTP
• HTTP/HTTPS
• HTTPS/HTTPS

HTTP request using HTTP proxy

In this case request is sent from your client in plain text thus allowing your local network administrator or an attacker to intercept it and access all the data being sent. This is the least secure option.

HTTPS request using HTTP proxy

Target host name alongside your BotProxy proxy user credentials are transferred in plain text. Your client requests to create a tunnel to the specific host and port with CONNECT command and then makes secure communication via this tunnel. Parties on local network cannot intercept or hijack any data being transferred between you and remote host.

HTTP request using Secure (HTTPS) proxy

In this case your client establishes secure connection with our proxy server and no one can know what data is being transferred between you and target host. Moreover nobody in local network can know what is the target host. Even though the target host can be accessed using plain text HTTP protocol.

HTTPS request using Secure (HTTPS) proxy

Requests and data are encrypted all the way through. Most secure option.

Browser API

The Browser API lets you control headless Chrome workers through simple HTTP requests. Navigate pages, interact with elements, extract data, take screenshots, generate PDFs, and connect directly via Chrome DevTools Protocol — all without running a browser locally.

Authentication

Browser API requests are authenticated with an API key. Generate keys in your account dashboard. Include the key as a query parameter (?key=YOUR_KEY) or HTTP header (X-Api-Key: YOUR_KEY).

Endpoints

For complete request/response schemas, parameters, and examples see the full API reference.

Python Example

import requests

API_KEY = "your-api-key"
API_URL = "https://api.botproxy.net"

# Navigate to a page and extract data
response = requests.post(f"{API_URL}/navigate", params={"key": API_KEY}, json={
    "url": "https://example.com",
    "proxyCountry": "US",
    "keepAlive": 30,
    "extract": {
        "title": "h1",
        "links": "a[href]"
    }
})
result = response.json()
session_id = result["session"]
print("Title:", result["data"]["title"][0]["text"])
print("Links:", [a["value"] for a in result["data"]["links"]])

# Interact with the page
response = requests.post(f"{API_URL}/interact", params={"key": API_KEY}, json={
    "session": session_id,
    "actions": [
        {"action": "click", "element": "a.nav-link"},
        {"action": "wait", "element": "#content"}
    ]
})

# Take a screenshot
response = requests.post(f"{API_URL}/screenshot", params={"key": API_KEY}, json={
    "session": session_id,
    "fullPage": True
})
with open("page.png", "wb") as f:
    f.write(response.content)

# Stop the session when done
requests.post(f"{API_URL}/session", params={"key": API_KEY}, json={
    "session": session_id,
    "stop": True
})
      

Proxy Countries

Route browser traffic through specific countries using proxyCountry:

• Datacenter: US, ES, DE, NL, FR, UK, SG, AU, CA, IN
• Residential (prefix RS-): RS-US, RS-DE, RS-UK, etc.
• Regional: RS-EU (Europe), RS-RSA (South Africa)
• Random residential: RS

Residential draws on the same balance as the rotating proxy and is subject to the same rules — see Residential traffic. Note that the upstream is bound when the browser session is created, so proxyCountry takes effect on the request that creates a session; passing a different value on a later request against the same session has no effect. Start a new session to switch.

Sessions & Concurrency

Sessions persist browser state (cookies, loaded page) across requests. Set keepAlive (1–65 seconds) to control how long a session stays alive after the last request. Your plan defines the maximum number of concurrent sessions. Stop sessions explicitly when done to free concurrency slots immediately.

MCP Server

The MCP (Model Context Protocol) server lets AI agents — Claude, ChatGPT, Cursor, and others — use BotProxy as a native tool. Agents can browse the web, extract data, take screenshots, and fetch pages through BotProxy's proxy network without any custom integration code.

Endpoint & Authentication

The MCP server is available at:

https://api.botproxy.net/mcp

Authentication uses the same API keys as the Browser API. Pass your key as a query parameter (?key=YOUR_KEY) or header (X-Api-Key: YOUR_KEY). Generate keys in your account dashboard.

Configuration

Add BotProxy to your MCP client config:

{
  "mcpServers": {
    "botproxy": {
      "type": "streamable-http",
      "url": "https://api.botproxy.net/mcp?key=YOUR_API_KEY"
    }
  }
}

Claude Desktop: ~/.claude/claude_desktop_config.json · Claude Code: .claude/settings.json

Available Tools

Protocol

The server implements the MCP specification over Streamable HTTP transport, and negotiates protocol revisions 2024-11-05, 2025-03-26 and 2025-06-18. Clients send JSON-RPC 2.0 requests via POST:

# Initialize the connection
$ curl -X POST 'https://api.botproxy.net/mcp?key=YOUR_KEY' \
    -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","id":1,"method":"initialize",
         "params":{"protocolVersion":"2025-03-26","capabilities":{}}}'

# List available tools
$ curl -X POST 'https://api.botproxy.net/mcp?key=YOUR_KEY' \
    -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}'

# Call a tool
$ curl -X POST 'https://api.botproxy.net/mcp?key=YOUR_KEY' \
    -H 'Content-Type: application/json' \
    -d '{"jsonrpc":"2.0","id":3,"method":"tools/call",
         "params":{"name":"fetch_url","arguments":{"url":"https://example.com"}}}'

Billing

MCP tool calls consume the same API request quota and residential balance as Browser API requests. Each tools/call invocation counts as one request. The fetch_url tool routes through a residential exit, and is billed as residential, when an RS or RS- country is specified.

Benchmark & Methodology

We publish a measured answer to the question customers ask before they buy: which class of proxy does this target actually require? The harness, the target list and every raw request record are on GitHub under an MIT licence, so the numbers can be checked, disputed, or re-run against your own targets.

github.com/botproxy/proxy-tier-benchmark

How it is measured

Thirty-two public-data targets are fetched through six arms that differ by one variable at a time — no proxy at all, a datacenter IP pinned to one exit, a datacenter IP rotating per request, each of those with Browser Impersonation Mode enabled, and a residential exit — interleaved by round so the hour of day cannot be confounded with the arm. A request counts as a success only if the expected content is present. A 200 carrying a challenge page or an empty body is a failure, because it is not a page you can use.

Every target is gated on robots.txt before and during the run, targets whose robots.txt cannot be read are excluded rather than assumed permitted, and each exclusion is published with its reason.

What it found

The results do not flatter us, which is the point — a vendor benchmark the vendor wins is not evidence:

• No proxied arm beat the unproxied control. On a modest number of fetches from a clean cloud host, plain direct access scored highest.
• 19 of 32 targets needed no proxy at all at that volume.
• 12 of 32 were unreachable at any tier, because the block is an interactive challenge that no plain HTTP client can solve. Those targets need a real browser — see the Browser API — not a different proxy.
• Exactly one target was decided by the proxy tier.
• Where an official API or bulk download exists, the report recommends it over scraping.

What a low-volume benchmark cannot show

Read the headline with its scope attached. A rotating pool is rarely needed because a target blocks datacenter ranges outright — it is needed because of per-IP rate and volume limits and geographic restrictions. Ten polite requests to a target never approach a rate limit, so the benchmark is structurally unable to measure the thing rotation exists to defeat.

Two effects sit outside a single-day run and matter in production:

• Per-IP throttling. On targets that limit per address, the same workload fails far more often when concentrated onto few exits than when spread across many. On one measured storefront the failure rate moves from 2.4% to 73.3% purely with concentration.
• Daily pool replacement. Our datacenter nodes cycle continuously; roughly 40% of the exit addresses a target saw yesterday are gone today. IP-level bans therefore have a short horizon and per-IP budgets reset with the address, which no one-day test can observe.

If you are evaluating for sustained collection rather than occasional fetches, those two dimensions matter more than the headline pass rate, and the harness is published so you can measure them on your own targets.

Knowledge Base

• Using Selenium Chromedriver with Rotating proxies
• Tunneling TCP traffic through HTTP proxy
• Using Puppeteer with Rotating Proxies
• Using Scrapy with Proxies (IP Rotating Proxy)
• more web scraping tutorials