Solutions › B2B data & business directories
B2B data collection — company profiles, directory listings, professional registries, and firmographic enrichment — works well with rotating datacenter IPs because these targets throttle per address rather than blocking datacenter IPs outright. What you need is a pool of addresses that keeps turning over, a browser-like TLS fingerprint, and a session strategy that keeps each exit IP under the target's per-address request budget across a crawl that runs for weeks.
The defining shape of this work is that it doesn't end. A one-off directory dump is a weekend project. An enrichment product re-checks the same records on a rolling schedule forever, which makes cost per gigabyte and predictable behaviour matter far more than peak throughput.
It varies more here than in any other segment. A plain HTML directory hands over its pages to any client that asks politely. A large aggregator, on the other hand, sits behind a commercial WAF and inspects the TLS handshake before serving a byte.
The official registers at the bottom of many enrichment pipelines are the hardest part, and they fail the same way government portals do. When we tested 50 US state and federal public-data URLs from a clean US datacenter IP, only 36% returned usable content. Of 22 state business registry portals, just three served content directly. The rest came back with Cloudflare challenges, Imperva or Incapsula interstitials, DataDome, a reCAPTCHA gate, a CloudFront 403, or a plain connection timeout. One pattern deserves a callout: several returned HTTP 202 with an empty body. A pipeline that validates on status codes records those as successes and silently produces empty rows. Check for the content you expected, not just the status line.
Across 268 targets in our production traffic, 28 show failure rates rising sharply when the same request volume is concentrated onto fewer exit IPs. The site isn't refusing datacenter traffic — it's refusing this address's eleventh request this minute.
That's good news for a B2B pipeline, because a request budget is something you can plan around. Throughput depends on how widely you spread the work and how you pace it, not on luck. And the fix for a rising error rate is usually spreading the same work over more sessions, not buying a more expensive class of IP.
Every request through BotProxy runs inside a session, and a session holds one exit IP. If you don't set a session ID, everything shares one default session and one IP — the worst possible setup for a crawl that runs all day. The session is encoded in the proxy login:
login + location + SESSIONID
pxu1000-0+US+enrich-07 # worker 7, US exit
pxu1000-0++enrich-07 # same session, no location constraint
Change the session ID to change the IP. Two limits control a session's lifetime: it's dropped
after 5 minutes idle, and limit_session_age caps its total life
— 60 seconds on most credentials, or 0 for no cap.
Practical patterns for enrichment:
The larger directories use the same WAF products as everyone else, and those products fingerprint the TLS and HTTP/2 handshake. Anti-Detect Mode replaces your client's handshake with a real browser's — that's what gets past AWS WAF, Imperva, and Cloudflare fingerprint checks. It's on by default. Two things to know: your client needs to accept the proxy's certificate, and the browser profile provides the User-Agent, overriding whatever your client sends — pick the profile instead of hand-writing a UA header.
Location is checked before fingerprints are. Two of the US state sites in our probe served a US
datacenter IP normally and returned 403 to a European IP for the same request.
If your enrichment stack runs in an EU region and hits US sources, that alone can look like bot
detection. Pin the country in the login (pxu1000-0+US). Every location is included
on every plan, so matching the exit to the source costs nothing extra.
A surprising amount of B2B ground truth is published properly, and pulling it from an API is cheaper, more complete, and more reliable than any crawl:
Use the proxy for sources with no API, for fields the official extract leaves out, and for daily changes a quarterly dump can't give you.
Directories that require a logged-in account, professional networks that treat scraping as a terms violation, and anything behind a Cloudflare managed challenge aren't the job for rotating datacenter IPs. Build on sources that publish to the public web, and check the target's terms before you scale a crawl against it.
Paste your URL into the live tester and see the real response before creating an account. Every plan includes all locations, Anti-Detect Mode, and the Browser API. You're billed on traffic volume only.
Test your target See plans Read the documentation
From our published proxy tier benchmark: what each of these targets actually required, measured rather than asserted.
Licensing boards, Secretary of State business registries, county assessors and recorders. About 1,450 separate government domains, each with its own stack and its own WAF.
Company and professional profiles, directories, and firmographic enrichment — long-running pipelines that re-check the same records on a schedule.
Marketplace and retailer product pages, price history, and stock tracking across many sites at once.
Search result pages by keyword and locale, rank monitoring, and share-of-voice reporting for agencies and in-house teams.
Airline and OTA fare and availability checks, where the price depends on the country the request comes from.
Listing portals plus the county assessor, recorder, and land-record sources that hold the official ownership and tax data.
Marketplace listings and product images collected on a schedule to find copies of your products and document them for takedowns.