Proxy tier benchmark › Targets

Is the North Carolina business registration search blocked to scrapers?

State business entity registries · minimum viable tier: no tier sufficient · measured 2026-08-18

Yes, and no tier tested changed anything. All 60 requests to the North Carolina Secretary of State business registration search returned HTTP 403, on every one of the six configurations including residential. Every arm scored 0%. Measured 2026-08-18.

What was measured
URL https://www.sosnc.gov/online_services/search/by_title/_Business_Registration
Content assertion Business Registration — never verified against a live page
Arms 6
Rounds per arm 10
Requests 60
Measured 2026-08-18, 15:38–18:09 UTC
Benchmark host DigitalOcean droplet, New York (nyc3)

A request counts as a success only if the expected content is present. An HTTP 200 carrying a challenge page, an empty body or the wrong page is a failure, because it is not a page you can use.

This measures whether a tier works, not long-run stability under sustained load. Ten requests per arm, at a two-second floor, never approach a per-IP rate limit — which is the thing a rotating pool exists to defeat.

Results by arm

Arm Success p50 p95 Bytes per success Outcomes
direct (no proxy) 0% 100 ms 135 ms n/a blocked 10
datacenter, one pinned exit 0% 469 ms 1,164 ms n/a blocked 10
datacenter, rotating exit 0% 453 ms 1,334 ms n/a blocked 10
anti-detect, one pinned exit 0% 534 ms 1,303 ms n/a blocked 10
anti-detect, rotating exit 0% 478 ms 1,233 ms n/a blocked 10
residential, rotating exit 0% 1,511 ms 2,408 ms n/a blocked 10

What protects this target

Cloudflare, refusing at the edge before any content is served.

The reconnaissance probe recorded a 403 whose page title is Just a moment... — Cloudflare's interstitial, returned with a refusal status rather than as a solvable challenge. In the run the bodies range from 4,543 to 6,227 bytes across the six arms, which is the size of an edge error page and not of a registry search application.

This is one of four targets where all 60 requests produced the same outcome, and the only one of the four where that outcome is a hard 403 on every tier. It is also the cleanest counter-example to the idea that a more expensive exit buys access: the residential arm, six times the price of datacenter traffic, was refused ten times out of ten exactly like the unproxied control.

Where a target refuses at the edge on every network and every fingerprint, the useful question is not which proxy to buy. The target list records no official API for this portal, so the route worth checking first is whether the Secretary of State publishes the register as a bulk download — not which exit to route the next 403 through.

robots.txt status

The host serves no robots.txt — the request returns HTTP 404. Nothing is disallowed, and nothing is granted either.
Status allowed
File https://www.sosnc.gov/robots.txt
Crawl-delay not declared
Fetched as the benchmark's own User-Agent
Note no robots.txt (HTTP 404)

Limits of this result

  • The content assertion was never verified against a live page. The result does not depend on it: an HTTP 403 is classified from the status code alone.
  • The search page was measured, not a submitted query. A 403 at the search page means no query was reachable.
  • Ten rounds per arm in a single three-hour window on 2026-08-18. Anti-bot behaviour drifts; a result older than a few months is a historical record, not a current one.
  • One exit geography. Every request left from a US datacenter host or a US exit, so nothing here describes how this target answers traffic from elsewhere.
  • The unproxied arm ran from a clean, previously unused cloud IP with no accumulated reputation. It is a best case a production scraper will not sustain.

Run date: 2026-08-18 · Full benchmark report · Harness and raw data · All 32 targets

The 60 rows behind this page are in results/six-arm/raw.csv; filter them with grep '^[^,]*,us_nc_sos_biz,' raw.csv.

Test your own target

The result above describes one URL on one day. Paste your own URL into the live tester and see the real response before creating an account.

Test your target See plans