Solutions › Travel & airline fare monitoring
Fare monitoring is different from most scraping in two ways: the price depends on what country the request comes from, and getting a single fare takes a sequence of requests the site expects from one visitor. That means you need precise exit-country control and sessions that stay sticky for an entire search flow but rotate between flows to stay within each address's request budget.
Airline and OTA pricing is set per point of sale. The same route on the same date can return a different fare, in a different currency, with different fare families available — all depending on which country the request appears to come from. The exit IP isn't a transport detail — it's one of the inputs to the measurement, alongside route, date, and cabin.
Two rules follow. Pin the exit country in the proxy login and record it with every price you store, so a fare is always tied to a specific market. And keep it stable across runs — a fare series that quietly drifts from a US to a European exit will show a "price change" that's really just a change of market. Every location is included on every plan, so covering multiple points of sale is a login change, not an upgrade.
pxu1000-0+US+route-JFKLHR-0312 # US point of sale
pxu1000-0+GB+route-JFKLHR-0312 # UK point of sale, same queryAccess can also be blocked by geography before anything else kicks in. When we probed US public-data sites, some servers answered a US datacenter IP normally and returned 403 to a European one. Matching the exit to the market is the first thing to check when a target starts refusing you.
Unlike a product page, a fare isn't a single request. A search posts an itinerary, the site issues a search token, results come back over several calls, and the fare detail is fetched against that token. If the exit IP changes mid-flow, the site sees the session jump countries and usually discards it.
BotProxy's session model maps directly onto this. Every request runs inside a session, a session holds one exit IP for its entire life, and the session ID is part of the proxy login:
login + location + SESSIONIDUse one session ID for one complete search flow, then a new ID for the next itinerary. Two timeouts matter here more than anywhere else:
limit_session_age caps total session life — 60 seconds on most credentials. A multi-step fare flow usually needs this raised, or set to 0 for no cap, on the proxy user you use for travel work.If you don't set a session ID at all, a single default session is used and every flow shares one IP — which causes both cross-contaminated searches and fast throttling.
A fare search is genuinely expensive for the operator — it hits inventory and pricing systems, not a cache. So carriers and OTAs meter searches per address aggressively. This is a request budget, not a ban on datacenter traffic — the same pattern we see elsewhere. Across 268 targets, 28 show failure rates rising sharply as the same volume is concentrated onto fewer exit IPs.
In practice: budget searches per IP per hour, not requests per second. Spread a route portfolio over the day instead of bursting it. And when a search starts returning empty results, rotate to a new session rather than retrying on an address that's already been cut off.
Travel sites rely heavily on commercial WAFs that inspect the TLS and HTTP/2 handshake before any application logic runs. Anti-Detect Mode rewrites your client's handshake to match a real browser — that's what gets past AWS WAF, Imperva, and Cloudflare fingerprint checks. It's on by default. It works as a man-in-the-middle, so your client needs to accept the proxy's certificate, and the browser profile provides the User-Agent, replacing your client's — worth knowing here because mobile and desktop fare pages can differ.
Many booking engines are single-page apps whose search state never shows up in a plain HTML response, and some gate results behind interactions a regular HTTP client can't handle. Every plan includes cloud Chrome browsers you control over HTTP: run the search, wait for results to load, read the fare, screenshot the quote for an audit trail. Same account, same IP pool, same bill — see the Browser API reference. Keep browser use to the flows that need it — a rendered search uses far more traffic than a plain fetch.
If an authorized feed exists, use it. Airline NDC APIs, GDS access, and hotel distribution APIs give exact, contractual fare data with no fragile parsing and no rate-limit game. They require a commercial relationship, which is exactly why scraping exists for the rest — competitive monitoring, market surveys, and coverage of carriers you don't have an agreement with. Both approaches coexist in most serious travel pipelines.
Anything that requires completing a booking, a logged-in loyalty account, or passing a Cloudflare managed challenge is outside what rotating datacenter IPs can do. Fare data is also unusually sensitive commercially — check the target's terms and keep your search rate defensible. A monitor that behaves like a heavy user is one that keeps working; one that looks like a denial-of-service attempt won't.
Paste your URL into the live tester and see the real response before creating an account. Every plan includes all locations, Anti-Detect Mode, and the Browser API. You're billed on traffic volume only.
Test your target See plans Read the documentation
Licensing boards, Secretary of State business registries, county assessors and recorders. About 1,450 separate government domains, each with its own stack and its own WAF.
Company and professional profiles, directories, and firmographic enrichment — long-running pipelines that re-check the same records on a schedule.
Marketplace and retailer product pages, price history, and stock tracking across many sites at once.
Search result pages by keyword and locale, rank monitoring, and share-of-voice reporting for agencies and in-house teams.
Airline and OTA fare and availability checks, where the price depends on the country the request comes from.
Listing portals plus the county assessor, recorder, and land-record sources that hold the official ownership and tax data.
Marketplace listings and product images collected on a schedule to find copies of your products and document them for takedowns.