Type “London to Bangkok” into a flight comparison site and you get a few hundred options back before you have finished reaching for your coffee. Different airlines, different stopovers, prices sorted to the cent.
It looks like one search. Behind the screen it is closer to a small relay race, with half a dozen systems passing data to each other, some of them older than the people using the site.
If you like knowing how your gadgets work, this is a fun one to take apart. It also explains a few things that annoy every traveler, like the fare that jumps the second you click on it.
It starts with a giant shared price list
Airlines don’t keep their fares in a drawer. Most of them publish prices and the rules attached to them (advance purchase, minimum stay, refund conditions) through an industry clearing house called ATPCO. Think of it as a wholesale price catalog that gets updated many times a day.
A fare on its own is not a ticket, though. You also need to know whether there is a seat left at that price on that flight. That is a separate piece of data called availability, and it lives with the airline.
So the first job of any travel site is boring but essential: match the published fares against live seat availability, then apply the rules to see which combinations are allowed.
The GDS, travel’s old plumbing
For decades the matching was done by global distribution systems, or GDSs. The three big names are Amadeus, Sabre and Travelport. They began life as airline reservation systems, and they still sit underneath a large share of the flights sold by online travel agencies.
A travel site plugs into a GDS, sends a route and dates, and gets back bookable itineraries. It works, and it covers a huge number of carriers in one connection.
It also has holes. Some low-cost airlines keep part or all of their inventory out of these systems because they would rather sell direct and skip the fees. That is why a comparison site occasionally misses a cheap flight you later find on the airline’s own page.
Direct lines into the airlines
To fill those holes, travel sites connect to airlines one by one. The modern way to do it is through a standard called NDC (New Distribution Capability), which lets an airline send offers straight to a seller, complete with extras like bags and seat selection.
Direct connections tend to carry prices and bundles that never appear in the older feeds. The downside is the workload. Every airline needs its own integration, its own testing and its own upkeep, so a comparison site with a few hundred airline partners is maintaining a lot of pipes.
The shopping engine that does the math
Here is the part that is properly hard. A trip with one stop each way can be built from thousands of flight combinations, and each combination can be priced with many different fares. The number of possibilities gets silly very quickly.
Fare shopping engines exist to search that space fast. The best known is the software built by ITA, a company started by MIT computer scientists and bought by Google, which went on to power Google Flights. Other sites build their own engines or license one.
When a result page loads in two seconds, you are looking at the output of software that threw away millions of bad combinations to show you the two hundred worth reading.
Caching, or why the price changed when you clicked
Asking every airline for a live quote on every search would be slow and expensive, and the airlines would not tolerate it. So travel sites save recent results and reuse them. If somebody searched your route an hour ago, you may well be shown their answer.
Most of the time that is fine. Sometimes the last cheap seat sold in the meantime, and the site only finds out when you click through and it runs a real check. The price goes up, you swear at the screen, and you assume you are being tracked.
Usually you are not. A stale cache is the more common explanation than a site punishing you for looking twice.
Reading the web like a customer would
Feeds and APIs still don’t cover everything. Some airlines offer no feed at all. Some run flash sales only on their own website.
And fares can differ depending on the country you appear to be shopping from, which the industry calls point of sale. The same seat can be listed at one price to a visitor in Mexico and another to a visitor in Germany.
To see those prices, fare trackers and travel data companies load the public pages the way a normal shopper would and read what comes back.
This is web scraping, and airlines are some of the most heavily guarded sites on the internet, so requests coming from a cloud server tend to get blocked or shown a captcha.
That is where proxy networks come in. A residential proxy sends the request through a real home internet connection in a chosen location, so the airline’s site responds as it would to a local customer.
Providers such as proxyempire.io offer rotating residential proxies in more than 170 countries with targeting down to the city, which is what lets a fare tool check a route as a shopper in Lisbon and then again as one in Chicago.
For a price tracker, this is the only practical way to answer a simple question: what would a person in that market pay right now?
Guessing where the price goes next
Once a site has stored years of fare history, it can do more than report today’s price. It can make a forecast.
The “buy now or wait” badges and price alerts you see on apps like Hopper and Google Flights come from models trained on how fares for a route have moved in the past: how far ahead prices tend to bottom out, how they behave around holidays, how quickly a route sells.
These are predictions, not promises. They are right often enough to be useful and wrong often enough that you should not postpone a trip you need to take because an app told you to hold out for twenty dollars.
What this means when you book?
Knowing the machinery gives you a few practical habits.
- Check more than one site. Each one has a different mix of feeds, direct connections and cache settings, so they don’t all see the same fares.
- Look at the airline’s own page before paying. If a carrier keeps inventory out of the big systems, its website is the only place the cheapest fare shows up.
- Don’t panic when a price jumps. Refresh, search again, and you will often find the real current fare was never the one in the list.
- Treat predictions as a hint. Set an alert, pick a price you are happy with, and book when you hit it.
Next time a results page appears in two seconds, you will know there was a filing system, a reservation network, a pile of airline integrations, a cache and a few proxy requests behind it. Cheap flights are a data problem long before they are a travel one.
