An Internet Exchange Point provides a meeting point for networks to connect and exchange traffic directly with one another, a process known as peering. This results in fewer hops and keeps traffic local. For AI providers this means lower latency between users and models, and allows interconnection capacity to be scaled as demand grows.
What Are the Main Types of AI Traffic?
Only part of what AI does sends traffic across the public internet. Models are trained on large amounts of data often gathered from across the internet and then used to answer requests from users, which is known as inference.
AI clusters consisting of GPUs exchange that data constantly, working on training the model. That traffic moves between servers inside data centres rather than across the public internet, which is known as east to west traffic.
Inference is the model being used, whether that be generating an image or video, transcribing a call, classifying a support ticket or more. Every new user adds more of these requests, which travel over the public internet.
Why Would an AI Provider Need to Peer?
Without peering, every request and response crosses one or more third-party networks on its way between the user and the model. Each network along the way adds delay and the route is not always direct. Traffic can leave the region and come back to reach someone a short distance from where the model is running.
For an AI provider, peering at an internet exchange can bring several benefits:
How Can LINX Help AI Providers?
LINX connects hundreds of networks across our global interconnection hubs in the UK, U.S. and Africa. For an AI provider, that means efficiently delivering inference, API access and AI-generated content directly to eyeball and enterprise networks.
Members also benefit from:
Our interconnection specialists are on hand to talk through how connecting to an internet exchange can support your needs, reach out today.
Frequently Asked Questions
What is AI Inference?
Inference is a model being used rather than trained. Every time an AI tool answers a question, generates an image or classifies something, that is inference. It is the stage that runs continuously once a model is live, and the stage that sends traffic across the public internet.
Does AI Need the Internet to Work?
It depends where the model runs. Smaller models can run on a phone or laptop with no connection at all. Most AI tools people use day to day run on remote infrastructure, so every use is a request that travels out and a response that travels back.
Does AI Training Use an Internet Exchange?
No. Training traffic moves between servers inside a data centre rather than across the public internet, so an internet exchange plays no part in it. The data used to train a model is gathered from the internet beforehand, but the training itself is not.
Where Does AI Inference Happen?
On servers in data centres and cloud regions, and increasingly at locations closer to users. Where those servers sit relative to the people making requests is what decides how far each request has to travel.
How Does Peering Reduce AI Latency?
By removing networks and geographic detours from the path between the user and the model. Providers whose traffic currently leaves the region before coming back tend to see the biggest improvement.
What is a Token in AI?
A token is the unit a model reads and writes in, roughly a word or part of a word. A response is produced one token at a time and streamed as it goes, which is why text appears gradually rather than all at once. The network path affects the wait before the first token arrives more than it affects the rest.
Do You Need an ASN to Peer at an Internet Exchange?
Yes. Any network connecting at LINX needs its own Autonomous System Number (ASN), which identifies it on the internet and allows it to exchange routing information with other networks.
< Go Back
