When the 2022 World Cup made its debut on Sunday, it kicked off one of the most significant in-game uses of technology in sports history.

All tournament long, match balls will contain a sensor that collects spatial positioning data in real time — the first World Cup to employ such a ball-tracking mechanism. This, combined with existing optical tracking tools, will make VAR (video assistant referees) and programs like offside reviews more accurate and streamlined than they’ve ever been. Combining these two forms of tracking has long been a holy grail of sorts in technology circles, and FIFA’s use of the ball sensor in particular will serve as a highly public test case over the next four weeks.

Like so many other parts of the burgeoning world of sports tech, the setup used at the World Cup is both an endpoint and the foundation of a whole new era. Years of research and testing were needed to get here — this particular ball sensor was in development and testing for six years before receiving full FIFA certification — but events like this could quickly catapult emerging technology into the public eye through applications that stretch well beyond officiating.

What went into the development of today’s tracking technology, and what are its key uses at this World Cup? How has the tech been tested, and how can players, teams and fans alike be confident that it’s accurate and consistent? And maybe most importantly, what does this same technology portend for the future of analysis, fan engagement and team data across the world’s most popular sport?

I spoke to people across the world of sports tech to find answers around one of the field’s boldest experiments to date.

Which technology is being used, and how does it work?

FIFA’s application of this technology at the 2022 World Cup is being termed a “semi-automated offside” program – one that’s largely run by AI features, but retains a vital element of human confirmation.

Within every match ball is a device designed by KINEXON, a major player in the performance-tracking world across several sports. Per the company, this device weighs 14 grams (just under 0.5 ounces), and actually houses two separate sensors operating simultaneously:

• Ultra-wideband (UWB) sensor: A type of technology that’s superior to GPS or Bluetooth for precise positional data, plus can transmit data in real time to constantly track the ball’s position.
• Inertial measurement unit (IMU) sensor: A sensor meant to detect nuanced movements of an object in space.

“While the ultra-wideband helps me to have the position of an object, the IMU gives me the granular movement in three dimensions,” said Maximillian Schmidt, co-founder and managing director of KINEXON.

So any time the ball is kicked, headed, thrown or even so much as tapped, the system picks it up at 500 frames per second. Data is sent in real time from sensors to a local positioning system (LPS), which involves a setup of network antennas installed around the playing field that take in and store the data for immediate use. When a ball flies out of bounds during the course of play, and a new ball is thrown or kicked in to replace it, KINEXON’s backend system automatically switches to the new ball’s data input without the need for human intervention.

KINEXON’s in-ball device is supported by suspension technology provided by Adidas, designed to house the sensor at the central interior point of the ball and keep it secure in a consistent location.¹

Paired with this ball sensor is optical camera tracking from Hawk-Eye, a system well known for its work in tennis. Twelve Hawk-Eye cameras are set up around the stadium, tracking both the ball itself and each player 50 times per second. Twenty-nine separate points of the body are tracked for players, including limbs.

When combined, these two data sources allow for offside decisions that are not just highly accurate, but also available much faster than in the past – a major priority for FIFA in this World Cup cycle.

“We debriefed in 2018 after the World Cup,” Nicolas Evans, head of football research and standards for FIFA Technology Innovation, said. Such a debrief is standard after each World Cup, according to Evans. “The biggest area for improvement we saw was the time it took to make offside decisions.”

With that in mind, data from both KINEXON and Hawk-Eye is run through artificial intelligence software that’s programmed to generate automated offside alerts to officials in the match’s video room. Instead of manually combing through plays, a time-consuming process, AI programs auto-generate alerts that can then be confirmed by video match officials.

The software also generates 3D renderings of the spatial data, which will be overlaid onto TV broadcasts and in-stadium monitors to give fans a direct look at how each reviewed call was decided.

How accurate is this data? How is it tested?

Some readers might have an understandable question at this point: “How do I know this data consistently reflects the true positioning of the players and ball?”

The first concept to understand here is refresh rate, which is measured in Hz and refers to the number of times per second that a given display is able to draw a new image. Standard 50Hz video, a common format used for HD monitors today, is generating a new image 50 times per second (or a “frame rate” of 50 frames per second, for those more familiar with that term). Anyone who has slowed a video down to frame-by-frame mode knows the sensation of slight periods of time passing between each frame – and for 50Hz video, that gap is 20 milliseconds per frame (for 60Hz video, it’s 16.66 milliseconds per frame, and so on).

With KINEXON’s ball-tracking system, that data is coming in at 500Hz, meaning any gaps in true positioning are below two milliseconds long – or 10 times shorter than standard 50Hz lags. Furthermore, the use of a PTP master clock allows synchronization between KINEXON and Hawk-Eye data that’s precise down to one-millionth of a second, per Evans, ensuring the two feeds are never out of whack with one another.

To check each component of the system, FIFA has conducted tests in both controlled and live settings.

For both the KINEXON ball sensors and Hawk-Eye’s optical camera setup, a format known as “ground truth testing” is required by FIFA’s Quality Programme for Electronic Performance Testing Systems (EPTS). The approach utilizes a minimum of 36 high-quality Vicon motion-capture cameras, combined with reflective markers designed for these cameras placed on the ball itself and every player for ultra-accurate detection. As players and the ball move around the testing pitch, both these Vicon cameras and the KINEXON/Hawk-Eye setups run simultaneously – and researchers compare the two outputs to assess the accuracy of the latter systems. This testing is augmented by the use of other tools, such as a laser to detect events like high-speed player sprints.

Another kind of test is also vital here, especially for one of the world’s most-watched sporting events: Ensuring that the addition of a sensor inside the ball is imperceptible to the players kicking it. Since it’s all too easy to envision a scenario in which a prominent player blames the feel of the “different” World Cup ball for a missed penalty or some other on-pitch failure — igniting a worldwide controversy — eliminating such concerns is naturally important.

Adidas is the key entity responsible for this kind of testing, which sources familiar with the program² report is done in two ways:

• Blind player testing: With the help of clubs in places like Spain, Germany and England, extensive blind testing was carried out to see if players could tell the difference between “normal” balls and balls infused with the sensor/suspension setup without knowing which was which ahead of time.
• Mechanical shooter testing: In a lab setting, robotic shooting devices can be programmed to “kick” the ball at varying speeds, spins and directions. High-speed cameras then evaluate the flight of the ball, ensuring that the presence of the sensor does not create abnormal flight paths.

This testing was performed in several professional and grassroots settings, such as the 2021 FIFA Arab Cup in Qatar and the 2021 FIFA Club World Cup in Abu Dhabi. In addition, Evans said FIFA ran several of its own additional blind tests, including at a team workshop in Qatar earlier in 2022.

“Together with our partner Adidas, large-scale sample blind tests have been carried out where no differences could be observed by players between the connected ball and the regular ball,” Evans said.

These and similar forms of tech also have been getting real-life test runs for years now. An earlier version of KINEXON’s sensor-infused ball was used alongside jersey tracking in 2018 in Germany’s fourth-tier Regionalliga to bring real-time data to a live TV broadcast; a newer setup was used earlier this year during a German academy club match. The Hawk-Eye camera setup, meanwhile, has been utilized in the UEFA Champions League group-stage matches throughout the early part of this season in the lead-up to the World Cup.

Of course, none of this guarantees an incident-free event. Unforeseen circumstances can come up in the world of tech. Equipment can malfunction. Even in the likely scenario that the tech works perfectly from start to finish, good luck convincing crazed sports fans of that if their country’s top player has trashed the tracked ball or suggested the system isn’t accurate.

Such possibilities are all but unavoidable in the end. But the companies involved are confident they’ve met every important threshold to introduce this new level of technology into the world’s most important soccer competition.

“[It’s] probably as good as it’s ever been in this type of technology,” Evans said.

This is the tip of the technological iceberg

The 2022 World Cup is barely scratching the surface of the potential capabilities of this sort of technology – which go well beyond officiating.

Before long, this combination of ball- and optical-tracking could be a familiar tool to everyone involved with the sport — much like Second Spectrum tracking in the NBA or Hawk-Eye in tennis. Teams and players could use the resulting data for next-generation tactical analysis; broadcasts could use it to visualize the game and draw in new viewers. Fans could eventually have access to a vast ocean of new stats that were never possible until now.³

In fact, this already has started happening.

Earlier in 2022, KINEXON’s ball sensor was used in a Liga Portugal relegation match, along with sensors in player jerseys to track their movements. More than 300 different metrics were collected in multiple categories, including technical data (shot and pass speed, ball possession time, distinct ball actions), performance data (dribbling speed, accelerations and sprints — both with and without the ball — plus measures of athlete “load”) and tactical data (for both teams and players, tactical concepts such as space control, pressing, ball gains/losses and counters can be tracked).

Possible applications of this data, especially once it’s a more standardized part of the game and the parties involved become familiar with using it, are virtually limitless.

“We can use that information in real time to tell new stories,” Schmidt said. “[We can use] them to create virtual worlds, augmented overlays, insights on performance of players.”

For one thing, expect soccer fans to have new ways of enjoying the game based on this advanced tech. The first layer here will be broadcast improvements: More 3D immersive graphics, similar to the offside renderings 2022 World Cup viewers will see, could be used in several new ways – even those that viewers control. We’ll also see overlaid on-screen data, much like the LA Clippers’ “CourtVision” broadcast option that displays tracking information and a digitized view of the court during play.

And the next wave after that is even more exciting to think about. The world of virtual reality offers tantalizing potential; imagine watching plays (or even an entire game) from the direct vantage point of Kylian Mbappe, Karim Benzema or any other player of your choosing. Evans and others in the space also expect this data to eventually make its way into the world of video games, merging real and digital realms.

That’s just on the fan side. As the 2022 World Cup tracking setup becomes more commonplace around soccer, players and teams will now have a wealth of new data, particularly surrounding the ball – the true heartbeat of the sport. Tactics, analytics and even all-important health and performance improvements are all on the table.

Said Schmidt: “More and more leagues are now thinking about, ‘How can we enhance the already existing dataset with new insights from the ball?’ Because it’s the head of the game.”

So as tech continues to change the way we view sports, the 2022 World Cup stands poised to serve as a watershed moment. When you see that first “semi-automated” offside call, remember what it took to get to this point – and daydream about what the near future holds.

Check out our latest World Cup predictions.