This article was originally published by Training Ground Guru.

Tracking data has transformed the way that top teams approach and analyse the game.

But what exactly is it? And how is it used? These are the questions Training Ground Guru set out to answer, in some depth, in Episode #68 of the TGG Podcast.

To do so, they engaged an expert in this field: Michael D’Auria, the Executive Vice President of Sports and Technology at Genius Sports, the official tracking data provider for the Premier League.

Michael D’Auria: Tracking data is a class of technology that’s trying to better capture the live action that’s going on on a football pitch. When we talk about it now, we refer to optical tracking data. So that’s a set of cameras installed in a stadium that are using computer vision to translate the live action into as robust a real data representation of the game as possible.

This state-of-the-art technology allows you to essentially create a real-time digital twin or digital replica of what’s going on in the game.

That opens up all types of exciting downstream possibilities for products, services and just having a better understanding of the game. Now we can track thousands of data points on the surface of every player and the ball hundreds of times a second. Over the course of a 90-minute match, you’re getting billions of data points to represent what’s going on.

What’s the origin of tracking data?

Some of this computer vision technology was from things like missile defence systems. Those are some of the early applications of it.

It’s the same type of technology that you see in self-driving cars and in lots of other applications now. It’s using cameras and different kinds of sensors to try to get the best representation of the real world.

Our specific application is sport, which was actually a bit late to adopt some of this technology. That’s because it’s hard, it’s live, it’s dynamic. This is the reason Second Spectrum was founded.

When we started, we were just trying to track the centre of mass of a player projected down onto the pitch in two dimensions. But as we’ve got better and AI has continued to evolve, we’ve been able to get a much more granular representation.

Back then it was a single data point per player, 25 times a second. And the data was delivered to you the next morning, a number of hours after the game. Over time, the systems have got much more mature and they can capture a huge amount more data, but also do it in real time, in less than a second.

A much earlier version of this system was first introduced in the NBA. It wasn’t league wide, but existed at a couple of different venues. Folks in the NBA ecosystem started to really get excited about the possibilities this could bring and how it could improve tactical analysis and how the game is coached and played.

The challenge at that time was that it was a huge amount of data and most NBA teams didn’t staff up teams of engineers. So we brought this combination of athletes and cutting-edge engineers into the space to say, ‘We can help you take this tracking data and turn it into something that’s going to be valuable for you as a coach, as a General Manager, as a fan.’

One of the early pieces of value we added was classifying all the pick and rolls that happened in a basketball game and on all the types of defences that a team could play against them. And it could be things that go beyond what a human can measure like shot probability or expected pass completion.

Even post acquisition by Genius, [our focus] is to create the best tracking data and then use it to create more value in the sports ecosystem, whether it’s a coach, a fan, a broadcaster, a player.

In football now, you can take that tracking data and instantly have hundred of new football metrics that fans and coaches have always wanted to talk about. Pressing and pressure and between the lines, passes and overlapping runs, things that are happening on the ball, things that are happening off the ball.

And you can deliver this much more robust and complicated language of football in real time.

This has unlocked a new era of how data could be used to inform sports. You really saw this in the NBA over the past decade – for all 30 NBA teams, their workflows now are totally reliant on this data.

They make a decision about how to prepare for a game and how to defend the team they’re playing, to how to value a player when they’re making a trade or acquiring somebody in free agency. This level of information just didn’t exist in sport before.

Entry into English football

You started to see individual clubs dabble with this about 10 years ago, a bit later than the NBA.

As a company, as a business, we really made the transition from being just a basketball company to working in football via our partnership with the Premier League (in 2019).

Now, every Premier League ground – and actually every Championship ground too – has our optical tracking system installed. So at a Premier League ground, you’re going to have about 28 or 30 cameras deployed throughout the stadium.

We actually use a smartphone as the core unit of our system we use. So there are 28 or 30 iPhones at every Premier League ground that are capturing data real-time for what’s going on in a game.

I think the Premier League has been a great example of how this data you can capture can touch every part of the football ecosystem. All of the clubs use this data on a day-in and day-out basis to evaluate players, figure out strategy, help their Manager, be informed about what’s going on in the game.

The media uses it as well. Once a week, Premier League Productions puts out something called Datazone, that uses data and video augmentation to have a much different presentation of the game. Sky Sports are huge users of our data from an editorial perspective to tell deeper stories about what’s going on.

And then this year, you may have seen that we’ve launched semi-automated offside technology, which is powered by the exact same data. And so you’re now moving into the world of impacting officiating of the game as well.

This is where we think there’s a lot of power when you can capture really high-quality data in real time and use AI to translate it into football language that someone’s going to care about.

Whether you’re a club or an official, you can really bring a lot of improvement to the game with that. That’s what we’re after.

How do you collect tracking data?

We actually use the camera on the smartphone. It sounds a little bit interesting, but one of the things about phones is that over the last decade or so they’ve had billions and billions of dollars of investment put into them.

It’s a very competitive market and the cameras just keep getting better and better with every evolution of the phone. We can capture up to 200 frames-per-second on the iPhone and, because it’s a small unit that you can point wherever you want, you can really have a lot of focus on certain areas of the pitch where you might need more coverage.

And because they’re relatively cost effective and easy to mount, you can really scale up or scale down the numbers of phones you need. And so we think it’s going to be the right system for the future. Whereas even if new use-cases come online, it’s not so hard for us to go back to a stadium and add another 4, 5, 6, 7 phones.

We tend to put them up pretty high and try to tuck them away in the rafters or in the higher parts of the stadium. Every ground is a little bit different and we’re pretty creative about where we can fix them.

Again, they’re small units, they just come in a bit of a weatherproofed housing, and can be affixed to just about any part of stadium infrastructure. But we generally try to just get as good a coverage as we can, all the way around the pitch.

For us, it’s really about having as many high-quality angles of the pitch as possible to make sure that in any given moment you have multiple cameras pointed at every part of the pitch.

How is the tracking data delivered to a team?

The real value is that we can deliver this tracking data FOR you. Then we can apply another layer of real time AI, to basically translate that tracking data.

We’ll process that video into tracking data and provide that as an API to clubs. Then we’ll have another layer of processing that turns it into that language of football.

Not just, ‘where was my right ankle for this 100th of a second,’ but, ‘this was a particular football action.’ That’s also delivered as a separate data API that clubs can consume.

This all goes into a software tool that we produce for them that can further help them process this data in a live environment. So it will have a live fitness tracker to help them manage substitutions and understand if players are getting a bit fatigued, it will allow them to chart data they might want to be looking at over the course of a game, it will allow them to query the data and precisely index it to video.

So if you want to look at every time your left back made an overlapping run and joined the attack, you can very quickly pull up the video playlist of all those moments.

It’s really using it to try to help video analysts and data analysts – whether it’s data, video or visualisations – in the most bespoke way possible.

Most clubs now have somebody who works in data science or a similar profession and their job is to connect to the live streams of this data and video, power it through whatever internal systems the club has and use it as a way to evaluate how they’re performing in the match or get ready for some reports to give to the Manager at half-time or as a post-game analysis.

We’re pretty tightly integrated with all those clubs, trying to make sure all that’s happening as smoothly as possible. They’re generally taking in a data API that might power an internal report that they’ve constructed for their club or their Manager.

A lot of the work does happen through our software tool, where we can help them process it and they can preset some things. You have a dashboard and you’re getting alerts about whatever things you might particularly care about during the match. And a huge part of football coaching and analysis is video analysis.

It’s a lot of the ways that the actual message finally gets translated to players. So a lot of this just helps them really quickly and really efficiently or fully automatically cut up video into a playlist, so a Manager can know at half-time they’re going to be able to walk in and have the seven video clips of their team not performing the way they wanted to, or show something their opponent’s doing that they need to respond to.

It’s a way we can really help clubs do that faster and more automatically.

Very often, our data will be powering some sort of report that they want to be looking at, or some sort of tracking of what was going on during the action. The greatest Managers have a really good eye test.

They can watch a game and just get a handle of what’s going on everywhere on the pitch intuitively, because they’ve watched so many games before.

What we can really do is give them a superpower to be able to do that with AI – so something they might be missing, or something that maybe allows them to focus on a different part of the game.

Liverpool: Early adopters

At this stage now, we see all 20 Premier League clubs as heavy users of tracking data. When technology is a bit more nascent, there are always clubs that are a bit more eager to lean into that and that’s sometimes dictated by who the staff are or what their resources are.

Liverpool were one of the early ones to invest in this, they were early adopters.

And it’s also one of the ways that technology has really evolved. When Will (Spearman) started at Liverpool, there was probably a number of seconds or even minutes of latency in the data feed.

Now we’re delivering it in less than a second. If you’re looking at the game and then down at an iPad or a device, you can’t discern the difference.

(Spearman has previously spoken about using tracking data to assess pitch control).

It’s one of the great things about football – it’s such a geometric game where space matters so much. It’s one of these great things that we can do.

We can very precisely measure exactly where every player is, exactly where the ball is, and so, by default, how much space they are occupying, or how much pitch control they might have, depending on how people are moving.

And we can represent that to an analyst. That’s something that, again, a great Manager might get a feel for, but we can actually mathematically measure it now and give you a real-time view of it and a report of how that has changed and evolved over the course of a half or 90 minutes of football.

I always enjoyed visiting Liverpool. I had an engineering background – I went to MIT many years ago – and they had had the most physicists I’d ever seen employed by a football team!

It was really a good fit, because I think the way that a physicist models 3D interactions in the real world actually has a lot of parallels with the type of data we’re trying to capture in a football match. Now we refer to it as spatiotemporal data – it’s data about how physical objects move through space and time.

There are quite a few parallels in that core level of tracking data and the way that physicists model interactions in the real world.

We look at a club and say, ‘You are the absolute world-leading experts on the game’. It’s our job to show them what’s possible with the technology and they are often the ones that come to us and have some really great ideas about the practical applications they would like. We will help bring them to life in our set of products.

That’s been a really fruitful cycle for us. The data is now able to touch more and more parts of the club and there’s more and more you can do with it.

The Sports Scientists use it to measure physical performance. The Video Analysts use it to automate their video workflows. You have these data science teams now that are managing different parts. The recruiting group, Head Coaches and Assistant Managers, are using this now as well.

Recruitment

The great part about the system is that once it’s installed, there’s really no limit to the kind of ways you can evaluate a player or a team. You can get precise information about their physical attributes: how fast are they running, their acceleration, their deceleration, how much they fatigue at what points in the match, how they perform under fatigue.

Because you have full 3D body modelled, you can really understand how their leg swings when they are trying to bend a cross in, how high can they get up when they’re trying to head a ball into the back of the net and really understand that dynamic way their body moves.

A lot of this stuff wasn’t possible before. For years, people would do an analysis on how many times a midfielder might have a head swivel or be checking for space as he’s receiving a pass and getting ready to distribute. These become things you can now mathematically model.

You can look at the likelihood that a player is going to complete a pass and how they actually perform against that. I think recruitment is probably the next big frontier here.

Sharing tracking data between countries and leagues

We have really great coverage across English football, in the Premier League and the Championship.

One of the big things we hope to do is expand the number of leagues where this type of tracking system exists. It was one of the real design principles behind our newest tracking system, where we can go to 28, 38, 50 phones in a Premier League ground, but you can also scale that down.

So if you get to smaller leagues that are a bit more resource constrained, we can still have that same core tracking system deployed. So a lot of what we’re spending our time on now is trying to promote getting this system out to more places where significant football is played.

We think it will democratise a lot of these tools and generally help the whole industry if there’s that level of data sharing. When we started in the Premier League, data wasn’t even shared across all 380 matches.

You would only get access to data for games that you played in. That’s changed now. If you’re a Premier League club, you get access to all 380 matches. And so we’d love to see a similar thing happen between leagues as well, between England and France and Belgium and Denmark (we work in all those leagues), to get a data exchange.

And then, as a club, you can really look across all the different leagues as you’re going through your recruiting process.

Data is becoming more and more portable and we’re becoming more and more comfortable sharing some (maybe not all!) of this information. And so we certainly support more cross-league sharing of data in the future.

We already know some leagues have frameworks for this. Ultimately, of course, that’s going to be a league-by-league decision, but we think there’s kind of just more value for everybody if that starts to become more of the norm.

I think we will start to see that pretty soon. It won’t be all at once, but I think we’ll start to see that happening a bit more.

Broadcast tracking is an opportunity as a stopgap but it is is never going to be as robust and detailed as having a fixed set of cameras in a stadium.

That’s just based on the kind of physics of occlusion and how many cameras you have, but that is still a better data source in some instances than traditional manual events.

Future of tracking data: mesh tracking

Mesh tracking is fully deployed in the Premier League and allows you to do a couple of really important things.

The first one is for officiating, to really make a call on off-sides. You don’t want to approximate where a player’s centre of mass is, or their shoulder joint is, you need the full surface of a human body mapped out with thousands and thousands of data points per frame.

On a really tight call that’s coming down to one centimetre, you need to know with certainty the moment the foot struck the ball, the precise kick point, and then exactly where the curvature of the shoulder or thigh or whatever part of the body is being evaluated in is with regards to the offside line.

Premier League players are really impressive athletes, but they have different body shapes, and so being able to get that level of resolution is very important. It has allowed us to really push into the officiating space where we can have the most accurate calls and again, do it in real time, so you’re not interrupting the game for a fan.

The second thing I expect is this world of 3D recreations of football for fans and for coaches. If a phenomenal goal is scored, we’re used to having a replay before the next kick-off, a couple of different angles from wherever coverage you have in the stadium.

With these really realistic 3D recreations, you could instantly go into a first-person view to see what it felt like to be Hojlund as he was scoring a goal, or you could go from the keeper’s view or travel the path as the ball. You can now watch and experience a game in this 3D environment that’s hyper realistic.

We’re already seeing it with football clubs: where if you’re trying to coach an athlete on what they were doing wrong or a different decision they could be making, it’s often much more effective to go into the 3D world and show them what it felt like to be on the pitch.

While we all watch the game from that midfield camera-one view, that’s not the way a player experiences the game at all. So if you can actually show them and speak to them in their own language – it’s a really potent tool.

It’s easy to critique from that overhead view, but that’s not what the players see. The better you can show a player what they were actually experiencing on the pitch, the better you’re going to be able to coach them. And the same thing goes for a referee too, right?

Sometimes it’s really instructive to put yourself in the position of a linesman who’s trying to call a close offside on the other side of the pitch and give a bit of perspective of how challenging it is to make that call in real time.

You think about how tracking evolved – one data point on each player 25 times a second, and then maybe 25 or 30 skeletal points, maybe 50 times a second.

That’s a pretty big leap. Now you’re going up to 10,000 data points per player, hundreds of times a second. And so the volume of data is just scaling up. And frankly, not every club needs that level of data or is going to instantly get value out of it.

Different clubs, different approaches

A club has more data to consume and play with if they want to go down that route. But we’re still going to package that data up into products that are easy to consume, so you, as the Data Analyst, get more value, more options, but don’t necessarily need to process all that data.

The 3D recreation is a great example. We produce those in our software tool. And so an analyst can just go around and navigate the 3D world. They don’t have to worry necessarily about the billions of data points that went into creating that.

We can also give you tactical data points about how the physical bodies are moving in kind of human language rather than having to parse through, you know, all these representations of again, an ankle rotating through space.

And so, for the clubs that want to go down and do it on their own, great, they can have at it. And for the clubs that maybe that’s a bit overwhelming for, we can help them turn it into kind of products and services that are more powerful than they were before.

They both give you more precision, they give you a better representation of what’s going on in the football world and then that leads to the possibility to do more with it.

If I go from knowing I’m a dot in two-dimensional space to actually knowing the full context of how my body’s moving, there’s just more you can analyse.

I could maybe look at a space when I was 2D world. But you can now get down to the technique of how you’re planting your foot and how you’re running and how you’re rotating your body – all these things that matter to the game.

Potential for competitive advantage

Then it creates opportunity for bigger staffs. We definitely saw that happen in the NBA. When we started in the NBA, two or three clubs had one or two people who were working on this in the background.

Now you have, across all 30 NBA teams, groups of 10, 15 people who are doing this every day. The other way you can think about this is it kind of creates a new area for competitive advantage.

If you are a club that is not at the top of the table or doesn’t have the biggest budget, but you want to get really smart about how you deploy technology across your organisation, it gives you a new place where you can try to find a competitive advantage.

And I think, particularly in football, there’s still a lot of opportunity there. It is such a dynamic and flowing game that I think there’s a lot more potential of how data can have an impact and evaluate what’s going on.

It’s one of the reasons we’re really excited about the future value we can hopefully bring to the ecosystem.

We love to chat about this stuff and are really trying to promote the deployment of technology and this exciting future all across sport.

Tracking data: Past, present & future

Discover the future of play

Products

Solutions

Learn

Company