Live from Super Bowl LI: Intel’s ‘Be the Player’ set to transform analysis

One of the more anticipated advances for Super Bowl LI is the ‘Be the Player’ technology that Fox Sports plans using to give analyst Troy Aikman an option no one has ever had before: to see the field through the eyes of a quarterback on the field. Intel’s freeD technology was used at Super Bowl 50 by CBS Sports but advances in technology in the past 12 months have taken the system to a whole new level.

“We had to redesign from the ground up the render algorithms to increase the render resolution,” says Diego Prilusky, Intel Sports Group, Director of Product and Creative. “We doubled the amount of zoom just by changing the software architecture and having a big leap in software optimisation on the hardware.”

The system requires 36 5K cameras to be placed strategically around the stadium to cover the action from all 360 degrees. Two additional cameras are on hand to capture background imaging. The advances in the past year have doubled the resolution and cut the processing time by 50% and 36 computer processing servers are on hand to bring all the image data together.

Scott Katz, Fox ‘Be the Player’ producer, will be working with a small team of eight from Intel to create the magic. Katz will be watching the game and if a play of interest occurs he has only seconds to decide whether to start building a clip as the system over writes itself. Once the decision is made to create a replay he notifies the ‘pilot’ who operates as the editor and helps identify the points of interest in the frame.

For example, Katz will look at a wide image of the field that includes all the players. He can tap the quarterback (or any player) to identify that the replay will be seen from the quarterback’s perspective. He then taps on the wide receivers to set those players as the place to start and end the POV replay. The system then takes about 60 seconds to render the replay.

The Be The Player replay system pilot is essentially the editor of the replays that will give viewers a POV view from anywhere on the field

What the viewer sees is based on the player chosen for the POV and the start and end points. In this case the replay would begin with a sweeping camera move from behind the quarterback down to his viewpoint and then a still image would freeze the players. The viewer would then see what the quarterback would see as he looked at each receiver. The replay ends with the virtual camera pulling back and then rolling a video of the rest of the actual play.

The interesting thing about the system is that the POV perspective can be from any player and even a referee.

“There are no real limitations for where you start, end and where you want to go,” says Prilusky. “We have been playing around with some other things. One of those things is showing what the referee sees on a play. We did a ‘Be the Ref’ and you can see what the refs are seeing, even something like a hold,” says Katz.

The production team also includes a navigator who plays the role of the engineer and then another half dozen staffers that work in the image processing and an EVS operator in the production truck who makes the clips available to the front bench production team.

Given that Aikman will be analysing the Be the Player replays odds are most of them will be from the QB perspective as he will easily be able to offer up his thoughts on what Tom Brady or Matt Ryan are seeing via the replays. The challenge for Katz will be to make sure that a replay of the quarterback’s progression from one side of the field to the other goes from right to left or left to right.

Intel has installed 38 5K cameras around NRG Stadium to capture images for the Intel Be The Player replay system

“It will be about two minutes from the time we grab the play to it makes into the EVS server so we aren’t going to try and get it in the first replay sequence,” says Katz. “But we can turn it around and have it ready after the next play.”

Getting to this point involved practicing with the system at Levi Stadium in San Jose, CA (home of the San Francisco 49ers) and M&T Bank Stadium in Baltimore (home of the Baltimore Ravens). The Foster Farms Bowl then served as the first true test of the system on hand for Super Bowl LI.

Prilusky says the company is already looking at applications for other sports, including basketball and soccer. The key to using the system is making sure there is a place to hang the 5K cameras around the field of play.

“This is where we always aimed to be and with the support from Intel we were able to take the hardware and software implementation to the next level,” says Prilusky.