Live from Super Bowl LII: NBC debuts virtual 3D graphics tech

Patriots quarterback Tom Brady being scanned

NBC Sports took its use of augmented reality (AR) and virtual graphics to a new level at Super Bowl LII with the debut of “volumetric-AR”, a technology that allows lifelike 3D player animations to be inserted into AR graphics.

In preparation for the broadcast, on Monday before the Super Bowl, the production team completed body scans of six players from the New England Patriots and the Philadelphia Eagles, including quarterbacks Tom Brady and Nick Foles.

These scans provided NBC with 3D player animations that would then replace 2D images in the on-screen AR elements superimposed on the field.

“During the game, when we did graphics on the players — not all the time, but a few special ones — viewers could see the players themselves in full three dimensions with their information designed into the shot,” explained NBC Sports producer Fred Gaudellisaid

Developed by NBC Sports in conjunction with Ross Video and an army of tech vendors from the US and Europe, the 3D virtual graphics were available on the same three cameras as Ross Video’s standard AR system: both end-zone goalpost cameras (with encoded lenses) and the lower SkyCam cable-cam system.

In addition, an exterior beauty shot from an encoded camera also featured the virtual 3D graphics and other AR elements.

The system was driven by Ross Video’s Frontier gaming video-graphics–rendering engine (powered by The Future Group).

Besides the NBC Sports team, a number of other vendors are involved in the project including: Rocket Surgery/Ross Creative Services (which creates NBC’s AR elements for Thursday Night Football and Sunday Night Football); Ross Virtual Solutions, SMT for virtual positioning data; The Future Group for Frontier development; Fanview and Repronauts for the player scans; and RealMedia to handle cleanup, rigging, and animation in the Unreal game engine.

The six players scanned were Tom Brady, Danny Amendola, and James Harrison of the Patriots and Nick Foles, Fletcher Cox, and Brandon Graham of the Eagles, using technology developed by Fanview/Repronauts.

The scans were sent to RealMedia’s facility in Kansas to clean them up and prep them for air.

Completed scans were imported into the Frontier system where the Ross Video Creative Services team onsite in Minneapolis created the in-game animations.

The final elements were then integrated into the AR elements that are traditionally deployed by NBC Sports for its American football coverage.

Standard 2D graphics were used for players who were not scanned.