Super Bowl LIV: THUMBWAR’s Michael Drazin on an historic HDR workflow
Plenty of planning and regular-season testing led to 1080p HDR production
THUMBWAR’s Michael Drazin played a key role in Fox Sports HDR Super Bowl efforts
Like many firsts, the UHD HDR production of Super Bowl LIV was the result of months of planning and, arguably, years of experiments done not only by Fox Sports but an entire industry.
Key to the Fox Sports efforts was a plan overseen by Michael Drazin, VP, special projects, THUMBWAR. The production format was 1080p HDR, with the HDR portion relying on the HLG BT.2100 (hybrid log-gamma) format, which was developed by the BBC and NHK.
“The goal of what we are doing,” he said, “is to make it so that production should have no idea they are working in HDR except that it looks better; all of the elements that are available to an SDR show are available to an HDR show.”
That is easier said than done as it requires plenty of planning. For those who are looking to try an HDR production, Drazin says that is where it all begins.
“The biggest thing is planning and understanding what you want to do, how you are going to do it, and why. And you have to make that plan before you get to the parking lot. If you have a plan in place when you get there, you can execute. But you really need to stop and think through the entire production.”
For Fox Sports, that planning began in the preseason and followed through the entire regular-season schedule of Thursday Night Football contests. Those preseason and regular-season contests allowed not only Fox Sports but also Game Creek Video and other partners to sort through workflows. The workflow begins by using only the HDR signal coming out of the cameras.
“All the HDR cameras are set in HLG BT.2100, and we are taking only HDR out of the CCUs,” said Drazin. “We are not using any SDR from the CCUs.”
Any source that originates as SDR is run through an AJA FS-HDR real-time HDR/wide-colour-gamut converter. The production is using BBC lookup tables (LUTs) to tone-map signals from SDR to HDR. The tone-mapped signals are then available via each truck’s router and to all replay devices, monitors, and the switchers. That allows cameras like the PylonCams and other specialty cameras to be ‘tone-mapped’ and integrated into the production with the HDR cameras.
“BBC and NHK are the founding fathers of HLG,” said Drazin. “The BBC’s lookup tables follow the science behind HDR and are available under license to the industry. We have seen many manufacturers and broadcasters license their work, and it has become the de facto standard. The team at the BBC has done excellent work and has actively collaborated with us as they have continued to improve the colour science.”
There were about 100 AJA FS-HDRs across the Fox compounds at Super Bowl LIV between Hard Rock Stadium and South Beach. In 1080p, each converter has the ability to handle four signals, including a color corrector where needed.
“Having all of the handoffs in HDR between the trucks and sites streamlines the entire operation, as there aren’t different iterations of signals floating around,” said Drazin. “That was a decision we made months ago, and it has made things efficient and is paying dividends.”
Making things cohesive
One challenge is bringing all the different elements into a single cohesive production. The industry has learned that two types of conversion are needed to get from SDR to HDR. SDR cameras use a scene-light conversion, in which the science closely resembles how a camera sensor works.
HDR graphics use a display-light conversion. A final product, like a graphic or a video element, is tone-mapped as a display-light conversion. Currently, the LUTs are doing what is known as a direct conversion, where the SDR is mapped into the HDR space without any expansion taking place. This enables the SDR content to make a roundtrip, going through the HDR production and then returning as SDR.
“Utilising two different LUTs to get from SDR to HDR, as well as the 75% anchor point, we can build a cohesive production in both HDR and SDR,” said Drazin.
Although the entire production benefited from the regular-season contests, the video operators, who are responsible for shading the cameras, had an opportunity to see how HDR enabled them to be more creative and also see how their work looked when returned to SDR via inverse tone mapping.
“Utilising two different LUTs to get from SDR to HDR, as well as the 75% anchor point, we can build a cohesive production in both HDR and SDR”
“Working with the video teams,” said Drazin, “we learned that they preferred to shade in HLG on an HDR monitor with an SDR monitor next to them with a predictive downconvert. This enables them to see how it will look when the signals go to air. We have seen the video teams quickly become accustomed to this workflow.”
The goal with the inverse tone mapping to SDR, he added, is to maintain the shader’s artistic intent.
The other big question facing an HDR production is monitoring for the entire team. Do you need to replace all the SDR monitors with HDR-capable versions, or can most of the team get by fine looking at HDR in an SDR monitor? For Super Bowl LIV, all the multiviewers in the compound were SDR monitors that inputted HDR signals, and the preview and programme monitors in the production trucks were inverse-tone-mapped so the SDR was what was going to air.
Improvements ahead
Drazin says the current state of HDR is similar to the early days of HD when the 4:3 needs of SD viewers trumped the potential ways the 16:9 screen could change the nature of production.
“Right now,” he noted, “we are protecting for SDR like we used to protect for 4:3. When we reached a tipping point in the market, the graphics slid out to 16:9, and we don’t even think about 4:3 anymore. How long will it be before we get to that point in HDR? Is it six months, a year, or 18 months? I have no idea, but it will be interesting to see.”
File-based workflows are another consideration, especially given the number of video elements used during the pregame and the game itself.
Fox Sports used Telestream’s Vantage transcoding to deliver SDR elements into the EVS servers. The same Vantage servers enabled delivery of both HDR and SDR archive material from the game. The Vantage system was loaded with the same LUTs being used in the baseband workflows, enabling consistency between the file-based and baseband environments.
“We work with the EVS operators and the technical directors to ensure they are empowered to QC HDR content on a waveform monitor to ensure that everything going to air is correct,” said Drazin.
The halftime production was also in HDR this year. The Fox team worked with that team and applied the same structure to the production that was applied to the rest of the Super Bowl. The entire halftime team from the engineers to the operators quickly adapted to the environment.
“The entertainment team brings a different approach to their execution, and it fit right into the structure that we have designed,” said Drazin. “It is amazing to see how the technology and structure empowers them to excel creatively.”
Post production considerations
Fox brought a large post-production operation to the game as well. Using the same production structure, the post team was empowered to use HDR in pieces where there was original content to support it.
“We built workflows so that the post team could work entirely in SDR from editing to colour grading through to delivery to the production via tone mapping,” said Drazin. “We also built workflows where the colourist could work in HDR using the same methodology as shading.
“By utilising a predictive tone map, they could see how their artistic decisions would be delivered in both viewing experiences. They then were able to utilise the additional latitude that exists in HDR to make creative decisions for the final grading of the piece. It is exciting to see how HDR empowers the colorists and how many of the finished pieces were graded in HDR that were delivered to the production.”
All the HDR efforts in Miami were ultimately delivered to an HDR master-control room in Los Angeles that integrated the HDR feeds from tone-mapped SDR commercials and promos, delivering a complete 1080p HLG BT.2100 HDR programme. This programme was then upconverted to 2160p (4K) for the distribution partners.
“Some of the distribution partners will deliver it in HLG; others will tone-map it to HDR10,” said Drazin. “There was a collaboration with the distribution team at Fox to ensure that both experiences were seamless.”
“Once you start watching your favourite content in HDR, it becomes really hard to go back to SDR”
While many in the industry are just beginning their journey to HDR, Drazin says the workflow for Super Bowl LIV is battle-tested and ready to go.
“It has been proven in all kinds of conditions: night, day, rain,” he said. “It works, and there are no worries about the workflows as it has been done all season. It will continue, evolve, and get better as everybody in the process continues to collaborate. The structure we have built to work in is solid.”
Drazin looks forward to seeing how HDR will continue to improve the experience for the audience as the tools continue to evolve and creatives are empowered with more latitude to work.
“There is a tremendous team of professionals here working for Fox putting this production together,” he said. “It says a lot about them for how fast they have adopted this technology and have driven it to improve the production and the experience being delivered to the audience. Once you start watching your favourite content in HDR, it becomes really hard to go back to SDR.”
Super Bowl LIV took place on 2 February 2020 in Miami