NEP UK scraps pen and paper lipsync calculations at Wimbledon 2021 with trial of Hitomi MatchBox Glass

NEP trialled Hitomi’s MatchBox Glass at Wimbledon Championships 2021

NEP recently ran a trial using Hitomi’s MatchBox Glass lip sync app at the Wimbledon Championships 2021 to reduce the need for pen and paper calculations to synchronise audio and video.

The issue at the Champs for NEP, which was providing technical facilities for host broadcaster, Wimbledon Broadcast Services (WBS), is the multiple and widely varied sources for audio and video from across the many courts.

Explains Adam Robbins, NEP UK senior broadcast engineer: “The problem is that on Wimbledon, we’ve got various different courts using different mixing technologies and different camera technologies, different super slows, different UHD, radio cameras, different ways of getting the cameras back to us; all kinds of stuff. The same way with sound. By the time all this comes back to us, none of it lines up anymore.

“There’s such a variety of UHD, HD, radio, hotheads, proper cameras [at Wimbledon]. It’s too much to try and [work out the delays to sync audio and video] on pen [and paper for each match]. Every single court has its own unique set up. There’s just a lot of different stuff.”

Uncomplicated set up

MatchBox Glass is a free iPhone app that allows measurement to be made from in front of the camera on location. It works with MatchBox Analyser to replace the need for human intervention to test audio and video links.

The trial with MatchBox Glass at Wimbledon involved an interview using a radio camera and a line camera with a radio microphone that was coming to NEP via the host broadcaster’s IT networks.

“Normally I see so many different pieces of scraps of paper with cameras and cables coming out them in different drawings and people have written what they think; all the delays, and it’s like, “that’s brilliant, but you’ve forgotten that””

Robbins comments on the complications this set up created: “We weren’t handling the microphone and the microphone is not un-delayed, and we’ve got a camera that may or may not have two frames of delay in it because it’s a radio camera, and then you’ve got a line camera, which you know has got one frame of delay. It’s all going through the vision mixer, which we know has got a frame of delay in it. There’s just too much going on here. Oh, and the audio router by the way has about 20 milliseconds of delay in it. For me, it’s just wanting to know the right answer.”

To use, an iPhone or iPad running the app is held up in front of up to four cameras. At the other end, the MatchBox Analyser, which has four inputs, when enabled with the licence for Glass, measures video and audio timing differences across all four cameras. The remote MatchBox Analyser in the OB truck or MCR looks and listens for a unique flash and audio tone generated onsite by MatchBox Glass, and in approximately two seconds it returns video and audio alignment and adjustment data, which is particularly useful when aiding remote productions.

Archaic processes

Robbins says the way this process of matching audio to video up till now has been somewhat archaic. “Lipsync is often done by eye. Regularly you’ll see, especially on unilateral interviews for football matches, people tapping their stick microphones,  and then somebody at the far end says, “oh, it looks good to me”. Literally the end of the solution is, “it looks good to me,” in so many cases.”

Frames of delay

On commonly used Valid generators, Robbins states: “You can measure how delayed the audio and the picture are, lined up to [Valid generators]; that’s been around for years. However, the camera itslef has got a frame of delay in it, which this method of measurement doesn’t account for. The inherent delay from the input device is especially important to account for now that we’re in IP land, where you’ve got to get it through to the router, and that’s not instant.

“With the new phone app [from Hitomi], you just hold the phone in front of the camera and put the microphone near it, and then it just comes out with, “this is the path delay, literal, beginning to end. There are no arguments. It is 42 milliseconds [delay]. Let’s fix it now”.”

Adds Robbins: “You don’t have to sit there at your sound desk or wherever you’re choosing to delay the audio or the video, whichever way round it is, trying to work out delays on scraps of paper. [Glass] gives you a definitive answer that says “this is the delay”.

“Normally I see so many different pieces of scraps of paper with cameras and cables coming out them in different drawings and people have written what they think; all the delays, and it’s like, “that’s brilliant, but you’ve forgotten that,” whereas with [Glass] there’s no arguing about it. Rather than people watching clapper boards, slowing the clapper boards down in an EVS machine to try and get it exactly right, [Glass] removes the subjectivity involved and gives a quantitative measurement of any delays that can be relied on.”

Robbins confirmed the trial was encouraging and that NEP UK will be using the technology again at future sporting events.