SportTech 2017: Fox Sports Netherlands ventures into virtual camera technology

After nearly two years of R&D, Fox Sports in the Netherlands recently went live with an innovative virtual camera system that makes it possible for viewers to feel more intensely immersed in the action of Eredevisie League matches. At a SportTech session on June 7, Fox Sports Netherlands Head of Production Henk van Meerkerk, along with partners Mark Bowden, Head of Sports Project Management at ChyronHego and Nicolaas Westerhof, Development Lead at Beyond Sports, discussed the challenges involved with this project in discussion with SVG Europe Editor David Davies.

On the panel (L/R): David Davies, SVG Europe; Nicolaas Westerhof, Beyond Sports; Mark Bowden, ChyronHego; and Henk van Meerkerk, Fox Sports NL

With the goal of providing the concept of VR in live sports and engaging millennial fans, Fox Sports NL teamed up with Beyond Sports and ChyronHego to broadcast the first-ever live football match enhanced with computer-generated VR (not 360° video) at the February match-up between Feyenoord and PSV. The VR version was streamed in real-time to Twitch and YouTube.

Key to the VR broadcast is ChyronHego’s TRACAB, which uses advanced image-processing technology to determine the position of all moving objects in the field of play. The result is a live data feed that contains the X, Y and Z positional coordinates for each identified object.

To create the VR component in February, ChyronHego and Beyond Sports created a workflow in which the live tracking data generated by TRACAB could be visualised in a VR format. The goal was to create a live, first-person, ‘virtual’ view of each player broadcast alongside the actual match footage – delivering virtual images of real, live situations on the field.

Beyond Sports core strength is the virtualisation of tracking data. Using computer-generated imagery, the Beyond Sports engine creates a full 3D virtual simulation of sports matches based on the X, Y and Z positional coordinates. Once a 3D simulation has been created, it gives the end-user the possibility to view and record the match from any perspective.

Beyond Sports created a fully automatic camera system, that follows the play around the field without any intervention from an operator. Fox Sports analyst and former star Ronald de Boer used Beyond Sports’ VR Analysis Tool in ‘Dit was het Weekend’ (Match of the Day). He viewed Feyenoord’s goal through the eyes of Dutch player Terence Kongolo, who gave the assist, to show the decision-making process Kongolo faced in that specific situation.

“We not only have the rights to the Eredevisie Dutch Premier League but we also in charge of game

View of large crowd attending SportTech 2017 at Arsenal Football Club June 7

coverage for all the games, so it makes it quite easy to do all kinds of new stuff,” said van Meerkerk. “We really wanted to show the point of view of the guy who’s taking the free kick, or the goalie. When we experimented before, in 2014, the avatars looked rather fake, and not something we could show on many games.

“So we thought, when you play FIFA, you’re sending data from your games console to the graphics engine. With your console you basically steer the avatars. So then we figured, we have TRACAB data — what would happen if we connected this data to an EA-type graphics engine? Then we bumped into Beyond Sports, who were already working with data from TRACAB.

“We wondered could we create a live broadcast, to get the avatars running. We had our first try-out in December 2016, where we created a 3D stadium and a VR camera, with the avatars moving. Not moving very well, but they were moving,” said van Meerkerk.

“We then picked out a big match in the Netherlands, Feyenoord against PSV, and decided to put in the live virtual camera. We were very unlucky with this match, because first of all there was a lot of confetti on the pitch so the TRACAB system had problems tracking the ball. The other problem was that it was a very intense game, so there were no free-kicks around the penalty area. However we did get a corner kick where we could do everything live, with the avatars and all the data,” he said.

“We had the data, and the 3D virtual stadium, so could provide analysis. We asked our analyst, former Dutch national team player Ronald de Boer, to take our viewers into a virtual experience. He was able to explain what the player should do in this situation. The animation was not great, but still the reaction we got from our viewers was really good – so for us it is something to move forward with. As soon as people put on the goggles they really feel they’re in the game – even though they are looking at avatars.

“We also created an EA-type stream and we put that on the Twitch platform for gamers,” said van Meerkerk. “We provided a FIFA-like live stream of the game, and we asked a couple of YouTubers to come over and do the voiceover. It’s an automated camera that follows the ball and zooms in and zooms out – really acting like it is in a FIFA game.

“From next season on we will launch our Fox Sports Netherlands VR, broadcasting the Twitch feed live and also we will bring all the clips from our VR into our analysis so people back home can experience everything the analyst is experiencing.

“We really believe in this virtual camera. The animations you’ve seen here today look good, but we’ve already made a lot of improvements. It’s going to be better when we start with it next season,” he said.