SportTech 2017: NEP The Netherlands makes case for augmented reality
NEP is known mainly as an OB truck company, “but in the Netherlands it’s quite different. We provide services from glass to glass,” explained Ralf van Vegten, Chief Information Officer, NEP The Netherlands at the Emirates Stadium on June 7.
NEP has been using photorealistic augmented reality in daily sports programmes, large entertainment shows and location sports productions, including carpeting and virtual advertising, but now it is taking AR to the next level. As part of its studio offering, it has developed (together with partners) an improved augmented reality system. While its components are not new, he thinks NEP is “using it in a very different way.”
Ralf van Vegten, Chief Information Officer, NEP Netherlands, addresses SportTech 2017
He believes that it “can help you to create better formats, better productions, more efficient predictions” and that “it’s possible to create a new operational financial standpoint for the production.” As all the AR material resides on a USB drive it makes it easy for a production to keep control of the format, “so you own everything that is used in the production and it’s impossible to change it so if you want to bring it abroad it’s just there as you created it.”
There are many studios in the Netherlands with just one set inside them and NEP employs augmented reality for a lot of its customers to maximise studio use. It also means there are no boundaries to a studio anymore and it is easy to change the set, to match the time of day, for example, or to add screens of any size, and even to build logic into the set (useful in a game show format or sports shows where set elements change depending on results).
NEP had used augmented reality for large entertainment or sports productions, but making it applicable to a much wider range of programming required close examination of the value chain, “and we had to fix few things to be able to,” he told the audience at Arsenal FC.
“The most important thing that we had to fix was the tracking.” Directors wanted it to be available on all cameras, whether pedestal or handheld. “Also, the rendering quality was really poor. We used a lot of rendering engines that are really familiar in the broadcast industry and although it looks cool it’s not real.” They couldn’t offer both live rendering and realism, and there were problems with keying. “It’s very complicated, can be tricky and can ruin a complete augmented reality set. The set can be really cool and just by keying the person inside the set it doesn’t work anymore.”
In looking for partners to develop a workflow and system that worked on every camera it found Mo-Sys, which uses a miniature camera attached to the broadcast camera, allowing the system to triangulate using lots of tiny reflective stickers on the studio ceiling. Being able to use a handheld camera in the virtual set makes it look much more realistic and alive.
For rendering NEP partnered with a Turkish company, using the Unreal engine based on a gaming system that is widely available for free, which has been transformed in to a broadcast system. “They have incorporated photorealistic rendering with all the stuff that was not working in regular sets,” allowing the use of real shadows that are perfectly keyed with no loss of detail, and reflections, without which many surfaces would look fake. They keyer delivers a lot more detail, such as people’s hair, or if a presenter picks up a bottle of water, the water, the plastic and the virtual set behind it still look completely real.
A drinks reception was held at Arsenal FC following the conclusion of SportTech 2017 proceedings June 7
For one sports customer, it was used in one of NEP’s smallest studios, with a low ceiling, but “with set extensions we created the environment that they wanted like a large studio,” ranging from a Champions League set to using extensions from any stadium in the Premier League with virtual screens inside the set. “But their limitation was that they were thinking in the more traditional way. They were thinking in the standard broadcast ‘what do we want to do with this television set way’ and that’s not what you want if you want to do something different with new technology where you don’t know what’s possible with the technology,”
In March NEP worked with Endemol Shine on a large entertainment pilot (a version of Deal or No Deal that has not aired yet), where they introduced a second contestant who had been playing the same game on a hidden set, and wanted to have that second small studio appear as if descending in a glass lift within a vast virtual set before emerging into the studio.
Since the pilot was shot the software has improved significantly. “The glass is looking more real and this is going week by week, we get more updates and it’s looking more real.” NEP used a version of the set to demonstrate one of its studios in New York recently, and built that set to fit the studio in two hours, with another two hours to set up the cameras. “It’s really easy to transport the set if you have a sports production that is covering more locations.”
He also showed some clips of a demo done in a really small studio, using an architectural model built using the same software. After downloading the model, it took just two minutes to have it running (using some real furniture for the presenter to interact with), with van Vegten doing the handheld camera work.