Bigger, more versatile and “less of a science project”: Inside Eurosport’s enhanced Cube at the US Open
Ahead of the US Open tennis finals this weekend, SVG Europe spoke to the team behind Eurosport’s updated Cube presentation studio to find out what is different and why, and to get the lowdown on how the new set was designed and built.
The Eurosport Cube is a live presentation studio that brings together a variety of technologies to create an interactive mixed-reality set.
First introduced for the 2018 Winter Olympics in Pyeongchang, the pan-European broadcaster is currently using it during coverage of the US Open tennis in Flushing Meadows and will do so again at Roland-Garros for the French Open (21 September to 11 October).
With virtual guests, players and pundits ‘beamed in’ from green screen studios at the stadium and around Europe, and making use of augmented reality (AR) and enhanced graphics, the Cube’s job is to help the presenter (in the case of the US Open, that’s Barbara Schett) and the Eurosport production team to provide viewers with detailed and insightful analysis of the action out on the court.
Although the first incarnation of the Cube won awards for its originality and ingenuity, the one being used for the tennis, which is currently located in a studio in London, is an enhanced version that was originally scheduled to be deployed at the Tokyo Summer Olympics. But, of course, COVID-19 pushed the games back 12 months.
Understandably, some of the new additions and enhancements have been held back for the rescheduled event. But there are still lots to get excited about: the new Cube is bigger, more versatile and more refined than its predecessor.
Watch the Cube in action
Cube version 2.0 features a virtual set extension, expanding the reach of the content outside the footprint of the physical LED video environment and placing the presenter and camera in a fully immersive 360-degree digital environment.
4m x 4m, and made up of over four million LED pixels, the improved version can also be constructed in a day, aided by an auto-calibration system that speeds up the lens and camera tracking setup.
It also includes tracking technology that allows the lighting to automatically follow the presenters in the space, which in turn enables presenters to interact with AR graphics or props.
This version of the Cube was designed by Toby Kalitowski, creative director at BK Design Projects, under the guidance of Eurosport head of graphics and innovation Alex Dinnin.
Dimension Studio, and its partner Nikon, were instrumental in the development of the new Cube which now makes use of Unreal Engine, another change from the original. Gravity Media Group supplied technical facilities.
“We removed a wall so that it’s now only two sides and a floor. This gave us better camera angles and it also allowed us to introduce a full AR world around it so that it completely enclosed the Cube but gave us a much bigger studio to work with.”
With the original version of the Cube having been widely praised, the team would have been forgiven for thinking, ‘well if it ain’t broke, don’t fix it’. But that is absolutely not how Dinnin saw it.
“The Cube was a major success in Pyeongchang,” he told SVG Europe during the first week of the US Open.
“It won awards, which we are all very proud of. But after the Winter Olympics, we did a deep dive to look at what was good and what was bad about it.
“The main complaint, certainly from the production point of view, was that it took a long time to set up. So we looked at the workflow. We tried to make it much more user-friendly so you could just walk into the studio and say: ‘OK, I want that screen there, I want to put this footage in here.” And this can now all be done by the mixer all in real-time.”
Size matters too, reveals Dinnin. “We removed a wall so that it’s now only two sides and a floor. This gave us better camera angles and it also allowed us to introduce a full AR world around it so that it completely enclosed the Cube but gave us a much bigger studio to work with.”
The enhanced Cube was once again built by White Light and makes use of Disguise hardware.
Andy Hook, technical solutions director at White Light, acknowledges that the original incarnation was a “bit of a science experiment.”
“It was the very first version of the tech with lots of different things pulled together to make it happen,” he says. “There were a lot of lessons learnt but we’ve held onto the biggest success which was that you can just bring an athlete into the Cube with zero rehearsal and let them connect with the content around them.
“In Pyeongchang, we got some really emotional and powerful content from people who were pretty scared of being on TV. Or maybe had never been on camera before. We wanted to continue to embrace that but make the technical and production workflow easier and make it less of a science experiment.”
That manifested itself in a brand new Cube.
“We went back to the drawing board and basically rebuilt the whole technical solution around it,” continues Hook. “So now, with just those two walls and opening the cube right up, you’ve still got that engagement with the content that’s around you and you’ve still got natural lighting and reflections and all those sorts of things, but you’ve got this virtual extension too. This extends the content outside of the LED walls so the camera can move around wherever it wants.”
“In a broadcast environment we were trying to up the game and make it as realistic as possible: taking full advantage of what the Unreal Engine provides but also including the capability to add additional functionality in.”
The team is now also able to use one unified system to put all of the content together. Irrespective of whether the image goes on the LED walls or onto the virtual set extension or within any AR elements, there is a single point of control.
And, crucially, it is now render engine agnostic which means that they can use the right render engine for the job. For this version that means gaming technology, which Dinnin says “gives you much much richer content.”
So, Dimension’s virtual world team was brought in to build the set in Unreal Engine.
“The Dimension team themselves come from a gaming background and they’re very used to building virtual worlds of a very high quality for immersive VR gaming and experiential worlds,” explains James Banfield, head of mixed reality solutions at Nikon.
“But in a broadcast environment we were trying to up the game again and make it as realistic as possible: taking full advantage of what the Unreal Engine provides in terms of the life-like nature of what you’re seeing, but also including the capability to add additional functionality in.
“For instance, the team at White Light have control not only of the lighting in the actual studio itself but also the lighting in the virtual set. It is that type of programming that we’ve integrated so that the production team if they want to tweak and adjust, they’ve got the flexibility to do so there and then.”
From a visual perspective, Eurosport designs were provided to Dimension and the virtual world team was then set to work on making them a (virtual) reality.
“The team had to visualise those beautiful designs and make them as realistic as possible, but ensure they worked in a real-time environment where you don’t have the luxury of the GPU slowing down. Making sure those assets were perfect for this type of environment, but look really rich on screen, was a key point.”
Banfield is excited about where this combination of real and virtual, combined with gaming technology, might eventually take the Cube.
“At the moment we’re trying to recreate an existing set design but the fact that is you’ve got the capability and the programmers that can create almost anything within this 3D set that is triggered by real physical buttons by the production team. This opens up an enormous world of creativity where previously that may be a little bit restricted by doing it the normal way.”
Some of the most effective differences between the old and the new Cube are also some of the smallest. The devil, as they say, is in the detail.
“Within the Cube,” continues Banfield, “Because you’ve got the LED screens in the background, you’ve got all of that rich content reflecting off the presenter and the presenter can see the world they are in. They don’t have to imagine the virtual world. They can physically see it on the screens. But on top of that, when there are reflections from jewellery or in certain points, we’ve had an AR table, you can actually see the reflection of the New York skyline from the background in the reflections of the table there and in the shadows. These small touches of realism make the viewer at home question less: is this actual or a virtual set?”
“Being able to bring the players from a flash interview straight off the court into a green screen in New York and beam them into the studio has been a revelation.”
Much like with movie visual effects, if the audience cannot tell what is real and what is a visual effect, then the artists have done their job. That certainly seems to be the case with the Cube. While all the talk on social media has been about ‘teleporting’ players into the studio, it means that everything else goes unnoticed.
“No one’s really paying any attention to the fact that this whole thing is completely virtual,” says Banfield proudly. “And that’s the wonder of what’s been delivered here. The team has got it to a point where the audience at home just isn’t questioning it at all. We’re super proud of how the worlds come together but it’s been a massive team effort in making that actually work well on air.”
While there is magic, there is pragmatism too. In response to the Eurosport production team’s desires to set-up in a speedier fashion, deployment has been improved, says Hook.
“The Cube can be packed down, deployed and then re-set-up very quickly and there’s a lot more automation in terms of the calibration workflows. It basically self calibrates: lenses, colour calibration. All those things are much more automated. So the whole thing is far more deployable.”
Once the US Open concludes on Sunday, the Cube will move on (figuratively speaking) to Roland-Garros.
“We’ve built a generic Eurosport set so we can re-brand to whatever sport or event we want,” says Dinnin. “When we finish the US Open, we’ll all sit down and look at what we do differently. Inevitably, however, it will be pretty much the same for the French Open.”
While the Cube is a hugely exciting development and incredibly useful in COVID-times, when presenters and reporters cannot come into close contact with players, and pundits cannot easily travel to sporting venues or studios, Dinnin does not see the Cube as being a replacement for physically being on-site, at a venue or stadium.
“What we don’t want to take away is being on-site and being at the heart of the action. I think that is crucial for sports broadcasting. Currently, we can’t do that because of COVID restrictions. However, we built the Cube so it can be quickly packaged up, folded away, put into a shipping crate and sent over to another event, fairly easily. The engineering department at Eurosport have been working on a flyaway pack so we’ve just got racks and racks we can ship off, turn it all back on, stick the LED wall and we’re ready to go.”
Eurosport’s Cube at the 2018 Winter Olympics provided a chance for athletes to describe the action in close detail
As for next year’s Summer Olympics, the team is keeping details of that close to their chests.
“There are certainly a lot more elements that we’re going to be bringing to the table that we’ve been working on for the past year or so,” says Dinnin, giving nothing specific away. “It’s awful that the Olympics was put on hold but it’s also given us another year to carry on working on it.”
“We’re always thinking and looking at least a year or two ahead,” adds Hook. “We were ready to go for Tokyo with some things that we’re very proud of – but the goalposts have moved. The Olympics is delayed and therefore we’ve pushed our goal posts even further forward because we’ve got more time. We’re always looking to innovate and keep ahead of the challenges and embrace new technologies out there.
“We’ve built something that’s so flexible that we can keep adding to it. We can keep improving content and keep improving the technical overlays.
Dinnin is hugely proud of the team and what has been achieved with the new version of the Cube.
“We knew we couldn’t be there [in person] so the ambition was to take the fans as close to the action as possible. We wanted to show off our Cube. We’ve created a seamless set with New York skylines in the background and the camera perspective is tracked. We can bring [pundits] into the studio so our presenter can talk to [them] and they’re standing alongside each other. And being able to bring the players from a flash interview straight off the court into a green screen in New York and beam them into the studio, has been a revelation.”
He reserves special praise too for Cube production manager Ann Keith who, he says, kept everyone in check and on budget.
“Without Anne, no one could do anything and we’d probably still be waiting to get out of Pyeongchang!”
The US Open finals from Flushing Meadows in New York will be screened on Eurosport across Europe in over 50 countries and territories, except the UK and Ireland, on Saturday 12 and Sunday 13 September 2020.