FutureSPORT 2019: World first case study as Sky Sports debuts volumetric capture
On the first day of The 148th Open at Royal Portrush, Sky Sports introduced Sky Scope 3D virtual reality in which presenter Nick Dougherty demonstrated the golf swings of Dustin Johnson and Henrik Stenson at The Open Zone, thanks to a brand new volumetric capture process.
Working closely in partnership with MRMC and Dimension, Sky Sports created the world’s first mobile volumetric capture studio that is also integrated with motion capture.
At SVG Europe’s FutureSPORT 2019 event at Chelsea FC’s Stamford Bridge football ground, attendees were taken on a journey through the introduction of volumetric capture, to where this technology is heading next.
“My phone [went] off like a Christmas tree. A lot of people within the broadcast industry wanted to know how the hell we did it.”
Duncan East, Sky Sports’ head of directors and creative, Jason Landau, design director at Sky Creative Agency within Sky Sports, and James Banfield, head of synergy at MRMC, took to the stage with SVG Europe’s editorial director, Fergal Ringrose, who had been at Royal Portrush this year to see the first live broadcast of this volumetric innovation, to discuss the project.
The demonstration itself at FutureSPORT was remarkable, with the avatar of Rory McIlroy seemingly live on stage and a steadicam able to walk around the avatar to see all the angles of his golf swing, pausing the avatar when needed to get that essential close up.
Sky Scope technology is created using The Polymotion Stage from MRMC, which consists of 106 video cameras that can capture video at either 2K or 4K resolution. The cameras are positioned with 96 around the capture volume facing inwards, and 10 above pointing down. With multi-directional microphones, it can capture spatial audio or reference tracks to be sent to sound design.
Landau recounted how the idea for this innovation got off the ground: “It all starts with relationships and conversations. We met up with James at NAB 2018 on the MRMC stand. The thought process led itself to volumetric capture and that’s where he got a bit excited, and he said ‘you’ve got to let us come in and present to you some ideas we’ve got, and also show some of the exciting work a company called Dimension have been doing’.”
Landau said MRMC came into Sky Sports and the meeting, while good, did not result in a definitive idea. Landau’s team then went to visit Dimension to see its technology in action and brainstorm, and it was in the car on the way home that the team realised that golf was going to be the perfect fit for this technology.
He explained: “The swing is rhythmical, it’s a singular sport, and we knew that at The Open this year we’d be in the Open Zone and we’d have the Ncam tracking system. We thought if we could bring the volumetric capture together with the tracking system, we would have a world first in swing analysis.”
Sky had already demonstrated AR previously in a “2D world,” said East, who added that this was the next step, taking that AR idea into a 3D world. “Everyone wants to be first, everyone wants to be best, everyone wants to take that next step in innovation so we’re always looking at what we can do and where, and The Open, what a platform that is.”
With the focus pinned to the golfers’ swing, volumetric capture was integrated with motion capture, with players golf clubs scanned in – because the players swung the clubs so quickly they were just a blur – and digitally added to the volumetric scan itself in a world first use of the two technologies within the workflow. The fast timeline was an added pressure, along with torrential rain on site, yet the debut was a success.
Since the debut of this technology at The Open the impact has been remarkable, Landau said: “At the event itself, there was certainly a buzz around the TV compound as word was getting out of what we were trying to achieve. Then I remember Thursday morning as it goes live with Henrich [Stenson], and very soon after Duncan comes into the truck and says a few kind words and then everyone in the truck starts applauding – not at Duncan! – but I thought that kind of reaction doesn’t generally happen that often, so we knew that we’d done something special.
“My phone [went] off like a Christmas tree and it was a lot of people within the broadcast industry wanting to know how the hell we did it. The social media reaction was really positive and people were talking about it around the world. At IBC the first question from people was how the hell did you do it, and now we’re here [at FutureSPORT],” he went on.
Banfield commented on where the technology was now going, after its successful debut: “From our perspective [The Open] really gave us confidence to move forward with and complete phase one of this project, which is to create three stages: there’s one in Wimbledon, Dimension themselves, the original [stage]; [then there’s] the Dome, which was used at Portrush; and we’ve just created a new truck which is just outside Heathrow at the moment going through its final testing and that’s a triple expander, so we actually have the ability to turn up with three hours [till broadcast] and be ready to scan.
“These three stages give us the capacity to get the price point to where the industry needs it to be to adopt the technology,” added Banfield. “I’ll probably regret saying it but our target’s a grand a scan, and that’s effectively what we’re able to achieve now with the volume that we’re at, and the stages give us the flexibility to be able to do that where the talent’s at, which is the super important part.”
“Phase two for us is introducing some of the other technologies that we’ve been involved in investing in,” he noted, mentioning areas including robotics and limb data to be able to animate puppets, or very lifelike avatars. Phase three is to take that latter technology and add artificial intelligence, Banfield concluded, showing the sports broadcasting industry a new virtual future.
Click here to see how Sky Sports and MRMC created Sky Scope.
FutureSPORT 2019 took place on 26 November at Chelsea FC.