Live from The Open: World first as Sky Sports debuts volumetric capture innovation
On the first day of The 148th Open at Royal Portrush, Sky Sports introduced what it calls Sky Scope, in which presenter Nick Dougherty demonstrated the golf swings of Dustin Johnson and Henrik Stenson at The Open Zone in 3D virtual reality, thanks to a brand new volumetric capture process.
Working closely in partnership with Mark Roberts Motion Control (MRMC) and ‘virtual human’ experts Dimension, Sky Sports has created the world’s first mobile volumetric capture studio that is also integrated with motion capture. See a video of the end result.
MRMC has also announced this innovation as Polymotion Stage, a mobile solution for high-end volumetric 3D video capture and avatar creation. Polymotion Stage is currently available as a Stage Dome (and soon a Stage Truck), and can be transported globally to wherever the talent is located, such as sports arenas or film sets.
Inside these soundproofed capture stages, 120 video cameras capture everything in the performance space at up to 4k resolution, making assets flexible for use across immersive and traditional media formats.
The Sky Sports team captured eleven top golfers in the build-up to The Open across Monday, Tuesday and Wednesday. The players whose swings are being played out and analysed across the weekend include Rory McIlroy, Francesco Molinari, John Rahm, Adam Scott, Matt Wallace and Gary Woodland.
Sky Sports head of golf Jason Wesseley put this innovation into perspective for SVG Europe in the TV compound at Royal Portrush. “From a golfing perspective it enables you to see a top level player in any environment you choose to put him in and see how the mechanics of his swing actually work. It’s huge.
Jason Wessely, head of golf, Sky Sports
“These players have limited time, and they don’t always want to talk. They can come in and get their swing mapped, and then it takes five to eight minutes and they just have to swing a golf club. Then you have that asset to do what you want with it, because that’s really key. This is just touching the surface of what’s possible.
“The 3D nature of it enables you to see the golf swing from positions that you’ve never seen before. You’ve never seen a golf swing from in front, because the ball is coming at you and it’s a dangerous place to be. Now you’re seeing it and that’s quite a wow factor.
“[Presenter] Nick Dougherty is very talented and can tell the story of the swing on multiple levels. That Dustin Johnson example was quite complicated and quite in-depth. What you saw today was quite a high level analysis,” said Wessely.
“The good thing about the Sky Scope is that not only is Nick able to describe the visuals, he can illuminate things that have never really been told properly before. In other forms of slow motion, you can’t freeze it to the level you can here and move around it at the same time.
“Nick probably had an hour’s worth of prep for each sequence. He gets the raw video asset, and he can pick out the points he wants to talk about and advises us on where to push in with the camera or what angle to shoot if from at any one time.
“I think it will probably take [golf] coaching to a new level. And I think we could take it to the first tee and put it there with an Ncam. It just so happens that The Open Zone is our environment of choice,” said Wessely.
“Hopefully the technology will also improve. Because they have already mapped the club head, that is really sharp visually. As of now, things like the hands on the club are a little less precise – but that will get better.
“The exciting thing is being able to put these players in whatever environment you want – whether out on the golf course or back at Sky Studios or even a VR application on your phone. It’s pretty exciting.
“Engagement with the players, talking to them and getting access, is critical in the modern age,” he said. “At the moment the key thing that separates broadcasters is our relationship with top level athletes.
“Golfers are mainly nice guys and they understand that golf is a sport that needs help to raise awareness, participation and image. Most of them are on board with that, and they’ve always given us pretty good access.
“We’ve got a group of reporters, commentators and talent who work very hard at maintaining those relationships. We’re trying to make golf more accessible, more attainable and more real,” said Wessely.
Getting closer to the golf action: the next step
Duncan East, head of directors & creative, Sky Sports, takes up the story of how this latest Sky Sports Golf innovation came into being. “The Open Zone that we have here has been a huge step forward as far as an innovative way of getting the real golfers here and showing you their techniques. But ultimately, when the tournament starts you get peak viewership because they’re all out on the course and you don’t get access to them.
“You can pre-record some lovely pieces on the Tuesday and Wednesday of course, but they might not be relevant come Friday or Saturday. As brilliant as The Zone is and has been, we’re always thinking what’s next — what’s next?
“We’ve done AR in The Zone before with our player walkthroughs at Opens and The Ryder Cup. Last year we took the AR out of the glass studio on-site down to The Zone, a great step forward to take it outside and place AR in an environment where is pushes you to really get it right.
“This is the next step,” said East. “What we’ve now done is partnered with Nikon and MRMC. They came
By the beach at Dunluce Links, Royal Portrush Golf Club: (L/R) Nicky Chauhan, senior creative, Sky Creative Agency – Sky Sports;
Duncan East, head of directors & creative, Sky Sports; and Jason Landau, design director, Sky Creative Agency – Sky Sports
to us with some products and ideas around AI and different strands of broadcasting. But one thing they showed us was a 3D capture of an individual. They teamed up with another company called Dimension Studios, and we were familiar with them from an Anthony Joshua 3D capture they did for Sky VR a couple of years ago which was fantastic.
“Technology has moved on: what this allowed us to do is capture a golfer’s swing in a relatively short time, and in 48 hours turn around that capture and put it into an AR broadcast environment. They pitched the idea to us around the capture of individuals and athletes originally – and straight away we thought ‘this would work best on golf’.
“Because with golf you can isolate the individual, you can capture that individual and study their swing in detail. There are some other areas where we think it can work really well, but with the platform we have for golf, with this opportunity to showcase it in The Zone, the lightbulb went on. Golf swing: this is where it’s going to come alive.
“[Design director] Jason Landau went away to get more detail on how it could be done and achieved, and came back with a slicker presentation in a Sky Sports form – which in fact, ultimately, is very close to what you’re seeing on-air today,” said East.
“We then very quickly included Jason Wesseley, our head of golf, telling him we have a fantastic piece of technology that will take The Open Zone onto the next level. In order to make it happen there was an incredible amount of collaboration between the R&A, Sky, Nikon (who are of course a sponsor of The Open as well), the golfers themselves, their sponsors and individual club manufacturers – there were so many people we needed to bring together to make this happen.
“Everyone worked really hard and it all culminated in building a dome just off the driving range here, which is the capture centre. Golfers came in on Monday, Tuesday and Wednesday and obviously we had all their clubs ready. They walk into the dome, do a swing with an iron, a swing with the driver and then they walk out. You need to do these things quickly.
“From that, Dimension took the capture and in a 48-hour turnaround time delivered a file to our graphics department which we play out in the same way we do with all of our AR, using Viz and Ncam tracking, which then delivers this 3D scan of a player – the first one being Henrik Stenson and the second Dustin Johnson,
“We track a Steadicam around an AR object (which isn’t there, obviously) in The Zone, and Nick Dougherty talks the viewer through the swing. Nick uses a monitor with output only: very difficult for the talent, because he has got to address the viewer, the camera, a 3D object which doesn’t exist, all while checking a monitor out of shot. No script.
“You can get a camera in there and freeze it, analyse it. When have you ever been able to stand in front of a golfer when he’s swinging the club? You can pause at any moment you want and analyse any intricacy that makes that golfer unique.
“Every millimeter makes a difference – and now you can talk the viewer through all of that, doing it in a 3D world. It’s a world first. I think it’s one of the most innovative steps forward we’ve ever had in sports broadcast analysis. It’s that good.
“It’s a world first. I think it’s one of the most innovative steps forward we’ve ever had in sports broadcast analysis. It’s that good” – Duncan East
“I’ve been thinking all day about what rivals this – like when we first started using touchscreens or super slow mo. This is one of those milestones for sports analysis.
“This is not a simple process. There’s a 48-hour turnaround time, and that is working at pace. In an ideal world you’d take longer, so that every detail would be perfect.
“And to be honest, there were elements of today that weren’t perfect. But we’re in a space where I don’t think it needs to be perfect, to deliver what we have delivered.
“There has been an incredible amount of people and an incredible amount of technology involved. It’s a huge learning curve for everyone. It’s a major milestone. It’s the first 3D analysis tool we’ve ever seen,” said East.
Jason Landau, design director, Sky Creative Agency – Sky Sports added, “There’s a big white dome on the practice range that looks like half a golf ball. Some of the players were really wowed when they went in. It’s basically a 360 greenscreen, with lighting and 120 cameras.
“We capture the player from all angles, and we’ve worked with Dimension in making sure we’ve got the best quality. Dimension are an entertainment capture company, so this is a new field for them – especially working with live broadcast. It’s an education: you learn as you go.
“We were put in touch with all the different golf club manufacturers, and they supplied clubs – a driver and a five iron — to us before we came out here. The guys swing the club so fast that you can’t pick up the swing. Before we turned up here, Dimension scanned a 3D model of each golfer and added all the motion capture points onto the golf club,” said Landau.
“We were then ready, so that when Molinari for example walked in we had his driver and five iron ready. We then explained why all these motion capture points were on the club. The players have been brilliant.
“I can honestly say, that when it went on air for the first time today, my phone lit up like a Christmas tree with people wanting to know how we did it!” – Jason Landau
“We then still had to get the data back to England. They could guarantee that they could get the file back to us 48 hours from the moment the data hit the studios in England.
“It was the first time ever that volumetric capture has integrated with motion capture. I believe this is a world first in sports analysis. There are 120 cameras: the bulk of the cameras are 30 frames per second, and the infrared motion capture cameras are 360.
“The capture points are only on the club – because of the speed of the swing. It means we then actually have club data, including the swing plane, that may have use in the future.
“I can honestly say, that when it went on air for the first time today, my phone lit up like a Christmas tree with people wanting to know how we did it! I believe we have hit the perfect balance between wow factor and editorial,” said Landau.
“There is a real reason why we’re doing this, but at the same time you’ve got to have the viewer going ‘that’s amazing’. I think we’ve absolutely got that balance right.”