No more heavy metal as Sony spins up Virtual Live Production as a service

The move to the cloud goes hand in hand with reducing reliance on the ‘heavy metal’ of at-the-venue hardware. More and more traditional machinery is being turned into software apps to run on generic computers for access over the web.

For example, complex, powerful and often expensive hardware has been at the core of live multi-camera productions. Sony has gone about as far as anyone in placing this hardware-intensive process in the cloud and replacing it at the coal face with a browser-based user interface

Its new Virtual Production system relies on processing in Amazon’s AWS and on there being enough consistent bandwidth to get the streams from remote site to an operations hub, but it points the way forward. Since the only physical kit required are the cameras, it alters the business model for outside broadcasts too since a producer need pay on demand rather than invest in kit.

Sony itself calls its Virtual Production a service.

“To date, high quality and reliable live production has always required serious infrastructure, major upfront investment and an army of talented creatives and technicians,” explains Nicolas Moreau, Live Media Solution Marketing Manager at Sony Professional Solutions Europe.

“Our aim was to provide a solution that would create a frictionless production workflow with access to a cloud based professional vision mixer, and then deliver it quickly to social, websites, apps or CDN platforms.”

He explains that Sony aims to bridge the market between full scale professional OB and what it describes as ‘consumer live streaming applications’.

“The idea is to help digital media groups that need to produce more live content to drive audiences to their website. Most of them couldn’t have this quality of streams easily available or as affordable without a third party production company.”

Unlike other live streaming solutions there’s no need to set-up a vision mixer, encoder and streaming box on site. “This is its beauty. You can remotely control and publish live streams to the internet and all you have to send are camera crew with cameras.”

How it works

You need up to six cameras. Sony cameras such as the PXW-Z90 or PXW-Z280 which have built-in WiFi or any other camera, Sony or not, with wireless capability either through 3G, 4G cell, WiFi connection or those fitted with a wireless adapter such as a Sony CBKWA100 box. Output can also be made via Ethernet.

The advantage of using a Sony camera is that connection to the cloud will work with Sony’s QoS, a protocol which will adapt the video quality to the level of available bandwidth.

Sony’s QoS technology was developed from the same technology as used in its video conference systems, and features two major improvements in order to cope with video and audio over mobile networks. Firstly, a new algorithm was developed. This Adaptive Rate Control algorithm follows any sharp fluctuation in network bandwidth. Secondly, Forward Error Correction (FEC) is performed with large FEC blocks, enabling correction of a large number of lost packets in an FEC block.

“QoS is our USP,” says Moreau.

However, if you don’t use camera equipped with QoS then Virtual Production will work with regular RTP.

At top quality, each of the six cameras supported by Sony Virtual Production would require bandwidth running at 6Mbps to deliver 720p. The service will still run at 1Mbps with QoS managing the drop in bandwidth.

Sony says it is working on a 1080p version which will work at 10Mbps.

The encoding of each camera stream is done locally in the camera in H.264 and pushed to the cloud for switching. The switcher’s functionality will be basic for those familiar with the bells and whistles of top of the range consoles but is still pretty impressive for what is version 1.0.

It’s linked to Sony Ci, the cloud-based collaborative media file exchange which emerged out of Sony Pictures, for upload and insertion into the live stream of pre and post rolls or still images. It enables graphics and logo insertion, integration with YouTube Live and Facebook Live and with Twitter whereby it’s possible to drag and drop tweets into the live stream.

“There’s the possibility to record every stream that comes in from every camera for a post file workflow and republishing as non-live content to a website,” says Moreau.

Replays and slo-mos are not enabled but Sony has received requests for this from companies which have begun evaluating the solution.

Road test with Red Bull

Sony began R&D on the project in 2016 and made a demo of it during last year’s Locarno Festival. It’s been refining the tech and the GUI ever since and is ready to take it to market after IBC2018.

Last weekend, Red Bull Switzerland used the solution to live stream the Alpenbrevet motorcycle race in Sarnen to its social media

Indeed, Sony has taken care to announce this product having road tested it with a high profile partner. Last weekend, Red Bull Switzerland used the solution to live stream the Alpenbrevet motorcycle race in Sarnen to its social media. The remote, if scenic, location of the race nestled deep in the Swiss mountains meant that deploying physical production infrastructure would have posed a significant challenge.

However, Red Bull was able to capture and broadcast the whole event using only a number of camcorders and a single laptop to access the Virtual Switcher GUI.

“Over the years, our Alpenbrevet event has become an important date in the calendars of casual motorcycle racers across Switzerland and Europe,” explained Hubert Zaech, Production Manager at Red Bull Switzerland. “Virtual Production gives us the freedom, flexibility and agility to capture and distribute content quickly, without having to deal with the logistical nightmare of installing physical infrastructure in these remote locations.”

Before Red Bull Switzerland, Portuguese TV network SIC trialled the system for coverage of the World News Media congress in Lisbon in June.

What else is out there?

The switcher market has been shaken up in recent times with innovations from encoding specialists like NewTek and its TriCaster which is particularly adept at pulling keys in virtual sets and Telestream. The latter’s Wirecast Gear product comes with four HD-SDI or HDMI camera inputs and the ability to switch between them while mixing in other media such as remote live guests, screen captures, movies, images, and audio tracks. FOX Sports used it to stream the a live daily during the FIFA World Cup directly to Twitter.

EVS acquired Belgium’s Scalable Video System in 2015 and with it the IT-based vision mixers Dyvi Live which is redeveloped and sold as DYVI. The programmable nature of DYVI is arguably its chief asset giving productions tremendous flexibility in designing macros for example. It also puts the processing at the edges of the IT network, while the operation appears local, to which more GPUs can be thrown depending on the demands of the production.

The Windows-based software only vMix offers multicam 4K switching, support for NewTek’s NDI, integrated PTZ camera control and direct web browser inputs which have been specifically designed for streaming esports.

The service cost

None of these processes the core production in the cloud and none is able to offer the software as a service model which Sony is setting up.

For a monthly fee you can have a limited number of hours for live streaming. Euros 300 entitles you to four hours per month; Euros 700 buys ten hours per month. An enterprise plan racks up 100 hours or more per month and is determined on request. Each package offers the ability to produce video streams with: Up to six live camera streams; 2 x playout servers; 2 x logo keyers, 2 x caption keyers and stream recording.

Sony isn’t supplying or specifying the cameras – it is selling access to the Virtual Production portal. The solution is locked into AWS for now.

“Production teams log into their personalised portal, synchronise their cameras and stream content on a pay per use basis,” says Moreau.

As you’d expect with core production over the open internet latency is an issue. There’s a 2 second or less delay in sending feeds from camera to cloud switcher and another 10 seconds for onward YouTube publication (which is standard).

“Some live content producers would be happy to trade slightly less workflow performance for the opportunity of going live and the flexibility of having this in their portfolio,” says Moreau. “This is for sports for sure but definitely also as a ‘go to’ solution for those not able to afford to go live in the past.”

He cites digital brands, the corporate and education markets and Tier 2 sports.

Nor does Sony believe this is spells the end of the traditional OB – at least for the Tier 1 events.

For the record, Sony has no intention of abandoning its own hardware line of switchers. “Absolutely we still continue development. The Virtual Production solution is an alternative direction that Sony is offering and there is no doubt we will learn a lot that we can apply in future to traditional hardware-based product.”