How Formula E and Virtually Live are pioneering a live broadcast-game hybrid
Formula E is arguably the most progressive of sports in its bid to capture a new generation of fans through media tech but its latest initiative could top the lot. Ghost Racing is the world’s first live racing game to allow gamers to race live, in real time, with real drivers, on the real tracks of the ABB FIA Formula E Championship.
While pioneered by the all-electric all city-circuit motorsport series, the technology behind the game could be applied to “speed bikes, yachts or camels,” according to the CEO of developer Virtually Live.
“With Formula E we have accomplished a quantum leap in gaming full of innovation including the world’s first in game live race commentary laying the foundation for Virtually Live’s success in the esport industry,” Markus Tellenbach told SVGEurope.
Ghost Racing launched in April to coincide with the Formula E-Prix in Paris. It is free-to-play and available on iOS and Android mobile devices. It combines live racing against real Formula E race drivers with hyper-realistic scenery and graphics, engaging online live commentary, upgradeable car collection and continuously updated challenges.
“The backbone of live is broadcast and without it a sport’s sponsors and share of fans nosedives. Live broadcast is the bloodstream of big sports events but it’s obvious that’s been under threat for some time from advances in distribution that have created an irreversible change in consumer behaviour.”
Tellenbach speaks with some authority as a veteran of the international media scene. Before launching Virtually Live he had been running Polish media group TVN for seven years, having previously been at companies including SBS Broadcasting, Sky Deutschland and Kirch.
“To Generation X, broadcast is non-existent,” he stressed. “That is extremely alarming for any sport because if you can’t create a fan following for your product then you have a long term structural issue.
“What you can do, though, is stream content across devices and match their expectation for snackable content. Without streamed and snackable content the younger generation are not interested. Period.”
Tellenbach launched Zurich-headquartered Virtually Live in 2017 having identified gamification of sports as the type of content that could match the expectations of Gen-X, and gamers in particular.
“Of course, gamification itself is not new,” Tellenbach said. “There is a great library of video games for each major sport out there. Anyone can create a fantastic sports video game but there’s never been a solution for live sport. That’s where it gets difficult.”
He found a partner in Alejandro Agag, founder and CEO of Formula E, willing to get onboard. “Alejandro is a visionary who thinks in a fan-centric way,” Tellenbach said. “When you ask younger demographics what Formula One means they will reply ‘Formula What?’ but Formula E has probably the most fan-centric strategy in sport since it reaches out to younger fans.
“Alejandro understood the idea of gamification as one way of reaching Gen X. Our vision was to create an immersive racing experience far beyond what existing racing games offer.”
Virtually Live’s starting point was much the same as any other computer game developer in rendering ‘scenery’ (race tracks) and ‘assets’ (cars) as accurately as possible in graphics.
Markus Tellenbach, Virtually Live, CEO
The main sources for the scenery are CAD drawings of each city track, its buildings and installations. Other sources included Google Earth, stills photos and drone video taken by Virtually Live at each circuit and information from Formula-E of the exact placement and content of billboards.
“We start about 4-5 weeks prior to each race to produce the basic scenery and we will continue to adjust and finalise it up to a few days before the race until we have 99% accuracy,” explains Tellenbach.
The next stage -and the critical element that makes the game live – is to ingest the assets into the CG scenery using realtime positional data.
For this they turned to Italian tech company Magneti Marelli which devised a GPS-based sensor, approved by the series’ regulatory body the FIA, for fitting to each of the cars.
“This allows us to track each car with near zero latency – a few milliseconds – and much faster than a broadcast signal.”
User can pilot their own car in real time in a real event
The GPS-driven data provides time and position and, therefore, speed. This is processed using Virtually Live’s proprietary machine learning engine to track the position of all 22 assets (cars) racing at 100mph or more, often wheel to wheel, and to deliver this contextualised in the scenery as a live stream.
“The way we treat the raw data from the sensor is vital in creating a continued CGI stream and one in which we have a lot of patents,” said Tellenbach. Treatment includes various algorithmic filters, validations and predictions to define the exact position of each car.
The resultant virtual stream can be applied in a number of ways. Users can, for example, select any viewing angle, including of their favourite driver, a seat in any cockpit live during the race, or watching from the tribune.
“You make your own viewing experience and if you wish you can rebroadcast it live on Twitch or YouTube. You can be a professional video-blogger or a car manufacturer or anyone can upload it and create a dedicated fan channel.”
The live CGI stream is an actual replication of the live event and allows Virtually Live to ingest a 23rd car onto the grid. This is Ghost Racing.
“A user can pilot their own car in real time in a real event,” said Tellenbach.
Formula-E’s broadcast production partner Aurora Media are also able to utilise the CGI feed within its output. This includes using the live CGI feed as a virtual track guide, highlighting ideal braking lines, tactical considerations or when to use Attack Mode. This feature, introduced this season, temporarily boosts energy levels from 200kW to 225kW and allows drivers a chance to chase down their rivals or build a lead to defend position.
“[The virtual stream] is a perfect tool to look at certain perspectives such as cars travelling through a tunnel where a slightly different angle gives you a different perspective on the driver’s performance.”
Braking points and angle of steering wheel are also used to create fidelity with the live action. “It will be increasingly used as an analytics tool integrated into the broadcast to highlight, for example, which driver had a better line and how impacted their race. The positional data was only used previously for marshalling purposes and not for broadcast. This is new to Formula-E and will develop massively to allow broadcasters to explain in richer detail to an audience why an event has happened.”
It is increasingly possible that the worlds of esports and actual physical sport will merge. Brendon Leigh, a 19-year-old from Oxfordshire, has won the F1 Esports Series tournament for the past couple of years playing 25-minute races simulated on existing Grand Prix circuits. Esports players like him are being scouted by motorsport teams alongside the traditional routes of karting.
“He has been posting lap times ahead of official drive times so I wouldn’t be surprised in the very foreseeable future if an esport player will find their way into a cockpit for real,” Tellenbach said.
It’s still very early days but Tellenbach reveals that the number of downloads for Ghost Racing has sped past 100,000. In development are plans to extend the product to PC and more serious rather than casual gamers. The quality of graphics will be upped too, including adding humidity, improved lighting and weather conditions for next season.
Nor is Virtually Live’s product restricted to automotive sports but could be applied to ‘Ghost Sailing’ events like a Volvo Ocean Race “or anything where participation make sense” – such as camel racing.
Other sports require a little more imagination. Tracking the position and action of football players during a live game, for example, can be done now.
“We need to understand the contextual side of how a gamer is involved in live football,” he said. “We have some ideas.”
The co-inventors of Virtually Live are Jesús Hormigo and Jamil El-Imad. Hormigo is the firm’s co-founder and CTO who previously led the technology R&D teams at NeuroPro, a Swiss based medical company that develops innovative solutions for brain data capture and analysis. El-Imad is a former software engineer developing code on IBM systems. He’s an Honorary Senior Research Fellow at the Institute of Biomedical Engineering at Imperial College where his team is developing algorithms for epilepsy prediction and other neurological disorders. He is also launching a bio-sensory cloud service offering big data management, storage and processing solutions.