EMG Italy makes positive inroads in live AR test at Milan versus Fiorentina match
EMG Italy and Boost Graphics tested augmented reality (AR) live at the San Siro stadium in Milan for the Milan versus Fiorentina match late last year
In November 2023, EMG Italy and its specialist department Boost Graphics successfully tested live augmented reality (AR) graphics during coverage of AC Milan versus Fiorentina in Serie A.
According to Tommaso Gargaro, technical manager at EMG Italy, who led the event production at the San Siro stadium in Milan, the test was designed to ascertain whether or not AR could be added to, and improve, the output without interfering with the standard broadcast set-up used for match coverage.
He said: “We tested AR in collaboration with the American company Ncam (now part of the Zeiss group) and the Italian distributor Video Progetti, a company at the forefront of innovation in virtual production and augmented reality with a team of specialists made available ‘on the field’ during the tests.
“We conducted this test at the San Siro stadium in Milan with the intention of adding AR in a real on-field condition and verifying the potential of these technologies to enhance the television product without interfering with the normal shooting set up. We also invited Serie A director, Angelo Carosi, to assess the opinion of a great expert on this new frontier of entertainment, capable of seamlessly combining live actors and three-dimensional graphic content in motion with the scene. The goal was also to assess the extent to which these effects can enrich the graphic landscape of sports, starting with football, and verify its live technical feasibility.”
Ncam technology was used in combination with the Vizrt graphics engine to evaluate the accuracy of real-time tracking in the scene and the system’s stability in complex conditions such as a football stadium. Virtual elements (3D graphics) were created ad hoc and aimed to enrich the narrative of the sports event by adding data and statistics. The system was based on a Sony 4300 camera in one of the typical configurations used in football matches and many sports events, with a Canon 12X lens.
Ncam technology was used in combination with the Vizrt graphics engine by EMG to evaluate the accuracy of real time tracking in the scene and the system’s stability in complex conditions such as a football stadium
Precision and reliability
“The results of the test were promising from a technical point of view,” said Gargaro. “Ncam technology was able to provide precise and reliable real time tracking, even in the presence of strong light contrasts or moving objects and subjects, as in the case of the journalist’s walk along the field. In particular, the test showed that Ncam technology is capable of precisely tracking camera movements, even in the presence of strong light contrasts or moving objects and overlaying virtual graphics realistically and without distortion.
“These results indicate that, in our opinion, Ncam technology has the potential to be used in new augmented reality applications in sports and television. In particular, it could be used to create new viewing experiences, such as overlaying live statistics or virtual images but with a decidedly innovative look.”
The test
In some ways, football offers the best opportunity to test the system because the shooting condition is a bit more critical here, given the size of the fields, compared to other sports where the arenas and work areas are smaller and usually uniformly lit. The “discontinuous” lighting could indeed challenge the system, which fundamentally generates real time tracking of images with respect to 3D graphic elements.
Another test that had to be carried out and gave positive results is related to the actual lack of reference points typical of the football field. Here, the “continuous” green of the grass prevents the placement of markers on the floor, which is typical of the studio setting. In similar systems it is precisely the markers that – kept out of the frame – make it possible to precisely track and synchronise the graphics with real elements; on the field it is not possible to place any because they would be visible in the shot. This makes the preliminary calibration part much more complex, as the grass field is typically a continuous expanse of… green.
In this test, the camera was located to the left of the field, on the longer side, and a couple of typical usage situations were chosen: inside a small set on the field, as in the typical unilateral commentary position with a micro-studio and guests, perhaps for the integration of the local broadcaster; and also in the typical walk of the journalist along the field.
Final effect
Instead of the usual overlay graphics, AR makes the inserts on stage more spectacular, as if they were autonomous elements on the field, becoming another protagonist alongside the talents and players.
Cups, statistics, shields, lineups, coaching assistance in the pre-match, and a thousand other “concepts” come to life and can move in perfect synchrony with the talents commenting on the highlights and who, in turn, can be static in a commentary position or walk along the field edge. These shots are ideal in moments of game stops, during substitutions, or in a corner kick, and become a driving element from an editorial point of view, flexible and highly creative in the hands of a capable director.
Gargaro concluded: “The Ncam machine acts as a tracker and manages all the aspects related to motion by identifying where the camera is in the shots. Then it transforms the visual information into position metadata relative to the external environment and represents them with coordinates in a three-dimensional space.”
“To do this, it uses two micro-cameras alongside the main camera, therefore in a stereoscopic configuration installed on the shooting support, [which is] in this case the Steadicam. The system is able to interpret reference points in real space and recreate metadata to manage the real image tracking, performing the magic with only six frames of latency. The operator is able to see in the viewfinder the live return of the graphics added to his real images, to make his movement perfect, for example, in a walk ahead.
“The test is the first of its kind as the Ncam system has already been used on cameras with fixed support in the studio, therefore in the presence of other equipment already capable of generating references in space using markers. In that case, it did not need to track the camera position in real time, as it happened here where there is no need to place markers on the scene,” he concludes.
Meanwhile, Carosi, an experienced and talented television director who has helped make Serie A one of the most spectacular and engaging football championships in the world, was present in the field to follow EMG Italy’s test.
“Considering that every graphic, even dynamic, is still partly static,” said Carosi, “we welcome this system that seemed very interesting to me because live, and I think its potential will increase even further with future integration with artificial intelligence. It is very interesting both in the international generation of the host feed and – perhaps above all – in the unilateral feed that the broadcaster wants to stand out and propose even in multi-platform broadcasts. I believe that excellent pre-production work and graphic taste are really essential to make the most of this innovation, also in light of the investment, now very accessible, required for those who must and want to innovate.
“Today, content producers or broadcasters must always offer something elite to discourage the subscriber churn rate,” Carosi continued. “Moreover, regarding the multi-platform streaming broadcast, it is potentially possible to customise each feed and make it unique for different user groups. In practice, I believe that this AR system allows the customised development of content through personalized graphics, which, thanks to careful and close work with the editorial team, makes such content even more unique for each customer. As such, I see truly great potential certainly in football but also in other sports.”