GENEVA — The European Broadcasting Union is the world’s leading alliance of public service media. Forty-seven of the EBU radio organizations have used the union’s Euroradio Live Network to broadcast concerts and events. In 2017, the EBU radio organizations and Eurovision Services engineers agreed to replace the Euroradio Live Network equipment, most of which had reached an end-of-life state.
The challenge was to find a detachment system that would simplify the exchange of live radio concerts; deliver the quality and reliability of a satellite solution; provide extra services such as bi-lateral exchanges and applications beyond concerts; have a latency of not more than 10 seconds; and a lower total cost of ownership.
The Secure Reliable Transport protocol would ensure IT security and stable operation for the provision and distribution of content. Features in its favor include:
- Security through AES 128/256-bit encryption
- The recovery mechanism for lost IP packets is efficient and economical regarding the recovery rate, bandwidth, and latency
- At the receivers, a timestamps recovery synchronizes the audio and video streams
- Each endpoint is configurable as a caller, listener or in rendezvous mode
- The monitoring tracks the network performance parameters between endpoints, such as IP packet loss, latency and jitter
The key to connecting the 47 EBU radio organizations to the EBU Flex System was an audio over IP codec that could both integrate live concerts into the orchestrating EBU Flex system and receive them. The RoIP project team shortlisted the minimum requirements to the following:
- Two channels for reducing the number of devices
- AES3/XLR interfaces for receiving live concerts
- Audio codecs such as PCM and Opus
- SRT and RTP/RTPC for IP streaming
- Remote control facilities via HTTPS, SSH or SNMP
- PTPv2 for time synchronization
- TS over SRT
- The use of shared resources and distribution via the public internet or CDN
With the vision for the future ecosystem set, the search started for a suitable audio over IP codec. In December 2018, there were three candidate codecs. The ES SA engineers tested them with an eye on the following:
- PCM streaming over six hours
- Cutting the internet line (buffer higher than one minute)
- Switching an audio over IP codec off and on
- Encoding and playing a short one-minute clip 60 times in a row
- Setting the buffer to 10 seconds
- Mixing Raw PCM and Opus in/out
- Shutting it all down for 30 seconds and restarting the equipment
- Internet line with at least 500 Kbps (Opus)
- Testing the new systems under different internet line conditions.
Key Points
Three key points tipped the scales in favor of 2wcom’s IP-4c four-channel audio over IP codec. These were 2wcom’s outstanding responsiveness in supporting requests, the codec’s ability to deliver the best technical test values, its intuitive configuration and the web interface’s monitoring function; plus, it was the only hardware device that supported SRT, as required.
There was a bonus: As only two of the four channels would be for Euroradio live concerts (defined configuration), members could use the other two channels for other purposes, such as radio news.
The proof of concept in the field started mid-August 2020. The system setup, containing the EBU Flex platform and 2wcom audio over IP codecs, enabled point-to-multipoint connections via SRT. Essentially, at the contribution site the IP-4c encoder packages the uncompressed PCM audio into the SRT protocol and establishes redundant point-to-point connections via SRT to the EBU FLEX headends in Geneva and Leuk.
Within the EBU FLEX headends, the incoming audio streams via SRT are converted into IP multicast by the codecs and routed again as unicast via the outgoing SRT streams to the IP-4c codecs of the EBU radio organizations members.
The redundant setup of the orchestrating EBU Flex server farms in Geneva and Leuk ensures transmission stability. The satellite backup ensures delivery should the IP connection go down. At that time, abandoning the parallel mode was scheduled for the end of December. It was a busy time with many challenges.
In autumn 2020, a periodic bug showed up in the IP-4c every three weeks. The long intervals between the failures complicated the problem, but the EBU Contact Engineers and the 2wcom team fixed it before the actual launch in January 2021.
The new RoIP network showed its worth during the Christmas Music Day in December 2020, when 15 concerts were broadcast live in a row without a single request for a replacement sound file due to transmission quality issues or dropouts — the first time in four years.
For the 2021 New Years’ concert, the project team used the old satellite network as a backup. An older version of the SRT protocols’ synchronization method, the Timestamp-Based Packet Delivery, proved sensitive to discontinuous jumps in the system time, leading to incorrect clocking and failures. The latest SRT versions solve this problem.
Some IP connection issues on the Flex Farms demanded further redundancies in the final architecture, including two extra IP-4cs and streams from two separate Icecast servers.
After some months of usage, EBU’s radio over IP solution shows its benefits: It is more open, robust and reliable, and members particularly appreciate real-time monitoring during live concerts via the IP-4c’s web interface.
In short, the EBU and its radio organizations are excited about the new way of exchanging live concerts over the public internet. The new SRT protocol combined with performant hardware, sound design and a dedicated project team were the keys to delivering this successful broadcast solution.
The author is head of the contribution and playout center at ARD-Sternpunkt in Germany. He is also manager of TCSP for the RoIP project of the European Broadcasting Union.