July 15, 2024
Adeia recently attended and exhibited at two of the premier conferences for research and development in the video industry: the ACM Mile High Video (MHV) 2024 Conference in Denver, and Streaming Media NYC 2024 in New York. At Streaming Media NYC, Adeia’s Chief Technology Officer, Serhad Doken, participated in a panel discussion on the need for energy efficiency in future video technologies.
Our team saw and heard about plenty of topics at both shows that are not only areas of interest for us, but also active areas of research throughout the video and streaming industries. Here’s a rundown of what we experienced this spring.
Our CTO, Serhad Doken, participated in the “Computing While Cooling” panel at Streaming Media NYC, alongside Tim Miglin, Executive Director at Help me Stream Research Foundation, Derek Powell, Director at Altman Solon, Dom Robinson, Founder of Greening of Streaming and Director at id3as / Norsk, and Sujana Sooreddy, Senior Software Engineer at Netflix.
The discussion centered on how to make streaming media’s full tech stack more energy efficient by implementing best practices throughout the supply chain. Doken offered some valuable insights, noting that sustainability must be integrated throughout the design process, not added as an afterthought. Everything from the choice of operating system to the cloud and network infrastructure to the application implementation and context awareness can make a difference in achieving industry-wide energy reduction.
The panel also discussed how the industry must adopt a scarcity mindset and build sustainability into product design in order to see the biggest effects, including a higher return on investment. But there are also low-hanging fruits that can provide immediate efficiencies and energy savings. And although there are plenty of misleading and inaccurate metrics out there, it is advisable for the industry to define standardized measures for reporting carbon footprint statistics when using streaming services to educate consumers.
We learned a few other things about the state of media tech and streaming media in particular. When it comes to monetization, one of our biggest takeaways was that the industry is in need of new ad formats. We also saw that there is high interest in interactivity and live shopping – and our clickable video technology can help bridge this gap.
There’s also a lot of potential for synergy and growth between the sports industry and streaming media. Sports fans are very passionate consumers because they are drawn to their teams. Sports streaming, in turn, creates emotional connections with audiences. Streaming players are drawn to sports as a product because there’s more opportunity for monetization compared to regular SVOD services.
Finally, one interesting takeaway we had is around the deployment of low latency streaming protocols (such as LL-HLS). These protocols have been available for some time, but the industry still has not deployed them.
You can’t have a conference about video technology without discussing codecs, and MHV 2024 was no exception. The key takeaway around codecs is: compression has been progressively improving.
There is also broad agreement among the video community that, despite the way coding has always worked (such as using block-based encoding on 2D pictures), new paradigms must come to the forefront. As the world moves toward the widespread use of immersive video technologies such as light fields and mesh and point clouds, the traditional block-based encoding methods that have worked for 2D video will no longer be sufficient or possibly even applicable.
Next-generation codecs will have to adapt to the type of content they are encoding, whether 2D video or some other type of 3D/volumetric/immersive video. Even different use cases such as traditional video streaming, live streaming, videoconferencing, cloud gaming, VR and others will all require versatile and adaptable approaches to encoding.
Networks are becoming more capable, but compression also needs to keep pace so that all types of networks — wired and wireless — can handle the delivery of these new types of data, which are expected to create an explosion in data volume.
Compression will also need to be “application aware,” meaning that the codecs will have to be adaptable to the type of content being compressed. For example, room-scale mobile and XR applications will be treated differently than those on more seated/fixed XR applications or even 3D video streaming of a live sporting event.
Another fascinating consideration for next-gen codecs is that, over time, an increasing amount of video traffic is expected to be viewed primarily by machines, not just humans. For instance, an AI-powered video surveillance system might be monitored by a machine that only alerts the human operator when it “sees” certain characteristics. Since computer vision senses video differently than the human eye, video intended for machine viewing can be encoded to optimize for machine-to-machine applications.
As streaming has become more widespread, especially for live sports and broadcasts, latency has become an area of focus. Typically, a live sports broadcast over cable TV carries a latency less than 10 seconds. But latency for streaming over-the-top (OTT) varies somewhat and may even be higher than cable.
With the advent of other technologies like real-time microbetting, improvements in latency become critical. Microbetting is wagering on the outcome of some highly specific aspect of a sports event. For example, in tennis you might bet on which player is going to win the next point. The betting period might only last a few seconds, and the odds change in real time based on a variety of metrics as well as on how many bets are placed.
In such scenarios even small delays in the broadcast can be costly. If I’m just watching the game and have no money on the outcome, I might be willing to tolerate quite a bit of latency. But if I’m placing bets on the outcome of small events, I need the stream to be as close to real time as possible, usually measured as one second or less.
This low level of latency is important for other video-based applications as well, such as cloud gaming and videoconferencing, which both need to feel natural to be usable. For VR applications, tolerances are even lower: users experience nausea if the latency is higher than about 20 milliseconds.
One exciting approach to reducing latency for a variety of streaming video-based applications is the use of transport protocols. While protocols like WebRTC continue to be of interest, QUIC has been gaining attention through the establishment in 2022 of an IETF working group on Media-over-QUIC (MOQ). QUIC can be used in browser (i.e., HTTP/3 web transport) and non-browser (i.e., raw QUIC) environments. The QUIC solution targets various applications including live streaming, cloud gaming, remote desktop, videoconferencing and eSports.
A promising study presented at MHV investigated the potential benefits of QUIC stream prioritization features, which can improve both latency and resource utilization in the transport layer. QUIC breaks a transmission into multiple streams and assigns priorities to each stream to optimize the delivery with much lower latency. An intelligent application-aware scheme that utilized this property allocated different video frame types (I, P and B) to streams with different priorities, resulting in improved metrics and a better user experience.
A next-generation streaming architecture, open caching, helps address some major bottlenecks inherent in traditional internet infrastructure that are exacerbated by high volumes of streaming media. Instead of serving streaming content from highly centralized locations, open caching places content caches deep within service provider networks.
This approach enables network operators to deliver content from locations closer to their customers, improving reliability and latency for users while enhancing operational efficiency for the service provider. In a lively panel discussion at MHV, participants agreed that enticing public content delivery networks (CDNs) to participate in a standards-setting discussion in this ecosystem with the internet service providers (ISPs) has been a challenge.
Speaking of CDNs, one fascinating topic at MHV was the concept of CDN leeching, an insidious new form of content piracy affecting OTT service providers. Using this method, pirates can procure a token that enables them to access content within the provider’s CDN and serve it to multiple unauthorized, illegitimate users.
Providers have been able to combat piracy by employing single-use tokens, which guarantee that a given piece of content cannot be replayed. However, the single-use token requires the infrastructure of security components to be scaled appropriately, so this approach has not become a widespread best practice.
Instead, a novel approach presented at the conference employed methods of rejecting tokens with a probabilistic frequency, meaning that anyone using an illegitimate token will encounter far more rejections than someone who is not reusing a token. This approach ensures, at a probability setting of 1/10000 for example, that a group of 5,000 pirates using the same token would face a near-certain rejection within one minute, while a legitimate user would face only around a 4 percent chance of rejection in a 15-minute window.
A growing amount of research is focused on improving the user experience. Video service providers today typically operate through multiple CDNs. Traditionally, choices were made about which CDN to use in each situation based on the service provider’s requirements. The recent focus on prioritizing user experience opens the door to “content steering”, i.e., dynamically switching video streaming sessions between CDNs.
CDNs themselves are independent, meaning that traditionally, there is no coordination between them. A new standard for content steering, however, can enable multi-CDN communication by allowing a CDN to manage its traffic load via communication with other competing CDNs over the steering server without exposing its internal performance data. As CDNs negotiate with each other based on their resources and load, this approach benefits the user: it helps coordinate a multi-CDN strategy to improve the experience.
Like content steering, this is also focused on the user experience. With active switching, the service provider selects the best CDN endpoint for each video segment. This can be dynamic, varying between the most cost-effective CDN endpoint, the fastest CDN endpoint or whichever endpoint is most capable of delivering the optimal user experience. By measuring real-time user conditions and using them as a basis for decision-making, this technology remains focused on optimizing the user experience.
While attending the MHV conference, our team had the opportunity to talk to a few of our colleagues in the industry, including those who are hard at work to foster diversity at Mile High Video and within the industry as a whole.
We heard from Tamar Shoham, Chief Technology Officer at Beamr: “Another trend that was obvious this year is the positive impact brought by MHV ongoing support of the Women In Streaming Media group. This was primarily evident in the diversity of the panelists on the excellent panels, as well as the introduction to the Women in Streaming Media Allyship program and the very vibrant Women in Streaming Media meetup.”
Victoria Tuzova, Business Operations at Elecard, shares insights from the speaker engagement team at Women in Streaming Media at MHV: “Our efforts at MHV 2024 aimed to elevate the presence of female speakers. Building on last year's focus on diversity and the need for greater integration, we successfully increased the participation of female speakers to 40% per panel this year. Moving forward, we are committed to maintaining this level of representation at future events to further enhance female involvement in the streaming industry.”
Our team also had a chance to exhibit and demo our graceful variation technology at both shows. Graceful variation is an innovation we’ve designed to deliver more consistent video quality over longer durations of time.
Abrupt changes in streaming quality, for example, jumping from 1080p to 420p, are often jarring and can significantly impact the user experience. They’re caused by sudden changes in connectivity or network quality. But by using dynamic playout through graceful variation, we can reduce abrupt jumps in quality and provide a smoother shift to mitigate perceivable changes for viewers.
.jpg)
Dhananjay (DJ) is responsible for roadmap definition, strategy and R&D activities in Adeia’s Media CTO office. Prior to Adeia, DJ was Senior Director for Emerging Technologies and Platforms at Charter Communication, where he built an R&D team focused on network-powered Gaming, AR/VR, holographic / light field communication and ML/AI applied to Quality-of-Experience delivery on the network. He has held positions across research, product engineering and product management across various organizations like Time Warner Cable, Eaton, Emerson and Bosch. He also served as Board Member and Network Architecture Workgroup Chair at the Immersive Digital Experiences Alliance (IDEA) and has 16 issued U.S. patents. DJ has a BE in Electronics and Communication Engineering from Indian Institute of Technology, a Ph.D. in Computer Science from University of Cincinnati and an MBA in general management from Carnegie Mellon University.
%20(1).png){kind=small}
%20(2).png){kind=small}