“Time Machine” – my talk at Demuxed 2018

I’ve just returned from a wonderful experience at Demuxed 2018.

I have had the honor to participate as a speaker alongside professionals from Twitter, Netflix, Youtube, Twitch, Comcast, Intel, Mux, Bitmovin, Akamai … and in general, the experience as both attendee and speaker has been amazing.

The event was streamed live on Twitch but today have been released also the individual VoD recordings (sessions list), including mine.

My session is:

“Time Machine” – how to reconstruct perceptually, during playback, part of the detail lost in encoding.

In the last years, I’ve focused my efforts on “joint” optimization of various elements of the streaming pipeline. Evolving from an intra-domain to a inter-domains optimization approach, it is possible to squeeze out much more efficiency.

I’ve worked on joint optimizations of encoding and players, for example. Sometimes throwing in the mix also “augmentations” of protocols. If the player knows how the encoder is optimized it’s possible to develop improved heuristics and vice-versa with a synergic effect. I’ve already discussed a bit about that trend in this previous post.

In this scenario, I have discussed during Demuxed about another un-usual possibility of joint optimization:

Reconstruct perceptually part of the detail loss in encoding using in the Player a GPU-based reconstruction model that uses information extracted by the encoder or ML to estimate the best parameters.

It’s an old idea I’ve been insisting on for years as a way to ultra-optimize the streaming pipeline, with different tunings for high quality and high-efficiency cases (es: mobile).
I proposed a proof-of-concept based on Flash in a 2010 trilogy of posts and spoke about it also at Adobe Max 2010 in Los Angeles.

After the decline of Flash I’ve waited for WebGL to be more generally available in browsers and devices to make the idea evolve. Now WebGL is very powerfull and filtering with complex pixel shaders also high-resolution content is not a problem.

I’ll elaborate more on the logic in a future post. By now, take a look at the recorded speak and/or at PDF presentation: Presentation-Demuxed2018-FabioSonnati.

I’ve been very satisfied with the level and quality of feedbacks on the topic and in general Demuxed has been a wonderful occasion to meet and chat with high level professionals of the streaming business.