Category: FMS

FCS and RTMP – Streaming Technologies from the future

I remember clearly the enthusiasm and excitement when I moved my first steps in streaming and real-time communication. I was already fond of video compression and interactive web and was working actively with Flash community to create interactive experiences, but my career took a turn when Macromedia released Flash Communication Server 1.0. It was September 2002 and after 20 years the RTMP protocol, one of its foundational technology, is still among us!.

Pritham Shetty (helped by Jonathan Gay – the father of Flash) was the ingenious main author of this milestone in the history of video streaming. Pritham had already an extensive expertise in real-time communication for the web and for example, in 1996 he developed for NTT a Java based web client for connecting multiple users in a synchronized experience. And again, in 1996 a personalization server he developed was used even by Netflix! (when it was still very distant from the company we use to know today).

FCS was an exceptional server capable to enable real-time communication, live and on-demand video streaming features in Flash Player 6.0. The architecture of FCS was really ahead of its time and when I started working with it I had only a 640Kbps down/128Kbps up ADSL connection and a 64Kbps GPRS phone and nonetheless it was possibile to communicate in real time with other users over such connections and create futuristic interactive applications.

As we all know, 18 years after this exceptional release, the entire world has depended on real-time communication technologies like Microsoft Teams, Google Meet or Zoom because of Covid-19 pandemic. Think of FMS as a playground where Flash developers could easily develop video conference applications similar to those, with multiple audio-video-data streams produced in the browser by Flash, transported via RTMP protocol, orchestrated server-side by FMS and consumed again on a Flash Client.

I think the main advantage of this stack was simplicity and elegance and I’ve always used the lesson learned with FCS in my career as an Architect of Media Solutions. At the FCS’s foundation there was a non-blocking I/O stack scriptable in Action Script 1.0 (essentially JavaScript). Every user connection, application startup, disconnection and in general interaction raised an event and the code responded with actions or async I/O operations to connect RTMP streams in publish mode to RTMP stream in subscription mode and orchestrate via script many other interactions and data sharing (Curiously the architecture is very similar to Node.js. When Flash was abandoned I started easily to work with an early Node and FFmpeg to substitute many of the use cases I used to serve with FCS).

The simplicity and high efficiency of RTMP is also the main reason why it is still used today. RTMP allows to stream interleaved audio, video and data tag on TCP, SSL or tunneled in HTTP(S) and it’s possible trasparently to pass from real-time (few ms), to live and to vod use cases, with RPC call interleaved in the stream and easily recordable for interactive reproduction of communication’s sessions.



When working with this stack you had literally infinite possibilities a decade before webRTC was conceived and I ended to be an expert in FCS (then known also as Flash Media Server / Adobe Media Server) developing many advanced applications in the next 10 years (for example, thanks to the flexibility of the duo Flash + FCS I was able to design one of the first implementation of adaptive bitrate streaming for the first catch-up TV in Italy in 2008). 

Unfortunately FCS/FMS/AMS has not had the success and the widespread diffusion it deserved because of an absurd and limiting pricing model by Adobe. Nonetheless it has left an undeniable contribution to the Internet streaming.

Happy 20th Birthday RTMP and kudos to the great FCS and its authors.

Advertisement

Adobe Media Server 5 announced

Adobe has announced that the new Adobe Media Server 5 (formerly Flash Media Server) will be available soon. The change in the name reflects the recent shift in strategy of Adobe [sarcastic mode on] that is running away from Flash as fast as possible to embrace more cool technolgies like, in this case, HLS [sarcastic mode off].

Irony apart, I’m very happy with this announcement because I have always stated that FMS could have interesting new possibilities supporting streaming for iOS with content protection. FMS supports HLS streaming since release 4.5, but this is not sufficient to keep the leadership.

I work for large media clients that need content protection and in the last 2 years I have seen such clients choose MS’s SmoothStreaming and PlayReady too much times because some independent vendors have been able to offer them native clients for iOS supporting PlayReady DRM inside HLS protocol.
So it’s impossible to remain on the technology and adoption edge if not supporting all the business needs for the most preminent mobile platform.

Now with AMS 5 we are able to use our preferite streaming server to deliver protected content to iOS device too, both in applications created with AIR as well as native ObjC apps.

We have two protection techniques: full blown DRM protection using Adobe Access 4 (again “Flash” is flashed away) or using PHLS (Protected HLS) the HLS version of PHDS (Protected Http Dynamic Streaming).

This last feature is indeed very interesting because offers a stronger protection than the very simple encryption possible with standard HLS without requiring the costs and worries of DRM servers.

More info in the Kevin Towes’ Blog

My presentation at MAX2011 is available on Adobe TV

Finally the recording of my presentation at MAX2011 (Encoding for performance on multiple devices) is available on Adobe TV.

You can also download the pdf version here. My using of FFmpeg for repurposing the streams of FMS has attracted quite a lot of interest and attention. I’m planning to extend the series of article dedicated to FFmpeg and also to transform it in a permanent knowledge-base on FFmpeg and related best-practices.

A dream comes true: H.264 encoding into Flash Player 11

Yes. A dream comes true. After almost 9 years the Flash Player will have a new video codec. Sorenson’s Spark is about to retire, finally. But let’s recap the whole story starting, obviously, from the beginning.

In 2002 Macromedia included in Flash Player 6 a video codec provided by Sorenson. The Spark video codec was a custom and simple implementation of the international standard H.263. Spark supported simple encoding techniques derived from H.263v1 (P-frames with motion estimation and compensation, half-pixel accuracy, 1 reference frame, +-16 pixel long reference frames, RLE and Huffman for entropy coding to name a few) plus some enhanced features like deblocking in post-processing and the special D-frames (Desposable frames) which are like P-frames but cannot be used as reference. Especially this latest technique was introduced to support the main objective of Spark: provide a low latency codec for video communication over the Internet.

Flash Player 6 was so capable to generate and consume video streams but only from a new Macromedia’s server product: The Flash Communication Server (FCS), a revolutionary product ahead of the market of several years. To be honest, everybody know the story: FCS had the potentialities to be a disruptive product but was heavily ruined by an absurdly inaccessible price tag  and strongly capped configurations (es: 4500$ for a 10Mbit/s capped version, 990$ for a 1Mbit/s capped version, no developer versions, etc…I’m not kidding). The result: years passed and only a few mad developers (I’m one of them) continued to support the product hoping for a brighter future.

With the successive Flash Player 7, Macromedia decided to unlock the use of Sorenson videos for progressive download. This is the spark that ignited the revolution of video on Internet. After some years of limbo even FCS/FMS was relaunced with less restrictive licenses and the product became more mature release after release (here we could start a different debate about the slowness of Adobe in improving FMS and the recent lawsuite with Wowza but this post has a different topic…).

After only 2-3 years Spark started to be obsolete, even because the implementation of the encoder in the Flash Player was not so optimized and used very simple approaches to rate-control in video encoding. The community started to ask for some improvement in this area but without response until now. I think to have asked for a new encoder for almost 5-6 years. I have also developed in the past some optimization to enhance the encoder performance for screen grabbing or webcam communication but 9-years in computer programming are a whole age and a 30% improvement in efficiency was still insufficient to compare Flash video with Skype video, for example.

So yes, a dream comes true because Adobe has introduced in the current Flash Incubator an H.264 video encoder. Oh yes! H.264 is the state of the art in video encoding, the presence of B-frames could be useful to replace the D-frames and the potentialities of this codec are excellent so that even a poor implementation can lead to excellent improvements over Spark. I’m only a bit afraid for real-time, but from some comments found in the Internet I think that it will be possible to opt for different configuration to address lantency and encoding efficiency.

From the Incubator’s Forum I have estracted this peace of code that show how to change the codec from Spark to H.264 and configure it.

 var h264Settings:H264VideoStreamSettings = new H264VideoStreamSettings();
 h264Settings.setProfileLevel(H264Profile.BASELINE, H264Level.LEVEL_2);
 stream.videoStreamSettings = h264Settings;

* stream is the NetStream istance that will perform the publish of the encoded stream to the FMS.

Notice the H264Profile enumeration which is probably capable to specify not only a BASELINE, but also a MAIN or perhaps HIGH profile for encoding. Similarly the H264Level specify the level (substancially reference frames number) presumibly from 2 to 5. I hope to be also able to define the number of consecutive B-frames, and/or something like an accuracy switch (suppose H264accuracy.SLOW, H264accuracy.FAST and so on).

I’m starting to do some testing by myself because I’m really excited of this future Flash Player feature. If you add the new EchoCancellation API, the support of the open source Speex codec, p2p, and mobile availability, I think that a new youth is starting for Flash based communication applications development.

Better late than never.

A Flash Streaming Success Story

I work with Flash Media Server since the very first release (2003) and in the last 8 years I have designed several streaming infrastructures, developed interactive programs, optimizations, services and provided consultancy for companies that wanted to leverage the flexibility and the power of Flash streaming.

In the late 2008 I was contacted as a FMS and streaming expert by Adobe Italy to help Finelco Group replacing Windows Media with Flash technologies for audio streaming.

Finelco Group controls the largest private radio network in Italy with 3 FM national radio stations (Radio 105, RMC, Virgin Radio Italy – over 8 Million daily listeners) and recently a Digital Terrestrial TV channel (Virgin Radio TV).

The decision to migrate from Windows Media to Flash streaming was motivated by the desire of innovate the offer of streaming services. Windows Media, Real Audio, Shoutcast are system that can offer only a fraction of the flexibility of the Flash Streaming Platform while they needed the maximum flexibility to create new services and offer more interaction.

Designing a private infrastructure

Since the beginning they were convinced to set up their own infrastructure for streaming instead that leveraging CDN. I think this has been a key factor for the success of the initiative, because the audience of the streaming services (radio re-broadcasting, webradio, 24h live video and vod) is almost exclusively in Italy and with a bandwidth requirement which has not much spikes a custom infrastructure can offer the best value for money.

I have had the opportunity to work with a  team of excellent professionals like Carmine Montella, Marcello De Palo, Francesco Bruni and Salvatore Sciacco, to name a few. Salvatore has been really precious. He is an excellent system administrator and now extremely expert in FMS tuning especially on Linux.

During this multi-step project we have had the necessity to push FMS to its limit thanks to the high popularity of the services that have been implemented and almost all the sub-systems of the streaming platform have been implemented using Adobe technologies:

1. Adobe Flash Media Streaming Servers for A/V live and vod streaming.
2. Flash Media Interactive Servers with custom server side scripts for stream replication and fail over.
3. Metadata distribution using dedicated data channels in RTMP.
4. System integration with Flash Projectors or AIR to integrate RTMP messaging to legacy systems.
5. Live encoding for audio and video using FMLE.
6. Dynamic webradio using server side playlists.

More recently we have added Wowza Servers to integrate support for iOS mobile devices (using live HTTP) and generic mobile applications (using RTP).
You can find some details about the FMS infrastructure in this old blog post.

The project involved also the development of client side players for audio and video streaming, the use of dynamic streaming, advanced Flash features like smart buffering, client side playlist for pseudo-live streaming and much more.

A really successful story

I helped them to design, deploy and tune one of the largest, privately held infrastructure for audio and video streaming in Italy. Now, using this platform, Finelco is offering the re-broadcasting of the 3 national radio channels plus over 30 webradio channels, 5 web TVs dedicated to music clips and the re-broadcasting of the national DTV channel.  Add a complete series of VOD, special live events and interactive services and you have a picture of the system.

The infrastructure has up to 8 Gbit/s of  bandwidth available. Usually there are 30/40.000 concurrent users that consume over 10 Million hours of audio streaming monthly.
To make a comparison BBC iPlayer delivers 17 Million hours of audio streaming.
Concluding, I think this is definitely a new little-big Flash based success story.

Adobe announces FMS4

Today Adobe announced the release of FMS4.
The most important new features are:

– IP Multicast
– Live and onDemand HTTP Dynamic Streaming
– Peer assisted Networking
– Enhanced buffering
– Enhanced Bitrate switching
– Absolute Timecode

The product is offered in four editions:

Adobe Flash Developer Edition
Adobe Flash Media Streaming Server
Adobe Flash Media Interactive Server
and the new Flash Media Enterprice Server

I’m very excited about the new features but also a bit disappointed about the price.
You know that FMS has a very dynamic and affordable competitor like Wowza server, so I disagree with the new price policy. Infact the first three editions have the same price of FMS3 while the Enterprice edition, which is the most interesting because of the p2p capabilities, has a different (higher) price (not disclosed yet).
With the diffusion of HTTP Dynamic Streaming, that does not require a streaming server but only an HTTP server to deliver video, the p2p feature is becoming the flagship of Flash Media Server, and a wrong price policy can be very dangerous for the product and for FMS developers.

Today launched La7.tv, the first Italian TV catch-up service

Today La7, the Italian TV broadcaster owned by Telecom Italia, launched La7.tv the first TV catch-up service entirely developed and delivered in Italy. In the last months I collaborated with ValueTeam at the development of the encoding pipeline and the streaming optimizations used by the Telecom Italia Group for the project.

The service offers in on-demand the tv programs of the last 7-days (a BBC-style catch-up), with the most succesful videos stored in a “cult” section.

The service is entirely Flash based (Video Player with Dynamic Streaming, Dynamic Buffering, Fast Protocol Negotiation and Auto-Recovery) and leverage a FMS 3.5 based CDN.

The custom encoding pipeline is integrated in the broadcaster’s publishing work-flow and after only 15-20min after the end of the program in TV, the encoded files (up to 3 Hrs long) are available for QA, inspection and meta-tagging.

The source, an high quality SD 16:9 video, is encoded in two bitrates: 600 and 350Kbit/s. Despite the low bitrate, quality is very good and is optimized for full-screen playback (see screenshots below). Obviously, a low bitrate to produce a target quality means lower costs and higher reach (see also my latest Adobe MAX presentation).

Maximise FMS’s VOD performance with SSDs

Finally I have started to play with my first SSD (Solid State Disk), an Intel X25-M. If you don’t already know, in the last months the price of SSDs is fallen and the average performances improved.

The main advantage of this kind of storage is the access time which is around 100 times lower than a traditional mechanical drive. This performance mark is usually not so important in a desktop environment, but on a server that handles tens or hundreds of concurrent access to different parts of the disk, it becomes crucial.

Have you ever try to understand why the OS (especially Windows) takes so many seconds to load even on an high end machine ?
Well, the problem is the degraded performance of a mechanical drive when multiple reads of little files are performed.
A fast 1TB traditional HD with a sequential read of 100MB/s can slow down to a few MB/s in such situations.

Usually a well designed software can compensate the limited access time of the storage, but there is a limit.

On the other hand these new SSDs are very fast both in sequential and random access. For example my X25-M sports an outstanding 260MB/s of sequential read speed and an absolutely excellent 50MB/s at random read access (4K files).

Random concurrent access performance is tens time higher !

This excellent technology has only two problems today: price and capacity. X25-M costed me 450€ and is only 160GB wide.

But now let’s talk about the performance increase that such tecnology can bring to Flash Media Server.

As you know, in live streaming a FMS installed on an high end server (now very affordable) can handle also thousands concurrent connections. This is because memory, cpu and especially disk usage are low for any new user. So the main limit is the connection card speed (but now the 10GbE standard is becoming affordable…).

What about VOD performance ? VOD is much different because the streamed data is specific, virtually, for every connected user (let’s exclude the effect of caching which is very sensible to the statistical distribution of the requested segments of data). So, even with two 15.000 rpm 300GB SAS disks, probably you will be able to serve at most few hundreds streams.

SSDs can enhance this capability. I think (and in the next weeks I’m going to verify…) that a RAID 10 with four X25 would be able to provide 1 – 1.5 Gbit/s of VOD streams. At least a 2-3x factor over mechanical drives.

Obviously the storage cost will rise, but the benefits are very high because more concurrent users per server means less delivery cost (fewer servers, fewer FMS licenses, fewer maintainance costs).

If you like to know more about SSD I suggest this reading: http://www.anandtech.com/storage/showdoc.aspx?i=3631

News from the Adobe’s Video Ecosystem

A couple interesting news:

Adobe has released the Open Source Media Framework, formerly “Strobe”. The aim of this new open framework is to standardize the several functionalities required by a modern Flash based media player.

:: From the home site: “Open Source Media Framework enables developers to easily assemble pluggable components to create high-quality, full-featured playback experiences. The open aspect of the framework enables collaborative development for web video monetization, with lower costs and faster turnaround”. The features covered by the framework are really exaustive, from playback to content monetization and reporting. A really must have for media specialists.

:: New information also on the future Flash Mobile. Kevin Lynch confirmed the availability of a beta for the end of the year and the support for accelerometers, gps and multitouch . Android based demos have been showed; what about iPhone ? time will tell…