Home > FMS, Video > A dream comes true: H.264 encoding into Flash Player 11

A dream comes true: H.264 encoding into Flash Player 11

Yes. A dream comes true. After almost 9 years the Flash Player will have a new video codec. Sorenson’s Spark is about to retire, finally. But let’s recap the whole story starting, obviously, from the beginning.

In 2002 Macromedia included in Flash Player 6 a video codec provided by Sorenson. The Spark video codec was a custom and simple implementation of the international standard H.263. Spark supported simple encoding techniques derived from H.263v1 (P-frames with motion estimation and compensation, half-pixel accuracy, 1 reference frame, +-16 pixel long reference frames, RLE and Huffman for entropy coding to name a few) plus some enhanced features like deblocking in post-processing and the special D-frames (Desposable frames) which are like P-frames but cannot be used as reference. Especially this latest technique was introduced to support the main objective of Spark: provide a low latency codec for video communication over the Internet.

Flash Player 6 was so capable to generate and consume video streams but only from a new Macromedia’s server product: The Flash Communication Server (FCS), a revolutionary product ahead of the market of several years. To be honest, everybody know the story: FCS had the potentialities to be a disruptive product but was heavily ruined by an absurdly inaccessible price tag  and strongly capped configurations (es: 4500$ for a 10Mbit/s capped version, 990$ for a 1Mbit/s capped version, no developer versions, etc…I’m not kidding). The result: years passed and only a few mad developers (I’m one of them) continued to support the product hoping for a brighter future.

With the successive Flash Player 7, Macromedia decided to unlock the use of Sorenson videos for progressive download. This is the spark that ignited the revolution of video on Internet. After some years of limbo even FCS/FMS was relaunced with less restrictive licenses and the product became more mature release after release (here we could start a different debate about the slowness of Adobe in improving FMS and the recent lawsuite with Wowza but this post has a different topic…).

After only 2-3 years Spark started to be obsolete, even because the implementation of the encoder in the Flash Player was not so optimized and used very simple approaches to rate-control in video encoding. The community started to ask for some improvement in this area but without response until now. I think to have asked for a new encoder for almost 5-6 years. I have also developed in the past some optimization to enhance the encoder performance for screen grabbing or webcam communication but 9-years in computer programming are a whole age and a 30% improvement in efficiency was still insufficient to compare Flash video with Skype video, for example.

So yes, a dream comes true because Adobe has introduced in the current Flash Incubator an H.264 video encoder. Oh yes! H.264 is the state of the art in video encoding, the presence of B-frames could be useful to replace the D-frames and the potentialities of this codec are excellent so that even a poor implementation can lead to excellent improvements over Spark. I’m only a bit afraid for real-time, but from some comments found in the Internet I think that it will be possible to opt for different configuration to address lantency and encoding efficiency.

From the Incubator’s Forum I have estracted this peace of code that show how to change the codec from Spark to H.264 and configure it.

 var h264Settings:H264VideoStreamSettings = new H264VideoStreamSettings();
 h264Settings.setProfileLevel(H264Profile.BASELINE, H264Level.LEVEL_2);
 stream.videoStreamSettings = h264Settings;

* stream is the NetStream istance that will perform the publish of the encoded stream to the FMS.

Notice the H264Profile enumeration which is probably capable to specify not only a BASELINE, but also a MAIN or perhaps HIGH profile for encoding. Similarly the H264Level specify the level (substancially reference frames number) presumibly from 2 to 5. I hope to be also able to define the number of consecutive B-frames, and/or something like an accuracy switch (suppose H264accuracy.SLOW, H264accuracy.FAST and so on).

I’m starting to do some testing by myself because I’m really excited of this future Flash Player feature. If you add the new EchoCancellation API, the support of the open source Speex codec, p2p, and mobile availability, I think that a new youth is starting for Flash based communication applications development.

Better late than never.

Categories: FMS, Video
  1. Niels
    14 May 2011 at 4:26 pm

    You mention p2p in the last paragraph. I haven’t yet heard of this feature in Flash, do you have any more info on this?

  2. Rajdeep Rath
    14 May 2011 at 8:37 pm

    Hi

    I just downloaded playerglobalrc1 from:
    http://download.macromedia.com/pub/labs/flashplatformruntimes/flashplayer10-3/flashplayer10-3_rc1_playerglobal_042011.swc

    There seems to be no such class as:
    var h264Settings:H264VideoStreamSettings = new H264VideoStreamSettings();

    • sonnati
      14 May 2011 at 9:56 pm

      Your are using the playerglobal of fp 10.3 rc. Its a future feature, Try in the download page of flash player incubator

  3. 14 May 2011 at 10:58 pm

    You can choose between BASELINE and MAIN profiles ;).
    There’s also a choice of levels: from 1 to 5.1 .

    However I’m not seeing any other options (xcept from the usual resolution, fps, etc.)

    • sonnati
      15 May 2011 at 10:53 am

      At least with the choice between baseline and main it is possible to enable b-frame for efficiency or disable to minimize latency

  4. 13 June 2011 at 12:34 pm

    你好,你的文章很受用,但是要实现的时候,总是报一个错,swc也已经导入,麻烦看看知不知道原因:
    报名语句:rtmpVideoStream.videoStreamSettings = h264Settings;
    报错信息:163 行1119: 访问可能未定义的属性 videoStreamSettings (通过 static 类型 flash.net:NetStream 引用)。

    编译的时候 videoStreamSettings 这个属性不会被识别,H264VideoStreamSettings像这些类均可以,不明原因,请作者指教。
    是不是跟编译器有关系,我用的是Flash Profession CS5.5

  5. 13 June 2011 at 1:48 pm

    I encountered this problem, could be resolved? Thank you。

    VerifyError: Error #1014: Class flash.media::H264VideoStreamSettings could not be found.

    the error : http://www.uini.net/wp-content/uploads/2011/06/err.png

  6. 14 June 2011 at 9:26 am

    Thank you for your reply, in accordance with your instructions, I have to solve this problem, the code is debugged, to achieve the effect I wanted, very much. The problem is because my version is not compiled FlashPlayer 11 lead.

  7. homayoon
    21 June 2011 at 2:00 pm

    Hi ,

    I get these errors at running time when connecting online.
    Can you guide me how solve this problem?

    typecheck MinCam/checkConnect()
    outer-scope = [global Object$ flash.events::EventDispatcher$ flash.display::DisplayObject$ flash.display::InteractiveObject$ flash.display::DisplayObjectContainer$ flash.display::Sprite$ MinCam$]
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {} ()
    0:getlocal0
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {} (MinCam~[O])
    1:pushscope
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {MinCam~[O]} ()
    2:pushnull
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {MinCam~[O]} (null[O])
    3:coerce flash.media::H264VideoStreamSettings
    VerifyError: Error #1014: Class flash.media::H264VideoStreamSettings could not be found.

    thanks.

  8. 3 August 2011 at 5:51 pm

    Hello
    what audio codec is used in FP 11? AAC?

  9. Ram
    5 August 2011 at 12:04 pm

    Hi Sonnati,

    I have implemented your code and when I am publishing stream then I am able save in mp4 format and codec is H.264 but my recorded files are not playing.

    So please can you share some code for publishing stream in MP4 format, video codec H264, audio codec AAC and 44khz

    Thanks
    Ram

  10. GatorEye
    11 August 2011 at 1:34 am

    Wondered if you got my email about H.264 encoding in the Flash Player. Had a few questions for you especially pertaining to the use of camera.setQuality. Thanks.

  11. 16 September 2011 at 9:28 am

    A dream comes true, with H264 video and AAC audio, but the dream goes out with Speex, G711 or Nellymoser audio code. hello world?

    • sonnati
      16 September 2011 at 9:38 am

      Yes, it’s a real pity. Indeed video encoding in Flash has always been aimed at RTC and AAC is not good at low latency. it requires at least 1024 samples which means 20-60ms of codec latency + network latency.
      Speex is a good codec but it is not implemented at its full. For example it support stereo but not in the version implemented in flash. If you need AAC for iPhone compatibility you can transcode on the fly using
      FFMPEG and only the audio track from speex to AAC. I have written an article about that.

  12. 30 September 2011 at 1:52 pm

    Is there a way to H.264 encode a local file? e.g. can I grab the webcam input and h.264 encode it locally (client side) without connecting to a server?

    • sonnati
      30 September 2011 at 6:37 pm

      no, you can’t

      • 4 October 2011 at 9:58 pm

        I don’t understand. If you can’t encode locally captured video then what is the purpose of H.264 encoding in the client?

      • sonnati
        5 October 2011 at 4:19 pm

        The purpose is to encode and stream on the fly the video to a FMS for real time video communication applications

      • 5 October 2011 at 6:53 pm

        Yes, that seems like a good use case. Thank you.

        Here’s hoping future enhancements will enable more local functionality. I would love to be able to offer client-side video recording and editing with the ability to save the results locally or upload finalized files.

  13. James Corbett
    14 November 2011 at 12:29 pm

    Brian Sexton :
    Yes, that seems like a good use case. Thank you.
    Here’s hoping future enhancements will enable more local functionality. I would love to be able to offer client-side video recording and editing with the ability to save the results locally or upload finalized files.

    A netstream can be written to disk, so I don’t see why you couldn’t write a file to the local computer that you have encoded using flash players h264 encoder.

    • 9 January 2012 at 6:06 am

      Could you explain how ? as far as i know there is no access to bytes from a netstream. http://www.bytearray.org/?p=955

      • JC
        11 January 2012 at 4:33 pm

        Sorry as you say I was mistaken, I merely assumed as you could append bytes that you could also grab them! Thank you for the correction that wouldn’t have been a nice surprise when I started writing my little app!

  14. Apurva
    2 December 2011 at 7:34 am

    Hi i’m getting the same errors can you please help me out?

    typecheck MinCam/checkConnect()
    outer-scope = [global Object$ flash.events::EventDispatcher$ flash.display::DisplayObject$ flash.display::InteractiveObject$ flash.display::DisplayObjectContainer$ flash.display::Sprite$ MinCam$]
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {} ()
    0:getlocal0
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {} (MinCam~[O])
    1:pushscope
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {MinCam~[O]} ()
    2:pushnull
    [MinCam~[O] flash.events::NetStatusEvent[O] *[A]] {MinCam~[O]} (null[O])
    3:coerce flash.media::H264VideoStreamSettings
    VerifyError: Error #1014: Class flash.media::H264VideoStreamSettings could not be found.

    • WingLi
      4 January 2012 at 5:51 am

      He he, I had the same problem as u and it took me 1.5 days to figure it out.
      It is not a compile error but a run time error, so the problem lies in the FP.
      It happens when u run with version 10 instead of 11.
      Go to Adobe website and grab the standalone (aka projector) versions of Flash Player 11.1 and try it again.

  15. WilliamX
    26 March 2012 at 11:34 pm

    On a one way communication, is it possible to have a MONSTER hardware machine to minimize the encoding time for H264? I am wondering if its posible to broadcast a livestream in h264 through the flash player with a minimal latency. Does anybody has experience with it yet?

  16. 18 September 2012 at 5:52 pm

    I’ve noticed memory leaks after applying h264settings to my net stream, any solutions?

  17. 3 April 2013 at 3:04 am

    some of the newer usb cameras (logitech c920) have hardware encoding….will the flash player grab this encoded stream from the camera or still do the encoding?

  18. 8 April 2013 at 11:17 am

    We have the same cam in the office (Logitech c920) and we can see
    1) a slight increase in CPU usage when we display the webcam feed in Flash Player
    2) a more pronounced increase in CPU usage when we display the stream + send it to the media server

    We left the loopback property on the _cam object untouched.

    So my conclusion is that even tough c920 has native H.264 encoding, Flash Player still uses the computer’s CPU for H.264 .

    I believe the native encoding on the webcam is used with Skype.

  19. 8 April 2013 at 11:39 am

    More on the c920:

    When initiating a Skype call with the c920 the call starts at 640×480 and the computer’s CPU is used for encoding.

    About 1 minute in Skype switches to 1280×720 and the H.264 encoding in the C920. At this moment you can clearly see in the task manager that the computer’s CPU usage drops (the native encoding in the webcam is being used).

    Same thing happens later on when Skype switches to Full HD.

    The Call > Call Technical Info window provides all the details.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: