Google's many positions on video codecs

Earlier this week, Google announced that it would drop support for the H.264 video codec from the HTML5 video tag in the Chrome browser, and replace it with support for its own WebM (VP8) codec. It also announced its intention to release WebM plug-ins for Apple's Safari and Microsoft's Internet Explorer browsers, which support H.264. Chrome joins Firefox and Opera in supporting WebM with the video tag.

Google suggests that Chrome users who want to access H.264 content use Adobe's Flash or Microsoft's Silverlight plug-ins. What's confusing about this is that the Chrome, Android and Google TV teams all have different views on the codec issue. Android's browser supports H.264, and there's no word from the Android team that they plan to drop it. Google TV doesn't support WebM, and in an interview last year, an executive from Intel was very noncommittal about future WebM support. Similarly, YouTube, which supports both H.264 and WebM, has said nothing about dropping support for H.264.

So, the message from Google seems to be "Use WebM for Chrome, H.264 for Android and Google TV, and either one for YouTube." It would be hard for them to be more confusing, or more confused. If you create or distribute Internet video to the desktop, set-top and mobile devices, you now have more to keep track of, and more transcoding to do.

It would be nice if Google would speak with a single voice, but its management doesn't seem capable of coordinating the actions of multiple product teams.

Extensive VP8/H.264 Comparison from Jan Ozer

Jan Ozer and StreamingMedia.com have done an extensive follow-up to their earlier comparison of VP8 (the video codec within Google's open-source WebM) and H.264. I'll briefly summarize their findings:

• In general, the playback CPU utilization of VP8 and H.264 are almost identical. However, many PCs and other devices have H.264 hardware acceleration; on those platforms, H.264's CPU utilization will be much less than that of VP8. This difference will continue until Google gets vendors to make VP8 hardware acceleration standard.
• VP8 requires much longer encoding times than H.264, but the difference can largely be mitigated by using an encoder that supports multiple cores and threads.
• Ozer compared the subjective output quality of Sorenson Squeeze's VP8 encoder, X264 and MainConcept's H.264 encoder. He concluded that X264 output looked best, followed by H.264 and then VP8. However, he noted that the differences between the three encoders were so small as to be virtually unnoticeable except in side-by-side testing.
• Ozer's conclusion: H.264 will remain royalty-free for free Internet content until 2015, and hardware acceleration for VP8 will take a long time to become widespread, so he sees no need for producers or distributors of free Internet content to switch. (He doesn't address the issue of paid content, and that's where VP8 is likely to have more impact when it's more widely adopted by encoder and browser vendors.)

Ozer also points to an extensive analysis by Jason Garrett-Glaser, one of the developers of X264. Garrett-Glaser's bottom line is that the encoding quality of VP8 is about the same as H.264 Baseline Profile and considerably less than Main or High Profile. He expects the decoding performance of VP8 and H.264 to be about the same, but that H.264 will have a significant advantage because of broad support for hardware acceleration of H.264 decoding.

Perhaps most significantly, Garrett-Glaser questions the claims that VP8 is patent-free; he believes that VP8 and H.264 use similar techniques in many parts of the encoding process. In fact, he says that calling VP8 "H.264 with a better entropy coder" would be an only slightly inaccurate description.

My bottom line? If you're producing or distributing free video content only, there's no reason to deal with VP8 or WebM for now. It's going to take at least a year for Google and its partners to optimize encoding and decoding and start getting VP8 hardware acceleration built into GPUs and other devices, and to resolve whether or not VP8 is truly patent-free. Unless you're an encoder, decoder or accelerator developer, I'd let these issues shake out before you adopt VP8 and WebM.

New WebM/H.264 Comparison

A new comparison of VP8 (the video codec in WebM), H.264 and XviD has been published by the Moscow State University's Graphics & Media Lab. The comparison is an addition to a comprehensive review of codecs. The team that performed the comparison makes it clear that it didn't do as extensive a test of VP8 as it did of the other codecs, since the study was apparently virtually completed by the time that Google announced that it was open-sourcing VP8 and including it in WebM. Nevertheless, the team came to a number of conclusions:

For movie encoding:

Comparing VP8 to XviD, VP8 is 5-25 times slower with 10-30% better quality (lower bitrate for the same quality). When comparing VP8 and x264 VP8 also shows 5-25 lower encoding speed with 20-30% lower quality at average. For example x264 High-Speed preset is faster and has higher quality than any of VP8 presets at average.

For HDTV:

Comparing VP8 to XviD, VP8 is 5-20 times slower with 10-20% better quality (lower bitrate for the same quality). When comapring VP8 and x264 VP8 shows 5-20 lower encoding speed with almost the same quality, excluding x624 High-Quality preset.

The source material that the team used for the Movie testing was clips from the films "Ice Age 3", "Raiders of the Lost Ark", "Enemy of the State" and "Up". For the HDTV test, the team used video shot in the Amazon taken from a Microsoft site, the trailer from "Iron Man 2", a close-up video of a calendar and a clip from the movie "Troy."

The WebM team responded to the Moscow State University results, admitting that a lot of work needs to be done to VP8 in order to improve its encoding speed. However, they contend that VP8 would have provided better-quality output had the source material not previously been encoded (only one sample, the calendar, was uncompressed.) Here are their comments:

We've been following the MSU tests since they began and respect the group's work. One issue we noticed in the test is that most input sequences were previously compressed using other codecs. These sequences have an inherent bias against VP8 in recompression tests. As pointed out by other developers, H.264 and MPEG-like encoders have slight advantages in reproducing some of their own typical artifacts, which helps their objective measurement numbers but not necessarily visual quality. This is reflected by relatively better results for VP8 on the only uncompressed input sequence, "mobile calendar.

Even with this limitation, VP8 delivered respectable results against other encoders, especially considering this is the first time VP8 has been included in the test and VP8 has not been specifically optimized for SSIM as some other codecs have.

To date, WebM developers have focused on the VP8 decoder performance and are only starting to optimize the encoder for speed. The WebM project has only been underway for three weeks, and we believe that our encoder speed will improve significantly in the near future.

The WebM team's comments about the source material have some merit, but in the real world, video that's previously been compressed is often included in projects. It may be difficult or impossible to get uncompressed source material--for example, AVCHD camcorders and some DSLRs output video using H.264. Therefore, the performance of VP8 on previously compressed video is important.

The Moscow State University comparison provides useful insight into the performance of VP8, and points out some of the areas that Google and its partners need to work on.

Big set-top box changes underway at Comcast

ESPN 3D, the cable network carrying 3D coverage of the World Cup in the U.S., is available to cable operators via both MPEG-2 and MPEG-4 compression. MPEG-4 is significantly more bandwidth-efficient than MPEG-2. and according to Cable360, Comcast customers who want 3D programming will have to use MPEG-4 compatible set-top boxes starting in August. Comcast has approximately 10 million MPEG-4 set-top boxes in the field, and 25 million set-top boxes that only support MPEG-2.

Most of Comcast's MPEG-4 set-top boxes are from Motorola, but the company is said to have chosen Pace to supply its next-generation set-top boxes. The Pace STBs will support MPEG-4 H.264 compression and Tru2Way applications, and they may be compatible with Switched Digital Video (SDV) services. Comcast's choice of Pace will have a major impact on both Motorola and Cisco in the U.S. market. Pace has been very strong everywhere but the U.S., but supplying the largest video service provider in the U.S. will dramatically increase its presence and shake up the market.

First WebM (VP8) to H.264 comparison published

Streamingmedia.com has published an initial comparison of Google's VP8 open-source video codec (part of WebM) and H.264, run by Jan Ozer, who's done many such codec and compressor comparisons over the years. In summary, at almost exactly the same bitrate (438kbps for VP8 vs. 439kbps for H.264) and the same resolution (480 x 360), H.264 provided slightly better overall visual quality, especially in clips with higher motion. However, Ozer states that the difference won't be noticeable in most applications.

Keep in mind that the tests were done at SD resolution and at one bitrate, so more definitive testing will be required to understand how the two codecs compare over a range of conditions. However, at first glance, VP8 seems to be well in the ballpark with H.264.

WebM might be a big deal after all

Earlier today, I wrote a post that said that Google's announcements at the I/O Conference this morning weren't all that exciting. But, the WebM announcement, might--might--just be a Big Thing. Google not only signed up most of the web browser leaders except for Microsoft and Apple (and after today's announcement, Microsoft said that it would support WebM in Internet Explorer 9 if the user installs the codecs themselves); it signed up just about all the major players in hardware encoders and decoders. They signed up almost all the major players in mobile phone chip sets, including ARM, Broadcom, Freescale Semiconductor, Marvell, MIPS Technologies, Texas Instruments and Qualcomm. They got many of the leaders in high-end video encoders, including Harmonic, Telestream, Digital Rapids, ViewCast, Inlet and Anystream. And, they got both of the leaders in GPUs, Nvidia and AMD.

These companies are important because it costs a lot to build hardware encoders and write tight, high-performance firmware. The fact that Google got all these players to sign on indicates that they see real potential in WebM. This represents a big problem for MPEG LA, the consortium that licenses patents related to H.264. Steve Jobs and MPEG LA executives have been doing more than hinting that both Ogg Theora and VP8, which is the video codec in WebM, infringe on MPEG LA's patents. But, I strongly doubt that these companies would expose themselves to potential liability to support an infringing codec. Google may well have agreed to indemnify at least some of these companies should MPEG LA, one or more of its consortium members, or a company with separate patents such as AT&T, takes legal action.

WebM isn't going to have any impact on Blu-Ray, which supports H.264 and Microsoft's VC-1 codec. Nor will it have much impact on cable, satellite or IPTV service providers; they're already heavily invested in MPEG-2 and MPEG-4. It's very unlikely to have a big impact on camcorders, because so many of the major camcorder manufacturers also contribute a large share of MPEG LA's patents and have cross-licensing agreements with each other to keep costs down.

Where WebM is likely to have its biggest impact is on the web, both in desktop and mobile applications. Acquisition--getting video from a camcorder into a digital form--probably isn't going to change, but the chain from encoding video after post-production to decoding it on your PC, tablet, phone or Internet-enabled set-top box can now be royalty-free.

The big "if" is whether or not WebM/VP8  is actually a good replacement for H.264. Is it in the same ballpark as H.264 for video quality, bandwidth efficiency, encoding and decoding speed and CPU utilization? Google claims it is, but we won;t know for sure until there's been third-party testing of a variety of VP8 encoders and decoders. If VP8 can stand up to H.264, it has an excellent chance of becoming the new standard for video on the web.

H.264 now comprises 66% of encoding.com's videos

In response to Steve Jobs' open letter about Flash last week, TechCrunch contacted encoding.com, a video encoding service, to get their statistics on what formats their clients are requesting. In Q1 2010, 66% of all videos that encoding.com processed were encoded into H.264, while On2 VP6 and .FLV (which could be any codec supported by Flash, but in this case probably means Sorenson) together add up to 26%. (Ogg Theora is barely 2%.) Here's the chart:

Keep in mind that encoding.com has encoded 5 million videos over the past year for a variety of clients, but it in no way represents the majority of video sites or content. Also, these numbers represent new or transcoded files, not the huge number of legacy video files that still exist on the web. Nevertheless, encoding.com's numbers suggest that H.264 has got major adoption momentum. However, that could change.

Google's rumored announcement later this month that it will make On2's VP8 format available as open source may change the balance, especially if YouTube starts encoding its videos in VP8. According to ComScore's traffic numbers for March, YouTube had more video viewing traffic than then next ten sites put together, so as YouTube goes, so goes a large part of the market.

Google to open source VP8?

NewTeeVee is reporting that Google will announce plans to make On2's VP8 codec open source as early as next month's Google I/O developers' conference. The move is no big surprise, and it's likely to dramatically increase adoption of VP8, which On2 claims is significantly more bandwidth-efficient than H.264. Adobe Flash currently doesn't support VP8, although given Adobe and Google's closer partnership, future support is likely. NewTeeVee reports that both Google's Chrome and Mozilla's Firefox will add VP8 support once Google makes its announcement.

VP8 could provide an alternative to H.264 for developers and content providers who are concerned about future changes in MPEG LA's licensing policies, and to Ogg Theora for those who believe that Theora requires significantly more bandwidth than H.264 for the same level of quality. (Google is also helping to fund an implementation of Ogg Theora for ARM processors.)

The bottom line is that VP8 and Ogg Theora would provide a solid base of royalty-free alternative video codecs, in much the same way as PNG served as a royalty-free alternative to GIF.