Is MPEG LA's "Call for Patents" for Google's VP8 a sign of weakness?

Yesterday, MPEG LA, the agency that licenses patents related to MPEG-4/H.264, issued a "Call for Patents" for Google's VP8 video codec. VP8 is the video codec incorporated in Google's WebM format, which Google is licensing at no charge. MPEG LA wasn't asked to round up potentially applicable patents by Google. Instead, MPEG LA sees an opportunity to make money, in much the same way that attorneys around the U.S. advertise on late-night television to find users of various medications or workers in particular industries who might have been harmed.

Update, March 3, 2011: PaidContent reports that the U.S. Justice Department has begun an investigation of MPEG LA's actions related to VP8 to determine if they violate antitrust laws. The article suggests that MPEG LA itself may not be the real target of the investigation, but rather, one or more members of MPEG LA who might have reason to sabotage Google's open video efforts. 

The interesting thing is that executives from MPEG LA have been claiming that VP8 infringes its MPEG-4/H.264 patents since last summer, when VP8 was first made available for free by Google. If that's the case, why do they now have to go out and find patents that cover VP8? Didn't they already know which patents were being infringed when they charged that VP8 was "undoubtedly" infringing their existing patents? And, if they examined their existing patent pool and decided that nothing they've got covers VP8, what are the chances that they're going to find anything now?

This move, which is being seen by some observers as a threat to WebM, looks to me more like a statement of weakness by MPEG LA. To this observer, it looks as though they don't think that they can prevail with their current patent pool, and they're desperately looking for a "submarine" patent somewhere that will stand up against court challenges.

I would have taken MPEG LA's threats much more seriously if they had filed suit for patent infringement against Google using the patents they already represent. At this point, Google is in a much stronger position than I thought they were.

Google's many positions on video codecs

Earlier this week, Google announced that it would drop support for the H.264 video codec from the HTML5 video tag in the Chrome browser, and replace it with support for its own WebM (VP8) codec. It also announced its intention to release WebM plug-ins for Apple's Safari and Microsoft's Internet Explorer browsers, which support H.264. Chrome joins Firefox and Opera in supporting WebM with the video tag.

Google suggests that Chrome users who want to access H.264 content use Adobe's Flash or Microsoft's Silverlight plug-ins. What's confusing about this is that the Chrome, Android and Google TV teams all have different views on the codec issue. Android's browser supports H.264, and there's no word from the Android team that they plan to drop it. Google TV doesn't support WebM, and in an interview last year, an executive from Intel was very noncommittal about future WebM support. Similarly, YouTube, which supports both H.264 and WebM, has said nothing about dropping support for H.264.

So, the message from Google seems to be "Use WebM for Chrome, H.264 for Android and Google TV, and either one for YouTube." It would be hard for them to be more confusing, or more confused. If you create or distribute Internet video to the desktop, set-top and mobile devices, you now have more to keep track of, and more transcoding to do.

It would be nice if Google would speak with a single voice, but its management doesn't seem capable of coordinating the actions of multiple product teams.

Extensive VP8/H.264 Comparison from Jan Ozer

Jan Ozer and StreamingMedia.com have done an extensive follow-up to their earlier comparison of VP8 (the video codec within Google's open-source WebM) and H.264. I'll briefly summarize their findings:

• In general, the playback CPU utilization of VP8 and H.264 are almost identical. However, many PCs and other devices have H.264 hardware acceleration; on those platforms, H.264's CPU utilization will be much less than that of VP8. This difference will continue until Google gets vendors to make VP8 hardware acceleration standard.
• VP8 requires much longer encoding times than H.264, but the difference can largely be mitigated by using an encoder that supports multiple cores and threads.
• Ozer compared the subjective output quality of Sorenson Squeeze's VP8 encoder, X264 and MainConcept's H.264 encoder. He concluded that X264 output looked best, followed by H.264 and then VP8. However, he noted that the differences between the three encoders were so small as to be virtually unnoticeable except in side-by-side testing.
• Ozer's conclusion: H.264 will remain royalty-free for free Internet content until 2015, and hardware acceleration for VP8 will take a long time to become widespread, so he sees no need for producers or distributors of free Internet content to switch. (He doesn't address the issue of paid content, and that's where VP8 is likely to have more impact when it's more widely adopted by encoder and browser vendors.)

Ozer also points to an extensive analysis by Jason Garrett-Glaser, one of the developers of X264. Garrett-Glaser's bottom line is that the encoding quality of VP8 is about the same as H.264 Baseline Profile and considerably less than Main or High Profile. He expects the decoding performance of VP8 and H.264 to be about the same, but that H.264 will have a significant advantage because of broad support for hardware acceleration of H.264 decoding.

Perhaps most significantly, Garrett-Glaser questions the claims that VP8 is patent-free; he believes that VP8 and H.264 use similar techniques in many parts of the encoding process. In fact, he says that calling VP8 "H.264 with a better entropy coder" would be an only slightly inaccurate description.

My bottom line? If you're producing or distributing free video content only, there's no reason to deal with VP8 or WebM for now. It's going to take at least a year for Google and its partners to optimize encoding and decoding and start getting VP8 hardware acceleration built into GPUs and other devices, and to resolve whether or not VP8 is truly patent-free. Unless you're an encoder, decoder or accelerator developer, I'd let these issues shake out before you adopt VP8 and WebM.

New WebM/H.264 Comparison

A new comparison of VP8 (the video codec in WebM), H.264 and XviD has been published by the Moscow State University's Graphics & Media Lab. The comparison is an addition to a comprehensive review of codecs. The team that performed the comparison makes it clear that it didn't do as extensive a test of VP8 as it did of the other codecs, since the study was apparently virtually completed by the time that Google announced that it was open-sourcing VP8 and including it in WebM. Nevertheless, the team came to a number of conclusions:

For movie encoding:

Comparing VP8 to XviD, VP8 is 5-25 times slower with 10-30% better quality (lower bitrate for the same quality). When comparing VP8 and x264 VP8 also shows 5-25 lower encoding speed with 20-30% lower quality at average. For example x264 High-Speed preset is faster and has higher quality than any of VP8 presets at average.

For HDTV:

Comparing VP8 to XviD, VP8 is 5-20 times slower with 10-20% better quality (lower bitrate for the same quality). When comapring VP8 and x264 VP8 shows 5-20 lower encoding speed with almost the same quality, excluding x624 High-Quality preset.

The source material that the team used for the Movie testing was clips from the films "Ice Age 3", "Raiders of the Lost Ark", "Enemy of the State" and "Up". For the HDTV test, the team used video shot in the Amazon taken from a Microsoft site, the trailer from "Iron Man 2", a close-up video of a calendar and a clip from the movie "Troy."

The WebM team responded to the Moscow State University results, admitting that a lot of work needs to be done to VP8 in order to improve its encoding speed. However, they contend that VP8 would have provided better-quality output had the source material not previously been encoded (only one sample, the calendar, was uncompressed.) Here are their comments:

We've been following the MSU tests since they began and respect the group's work. One issue we noticed in the test is that most input sequences were previously compressed using other codecs. These sequences have an inherent bias against VP8 in recompression tests. As pointed out by other developers, H.264 and MPEG-like encoders have slight advantages in reproducing some of their own typical artifacts, which helps their objective measurement numbers but not necessarily visual quality. This is reflected by relatively better results for VP8 on the only uncompressed input sequence, "mobile calendar.

Even with this limitation, VP8 delivered respectable results against other encoders, especially considering this is the first time VP8 has been included in the test and VP8 has not been specifically optimized for SSIM as some other codecs have.

To date, WebM developers have focused on the VP8 decoder performance and are only starting to optimize the encoder for speed. The WebM project has only been underway for three weeks, and we believe that our encoder speed will improve significantly in the near future.

The WebM team's comments about the source material have some merit, but in the real world, video that's previously been compressed is often included in projects. It may be difficult or impossible to get uncompressed source material--for example, AVCHD camcorders and some DSLRs output video using H.264. Therefore, the performance of VP8 on previously compressed video is important.

The Moscow State University comparison provides useful insight into the performance of VP8, and points out some of the areas that Google and its partners need to work on.

First WebM (VP8) to H.264 comparison published

Streamingmedia.com has published an initial comparison of Google's VP8 open-source video codec (part of WebM) and H.264, run by Jan Ozer, who's done many such codec and compressor comparisons over the years. In summary, at almost exactly the same bitrate (438kbps for VP8 vs. 439kbps for H.264) and the same resolution (480 x 360), H.264 provided slightly better overall visual quality, especially in clips with higher motion. However, Ozer states that the difference won't be noticeable in most applications.

Keep in mind that the tests were done at SD resolution and at one bitrate, so more definitive testing will be required to understand how the two codecs compare over a range of conditions. However, at first glance, VP8 seems to be well in the ballpark with H.264.

WebM might be a big deal after all

Earlier today, I wrote a post that said that Google's announcements at the I/O Conference this morning weren't all that exciting. But, the WebM announcement, might--might--just be a Big Thing. Google not only signed up most of the web browser leaders except for Microsoft and Apple (and after today's announcement, Microsoft said that it would support WebM in Internet Explorer 9 if the user installs the codecs themselves); it signed up just about all the major players in hardware encoders and decoders. They signed up almost all the major players in mobile phone chip sets, including ARM, Broadcom, Freescale Semiconductor, Marvell, MIPS Technologies, Texas Instruments and Qualcomm. They got many of the leaders in high-end video encoders, including Harmonic, Telestream, Digital Rapids, ViewCast, Inlet and Anystream. And, they got both of the leaders in GPUs, Nvidia and AMD.

These companies are important because it costs a lot to build hardware encoders and write tight, high-performance firmware. The fact that Google got all these players to sign on indicates that they see real potential in WebM. This represents a big problem for MPEG LA, the consortium that licenses patents related to H.264. Steve Jobs and MPEG LA executives have been doing more than hinting that both Ogg Theora and VP8, which is the video codec in WebM, infringe on MPEG LA's patents. But, I strongly doubt that these companies would expose themselves to potential liability to support an infringing codec. Google may well have agreed to indemnify at least some of these companies should MPEG LA, one or more of its consortium members, or a company with separate patents such as AT&T, takes legal action.

WebM isn't going to have any impact on Blu-Ray, which supports H.264 and Microsoft's VC-1 codec. Nor will it have much impact on cable, satellite or IPTV service providers; they're already heavily invested in MPEG-2 and MPEG-4. It's very unlikely to have a big impact on camcorders, because so many of the major camcorder manufacturers also contribute a large share of MPEG LA's patents and have cross-licensing agreements with each other to keep costs down.

Where WebM is likely to have its biggest impact is on the web, both in desktop and mobile applications. Acquisition--getting video from a camcorder into a digital form--probably isn't going to change, but the chain from encoding video after post-production to decoding it on your PC, tablet, phone or Internet-enabled set-top box can now be royalty-free.

The big "if" is whether or not WebM/VP8  is actually a good replacement for H.264. Is it in the same ballpark as H.264 for video quality, bandwidth efficiency, encoding and decoding speed and CPU utilization? Google claims it is, but we won;t know for sure until there's been third-party testing of a variety of VP8 encoders and decoders. If VP8 can stand up to H.264, it has an excellent chance of becoming the new standard for video on the web.

Google I/O Day 1: Perhaps more excitement in tomorrow's keynote

Anyone who was looking for earth-shaking news from today's keynote presentation at Google's I/O Conference didn't find it. The biggest news had already leaked weeks ago--Google's decision to open-source the VP8 video codec from On2 Technologies. Under the name "The WebM Project", Google will make VP8 available royalty-free and open-source, and will pair it with the Ogg Vorbis audio codec, which has been added to the project by Xiph.org, to create WebM. YouTube has begun to make WebM versions of some of its videos available for testing, and WebM will become a standard format supported by YouTube.

Google, Mozilla and Opera will add support for WebM to their browsers (currently in nightly developer Chromium and Firefox builds, and in a Lab build of Opera). Adobe also announced its intention to add WebM support to Flash within the next year. More than 20 other software, hardware and services companies also announced their plans to support WebM. Not surprisingly, neither Apple nor Microsoft announced support for WebM.

There were a lot of demonstrations of applications built on HTML5, and Adobe demonstrated some ways to use the current Dreamweaver CS5 and CSS to build platform-aware HTML5 content. The company also demonstrated an early prototype of software that can create interactive graphic HTML5 content. Google announced that Google Wave is now open for general use as a Google Labs project (no invitation needed). The company also announced a web app store for Chrome; support for the store will be first available in developer builds of Chrome to be released later this year. Applications available in the store will run in other HTML5-compliant browsers.

Finally, Google spent the last 40 minutes of the keynote discussing and demonstrating application development tools and platforms. The announcements including the Google App Engine for Business, which is not yet available, and significant extensions to Google App Engine itself to make it more competitive with Amazon Web Services.

It's now expected that tomorrow's keynote will focus on Android, Flash and Google's TV initiative with Intel, Sony and Logitech.