X

Photo format from Google and Mozilla could leave JPEG in the dust

Apple's got one idea for next-gen photo technology, but a rival approach based on the new AV1 video compression tech could go a step further.

Stephen Shankland Former Principal Writer
Stephen Shankland worked at CNET from 1998 to 2024 and wrote about processors, digital photography, AI, quantum computing, computer science, materials science, supercomputers, drones, browsers, 3D printing, USB, and new computing technology in general. He has a soft spot in his heart for standards groups and I/O interfaces. His first big scoop was about radioactive cat poop.
Expertise Processors, semiconductors, web browsers, quantum computing, supercomputers, AI, 3D printing, drones, computer science, physics, programming, materials science, USB, UWB, Android, digital photography, science. Credentials
  • Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
Stephen Shankland
5 min read

Last year Apple managed to shrink photo file sizes dramatically so you could cram twice as many photos onto your iPhone. But hold on, because that might not be the only game in town.

Google, Mozilla and others in a group called the Alliance for Open Media are working on a rival photo technology. In testing so far, the images are 15 percent smaller than Apple's HEIC photo format, said Tim Terriberry, a Mozilla principal research engineer working on the project. But smaller sizes are just the beginning.

The project is still young -- it doesn't even have a name, much less a certain future. But it's got a strong list of allies, an affinity for web publishing and modern features that could make it the best contender yet for overcoming JPEG's 1990s-era shortcomings.

"It seems downright silly that we're still relying on compression tech from 20 years ago," said Kelly Thompson, general manager at 500px, a photo sharing and sales site. "The equipment we're using to capture and display images is now exceeding JPEG's upper limits."

One way or another, photos are going to get better. It's not clear what path we'll take or how long it'll take, but it is clear that photos are turning into something more than prints you can stick in a photo album.

JPEG isn't just limited by needlessly large file sizes. It's also weak when it comes to supporting a wider range of bright and dark tones, a broader spectrum of colors, and graphic elements like text and logos. On top of that, photos these days are bursting beyond the rectangle-of-pixels limits, but JPEG can't handle new photo technology like bursts of shots, panoramas, live photos and 3D scene data.

Comparison of JPEG, HEIC, AV1
Enlarge Image
Comparison of JPEG, HEIC, AV1

These three images, each compressed to 28KB, compare JPEG, the HEVC encoder used in Apple's HEIC format, and an experimental photo format based on the forthcoming AV1 video technology. The two newer compression methods clearly outpace JPEG.

Mozilla

HEIC arrives

Apple's photo format, called HEIC, arrived last year with Apple's iOS software for iPhones and iPads and in its MacOS software for its personal computers. It's an offshoot of the HEVC video compression technology developed by an industry consortium called MPEG with years of experience in squeezing video file sizes so they're easier to store and to send over networks.

HEIC uses HEVC's compression technology. An HEIC photo takes about half the space of a JPEG of equivalent quality, and it answers many of these next-gen photo needs. For example, by recording 3D scene data captured with newer iPhones, photo apps can create special effects that play with the background in your selfie.

This close-up view of the above balloon image, magnified by a factor of 6, shows JPEG's visible problems at top. The technology for Apple's HEIC, center, shows more blotchy artifacts along edges than the compression for the AV1 video technology, bottom.
Enlarge Image
This close-up view of the above balloon image, magnified by a factor of 6, shows JPEG's visible problems at top. The technology for Apple's HEIC, center, shows more blotchy artifacts along edges than the compression for the AV1 video technology, bottom.

This close-up view of the above balloon image, magnified by a factor of 6, shows JPEG's visible problems at top. The technology for Apple's HEIC, center, shows more blotchy artifacts along edges than the compression for the AV1 video technology, bottom. Click to view enlarged.

Stephen Shankland/CNET

Apple promoted HEIC publicly and to developers at its 2017 developer conference. In practice, though, HEIC remains somewhat invisible. That's because Apple, recognizing that most computing hardware and software around today has no ability to display HEIC, generally converts photos into JPEG formats when they make their way out of your camera roll and into something like a Facebook post or email attachment.

But to use HEIC, makers of software, processors and phones must  jump through a lot of hoops to license patents. That's not just complicated and expensive. It also means HEIC will have trouble succeeding on the web: patent barriers are antithetical to the web's open nature.

Enter the AV1 photo format

The HEIC's new rival is from the Alliance for Open Media, a group whose top priority is a video compression technology called AV1 that's free of patent licensing requirements. It's got heavy hitters on board, including top browser makers Google, Microsoft, Mozilla and the most recent new member, Apple -- though Apple's plans haven't been made public. And it's got major streaming-video companies, too: Netflix, Amazon, Hulu, Facebook, videoconferencing powerhouse Intel and Google's YouTube. And with the support of chip designers Intel, Nvidia and Arm, AV1 should get the hardware acceleration that's crucial to making video easy on our laptop and phone batteries.

Video is the alliance's priority, and it's not yet clear which members will want to move an AV1-based photo format beyond the experimental stage. But Matt Frost, a Chrome strategy leader, says there's definitely enthusiasm.

"This is not something Google and Mozilla are cooking up. It's a request we've heard from multiple members of the alliance," Frost said.

If nothing else, AV1 photos should be a capable successor to WebP, the image format that Google introduced to speed up websites with images that typically are smaller than JPEGs or another format called PNG that's good for text and logos. WebP won over eBay, Alibaba, Yahoo and other major websites, but Google failed to convince other browser makers to support it. Already, though, AV1 photos have broader support than just the internet giant.

And 500px's Thompson is eager for something that's cheaper to store and send over networks. "We'd love to see a new format widely supported," he said. "For a photo-based site like 500px, storage and bandwidth costs are astronomical."

AV1 photo features

Outdoing JPEG is a great start. But AV1 photos can handle some other challenges, too:

These three views, edited to exaggerate otherwise difficult-to-see differences, show JPEG's imperfections at top, the significantly better technology behind Apple's HEIC image technology center, and the cleaner lines but perhaps lost textures of the AV1 photo compression technology in its current experimental state.
Enlarge Image
These three views, edited to exaggerate otherwise difficult-to-see differences, show JPEG's imperfections at top, the significantly better technology behind Apple's HEIC image technology center, and the cleaner lines but perhaps lost textures of the AV1 photo compression technology in its current experimental state.

These three views, edited to exaggerate otherwise difficult-to-see differences, show JPEG's imperfections at top, the significantly better technology behind Apple's HEIC image technology at center, and the cleaner lines but perhaps lost textures of the AV1 photo compression technology in its current experimental state.

Stephen Shankland/CNET
  • Graphical elements like logos, text and charts where JPEG is terrible.
  • Lossless compression that shrinks a photo but still preserves all its original pixels.
  • High dynamic range for a better span of bright and dark tones, something that could, for example, help Netflix show thumbnail images that better match its videos.

It's also possible the format could handle live photos -- those that package a snippet of video to give some vitality to an otherwise still shot.

"The nature of image scenarios is changing to include motion," said Gabe Frost, the alliance's executive director and a Microsoft Windows program manager. And there's demand for an AV1 photo format, he said: "AOMedia has heard strong industry feedback about participating for all the same reasons we created AV1 for video."

So far, though, AV1 photo work remains an exploration. "AOMedia is laser-focused on finalizing the AV1 video codec," the group's leader said. "The alliance doesn't have anything to announce with respect to still images."

Not so fast

One sticking point is how closely to follow the AV1 video compression format. Deviating could let AOMedia allies add photo-squeezing optimizations that aren't suitable for video that has to fly by at 30 or more frames per second, said Jai Krishnan, Google's product manager for still image technology. And new features could be added, too, like the ability to show just a portion of a huge image.

But any deviation from the video format would break hardware acceleration -- unless chipmakers added a few extra features for handling photos, too. Following AV1 video would speed completion of the AV1 photo format, but adding photo-specific features would add time. "I don't think we've nailed down which path to take," Mozilla's Terriberry said.

Apple is another unknown. It's the newest browser maker to join the AOMedia party, but it's sold on HEIC's advantages. One interesting possibility: HEIC is just one variety of a broader standard called HEIF, and it's possible the AV1 format could slot in alongside HEIC as just another variety that uses AV1 instead of HEVC technology. Apple didn't respond to a request for comment.

The biggest challenge, though, is the fact that JPEG, however imperfect, is overwhelmingly successful.

"Apple already is deploying a replacement, and other people have come up with other JPEG alternatives over the years," Terriberry said. "But JPEG has had incredible staying power."

First published Jan. 19, 5 a.m. PT.
Update, 8:52 a.m.: Adds further comment from the Alliance for Open Media.

The Smartest Stuff: Innovators are thinking up new ways to make you, and the things around you, smarter.

Rebooting the Reef: CNET dives deep into how tech can help save Australia's Great Barrier Reef.