Skip to main content

The context you need, when you need it

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

Join now

Microsoft thinks it can do a better iPhone camera app than Apple

Born in the company’s research labs, Microsoft Pix focuses on taking better pictures of people.

A side by side comparison of two shots taken simultaneously on identical iPhones, the one on the left using Microsoft Pix and the one on the right using Apple’s camera app.
A side by side comparison of two shots taken simultaneously on identical iPhones, the one on the left using Microsoft Pix and the one on the right using Apple’s camera app.
A side-by-side comparison of two shots taken simultaneously on identical iPhones, the one on the left using Microsoft Pix and the one on the right using Apple’s camera app.
Microsoft

Don’t look now, but Microsoft is becoming a serious player in the world of iPhone apps.

It has already brought over Office and subsumed well-regarded email app Acompli and calendar app Sunrise into a revamped Outlook for iPhone. Now, Microsoft is looking to offer up a rival to the built-in camera app.

Microsoft Pix, a free app from Microsoft Research, focuses first and foremost on delivering better pictures of people. It does this by continuously taking pictures when the app is open and using an algorithm to choose the best shot or shots from among 10 images (seven just before the camera button is pressed and three after).

Microsoft Pix chooses the best image from among 10 shots taken just before and after a user presses the app’s shutter button.

“If you see it happen and you aren’t already taking pictures, it’s too late, you missed the shot,” said Josh Weisberg, a Microsoft researcher and former team photographer for the Seattle Seahawks.

It turns out often the shot we really want is one just before our finger manages to hit the capture button, both in terms of composition as well as the shake that can occur when a finger hits the screen. Choosing from multiple shots also allows the app to weed out the ones where someone is blinking or goes from a smile to a grimace, serving up the best image, or the best couple of images if they are significantly different.

Microsoft uses artificial intelligence in a variety of ways to provide a better picture. It draws from stills it discards to improve the best shots by reducing noise and also automatically adjusts contrast and white balance with a focus on making sure the faces in shots look great.

The app is part of two big trends at Microsoft. First, and best publicized, has been Microsoft’s move into iOS and Android. Less well known is a big shift inside Microsoft Research, which for a long time was seen as a pure research house. Historically, one of Microsoft’s existing product groups would have to decide it liked a research concept and then do the work of commercialization. With Fix, Hyperlapse and other recent releases, Microsoft is showing it is willing to let the research team directly bring products to market.

Weisberg said that while Microsoft isn’t charging for Pix, it still reaps benefits as more consumers gain an appreciation for the company’s ability to lead in the burgeoning field of machine intelligence.

Weisberg and Microsoft Research colleague Neel Joshi showed off the app at a tricked-out loft in San Francisco’s South of Market neighborhood last week, using a specially built rig that allows two iPhones to shoot simultaneously, one using Pix and the other using Apple’s camera app.

The special rig Microsoft built to show the difference between the images captured by its app and the iPhone’s built-in camera.

Using a combination of backlighting, some breakdancers and pyrotechnics, the two showed off Pix at its best — capturing the people you want in complicated scenes and lighting conditions.

Microsoft Pix is not just for still photos. Fix also incorporates Microsoft’s Hyperlapse technology to turn any video into a high-speed time-lapse video. That works whether or not the video was shot in Microsoft Pix. Hyperlapse can also be used just to stabilize video at normal speed.

Microsoft also has its take on the Live Photo feature Apple introduced with the iPhone 6s. Microsoft’s “live images” represent the same general concept — mixing still and live imagery — but not the exact same approach. Microsoft’s versions are stored as short video clips and lack sound, but can be compelling in their own right, especially when the result fixes much of the action with only some of the image moving.

Unlike Apple Live Photos, Pix only takes live images when it thinks there is enough action to be compelling. Plus, Microsoft Pix works on the iPhone 5s, 6 and 6s, as well as newer iPads.

In some cases, Microsoft Pix creates a live image with some parts still and other parts moving.

And while there are other alternative camera apps, one thing Microsoft Pix has on its side is that it seamlessly saves images and videos in Apple’s camera roll as well as in its own gallery. Microsoft says it does so by synchronizing its photo database with Apple’s native camera roll.

Microsoft says it isn’t done with Pix and has a bunch of things on its roadmap, including adding support for Android and slower-speed video, adding panorama capabilities and bringing over the beauty mode originally developed in the Microsoft Selfie app.

Here’s a look at a shot taken at the same time on identical iPhones using the iPhone’s camera app and Microsoft Pix.

This article originally appeared on Recode.net.

More in Technology

Technology
The case for AI realismThe case for AI realism
Technology

AI isn’t going to be the end of the world — no matter what this documentary sometimes argues.

By Shayna Korol
Politics
OpenAI’s oddly socialist, wildly hypocritical new economic agendaOpenAI’s oddly socialist, wildly hypocritical new economic agenda
Politics

The AI company released a set of highly progressive policy ideas. There’s just one small problem.

By Eric Levitz
Future Perfect
Human bodies aren’t ready to travel to Mars. Space medicine can help.Human bodies aren’t ready to travel to Mars. Space medicine can help.
Future Perfect

Protecting astronauts in space — and maybe even Mars — will help transform health on Earth.

By Shayna Korol
Podcasts
The importance of space toilets, explainedThe importance of space toilets, explained
Podcast
Podcasts

Houston, we have a plumbing problem.

By Peter Balonon-Rosen and Sean Rameswaram
Technology
What happened when they installed ChatGPT on a nuclear supercomputerWhat happened when they installed ChatGPT on a nuclear supercomputer
Technology

How they’re using AI at the lab that created the atom bomb.

By Joshua Keating
Future Perfect
Humanity’s return to the moon is a deeply religious missionHumanity’s return to the moon is a deeply religious mission
Future Perfect

Space barons like Jeff Bezos and Elon Musk don’t seem religious. But their quest to colonize outer space is.

By Sigal Samuel