Google partners with Dr. Ellis Monk to improve skin tone representation across its products and technology: Digital Photography Review

As Tulsee Doshi, Head of Product for Responsible AI and Product Inclusion in Search at Google, writes, ‘Seeing yourself reflected in the world around you — in real life, media or online — is so important.’ Last year, Google announced Real Tone for Pixel, a color science technology that helps the cameras inside the Pixel 6 capture more accurate images of subjects, no matter their skin tone. Now, Google is furthering its goal of inclusion via a partnership with Harvard professor and sociologist Dr. Ellis Monk, whose helped design a new skin tone scale meant to be more inclusive of the spectrum of skin tones we see in the real world.

The Monk Skin Tone Scale is a free development tool from Google Responsible AI. Studies show products using current artificial intelligence (AI) and machine learning (ML) technologies can ‘perpetuate unfair biases and not work well for people with darker skin tones.’ Google is using the MST Scale to improve the computer vision (CV) technology that allows AI to ‘see and understand’ images of people and environments. Building more inclusive CV systems will help produce AI technology that better reflects real people—all people.

Doshi continues, ‘The culmination of Dr. Monk’s research is the MST (MST) Scale, a 10-shade scale that will be incorporated into various Google products over the coming months. We’re openly releasing the scale so anyone can use it for research and product development. Our goal is for the scale to support inclusive products and research across the industry — we see this as a chance to share, learn and evolve our work with the help of others.’

Simply updating its approach to skin tones has allowed Google to understand representation in imagery better and evaluate its current suite of products and technologies. By incorporating the MST Scale, Google has improved the effectiveness of its computer vision technology and built more representative datasets to train its AI models.

Google Search results will incorporate new features using the MST Scale that make it easier for people to find more relevant and helpful results. For example, if you’re searching for makeup results in Google Images, you can refine your results by skin tone to find results that better match your skin tone. Not every makeup approach works equally well for all skin tones, and it’s important for users, especially those with darker skin tones, to find results relevant to their needs.

Google will incorporate the MST Scale in its search results, allowing users to filter results based on skin tone, allowing users to easily find results relevant to their needs.

In the coming months, creators, brands and publishers will be provided new tools to label their web content more accurately. For example, you will be able to provide information about a person’s hair texture and skin tone in web content. Like with skin tone, there’s a wide range of hair textures that may affect the usefulness of search results for someone using Google. Users want, and should be able to find, results relevant to them.

Google will also use the MST Scale to improve auto-enhancing Real Tone filters in Google Photos. These filters use AI to improve skin and the overall look of a photo. Doshi writes, ‘We’ll also be using the MST Scale to improve Google Photos. Last year, we introduced an improvement to our auto enhance feature in partnership with professional image makers. Now we’re launching a new set of Real Tone filters that are designed to work well across skin tones and evaluated using the MST Scale. We worked with a diverse range of renowned image makers, like Kennedi Carter and Joshua Kissi, who are celebrated for beautiful and accurate depictions of their subjects, to evaluate, test and build these filters. These new Real Tone filters allow you to choose from a wider assortment of looks and find one that reflects your style. Real Tone filters will be rolling out on Google Photos across Android, iOS and Web in the coming weeks.’

Google is openly releasing the MST Scale and wants as many companies as possible to use it and provide feedback. Google is continuing its work with Dr. Monk to evaluate the MST Scale across different regions and products.