News Release
Your AI stylist will see you now
 |
| A flow chart showing how Complete the Look works. |
Computer scientists help teach an AI-powered tool a sense of style
San Diego, Calif., Feb. 13, 2020 -- Do you ever wonder what shoes or purse you should wear with your outfit? Well, have no fear: AI is here to help. Computer scientists at UC San Diego, in collaboration with Pinterest, developed “Complete the Look,” an AI-powered tool that recommends accessories and other fashion items to match your outfit based on just one photo.
Most recommendation systems for clothing are based on pictures of individual pieces of clothing or accessories (think product pictures on Amazon). But they lack important information, such as the season, the person’s body type and the clothes they usually wear--the kind of information that can be extracted from real-life images (think selfies). Most systems also focus on whether items are similar, as opposed to whether they are compatible.
To train the Complete the Look tool, computer science Ph.D. student Wang-Cheng Kang used data from Pinterest’s Shop the Look service, which finds clothing items for sale similar to those displayed in any given picture. Kang was an intern with the company at the time. The process was a little bit like helping the AI solve a puzzle. First researchers divided the images into pieces: clothes, shoes, accessories, and so on. Then they taught the AI which pieces are compatible to make a complete puzzle.
To test the tool’s performance, researchers had the AI solve more of these puzzles, this time without help. Complete the Look was able to identify compatible items 86.5 to 75.3 percent of the time for fashion and 80 percent of the time for home decor--better than all other similar computer tools.
In addition, researchers wanted to know how well Complete the Look did when compared to a real person. So they asked four fashion experts to complete the same compatibility task. They did not fare better than the tool.
Complete the Look still needs to be refined further before fashionistas--and the rest of us--can start using it. But in one aspect at least, it’s met the goal researchers set out to achieve. “The model appears to have learned a complex notion of style,” researchers write.
Kang is part of the research group led by computer science professor Julian McAuley, whose work focuses on better understanding how people behave and express opinions in online communities, including Amazon, Yelp, Reddit and Facebook. One application of his work is improving recommendation algorithms that power the likes of Amazon and Netflix.
Complete the Look: Scene-based Complementary Product Recommendation
Wang-Cheng Kang, Julian McAuley, UC San Diego
Eric Kim, Charles Rosenberg, Pinterest
Jure Leskovec Pinterest and Stanford University
CVPR conference, 2019
Media Contacts
Ioana Patringenaru
Jacobs School of Engineering
858-822-0899
ipatrin@ucsd.edu