Adobe's AI Virtual Fitting Room Launched, Say Goodbye to Awkward Seller Shows

2020-01-23

In the past decade, virtual try-on of clothing and accessories has become popular on shopping platforms. According to a survey conducted by Klarna, 29% of shoppers prefer to browse products online before making a purchase, while 49% of consumers are interested in pre-styling options to determine the right fit before buying.


Therefore, a group of researchers from Adobe, Indian Institute of Technology, and Stanford University explored their so-called "image-based virtual try-on" technology, named SieveNet. It can map a piece of clothing onto a virtual object, preserving the characteristics of the clothing, including wrinkles and folds, without causing texture blurring or bleeding.SieveNet aims to capture images of clothing and human body models and generate new images of models wearing the clothing while preserving the original body shape, pose, and other details. To achieve this, it employs a multi-stage technique that involves deforming the clothing to match the pose and shape of the body model, and then transferring the deformed texture onto the model.


Deforming clothing requires considering shape or pose variations between clothing images, as well as occlusions in the model image (e.g., hair or crossed arms). Dedicated modules in SieveNet can predict coarse transformations and fine corrections on top of earlier coarse transformations, while another module can compute rendered images and masks on the human body model.


In experiments conducted using a PC with 16GB RAM and four Nvidia 1080Ti graphics cards, the researchers trained SieveNet on a dataset consisting of approximately 19,000 images of frontal female models and upper-body clothing products. They reported that in qualitative tests, the system handled occlusions, pose variations, bleeding, geometric distortions, and overall quality preservation better than the baseline. Additionally, it achieved state-of-the-art results on qualitative metrics, including Fréchet Inception Distance (FID), which measures the similarity between the target distribution and the evaluated system (in this case, SieveNet) by capturing important features using AI object recognition.SieveNet is not the first of its kind. L'Oréal's ModiFace (recently featured in Amazon's mobile app) allows customers to test different shades of lipstick on their own real-time photos and videos. Vue.ai's AI system masters the characteristics of clothing, learns to generate realistic poses, skin tones, and other features, and produces model images of various sizes five times faster than traditional photoshoots. And both Gucci and Nike offer apps that enable people to try on shoes.


However, the researchers assert that systems like SieveNet can be more easily integrated into existing applications and websites. They write, "Virtual try-on, visualizing fashion products in personalized settings, is particularly important for online fashion transactions as it compensates for the lack of direct physical store shopping experience. Compared to the current state-of-the-art methods for image-based virtual try-on, we have made significant improvements."

RECENT AI NEWS

RECENT AI TOOLS