Amazon Launches Diffuse to Choose: Virtual Product Try-On Made Real

Online shopping can often be unpredictable because it is not easy to visually see the products before purchasing. Amazon hopes to change this by introducing Diffuse to Choose (DTC), a new AI system that allows you to virtually try on products in any environment.

In a new paper published by Amazon researchers, DTC allows shoppers to seamlessly blend product images into their personal photos. This creates a "virtual try-on" experience where items appear realistically integrated into the desired scene or setting.

The paper explains, "Recent diffusion models inherently contain a world model that makes them suitable for performing this task in a context-aware manner. However, traditional image-conditioned diffusion models often fail to capture the fine details of products."

DTC addresses this issue by adopting a novel diffusion-based image-conditioned inpainting model. The model uniquely balances fast inference with preserving high-fidelity details, ensuring accurate semantic manipulation in any given scene. Here, the use of a secondary U-Net encoder to inject fine-grained details into the diffusion process is a key innovation. This process involves masking the source image and inserting the reference image into the masked region, followed by a series of complex adaptations and alignments within the model.

The paper validates DTC on proprietary and public datasets, demonstrating its advantages over Paint By Example, a popular diffusion-based inpainting technique, in both quantitative and qualitative aspects. Importantly, DTC matches few-shot personalized methods without the need for fine-tuning on each product.

DTC has broad and diverse practical implications. For consumers, it means being able to place any item from an online store into their own living space or personal environment, and view the product's combination and appearance in real-time. This not only enhances the shopping experience but also helps make more informed purchasing decisions. For retailers, it offers a new level of customer interaction by providing more accurate product representations, potentially reducing return rates.

Early examples show that DTC is capable of realistically blending various products, from clothing to furniture and decor. Consistent lighting and shadows make the items truly appear in the inserted scenes.

DTC also supports iterative virtual decoration and styling. Users can adjust the mask to change the clothing style, such as tucking in a shirt or rolling up sleeves. This flexibility positions DTC as an engaging platform for personalized e-commerce experiences.

Amazon is not the only company exploring virtual try-on options. Last year, Google AI also conducted similar research with their TryonDiffusion model.

While Google's research primarily focused on enhancing the realism of clothing, Amazon's DTC aims to work across product categories. Both highlight the growing interest in using AI to create more immersive experiences for online shopping.

Amazon states that it will soon release the code and demos. If the research lives up to its promises, integrating tools like DTC into the online shopping experience could lead the next evolution of e-commerce.