Meta launches AI system 3D Gen, rapidly generating high-quality 3D assets

Meta has launched a revolutionary AI system called 3D Gen, which can generate stunning high-quality 3D assets based on text descriptions in less than a minute. This innovation represents a significant leap forward for AI in the field of 3D content creation, promising unprecedented convenience and efficiency for game developers, artists, and industry experts in building virtual worlds.

The birth of 3D Gen is based on Meta's two heavyweight research papers: Meta 3D AssetGen and Meta 3D TextureGen, which respectively address the technical challenges of generating 3D from text and refining textures, setting new benchmarks for the industry.

Specifically, 3D Gen employs a two-stage strategy to create 3D assets. In the first stage, AssetGen takes the spotlight by quickly generating 3D meshes with physically based rendering (PBR) materials based on input text instructions. This process cleverly transforms abstract 2D concepts into tangible 3D structures, achieving a qualitative leap.

"AssetGen not only generates the basic shape of objects but also considers details such as shadows and reflectance. This dual strategy allows our system to accurately capture the 3D structure and material characteristics, resulting in higher quality 3D assets," explained a researcher from Meta.

In the second stage, TextureGen takes over and optimizes the initially generated textures. It ingeniously combines view space and UV space generation techniques to ensure that the textures not only match the original text description but also exhibit a high level of coherence and detail richness on the 3D model.

What makes 3D Gen remarkable is not only its astonishing speed but also its outstanding adaptability. Whether it's a simple object, a complex character, or a detailed scene, it can accurately generate them according to user instructions with extremely high fidelity, earning high praise from professional 3D artists and designers.

In practical applications, 3D Gen demonstrates tremendous potential. Game developers can use it to quickly build game scenes and character prototypes, greatly accelerating the game development cycle. Architectural visualization companies can easily generate detailed 3D models of buildings and interior environments based on text descriptions, optimizing the design process. In the field of virtual reality and augmented reality, 3D Gen provides powerful assistance in creating immersive environments and objects, potentially accelerating the arrival of the metaverse era.