Stability AI Unveils Third-Generation Image Generation Model: Stable Diffusion 3

Stability AI has released an early preview version of Stable Diffusion 3.0, its next-generation text-to-image generation AI model, aimed at significantly improving image quality, spelling ability, and performance of multi-topic prompts.

Over the past year, the company has steadily improved its image synthesis capabilities through multiple model iterations. Stable Diffusion 3.0 is a major upgrade that adopts a new diffusion transformer architecture and flow matching, a simulation-free method for training models, which is expected to accelerate the performance of diffusion models.

Emad Mostaque, CEO of Stability AI, said on X, "Stable Diffusion 3 is a diffusion transformer, a new architecture similar to the one used in the recent OpenAI Sora model. It is the true successor to the original Stable Diffusion."

While transformers have become the mainstay of natural language models, diffusion transformers operate specifically on generating images by manipulating latent image patches. Combined with flow matching, Stability AI points out that these methods can improve training speed, sampling efficiency, and overall output quality.

Accurate text rendering has always been a challenge for image synthesis models (although DALLE-3 performs well in this regard). Mostaque emphasized the improvements in spelling of the new model, which stem from its transformer architecture and additional text encoders. He said, "Complete sentences can now be generated, and the style is coherent."

The company states that the parameter range of the Stable Diffusion 3 series models ranges from 800M to 8B, providing users with a range of quality and scalability options, and promises to release detailed technical reports soon.

Although introduced as a text-to-image model, Mostaque stated that Stable Diffusion 3.0 will serve as the foundation for Stability AI's upcoming video, 3D, and multimodal generation AI systems. "We create open models that can be used anywhere and adapt to any needs," he said.

Mostaque also took the opportunity to share the editing and control features that the company will soon launch. In a demonstration video, he showcased various advanced granularity control features that users will have when modifying specific parts of an image.