Stability AI Launches Its First Japanese Image Vision Model: Japanese InstructBLIP Alpha

2023-09-04

Stability AI has released its first Japanese visual language model, Japanese InstructBLIP Alpha. It can generate text descriptions for input images and generate answers based on the images. Japanese InstructBLIP Alpha is a visual language model that can generate conditional text for given images based on the recently released Japanese large language model, Japanese StableLM Instruct Alpha 7B. Japanese InstructBLIP Alpha utilizes the InstructBLIP architecture, which has shown excellent performance on various visual language datasets. To create a high-performing model, we used a portion of InstructBLIP pretrained on a large English dataset as the initial value for the model and then fine-tuned it using a limited Japanese dataset. Examples of applications for this model include image-based search engines, scene description/question-answering, and image text description for the visually impaired. In terms of performance, in addition to generating Japanese text, Japanese InstructBLIP Alpha can accurately identify objects specific to Japan, such as Tokyo Skytree or Kinkaku-ji. Furthermore, the input can also include text, such as questions. For example, as shown in the following example, it can answer questions about the input image. The model can be used for inference and additional training on the Hugging Face Hub.

RECENT AI NEWS

RECENT AI TOOLS