Google releases MediaPipe Image Generator now available on Android devices

A few months ago, Google announced the release of the MediaPipe diffusion plugin, which is now available as an experimental tool on high-end Android devices. The plugin, called Image Generator, is capable of generating images entirely on the device, taking approximately 15 seconds on high-end devices. The new MediaPipe Image Generator can generate images based on textual prompts using standard diffusion models. The Image Generator API supports any model that conforms to the stable diffusion v1.5 architecture. In addition to using pre-trained models, you can also fine-tune your own models using the conversion scripts provided by Google and convert them into supported model formats. This allows for fine-tuning the injected image into the model, providing better control over the generation process and the resulting image. Furthermore, you can use low-rank adaptation (LoRA) weights to create images with specific predefined concepts. To directly use the diffusion model, you pass the textual prompt to the Image Generator API by first creating an object and passing the path of the base model file on the device to the constructor. Once you have the instance, you pass the prompt, number of iterations, and seed value to it and obtain the generated image. ```kotlin val options = ImageGeneratorOptions.builder() .setImageGeneratorModelDirectory(MODEL_PATH) .build() val imageGenerator = ImageGenerator.createFromOptions(context, options) val result = imageGenerator.generate(prompt_string, iterations, seed) val bitmap = BitmapExtractor.extract(result?.generatedImage) ``` Additionally, you can use Google's newly developed plugin system to simplify the process of passing adjustments to the generated image. "We currently support three different ways to provide a basis for your generation: face structure, edge detection, and depth perception. These plugins allow you to provide an image and extract specific structures from it, then use these structures to create new images." Google provides multiple plugins that should be used in conjunction with the base model, each customized for specific desired effects. Specifically, the Canny Edge plugin utilizes the edges implied by the conditional image and generates a new image based on the textual prompt. The Face Landmark plugin provides a detailed facial mesh for a single face and generates new faces on the mesh. Lastly, the Depth plugin infers the size and depth of the object to be generated based on the conditional image. Each plugin supports multiple options that can be customized to modify their behavior. Using LoRA, the Image Generator can also be extended beyond the base model by teaching it new concepts. "With the new LoRA weights, the Image Generator becomes a specialized generator capable of injecting specific concepts into the generated images." For example, you can create LoRA weights for several given themes of images and then use these weights to generate new images of the same theme in different environments.