Spellburst: A Large-Scale Language Model Interactive Canvas for Generative Artists

2023-09-18

Creative artists engage in coding work. They use programming languages such as Processing or AI text-to-image tools to convert expressive semantics into lines of code, forming swirling, colorful patterns or surreal landscapes.

However, coding art is a time-consuming and complex process. While an eraser on a pencil can correct erroneous lines or a touch of yellow can brighten a dark horizon in a painting, improving generated art requires trial and error through multiple iterations, which can be a frustrating process.

After interviewing professional digital artists, scholars at Stanford developed a tool called Spellburst to improve the creative ideation and editing process.

Hariharan Subramonyam, an assistant professor at the Stanford Graduate School of Education and a researcher at the Stanford Institute for Human-Centered Artificial Intelligence, said, "Translating an artist's imagination into code takes a long time and is very challenging."

"Large language models can provide you with a good starting point. However, when artists want to explore different textures, colors, or patterns, they want to have more precise control, which large language models cannot provide. Spellburst essentially helps artists seamlessly switch between semantic space and code."

Spellburst uses the large language model GPT-4, allowing artists to input an initial prompt, such as "an image of a beautiful and bright stained glass window." The model then generates the code to render that concept. But what if the flowers are too pink or the stained glass window doesn't look right? Artists can open a dynamic slider panel to change any aspect of the image based on previous prompts or add modification instructions ("turn the flowers into deep red").

These creators can merge different versions ("combine the flower color from version 4 with the vase shape from version 9"). The tool also allows artists to transition from prompt-based exploration to program editing, where they can click on the image to display the code for more precise fine-tuning.

"A Greater Leap in Creativity"

To better design Spellburst, the research team interviewed 10 professional creative coders to understand how they develop concepts, create workflows, and the biggest challenges they face. Subsequently, the team tested the tool with professional generative artists.

Subramonyam said, "The feedback has been overall positive. The large language model helps artists transition from semantic space to code more quickly, and it also helps them explore many different variations and make a greater leap in creativity."

Of course, the tool itself has some limitations. The research team observed errors and unexpected results in some prompts, especially during version merging, and it is unclear which prompts will produce the desired results. Additionally, the sample size of artists providing feedback is small and may not represent the entire generative artist community.

However, Subramonyam expressed hope that this tool will be helpful to coding artists and even a broader user base.

"We hope to open-source this tool to artists later this year, so they can start using it, and we also hope to study how such tools can help beginners learn how to create art with code."

RECENT AI NEWS

RECENT AI TOOLS