Microsoft Launches Data Formulator: A Concept-Driven Visual Authoring Tool

Microsoft launches Data Formulator: a concept-driven visual authoring tool that uses AI agents to address data transformation challenges in visual creation.

Data visualization represents data in graphical or image formats, helping people understand patterns, trends, and insights in data. Effective data visualization plays a crucial role in data analysis. It enables data analysts to explore complex datasets, understand patterns, and communicate meaningful insights to various stakeholders. Today, there are many tools available to create visual representations of data. However, data must be transformed into a structured format to create visualizations.

Because this requires programming experience or separate data processing tools, data transformation remains a barrier in visual creation. It requires a certain level of programming expertise or familiarity with additional data processing tools. It highlights the complexity of data visualization and emphasizes the need for data analysts to have a simpler, more seamless process to create impactful visualizations, regardless of their technical background.

Therefore, researchers have made significant progress in overcoming data visualization barriers. Their collaborative efforts in AI and human-computer interaction (HCI) have resulted in an innovative tool called Data Formulator: an AI-driven visual authoring tool that introduces a breakthrough paradigm for data representation.

Data Formulator aims to simplify the complex process of data visualization. Compared to traditional approaches, this tool separates advanced visualization concepts from the details of data processing. Data analysts describe their visualization ideas as "data concepts" - specific fields or categories they want to see represented, even if they don't exist in the original data. By expressing their ideas in this way, analysts help AI agents understand their intentions, allowing the agents to associate concepts with relevant visuals to facilitate the visualization process.

Data Formulator recognizes that analysts may not be clear about what they want, so it presents different ways of visualizing data. It gives analysts many choices, making everything more transparent. Moreover, Data Formulator provides AI programs for analysts to manipulate data and generate new data itself. This makes it easier for analysts to examine everything and come up with better ideas for the next iteration.

AI agents dispatched by Data Formulator transform input data to highlight these concepts and create the desired visualizations. When presenting the results, Data Formulator provides feedback to help authors understand and inspect the transformed data and visualizations.

Data Formulator adopts two different approaches based on analysts' input: instance-based concept definition and natural language queries. For the former, program synthesizers generate specialized data shaping programs, while the latter invokes language models (LLMs) to generate code that creates the described new data categories. The resulting transformed data is compiled into structured tables alongside the corresponding visualizations.

In addition to visualization, this approach can also be used for data cleaning, integration, exploration, and storytelling. An ideal AI system would follow high-level instructions, suggest actions throughout the data analysis pipeline, and encourage collaboration between users and AI agents to achieve data visualization goals. Researchers state that creating AI tools capable of successfully conveying results to analysts - even in unclear, ambiguous, or inaccurate situations - is crucial for such collaboration.