Challenges and Best Practices of Generative AI in Software Development

2023-09-12

Rethinking Software Construction

Jon Noronha, co-founder of Gamma, said, "The biggest advantage and disadvantage of LLM (large language models) is their creativity. Creativity is great, but it also means unpredictability. You can ask LLM the same question and get very different answers based on subtle differences in wording."

Companies building production applications around LLM face challenges in predictable troubleshooting, software testing, and monitoring.

He added, "When building these applications at scale, we found that we need to rethink the entire software development process and try to create analogies to traditional practices like debugging and monitoring for LLM. The way to solve this problem will be different, but it requires a new generation of infrastructure tools to help development teams understand how their LLM performs in widespread applications."

This is a new technology, said Irfan Ganchi, Chief Product Officer at Oportun, and engineers encounter new problems every day. For example, considering the length of time it takes to train LLM, especially when training it on your own knowledge base and trying to maintain brand consistency across various touchpoints in different contexts.

He said, "You need to filter almost everything on the input and output sides, placing humans in the loop to validate and ensure consistency with the content generated by AI. There is still a long way to go, but it is a promising technology."

Working with LLM is different from software engineering, added Shailesh Nalawadi, Product Lead at Sendbird.

He said, "LLM is not software engineering. It is not deterministic. Small changes in input can lead to completely different outputs. What makes it more challenging is that you cannot trace LLM to figure out why it produced a specific output, which is something we as software engineers traditionally could do. Polishing LLM and putting it into production involves a lot of trial and error. And then, there are no tools, test automation, and CI/CD processes for updating LLM. When launching applications based on generative AI built on top of LLM today, we need to be aware of all the missing pieces and proceed with caution."

Misconceptions about Using Generative AI in Production

Nalawadi said one of the biggest misconceptions is that many people think LLM is similar to Google search: a database with real-time indexed information. Unfortunately, that is not correct. LLM is usually trained on datasets that could be 6 to 12 to 18 months old. To make it respond to users with specific desired information, users need to prompt the model with specific information from the data.

He said, "This means that in an enterprise setting, enabling the right prompts and ensuring that all information relevant to the desired response is packaged will be crucial. Guided engineering here is a very relevant and important topic."

Another misconception comes from the term itself, said Noronha. The term "generative" implies creating from scratch, which can be interesting but not usually where the greatest business value lies.

He said, "We find that generative almost always combines with your own starting data and then combines with generative AI. The key is bridging these two worlds, the creative, unpredictable model and the structure and knowledge you already have. In many ways, I think 'transformative AI' is a better term because the real value comes from here."

Best Practices for Applying Generative AI in Production

When applying generative AI, it is crucial to be very intentional, said Ganchi, and have a basic strategy within the organization and the ability to gradually test its value within the organization.

He said, "We find that once generative AI is introduced, there are concerns at both the employee and organizational executive levels. How to be goal-oriented? How to consciously face it? You need a strategy to gradually test, demonstrate value, and improve productivity within the organization."

Before deploying, Nalawadi added, you need to establish infrastructure to evaluate the performance of the generative AI system.

He said, "Does it produce the desired output? Does it meet expectations? Is it satisfactory? Perhaps a human evaluation framework is needed. Then, keep this gold standard as you develop LLM and improve prompts and ensure that it is indeed improving. Use it not just relying on qualitative metrics to assess its performance. Plan well and ensure you have testing infrastructure and a quantifiable evaluation framework."

In many ways, the most important part is choosing which problems to apply generative AI to, said Noronha.

He said, "There will certainly be some problems, but everyone wants to sprinkle AI magic dust on their product, rather than everyone thinking about where to place it correctly. We are looking for those tasks that no one is doing or no one wants to do, like formatting presentations. I encourage finding such cases and leveraging them fully. Another thing we bring when we focus on these aspects is that it not only changes efficiency but also guides people to create things they wouldn't have created before."

RECENT AI NEWS

RECENT AI TOOLS