How to Choose a Generative AI Model
With so many generative AI tools on the market, software companies must determine the right models for their use cases.
Technologists must weigh several competing factors to find the right solution:
- Cost
- Domain knowledge
- Ease of adoption
- Privacy
- Data prerequisites
That said, technologists should consider different models for different purposes. Rather than sticking to a single model, teams should first determine what they’re trying to accomplish and choose a model accordingly.
While there are many nuances and variations in generative AI models, we’ll group them into public and custom (or private) models to simplify the discussion. Deciding between a public and custom model is one of the most significant decisions in leveraging generative AI, as it will determine the type of outputs you’ll receive and how you can use them.
Let’s dive in.
Public Generative AI Models
Public generative AI models like ChatGPT are available for companies to leverage as-is. While the barrier to entry is incredibly low (all you need is an account and the ability to ask questions to get started), public models are quite generic and may pose some risks.
Because ChatGPT has been trained on the entirety of the internet, it struggles with specialization, nuance, and complex “thinking,” especially when tasked with niche problems. A human expert in the field is necessary to guide ChatGPT to useful outputs, to verify them, and to turn the raw text into a polished final version.
Additionally, organizations risk copyright infringement if models generate code or content wholesale from a copyrighted source—a problem that’s not often apparent to the end user (seen in this lawsuit against Microsoft). Organizations also risk open-sourcing their software or content if it’s not sufficiently human-directed, as AI outputs cannot be copyrighted as of May 2023.
There are other risks besides liability. While OpenAI does allow organizations to turn off logging, the default ChatGPT settings capture inputs and outputs for training data—meaning proprietary information can be accessed by other parties.
Organizations should also consider the implications of relying too much on public models. Because the outputs tend to be generic (even with advanced prompting), organizations risk losing a competitive advantage if other entities take advantage of these models in the same way. Even if content and code outputs aren’t identical, public models struggle with true innovation that can set your products and company apart.
That said, tools like ChatGPT are great introductions to generative AI. The natural language interface is incredibly intuitive, and a quick way to start familiarizing and training teams on this emerging technology. Companies can immediately ramp up and start gaining the benefits of generative AI without investing the significant time and money into custom models.
Another benefit of public models is the vast library of plug-ins that are constantly developed by third-parties. Plug-ins can manage the heavy lifting of specialized tasks without the significant investment involved in building custom models; that said, these plug-ins will have their own limitations and security risks.
Pros
- No or low cost
- Allows you to start immediately
- Accessible APIs
- Intuitive approach
Cons
- Doesn’t add unique value to your organization
- Potential security and IP risks
- Potential liabilities with unlicensed training data
Custom Generative AI Models
Custom generative AI models are trained on your company’s data, enabling much more subtle and nuanced responses specific to your industry and organization. With custom models, AI can act in a much more advanced capacity, replicating the work of a human in the field.
For example, a chat support feature could be enhanced with custom models trained on an entire library of product release notes, technical documentation, and internal guidelines. Armed with all internal knowledge, the chatbot can serve customers in a way that simply isn’t possible with public models.
Custom models can be built from scratch or on top of open source models.
There are two approaches to augment an existing model:
- Fine-Tuning: Re-train an existing model with specific weights based on your organization, using internal data. As a result, the model can respond with the same capabilities of ChatGPT, but with more specificity.
- Embedding: Less expensive than fine-tuning, embedding also produces a less refined result. It narrows the scope of what a model can respond to, and the model can’t respond if asked something outside of its knowledge base. Embeddings are large vectors that summarize text, used to inject context into the model without extensive training.
Both of these approaches have their benefits, and the right choice depends on your use case. However, they also add significant value not achieved by public models. Custom models offer a stronger competitive advantage because of their ability to specialize and generate unique outputs based on your data.
That said, the quality of outputs depends on the quality of your data. Without strong data governance in place, your models will be ineffective, prone to hallucinations or the inability to respond. Thus, organizations must have a solid foundation before they can take advantage of custom models, adding significant time and cost increases for those who need to catch up.
Pros
- Domain expertise
- Better responses that mimic a human in the field
- Enables monetization
- Enhanced security and IP protection
- Long-term cost savings
Cons
- High upfront costs that increase with the amount of data
- Requires robust data governance
- Time-consuming to build
Learn more about the difference between custom and public models with our CTO Roundtable Recap: Harnessing Generative AI in Software Development.
Choosing a Generative AI Model for Your Use Case
By understanding the types of models available, organizations can better serve their customers and employees with this emerging technology. The various capabilities have near endless use cases, but they each require a different model and approach:
- An HR assistant that can answer policy questions for employees → Need a custom model trained on internal policies
- Support chatbots that can answer level one technical questions → Need a custom model trained on internal documentation
- Marketing assistance for content generation → Public models can suffice, with significant editing and revision from a human to minimize risk of copyright infringement.
- Software development assistance for code generation, verification, and optimization → Public models can suffice, with significant revision from a human. However, a custom model would be far superior by understanding internal code structure and protecting your IP.
Each organization will have different needs and different standards when it comes to protecting proprietary information. Ultimately, a generative AI consultant can help guide you to the right decision.
How KMS Can Help
It’s nearly impossible to keep up with the rapidly shifting world of generative AI, the litigation in flight, and the new tools available while working your day job.
Our generative AI consultants can help. Our dedicated experts are on top of the latest research and developments in generative AI, with decades of experience in implementing a myriad of AI solutions. We’re driven by real, practical use cases that create ROI—not hype.
We can help you roadmap and build your generative AI solutions, all while protecting your IP and data. In fact, we’ve written guidance on how best to do just that.
Talk to our team to learn more.
