In the rapidly evolving generative AI landscape, OpenAI has revolutionized the way developers build prototypes, create demos, and achieve remarkable results with large language models (LLMs). However, when it’s time to put LLMs into production, organizations are increasingly moving away from commercial LLMs like OpenAI in favor of fine-tuned open-source models. What’s driving this shift, and why are developers embracing it?

The primary motivations are simple: 1. efficiency, and 2. avoiding vendor lock-in while safeguarding intellectual property associated with both the data and models. Open-source models like Llama2 and Mistral now match and, in some cases, even surpass commercial LLMs in performance, while also boasting a significantly smaller size. The shift towards open-source models not only ensures substantial cost savings, but it also grants developers greater control and oversight over their models.

Safeguarding Intellectual Property and Avoiding Vendor Lock-In

For most organizations, commercial LLMs are a black box, since they fail to provide access to the model source code or the ability to export model artifacts. Relying solely on black box models accessible through an API is no longer ideal for mission-critical and commercial applications. Organizations must ascertain model ownership and differentiate their product from competitors, while retaining their AI and data intellectual property. According to a recent survey by my company, three-quarters of respondents would not be comfortable using a commercial LLM in production. These respondents cited ownership, privacy, and cost as their primary concerns.

Ensuring compliance and privacy remains paramount, and developers are faced with the challenge of verifying that end-user data is protected from malicious entities when passed into a black box system. Moreover, reliance on third-party platforms raises concerns about latency and maintaining production-grade service-level agreements for commercial applications (SLAs). Finally, business leaders increasingly see AI as core to their IP, and they increasingly see customized models with proprietary data as a key differentiator that will set them apart from competitors. Put simply, enterprises are no longer happy with the idea of entrusting intellectual property to a third party and being just a thin layer on top of someone else’s API.

Specialized Models: Performance and Cost Efficiency

Once considered lacking in performance, open-source models have experienced a remarkable transformation through fine-tuning, and they are now emerging as powerful contenders. Fine-tuned open-source models are now meeting, if not exceeding, commercial models’ level of performance, while retaining a substantially smaller footprint. 

This represents a massive opportunity, since productionizing massive commercial LLMs has caused difficulties for numerous organizations due to the LLMs’ size and associated costs. By leveraging fine-tuned models, developers can achieve excellent results while dealing with models that are two to three orders of magnitude smaller than their commercial counterparts, and therefore significantly cheaper and faster. 

Consider the case of an organization using an LLM to process hundreds of thousands of messages from front-line workers. The organization could reduce their costs by utilizing a fine-tuned model rather than a large-scale LLM. The ability to achieve remarkable results at a fraction of the cost makes fine-tuning an attractive option for organizations seeking to optimize their AI implementations.

Conclusion

The transition from OpenAI to open-source models represents the next phase for companies seeking to retain ownership of their information and models, ensure privacy, and avoid vendor lock-in. Open-source models, as they continue to evolve, offer an attractive alternative for developers who aspire to introduce AI in production environments. In the era of custom AI, specialized models not only deliver optimal performance but also drive considerable cost reductions, pointing towards a bright future.

However, challenges remain in terms of simplifying and managing the fine-tuning process, establishing robust production infrastructure, and ensuring the quality, reliability, safety, and ethics of AI applications. To address these challenges, innovative platforms offer declarative solutions that assist organizations in building custom AI applications. By providing easy-to-use fine-tuning capabilities and production-ready infrastructure, these platforms empower organizations to unlock the tremendous potential of open-source models while maintaining the utmost control and achieving optimal performance.