Contributor: [Your Name]
This project aims to provide a lightweight alternative to OpenAI's paid API for deploying Language Models (LLMs) with an OpenAI compatible API. It offers the advantage of greater control over the deployed LLMs and their usage. The project also involves developing applications based on Langchain that utilize these APIs. Additionally, the project focuses on comparing various LLMs' performance, response quality, and other key performance indicators (KPIs). The ultimate goal is to gain insights into the cost-effectiveness of using LLMs along with the applications.
- ➜ Deploy OpenAI-compatible Language Models (LLMs) using an accessible API. start12/08
- Develop applications using the Langchain framework, leveraging the LLM APIs.
- Compare the performance and quality of responses from different LLMs.
- Assess the costs associated with utilizing LLMs in conjunction with applications.
- FastChat: Offers a user-friendly interface for deploying a range of popular LLMs, including Vicuna and potentially LLaMA. It supports PERF (Performance Enhancement and Response Fidelity).
- TBD: Explore other methods for easy LLM deployment.
- Vicuna: Chosen for its compatibility with FastChat. Note that while it's considered safe, its quality is relatively lower with the 7b model, particularly regarding Oneshot_REACT agents' adherence to the REACT framework.
- LLaMA-2: [Add reasoning for choosing LLaMA-2]
- Dolly-2: [Add reasoning for choosing Dolly-2]
- Falcon: [Add reasoning for choosing Falcon]
[Provide instructions for getting started with the project, including setup, installation, and any prerequisites.]
[Explain how to use the project, including API endpoints, examples, and expected responses.]
Contributions are welcome! If you'd like to contribute to this project, please follow the guidelines in CONTRIBUTING.md.
Initial commit by Ashish GCP, [Date of Initial Commit]. [Add any additional credits for resources or contributions.]
This project is licensed under the License Name.