# FastChat

## Docs

- [FastChat](https://fastchat.mintlify.app/introduction.md): FastChat is an open platform for training, serving, and evaluating large language model based chatbots
- [Install](https://fastchat.mintlify.app/install.md)
- [API](https://fastchat.mintlify.app/api-reference.md)
- [Chatbot Arena](https://fastchat.mintlify.app/arena.md): Chatbot Arena is an LLM benchmark platform featuring anonymous, randomized battles, available at https://chat.lmsys.org. We invite the entire community to join this benchmarking effort by contributing your votes and models.
- [AWQ 4bit Inference](https://fastchat.mintlify.app/awq.md): We integrated [AWQ](https://github.com/mit-han-lab/llm-awq) into FastChat to provide **efficient and accurate** 4bit LLM inference.
- [GPTQ 4bit Inference](https://fastchat.mintlify.app/gptq.md): Support GPTQ 4bit inference with [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
- [Local LangChain with FastChat](https://fastchat.mintlify.app/langchain_integration.md): [LangChain](https://python.langchain.com/en/latest/index.html) is a library that facilitates the development of applications by leveraging large language models (LLMs) and enabling their composition with other sources of computation or knowledge. FastChat's OpenAI-compatible [API server](openai_api)…
- [Model Support](https://fastchat.mintlify.app/model_support.md)
- [FastChat Server Architecture](https://fastchat.mintlify.app/server_arch.md)
- [Fine-tuning FastChat-T5](https://fastchat.mintlify.app/training.md): You can use the following command to train FastChat-T5 with 4 x A100 (40GB).
- [Vicuna Weights](https://fastchat.mintlify.app/vicuna_weights_version.md)
- [vLLM Integration](https://fastchat.mintlify.app/vllm_integration.md): You can use [vLLM](https://vllm.ai/) as an optimized worker implementation in FastChat. It offers advanced continuous batching and a much higher (~10x) throughput. See the supported models [here](https://vllm.readthedocs.io/en/latest/models/supported_models.html).

## Optional

- [Discord](https://discord.gg/HSWAKCrnFx)
- [Demo](https://chat.lmsys.org/)