codefuse-chatbot/sources/readme_docs/start-en.md

112 lines
3.9 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

If you need to deploy a privatized model, please install the NVIDIA driver yourself.
### Preparation of Python environment
- It is recommended to use conda to manage the python environment (optional)
```bash
# Prepare conda environment
conda create --name Codefusegpt python=3.9
conda activate Codefusegpt
```
- Install related dependencies
```bash
cd Codefuse-ChatBot
pip install -r requirements.txt
```
### Sandbox Environment Preparation
- Windows Docker installation:
[Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/) supports 64-bit versions of Windows 10 Pro with Hyper-V enabled (Hyper-V is not required for versions v1903 and above), or 64-bit versions of Windows 10 Home v1903 and above.
- [【全面详细】Windows10 Docker安装详细教程](https://zhuanlan.zhihu.com/p/441965046)
- [Docker 从入门到实践](https://yeasy.gitbook.io/docker_practice/install/windows)
- [Handling 'Docker Desktop requires the Server service to be enabled'](https://blog.csdn.net/sunhy_csdn/article/details/106526991)
- [安装wsl或者等报错提示](https://learn.microsoft.com/zh-cn/windows/wsl/install)
- Linux Docker installation
Linux installation is relatively simple, please search Baidu/Google for installation guides.
- Mac Docker installation
- [Docker 从入门到实践](https://yeasy.gitbook.io/docker_practice/install/mac)
```bash
# Build the image for the sandbox environment, see above for notebook version issues
bash docker_build.sh
```
### Model Download (Optional)
If you need to use open-source LLM and Embedding models, you can download them from HuggingFace.
Here we take THUDM/chatglm2-6b and text2vec-base-chinese as examples:
```
# install git-lfs
git lfs install
# install LLM-model
git lfs clone https://huggingface.co/THUDM/chatglm2-6b
cp ~/THUDM/chatglm2-6b ~/codefuse-chatbot/llm_models/
# install Embedding-model
git lfs clone https://huggingface.co/shibing624/text2vec-base-chinese
cp ~/shibing624/text2vec-base-chinese ~/codefuse-chatbot/embedding_models/
```
### Basic Configuration
```bash
# Modify the basic configuration for service startup
cd configs
cp model_config.py.example model_config.py
cp server_config.py.example server_config.py
# model_config#11~12 If you need to use the OpenAI interface, the OpenAI interface key
os.environ["OPENAI_API_KEY"] = "sk-xxx"
# Replace with the api_base_url you need
os.environ["API_BASE_URL"] = "https://api.openai.com/v1"
# vi model_config#LLM_MODEL The language model you need to choose
LLM_MODEL = "gpt-3.5-turbo"
LLM_MODELs = ["gpt-3.5-turbo"]
# vi model_config#EMBEDDING_MODEL The private vector model you need to choose
EMBEDDING_ENGINE = 'model'
EMBEDDING_MODEL = "text2vec-base"
# Example of vector model access, modify model_config#embedding_model_dict
# If the model directory is:
model_dir: ~/codefuse-chatbot/embedding_models/shibing624/text2vec-base-chinese
# Configure as follows
"text2vec-base": "shibing624/text2vec-base-chinese"
# vi server_config#8~14, It's recommended to use a container to start the service to prevent environment conflicts when installing other dependencies using the codeInterpreter feature
DOCKER_SERVICE = True
# Whether to use a container sandbox
SANDBOX_DO_REMOTE = True
```
### Starting the Service
By default, only the webui-related services are started, and fastchat is not started (optional).
```bash
# If you need to support the codellama-34b-int4 model, you need to patch fastchat
# cp examples/gptq.py ~/site-packages/fastchat/modules/gptq.py
# Modify examples/llm_api.py#258 to kwargs={"gptq_wbits": 4},
# start llm-service (optional)
python examples/llm_api.py
```
For more LLM integration methods, see[more details...](sources/readme_docs/fastchat-en.md)
<br>
```bash
# After completing the server_config.py configuration, you can start with one click
cd examples
python start.py
```