GPT4All# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Gpt4all

GPT4All#
GitHub：nomic-ai/gpt4all (opens in a new tab) 是一个基于大量干净的助手数据集训练的开源聊天机器人生态系统，其中包括代码、故事和对话。
此示例介绍如何使用LangChain与GPT4All模型交互。

```code
%pip install pygpt4all > /dev/null

```

```code
Note: you may need to restart the kernel to use updated packages.

```

```code
from langchain import PromptTemplate, LLMChain
from langchain.llms import GPT4All
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

指定模型#
要在本地运行，请下载兼容的ggml格式模型。有关更多信息，请访问https://github.com/nomic-ai/pygpt4all (opens in a new tab)
有关完整的安装说明，请单击此处 (opens in a new tab)。
GPT4All Chat安装程序需要在安装过程中解压缩3GB的LLM模型！
请注意，新模型会定期上传——请查看上面的链接以获取最新的.bin URL

```code
local_path = './models/ggml-gpt4all-l13b-snoozy.bin'  # replace with your desired local file path

```

取消下面的块以下载模型。您可能需要更新
```code
url
```
以获取新版本。

```code
# import requests

# from pathlib import Path
# from tqdm import tqdm

# Path(local_path).parent.mkdir(parents=True, exist_ok=True)

# # Example model. Check https://github.com/nomic-ai/pygpt4all for the latest models.
# url = 'http://gpt4all.io/models/ggml-gpt4all-l13b-snoozy.bin'

# # send a GET request to the URL to download the file. Stream since it's large
# response = requests.get(url, stream=True)

# # open the file in binary mode and write the contents of the response to it in chunks
# # This is a large file, so be prepared to wait.
# with open(local_path, 'wb') as f:
# for chunk in tqdm(response.iter_content(chunk_size=8192)):
# if chunk:
# f.write(chunk)

```

```code
# Callbacks support token-wise streaming
callbacks = [StreamingStdOutCallbackHandler()]
# Verbose is required to pass to the callback manager
llm = GPT4All(model=local_path, callbacks=callbacks, verbose=True)

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)

```
Gooseai ExampleHuggingface Hub


抱抱脸（Huggingface）# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Huggingface Hub

抱抱脸（Huggingface）#
抱抱脸（Huggingface） (opens in a new tab)是一个平台，拥有超过120k个模型、20k个数据集和50k个演示应用程序（空间)，所有内容都是开源和公开的，在这个在线平台上，人们可以轻松合作和构建机器学习。
此示例展示了如何连接到抱抱脸（Huggingface）。
要使用，您应该安装了
```code
huggingface_hub
```
的python软件包 (opens in a new tab)。

```code
!pip install huggingface_hub > /dev/null

```

```code
# get a token: https://huggingface.co/docs/api-inference/quicktour#get-your-api-token

from getpass import getpass

HUGGINGFACEHUB_API_TOKEN = getpass()

```

```code
import os
os.environ["HUGGINGFACEHUB_API_TOKEN"] = HUGGINGFACEHUB_API_TOKEN

```

选择模型

```code
from langchain import HuggingFaceHub

repo_id = "google/flan-t5-xl" # See https://huggingface.co/models?pipeline_tag=text-generation&sort=downloads for some other options

llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

```

```code
from langchain import PromptTemplate, LLMChain

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=llm)

question = "Who won the FIFA World Cup in the year 1994? "

print(llm_chain.run(question))

```

示例#
以下是通过抱抱脸（Huggingface）集成可以访问的一些模型示例。
由稳定性AI提供的StableLM#
请参阅稳定性AI (opens in a new tab)的组织页面以获取可用模型列表。

```code
repo_id = "stabilityai/stablelm-tuned-alpha-3b"
# Others include stabilityai/stablelm-base-alpha-3b
# as well as 7B parameter versions

```

```code
llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

```

```code
# Reuse the prompt and question from above.
llm_chain = LLMChain(prompt=prompt, llm=llm)
print(llm_chain.run(question))

```

DataBricks的Dolly#
请查看DataBricks (opens in a new tab)组织页面，了解可用模型列表。

```code
from langchain import HuggingFaceHub

repo_id = "databricks/dolly-v2-3b"

llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

```

```code
# Reuse the prompt and question from above.
llm_chain = LLMChain(prompt=prompt, llm=llm)
print(llm_chain.run(question))

```

Writer的Camel#
请查看Writer (opens in a new tab)组织页面，了解可用模型列表。

```code
from langchain import HuggingFaceHub

repo_id = "Writer/camel-5b-hf" # See https://huggingface.co/Writer for other options
llm = HuggingFaceHub(repo_id=repo_id, model_kwargs={"temperature":0, "max_length":64})

```

```code
# Reuse the prompt and question from above.
llm_chain = LLMChain(prompt=prompt, llm=llm)
print(llm_chain.run(question))

```

还有更多！Gpt4allHuggingface Pipelines


抱抱脸（Huggingface）本地管道# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Huggingface Pipelines

抱抱脸（Huggingface）本地管道#
Hugging Face 模型可以通过
```code
HuggingFacePipeline
```
类在本地运行。
Hugging Face 模型中心 (opens in a new tab) 托管超过 120k 个模型、20k 个数据集和 50k 个演示应用程序（Spaces)，全部都是开源且公开可用的，是一个在线平台，人们可以轻松协作和构建机器学习。
这些模型可以通过本地管道包装器或通过 HuggingFaceHub 类调用其托管的推断端点从 LangChain 中调用。有关托管管道的更多信息，请参见 HuggingFaceHub 教程。
要使用，您应该安装
```code
transformers
```
python 包。 (opens in a new tab)

```code
!pip install transformers > /dev/null

```

加载模型#

```code
from langchain import HuggingFacePipeline

llm = HuggingFacePipeline.from_model_id(model_id="bigscience/bloom-1b7", task="text-generation", model_kwargs={"temperature":0, "max_length":64})

```

```code
WARNING:root:Failed to default session, using empty session: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /sessions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1117f9790>: Failed to establish a new connection: [Errno 61] Connection refused'))

```

将模型集成到LLMChain中#

```code
from langchain import PromptTemplate,  LLMChain

template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

question = "What is electroencephalography?"

print(llm_chain.run(question))

```

```code
/Users/wfh/code/lc/lckg/.venv/lib/python3.11/site-packages/transformers/generation/utils.py:1288: UserWarning: Using `max_length`'s default (64) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.
warnings.warn(
WARNING:root:Failed to persist run: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /chain-runs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x144d06910>: Failed to establish a new connection: [Errno 61] Connection refused'))

```

```code
First, we need to understand what is an electroencephalogram. An electroencephalogram is a recording of brain activity. It is a recording of brain activity that is made by placing electrodes on the scalp. The electrodes are placed

```
Huggingface HubLlamacpp


Llama-cpp# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Llamacpp

Llama-cpp#
llama-cpp (opens in a new tab) 是 llama.cpp (opens in a new tab) 的 Python 绑定。
它支持 多个 LLMs (opens in a new tab)。
本教程介绍如何在 LangChain 中运行
```code
llama-cpp
```
。

```code
!pip install llama-cpp-python

```

请确保您遵循所有说明以安装所有必要的模型文件 (opens in a new tab)。
您不需要一个
```code
API_TOKEN
```
！

```code
from langchain.llms import LlamaCpp
from langchain import PromptTemplate, LLMChain
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
# Callbacks support token-wise streaming
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
# Verbose is required to pass to the callback manager

# Make sure the model path is correct for your system!
llm = LlamaCpp(
model_path="./ggml-model-q4_0.bin", callback_manager=callback_manager, verbose=True
)

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Bieber was born?"

llm_chain.run(question)

```

```code
First we need to identify what year Justin Beiber was born in. A quick google search reveals that he was born on March 1st, 1994. Now we know when the Super Bowl was played in, so we can look up which NFL team won it. The NFL Superbowl of the year 1994 was won by the San Francisco 49ers against the San Diego Chargers.

```

```code
' First we need to identify what year Justin Beiber was born in. A quick google search reveals that he was born on March 1st, 1994. Now we know when the Super Bowl was played in, so we can look up which NFL team won it. The NFL Superbowl of the year 1994 was won by the San Francisco 49ers against the San Diego Chargers.'

```
Huggingface PipelinesManifest


Manifest 清单# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Manifest

Manifest 清单#
本教程介绍了如何使用Manifest和LangChain。
有关更详细的信息
```code
清单
```
，以及如何像本示例中一样在本地hugginface模型中使用它，请参见https://github.com/HazyResearch/manifest (opens in a new tab)
使用Manifest和Langchain的另一个示例。 (opens in a new tab)

```code
!pip install manifest-ml

```

```code
from manifest import Manifest
from langchain.llms.manifest import ManifestWrapper

```

```code
manifest = Manifest(
client_name = "huggingface",
client_connection = "http://127.0.0.1:5000"
)
print(manifest.client.get_model_params())

```

```code
llm = ManifestWrapper(client=manifest, llm_kwargs={"temperature": 0.001, "max_tokens": 256})

```

```code
# Map reduce example
from langchain import PromptTemplate
from langchain.text_splitter import CharacterTextSplitter
from langchain.chains.mapreduce import MapReduceChain

_prompt = """Write a concise summary of the following:

{text}

CONCISE SUMMARY:"""
prompt = PromptTemplate(template=_prompt, input_variables=["text"])

text_splitter = CharacterTextSplitter()

mp_chain = MapReduceChain.from_params(llm, prompt, text_splitter)

```

```code
with open('../../../state_of_the_union.txt') as f:
state_of_the_union = f.read()
mp_chain.run(state_of_the_union)

```

```code
'President Obama delivered his annual State of the Union address on Tuesday night, laying out his priorities for the coming year. Obama said the government will provide free flu vaccines to all Americans, ending the government shutdown and allowing businesses to reopen. The president also said that the government will continue to send vaccines to 112 countries, more than any other nation. "We have lost so much to COVID-19," Trump said. "Time with one another. And worst of all, so much loss of life." He said the CDC is working on a vaccine for kids under 5, and that the government will be ready with plenty of vaccines when they are available. Obama says the new guidelines are a "great step forward" and that the virus is no longer a threat. He says the government is launching a "Test to Treat" initiative that will allow people to get tested at a pharmacy and get antiviral pills on the spot at no cost. Obama says the new guidelines are a "great step forward" and that the virus is no longer a threat. He says the government will continue to send vaccines to 112 countries, more than any other nation. "We are coming for your'

```

比较HF模型#

```code
from langchain.model_laboratory import ModelLaboratory

manifest1 = ManifestWrapper(
client=Manifest(
client_name="huggingface",
client_connection="http://127.0.0.1:5000"
),
llm_kwargs={"temperature": 0.01}
)
manifest2 = ManifestWrapper(
client=Manifest(
client_name="huggingface",
client_connection="http://127.0.0.1:5001"
),
llm_kwargs={"temperature": 0.01}
)
manifest3 = ManifestWrapper(
client=Manifest(
client_name="huggingface",
client_connection="http://127.0.0.1:5002"
),
llm_kwargs={"temperature": 0.01}
)
llms = [manifest1, manifest2, manifest3]
model_lab = ModelLaboratory(llms)

```

```code
model_lab.compare("What color is a flamingo?")

```

```code
Input:
What color is a flamingo?

ManifestWrapper
Params: {'model_name': 'bigscience/T0_3B', 'model_path': 'bigscience/T0_3B', 'temperature': 0.01}
pink

ManifestWrapper
Params: {'model_name': 'EleutherAI/gpt-neo-125M', 'model_path': 'EleutherAI/gpt-neo-125M', 'temperature': 0.01}
A flamingo is a small, round

ManifestWrapper
Params: {'model_name': 'google/flan-t5-xl', 'model_path': 'google/flan-t5-xl', 'temperature': 0.01}
pink

```
LlamacppModal


使用LangChain与Modal交互# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Modal

使用LangChain与
```code
Modal
```
交互#
Modal Python Library (opens in a new tab)提供了方便的、按需的从本地计算机上的Python脚本访问无服务器云计算的途径。

```code
Modal
```
本身并不提供任何LLMs，只提供基础设施。
这个例子介绍了如何使用LangChain与
```code
Modal
```
交互。
这里 (opens in a new tab)是另一个使用LangChain与
```code
Modal
```
交互的例子。

```code
!pip install modal-client

```

```code
# register and get a new token

!modal token new

```

```code
[?25lLaunching login page in your browser window...
[2KIf this is not showing up, please copy this URL into your web browser manually:
[2Km⠙ Waiting for authentication in the web browser...
]8;id=417802;https://modal.com/token-flow/tf-ptEuGecm7T1T5YQe42kwM1\[4;94mhttps://modal.com/token-flow/tf-ptEuGecm7T1T5YQe42kwM1]8;;\

[2K⠙ Waiting for authentication in the web browser...
[1A[2K^C

Aborted.

```

请按照这些说明 (opens in a new tab)处理密钥。

```code
from langchain.llms import Modal
from langchain import PromptTemplate, LLMChain

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
llm = Modal(endpoint_url="YOUR_ENDPOINT_URL")

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```
ManifestNlpcloud


NLP云# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Nlpcloud

NLP云#
NLP云 (opens in a new tab)提供高性能的预训练或自定义模型，用于命名实体识别、情感分析、分类、摘要、改写、文法和拼写纠正、关键词和关键词短语提取、聊天机器人、产品描述和广告生成、意图分类、文本生成、图像生成、博客文章生成、代码生成、问答、自动语音识别、机器翻译、语言检测、语义搜索、语义相似度、标记化、词性标注、嵌入和依赖解析。它已经准备好投入生产，通过REST API提供服务。
此示例介绍如何使用LangChain与
```code
NLP Cloud
```
模型 (opens in a new tab)交互。

```code
!pip install nlpcloud

```

```code
# get a token: https://docs.nlpcloud.com/#authentication

from getpass import getpass

NLPCLOUD_API_KEY = getpass()

```

```code
import os

os.environ["NLPCLOUD_API_KEY"] = NLPCLOUD_API_KEY

```

```code
from langchain.llms import NLPCloud
from langchain import PromptTemplate, LLMChain

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
llm = NLPCloud()

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```

```code
' Justin Bieber was born in 1994, so the team that won the Super Bowl that year was the San Francisco 49ers.'

```
ModalOpenai


Openai – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Openai

OpenAI (opens in a new tab)提供了不同级别的模型，适用于不同的任务。
此示例介绍了如何使用LangChain与
```code
OpenAI
```
models (opens in a new tab) 进行交互。

```code
# get a token: https://platform.openai.com/account/api-keys

from getpass import getpass

OPENAI_API_KEY = getpass()
```

```code
import os

os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
```

```code
from langchain.llms import OpenAI
from langchain import PromptTemplate, LLMChain
```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])
```

```code
llm = OpenAI()
```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)
```

```code
' Justin Bieber was born in 1994, so we are looking for the Super Bowl winner from that year. The Super Bowl in 1994 was Super Bowl XXVIII, and the winner was the Dallas Cowboys.'
```
NlpcloudPetals Example


Petals Example – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Petals Example

```code
Petals
```
以BitTorrent方式在家中运行超过100B的语言模型。
本教程介绍如何使用Langchain和Petals (opens in a new tab)。
安装Petals#
要使用Petals API，需要安装
```code
petals
```
包。使用
```code
pip3 install petals
```
进行安装。

```code
!pip3 install petals

```

导入#

```code
import os
from langchain.llms import Petals
from langchain import PromptTemplate, LLMChain

```

设置环境API密钥#
请确保从抱抱脸（Huggingface）获取API密钥 (opens in a new tab)。

```code
from getpass import getpass

HUGGINGFACE_API_KEY = getpass()

```

```code
os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY

```

Create the Petals instance#
You can specify different parameters such as the model name, max new tokens, temperature, etc.

```code
# this can take several minutes to download big files!

llm = Petals(model_name="bigscience/bloom-petals")

```

```code
Downloading:   1%|▏                        | 40.8M/7.19G [00:24<15:44, 7.57MB/s]

```

创建提示词模板Create a Prompt Template#
我们将添加一个QA提示词模板
We will create a prompt template for Question and Answer.

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

初始化LLMChain Initiate the LLMChain#

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

启动LLMChain Run the LLMChain#
启动LLMChain问一个问题。

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```
OpenaiPipelineai Example


Pipeline – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Pipelineai Example

Pipeline
PipelineAI允许您在云中规模运行您的ML模型。它还提供API访问多个LLM模型 (opens in a new tab)。
本教程介绍了如何使用PipelineAI (opens in a new tab)来使用Langchain。
安装pipeline-ai#
使用
```code
pip install pipeline-ai
```
安装
```code
pipeline-ai
```
库是使用
```code
PipelineAI
```
API，也称为
```code
Pipeline Cloud
```
所必需的。

```code
# Install the package
!pip install pipeline-ai

```

导入#

```code
import os
from langchain.llms import PipelineAI
from langchain import PromptTemplate, LLMChain

```

设置环境API密钥#
Make sure to get your API key from PipelineAI. Check out the cloud quickstart guide (opens in a new tab). You’ll be given a 30 day free trial with 10 hours of serverless GPU compute to test different models.

```code
os.environ["PIPELINE_API_KEY"] = "YOUR_API_KEY_HERE"

```

实例化PipelineAI #
当实例化PipelineAI时，您需要指定要使用的管道的ID或标签，例如
```code
pipeline_key = "public/gpt-j:base"
```
。
然后，您可以选择传递其他与管道特定的关键字参数：

```code
llm = PipelineAI(pipeline_key="YOUR_PIPELINE_KEY", pipeline_kwargs={...})

```

问答提示模板#
我们将创建一个问答提示模板。

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

初始化LLMChain#

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

Run the LLMChain#
提供一个问题并运行LLMChain。

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```
Petals ExamplePredictionguard


如何使用 PredictionGuard wrapper – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Predictionguard

如何使用 PredictionGuard wrapper

```code
! pip install predictionguard langchain

```

```code
import predictionguard as pg
from langchain.llms import PredictionGuard

```

基本的LLM用法#

```code
pgllm = PredictionGuard(name="default-text-gen", token="<your access token>")

```

```code
pgllm("Tell me a joke")

```

链#

```code
from langchain import PromptTemplate, LLMChain

```

```code
template = """Question: {question}

Answer: Let's think step by step."""
prompt = PromptTemplate(template=template, input_variables=["question"])
llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.predict(question=question)

```

```code
template = """Write a {adjective} poem about {subject}."""
prompt = PromptTemplate(template=template, input_variables=["adjective", "subject"])
llm_chain = LLMChain(prompt=prompt, llm=pgllm, verbose=True)

llm_chain.predict(adjective="sad", subject="ducks")

```
Pipelineai ExamplePromptlayer Openai


PromptLayer – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Promptlayer Openai

PromptLayer

```code
PromptLayer
```
是第一个允许您跟踪、管理和共享GPT提示工程的平台。
```code
PromptLayer
```
充当您的代码和
```code
OpenAI Python
```
库之间的中间件。

```code
PromptLayer
```
记录所有您的
```code
OpenAI API
```
请求，允许您在
```code
PromptLayer
```
仪表板中搜索和探索请求历史记录。
此示例演示了如何连接到[PromptLayer]（https://www.promptlayer.com)，以开始记录您的OpenAI请求。 (opens in a new tab)
另一个示例在[这里]（https://python.langchain.com/en/latest/ecosystem/promptlayer.html (opens in a new tab)) 。
安装PromptLayer#
要使用PromptLayer与OpenAI，需要安装
```code
promptlayer
```
包。使用pip安装
```code
promptlayer
```
。

```code
!pip install promptlayer

```

引入Imports#

```code
import os
from langchain.llms import PromptLayerOpenAI
import promptlayer

```

创建一个PromptLayer API密钥#
您可以在www.promptlayer.com (opens in a new tab)上单击导航栏中的设置齿轮，创建一个PromptLayer API密钥。
将其设置为名为
```code
PROMPTLAYER_API_KEY
```
的环境变量。
您还需要一个名为
```code
OPENAI_API_KEY
```
的OpenAI密钥。

```code
from getpass import getpass

PROMPTLAYER_API_KEY = getpass()

```

```code
os.environ["PROMPTLAYER_API_KEY"] = PROMPTLAYER_API_KEY

```

```code
from getpass import getpass

OPENAI_API_KEY = getpass()

```

```code
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

```

使用 PromptLayerOpenAI LLM#
您可以选择传递
```code
pl_tags
```
，以便使用PromptLayer的标记功能跟踪您的请求。

```code
llm = PromptLayerOpenAI(pl_tags=["langchain"])
llm("I am a cat and I want")

```

上述请求现在应该出现在您的PromptLayer仪表板 (opens in a new tab)中。
使用PromptLayer Track#
如果您想要使用任何PromptLayer跟踪功能 (opens in a new tab)，您需要在实例化PromptLayer LLM时传递参数
```code
return_pl_id
```
以获取请求ID。

```code
llm = PromptLayerOpenAI(return_pl_id=True)
llm_results = llm.generate(["Tell me a joke"])

for res in llm_results.generations:
pl_request_id = res[0].generation_info["pl_request_id"]
promptlayer.track.score(request_id=pl_request_id, score=100)

```

使用此功能可以让您在PromptLayer仪表板中跟踪您的模型性能。如果您正在使用提示模板，您还可以将模板附加到请求中。
总的来说，这为您提供了在PromptLayer仪表板中跟踪不同模板和模型性能的机会。PredictionguardReplicate


Replicate – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Replicate

Replicate

Replicate (opens in a new tab)在云端运行机器学习模型。我们拥有一系列开源模型，只需几行代码即可运行。如果您正在构建自己的机器学习模型，Replicate可以轻松实现大规模部署。

这个例子介绍了如何使用LangChain与
```code
Replicate
```
模型 (opens in a new tab)进行交互。
设置#
要运行此教程电脑，您需要创建一个Replicate (opens in a new tab)账户并安装replicate python客户端 (opens in a new tab)。

```code
!pip install replicate

```

```code
# get a token: https://replicate.com/account

from getpass import getpass

REPLICATE_API_TOKEN = getpass()

```

```code
········

```

```code
import os

os.environ["REPLICATE_API_TOKEN"] = REPLICATE_API_TOKEN

```

```code
from langchain.llms import Replicate
from langchain import PromptTemplate, LLMChain

```

调用模型#
在复制探索页面（replicate explore page） (opens in a new tab)上找到一个模型，然后按照
```code
model_name/version
```
的格式将模型名称和版本粘贴在此处。
例如，对于此dolly模型 (opens in a new tab)，点击API选项卡。模型名称/版本将是：
```code
replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5
```
。
只需要
```code
model
```
参数，但是我们可以在初始化时添加其他模型参数。
例如，如果我们运行稳定扩散并想要更改图像尺寸：

```code
Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf", input={'image_dimensions': '512x512'})

```

请注意，模型仅返回第一个输出。

```code
llm = Replicate(model="replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5")

```

```code
prompt = """
Answer the following yes/no question by reasoning step by step.
Can a dog drive a car?
"""
llm(prompt)

```

```code
'The legal driving age of dogs is 2. Cars are designed for humans to drive. Therefore, the final answer is yes.'

```

我们可以使用此语法调用任何复制模型。例如，我们可以调用stable-diffusion。

```code
text2image = Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf",
input={'image_dimensions': '512x512'})

```

```code
image_output = text2image("A cat riding a motorcycle by Picasso")
image_output

```

```code
'https://replicate.delivery/pbxt/Cf07B1zqzFQLOSBQcKG7m9beE74wf7kuip5W9VxHJFembefKE/out-0.png'

```

该模型会输出一个URL。让我们将其呈现出来。

```code
from PIL import Image
import requests
from io import BytesIO

response = requests.get(image_output)
img = Image.open(BytesIO(response.content))

img

```

链式 Chaining Calls#
Langchain的整个重点在于...链式！这里有一个示例，说明如何做到这一点。

```code
from langchain.chains import SimpleSequentialChain

```

首先，让我们将这个模型的LLM定义为flan-5，将text2image定义为stable-diffusion。

```code
dolly_llm = Replicate(model="replicate/dolly-v2-12b:ef0e1aefc61f8e096ebe4db6b2bacc297daf2ef6899f0f7e001ec445893500e5")
text2image = Replicate(model="stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf")

```

链中的第一个提示

```code
prompt = PromptTemplate(
input_variables=["product"],
template="What is a good name for a company that makes {product}?",
)

chain = LLMChain(llm=dolly_llm, prompt=prompt)

```

链中的第二个提示，获得公司描述的logo。

```code
second_prompt = PromptTemplate(
input_variables=["company_name"],
template="Write a description of a logo for this company: {company_name}",
)
chain_two = LLMChain(llm=dolly_llm, prompt=second_prompt)

```

第三个提示，根据第二个提示输出的描述来创建图片

```code
third_prompt = PromptTemplate(
input_variables=["company_logo_description"],
template="{company_logo_description}",
)
chain_three = LLMChain(llm=text2image, prompt=third_prompt)

```

现在让我们运行它！

```code
# Run the chain specifying only the input variable for the first chain.
overall_chain = SimpleSequentialChain(chains=[chain, chain_two, chain_three], verbose=True)
catchphrase = overall_chain.run("colorful socks")
print(catchphrase)

```

```code
> Entering new SimpleSequentialChain chain...
novelty socks
todd & co.
https://replicate.delivery/pbxt/BedAP1PPBwXFfkmeD7xDygXO4BcvApp1uvWOwUdHM4tcQfvCB/out-0.png

> Finished chain.
https://replicate.delivery/pbxt/BedAP1PPBwXFfkmeD7xDygXO4BcvApp1uvWOwUdHM4tcQfvCB/out-0.png

```

```code
response = requests.get("https://replicate.delivery/pbxt/eq6foRJngThCAEBqse3nL3Km2MBfLnWQNd0Hy2SQRo2LuprCB/out-0.png")
img = Image.open(BytesIO(response.content))
img

```
Promptlayer OpenaiRunhouse


Runhouse – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Runhouse

Runhouse
Runhouse (opens in a new tab) 允许在环境和用户之间进行远程计算和数据处理。请参阅 Runhouse docs (opens in a new tab)。
此示例介绍了如何使用LangChain和 Runhouse (opens in a new tab)，与托管在您自己的GPU上，或在AWS，GCP，AWS或Lambda上提供的按需GPU交互的模型。
注意：此代码中使用
```code
SelfHosted
```
而非
```code
Runhouse
```
作为名称。

```code
!pip install runhouse

```

```code
from langchain.llms import SelfHostedPipeline, SelfHostedHuggingFaceLLM
from langchain import PromptTemplate, LLMChain
import runhouse as rh

```

```code
INFO | 2023-04-17 16:47:36,173 | No auth token provided, so not using RNS API to save and load configs

```

```code
# For an on-demand A100 with GCP, Azure, or Lambda
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1", use_spot=False)

# For an on-demand A10G with AWS (no single A100s on AWS)
# gpu = rh.cluster(name='rh-a10x', instance_type='g5.2xlarge', provider='aws')

# For an existing cluster
# gpu = rh.cluster(ips=['<ip of the cluster>'],
# ssh_creds={'ssh_user': '...', 'ssh_private_key':'<path_to_key>'},
# name='rh-a10x')

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
llm = SelfHostedHuggingFaceLLM(model_id="gpt2", hardware=gpu, model_reqs=["pip:./", "transformers", "torch"])

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```

```code
INFO | 2023-02-17 05:42:23,537 | Running _generate_text via gRPC
INFO | 2023-02-17 05:42:24,016 | Time to send message: 0.48 seconds

```

```code
"  Let's say we're talking sports teams who won the Super Bowl in the year Justin Beiber"

```

您还可以通过SelfHostedHuggingFaceLLM接口加载更多自定义模型：

```code
llm = SelfHostedHuggingFaceLLM(
model_id="google/flan-t5-small",
task="text2text-generation",
hardware=gpu,
)

```

```code
llm("What is the capital of Germany?")

```

```code
INFO | 2023-02-17 05:54:21,681 | Running _generate_text via gRPC
INFO | 2023-02-17 05:54:21,937 | Time to send message: 0.25 seconds

```

```code
'berlin'

```

使用自定义加载函数，我们可以直接在远程硬件上加载自定义流水线：

```code
def load_pipeline():
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline  # Need to be inside the fn in notebooks
model_id = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
pipe = pipeline(
"text-generation", model=model, tokenizer=tokenizer, max_new_tokens=10
)
return pipe

def inference_fn(pipeline, prompt, stop = None):
return pipeline(prompt)[0]["generated_text"][len(prompt):]

```

```code
llm = SelfHostedHuggingFaceLLM(model_load_fn=load_pipeline, hardware=gpu, inference_fn=inference_fn)

```

```code
llm("Who is the current US president?")

```

```code
INFO | 2023-02-17 05:42:59,219 | Running _generate_text via gRPC
INFO | 2023-02-17 05:42:59,522 | Time to send message: 0.3 seconds

```

```code
'john w. bush'

```

您可以直接通过网络将您的流水线发送给您的模型，但这仅适用于小模型
```code
（'<2 Gb'）
```
，并且速度较慢：

```code
pipeline = load_pipeline()
llm = SelfHostedPipeline.from_pipeline(
pipeline=pipeline, hardware=gpu, model_reqs=model_reqs
)

```

相反，我们还可以将其发送到硬件的文件系统，这将更快。

```code
rh.blob(pickle.dumps(pipeline), path="models/pipeline.pkl").save().to(gpu, path="models")

llm = SelfHostedPipeline.from_pipeline(pipeline="models/pipeline.pkl", hardware=gpu)

```
ReplicateSagemaker


SageMaker – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Sagemaker

SageMaker
Amazon SageMaker (opens in a new tab) 是一个系统，可以使用完全托管的基础设施、工具和工作流程构建、训练和部署任何用例的机器学习（ML)模型。
本教程将介绍如何使用托管在
```code
SageMaker endpoint
```
上的LLM。

```code
!pip3 install langchain boto3

```

设置#
您必须设置
```code
SagemakerEndpoint
```
调用的以下必需参数：

```code
endpoint_name
```
：已部署的Sagemaker模型的端点名称。必须在AWS区域内是唯一的。

```code
credentials_profile_name
```
：位于~/.aws/credentials或~/.aws/config文件中的配置文件名称，其中指定了访问密钥或角色信息。
如果未指定，将使用默认凭据文件配置文件或，如果在EC2实例上，则使用来自IMDS的凭据。
参见：https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html (opens in a new tab)

Example#

```code
from langchain.docstore.document import Document

```

```code
example_doc_1 = """
Peter and Elizabeth took a taxi to attend the night party in the city. While in the party, Elizabeth collapsed and was rushed to the hospital.
Since she was diagnosed with a brain injury, the doctor told Peter to stay besides her until she gets well.
Therefore, Peter stayed with her at the hospital for 3 days without leaving.
"""

docs = [
Document(
page_content=example_doc_1,
)
]

```

```code
from typing import Dict

from langchain import PromptTemplate, SagemakerEndpoint
from langchain.llms.sagemaker_endpoint import ContentHandlerBase
from langchain.chains.question_answering import load_qa_chain
import json

query = """How long was Elizabeth hospitalized?
"""

prompt_template = """Use the following pieces of context to answer the question at the end.

{context}

Question: {question}
Answer:"""
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
)

class ContentHandler(ContentHandlerBase):
content_type = "application/json"
accepts = "application/json"

def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:
input_str = json.dumps({prompt: prompt, **model_kwargs})
return input_str.encode('utf-8')

def transform_output(self, output: bytes) -> str:
response_json = json.loads(output.read().decode("utf-8"))
return response_json[0]["generated_text"]

content_handler = ContentHandler()

chain = load_qa_chain(
llm=SagemakerEndpoint(
endpoint_name="endpoint-name",
credentials_profile_name="credentials-profile-name",
region_name="us-west-2",
model_kwargs={"temperature":1e-10},
content_handler=content_handler
),
prompt=PROMPT
)

chain({"input_documents": docs, "question": query}, return_only_outputs=True)

```
RunhouseStochasticai


StochasticAI# – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Stochasticai

StochasticAI#

随机加速平台 (opens in a new tab)旨在简化深度学习模型的生命周期。

从上传和版本控制模型，到训练、压缩和加速，最终投入生产。

本示例介绍如何使用LangChain与
```code
StochasticAI
```
模型进行交互。
您需要在这里 (opens in a new tab)获取API_KEY和API_URL。

```code
from getpass import getpass

STOCHASTICAI_API_KEY = getpass()

```

```code
import os

os.environ["STOCHASTICAI_API_KEY"] = STOCHASTICAI_API_KEY

```

```code
YOUR_API_URL = getpass()

```

```code
from langchain.llms import StochasticAI
from langchain import PromptTemplate, LLMChain

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
llm = StochasticAI(api_url=YOUR_API_URL)

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```

```code
"  Step 1: In 1999, the St. Louis Rams won the Super Bowl.  Step 2: In 1999, Beiber was born.  Step 3: The Rams were in Los Angeles at the time.  Step 4: So they didn't play in the Super Bowl that year.\n"

```
SagemakerWriter


Writer – LangChain中文网

6大核心模块（Modules)模型（Models)LLMS集成（Integrations）Writer

Writer
Writer (opens in a new tab) 是一个生成不同语言内容的平台。
本示例将介绍如何使用LangChain与
```code
Writer
```
models (opens in a new tab)进行交互。
您需要从此处 (opens in a new tab)获取
```code
WRITER_API_KEY
```
。

```code
from getpass import getpass

WRITER_API_KEY = getpass()

```

```code
import os

os.environ["WRITER_API_KEY"] = WRITER_API_KEY

```

```code
from langchain.llms import Writer
from langchain import PromptTemplate, LLMChain

```

```code
template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate(template=template, input_variables=["question"])

```

```code
# If you get an error, probably, you need to set up the "base_url" parameter that can be taken from the error log.

llm = Writer()

```

```code
llm_chain = LLMChain(prompt=prompt, llm=llm)

```

```code
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

```
Stochasticai文本嵌入（Text Embedding）


文本嵌入模型 text-embedding-model# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）

文本嵌入模型 text-embedding-model#

概念指南 (opens in a new tab)

本文档介绍了如何在LangChain中使用Embedding类。
Embedding类是一个用于与嵌入进行交互的类。有许多嵌入提供商（OpenAI、Cohere、Hugging Face等)- 这个类旨在为所有这些提供商提供一个标准接口。
嵌入会创建文本的向量表示。这很有用，因为这意味着我们可以在向量空间中考虑文本，并执行诸如语义搜索之类的操作，其中我们在向量空间中寻找最相似的文本片段。
LangChain中的基本Embedding类公开了两种方法：embed_documents和embed_query。最大的区别在于这两种方法具有不同的接口：一个适用于多个文档，而另一个适用于单个文档。除此之外，将这两个方法作为两个单独的方法的另一个原因是，某些嵌入提供商针对要搜索的文档与查询本身具有不同的嵌入方法。
以下是文本嵌入的集成。

Aleph Alpha

AzureOpenAI

Cohere

Fake Embeddings

Hugging Face Hub

InstructEmbeddings

Jina

Llama-cpp

OpenAI

SageMaker Endpoint Embeddings

Self Hosted Embeddings

Sentence Transformers Embeddings

TensorflowHub

WriterAleph Alpha


阿勒夫·阿尔法# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例Aleph Alpha

阿勒夫·阿尔法#
使用阿勒夫·阿尔法的语义嵌入有两种可能的方法。如果您有结构不同的文本（例如文档和查询)，则应使用不对称嵌入。相反，对于结构可比较的文本，建议使用对称嵌入。
不对称#

```code
from langchain.embeddings import AlephAlphaAsymmetricSemanticEmbedding

```

```code
document = "This is a content of the document"
query = "What is the contnt of the document?"

```

```code
embeddings = AlephAlphaAsymmetricSemanticEmbedding()

```

```code
doc_result = embeddings.embed_documents([document])

```

```code
query_result = embeddings.embed_query(query)

```

对称#

```code
from langchain.embeddings import AlephAlphaSymmetricSemanticEmbedding

```

```code
text = "This is a test text"

```

```code
embeddings = AlephAlphaSymmetricSemanticEmbedding()

```

```code
doc_result = embeddings.embed_documents([text])

```

```code
query_result = embeddings.embed_query(text)

```
文本嵌入（Text Embedding）Azure OpenAI


AzureOpenAI# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例Azure OpenAI

AzureOpenAI#
让我们加载OpenAI嵌入类，并设置环境变量以指示使用Azure端点。

```code
# set the environment variables needed for openai package to know to reach out to azure
import os

os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_BASE"] = "https://<your-endpoint.openai.azure.com/"
os.environ["OPENAI_API_KEY"] = "your AzureOpenAI key"

```

```code
from langchain.embeddings import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="your-embeddings-deployment-name")

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```
Aleph AlphaCohere


Cohere# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例Cohere

Cohere#
让我们加载Cohere嵌入类。

```code
from langchain.embeddings import CohereEmbeddings

```

```code
embeddings = CohereEmbeddings(cohere_api_key=cohere_api_key)

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```
Azure OpenAI虚假（Fake）


伪嵌入FakeEmbeddings# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例虚假（Fake）

伪嵌入FakeEmbeddings#
LangChain还提供了一个伪嵌入类。您可以使用它来测试您的pipeline。

```code
from langchain.embeddings import FakeEmbeddings

```

```code
embeddings = FakeEmbeddings(size=1352)

```

```code
query_result = embeddings.embed_query("foo")

```

```code
doc_results = embeddings.embed_documents(["foo"])

```
CohereHuggingFace Hub


Hugging Face嵌入类 – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例HuggingFace Hub

Hugging Face嵌入类
让我们加载Hugging Face嵌入类。

```code
from langchain.embeddings import HuggingFaceEmbeddings

```

```code
embeddings = HuggingFaceEmbeddings()

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```
虚假（Fake）指令嵌入（Instruct Embeddings）


InstructEmbeddings# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例指令嵌入（Instruct Embeddings）

InstructEmbeddings#
让我们加载HuggingFace instruct嵌入类。

```code
from langchain.embeddings import HuggingFaceInstructEmbeddings

```

```code
embeddings = HuggingFaceInstructEmbeddings(
query_instruction="Represent the query for retrieval: "
)

```

```code
load INSTRUCTOR_Transformer
max_seq_length  512

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```
HuggingFace HubJina


Jina Embedding – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例Jina

Jina Embedding
让我们加载
```code
Jina Embedding
```
类.

```code
from langchain.embeddings import JinaEmbeddings

```

```code
embeddings = JinaEmbeddings(jina_auth_token=jina_auth_token, model_name="ViT-B-32::openai")

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```

In the above example,
```code
ViT-B-32::openai
```
, OpenAI’s pretrained
```code
ViT-B-32
```
model is used. For a full list of models, see here (opens in a new tab).指令嵌入（Instruct Embeddings）Llama CPP


Llama-cpp嵌入 – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例Llama CPP

Llama-cpp嵌入
本教程将介绍如何在LangChain中使用Llama-cpp嵌入。

```code
!pip install llama-cpp-python

```

```code
from langchain.embeddings import LlamaCppEmbeddings

```

```code
llama = LlamaCppEmbeddings(model_path="/path/to/model/ggml-model-q4_0.bin")

```

```code
text = "This is a test document."

```

```code
query_result = llama.embed_query(text)

```

```code
doc_result = llama.embed_documents([text])

```
JinaOpenAI


OpenAI# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例OpenAI

OpenAI#
让我们加载OpenAI嵌入类。

```code
from langchain.embeddings import OpenAIEmbeddings

```

```code
embeddings = OpenAIEmbeddings()

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```

让我们加载带有第一代模型（例如“text-search-ada-doc-001 / text-search-ada-query-001”)的OpenAI嵌入类。注意：这些不是推荐的模型-请参见此处 (opens in a new tab)。

```code
from langchain.embeddings.openai import OpenAIEmbeddings

```

```code
embeddings = OpenAIEmbeddings()

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text])

```
Llama CPPSageMaker 端点（Sagemaker-endpoint）


SageMaker Endpoints Embeddings类 – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例SageMaker 端点（Sagemaker-endpoint）

SageMaker Endpoints Embeddings类
让我们加载SageMaker Endpoints Embeddings类。
如果您在SageMaker上托管自己的Hugging Face模型，可以使用此类。
有关如何执行此操作的说明，请单击此处 (opens in a new tab)。
注意：为了处理批处理请求，您需要在自定义的
```code
inference.py
```
脚本中的
```code
predict_fn（)
```
函数的返回行中进行调整：
将

```code
return {"vectors": sentence_embeddings[0].tolist()}
```

更改为：

```code
return {"vectors": sentence_embeddings.tolist()}
```
。

```code
!pip3 install langchain boto3

```

```code
from typing import Dict, List
from langchain.embeddings import SagemakerEndpointEmbeddings
from langchain.llms.sagemaker_endpoint import ContentHandlerBase
import json

class ContentHandler(ContentHandlerBase):
content_type = "application/json"
accepts = "application/json"

def transform_input(self, inputs: list[str], model_kwargs: Dict) -> bytes:
input_str = json.dumps({"inputs": inputs, **model_kwargs})
return input_str.encode('utf-8')

def transform_output(self, output: bytes) -> List[List[float]]:
response_json = json.loads(output.read().decode("utf-8"))
return response_json["vectors"]

content_handler = ContentHandler()

embeddings = SagemakerEndpointEmbeddings(
# endpoint_name="endpoint-name",
# credentials_profile_name="credentials-profile-name",
endpoint_name="huggingface-pytorch-inference-2023-03-21-16-14-03-834",
region_name="us-east-1",
content_handler=content_handler
)

```

```code
query_result = embeddings.embed_query("foo")

```

```code
doc_results = embeddings.embed_documents(["foo"])

```

```code
doc_results

```
OpenAI自托管（Self-hosted）


自托管嵌入# – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例自托管（Self-hosted）

自托管嵌入#
让我们加载
```code
SelfHostedEmbeddings
```
、
```code
SelfHostedHuggingFaceEmbeddings
```
和
```code
SelfHostedHuggingFaceInstructEmbeddings
```
类。

```code
from langchain.embeddings import (
SelfHostedEmbeddings,
SelfHostedHuggingFaceEmbeddings,
SelfHostedHuggingFaceInstructEmbeddings,
)
import runhouse as rh

```

```code
# For an on-demand A100 with GCP, Azure, or Lambda
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1", use_spot=False)

# For an on-demand A10G with AWS (no single A100s on AWS)
# gpu = rh.cluster(name='rh-a10x', instance_type='g5.2xlarge', provider='aws')

# For an existing cluster
# gpu = rh.cluster(ips=['<ip of the cluster>'],
# ssh_creds={'ssh_user': '...', 'ssh_private_key':'<path_to_key>'},
# name='my-cluster')

```

```code
embeddings = SelfHostedHuggingFaceEmbeddings(hardware=gpu)

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

SelfHostedHuggingFaceInstructEmbeddings同理：

```code
embeddings = SelfHostedHuggingFaceInstructEmbeddings(hardware=gpu)

```

现在让我们使用自定义的加载函数加载嵌入模型：

```code
def get_pipeline():
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
pipeline,
)  # Must be inside the function in notebooks

model_id = "facebook/bart-base"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
return pipeline("feature-extraction", model=model, tokenizer=tokenizer)

def inference_fn(pipeline, prompt):
# Return last hidden state of the model
if isinstance(prompt, list):
return [emb[0][-1] for emb in pipeline(prompt)]
return pipeline(prompt)[0][-1]

```

```code
embeddings = SelfHostedEmbeddings(
model_load_fn=get_pipeline,
hardware=gpu,
model_reqs=["./", "torch", "transformers"],
inference_fn=inference_fn,
)

```

```code
query_result = embeddings.embed_query(text)

```
SageMaker 端点（Sagemaker-endpoint）句子嵌入（Sentence Transformers）


SentenceTransformers – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例句子嵌入（Sentence Transformers）

SentenceTransformers
SentenceTransformers (opens in a new tab)嵌入是使用
```code
HuggingFaceEmbeddings
```
集成调用的。 我们还为那些更熟悉直接使用该包的用户添加了
```code
SentenceTransformerEmbeddings
```
的别名。
SentenceTransformers是一个Python软件包，它可以生成文本和图像嵌入，源自于Sentence-BERT (opens in a new tab)。

```code
!pip install sentence_transformers > /dev/null

```

```code
[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip

```

```code
from langchain.embeddings import HuggingFaceEmbeddings, SentenceTransformerEmbeddings

```

```code
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
# Equivalent to SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_result = embeddings.embed_documents([text, "This is not a test document."])

```
自托管（Self-hosted）TensorFlow Hub


TensorflowHub – LangChain中文网

6大核心模块（Modules)模型（Models)文本嵌入（Text Embedding）示例TensorFlow Hub

TensorflowHub
让我们加载TensorflowHub嵌入类。

```code
from langchain.embeddings import TensorflowHubEmbeddings

```

```code
embeddings = TensorflowHubEmbeddings()

```

```code
2023-01-30 23:53:01.652176: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-30 23:53:34.362802: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

```

```code
text = "This is a test document."

```

```code
query_result = embeddings.embed_query(text)

```

```code
doc_results = embeddings.embed_documents(["foo"])

```

```code
doc_results

```
句子嵌入（Sentence Transformers）提示工程（Prompts)


提示# – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)

提示#
概念指南 (opens in a new tab)
编程模型的新方式是通过提示进行的。
“提示”指的是模型的输入。
这个输入很少是硬编码的，而是通常从多个组件构建而成的。
PromptTemplate负责构建这个输入。
LangChain提供了几个类和函数，使构建和处理提示变得容易。
本文档的这一部分分为四个部分：
LLM提示模板
如何使用PromptTemplates提示语言模型。
聊天提示模板
如何使用PromptTemplates提示聊天模型。
示例选择器
通常情况下，在提示中包含示例很有用。
这些示例可以是硬编码的，但如果它们是动态选择的，则通常更有力。
本节介绍示例选择。
输出解析器
语言模型（和聊天模型)输出文本。
但是，很多时候，您可能希望获得比仅文本更结构化的信息。
这就是输出解析器的作用。
输出解析器负责（1)指示模型应该如何格式化输出，
（2)将输出解析为所需的格式（如果必要，包括重试)。
深入#

Prompt Templates

Chat Prompt Template

Example Selectors

Output Parsers

TensorFlow Hub聊天提示模板（Chat Prompt Template）


与聊天相关的提示模板 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)聊天提示模板（Chat Prompt Template）

与聊天相关的提示模板
Chat Models以聊天消息列表作为输入——这个列表通常称为提示。这些聊天消息与原始字符串（您将传递给LLM模型的字符串)不同，因为每个消息都与一个角色相关联。
例如，在OpenAI的Chat Completion API (opens in a new tab)中，聊天消息可以与AI、人类或系统角色相关联。模型应更密切地遵循系统聊天消息的指示。
因此，LangChain提供了几个相关的提示模板，以便轻松构建和处理提示。在查询聊天模型时，建议您使用这些与聊天相关的提示模板，而不是
```code
PromptTemplate
```
，以充分发挥基础聊天模型的潜力。

```code
from langchain.prompts import (
ChatPromptTemplate,
PromptTemplate,
SystemMessagePromptTemplate,
AIMessagePromptTemplate,
HumanMessagePromptTemplate,
)
from langchain.schema import (
AIMessage,
HumanMessage,
SystemMessage
)

```

要创建与角色相关联的消息模板，您可以使用
```code
MessagePromptTemplate
```
。
为了方便起见，在模板上公开了
```code
from_template
```
方法。如果您要使用此模板，它的外观如下：

```code
template="You are a helpful assistant that translates {input_language} to {output_language}."
system_message_prompt = SystemMessagePromptTemplate.from_template(template)
human_template="{text}"
human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)

```

如果您想更直接地构建
```code
MessagePromptTemplate
```
，您可以在外部创建一个
```code
PromptTemplate
```
，然后将其传递进去，例如：

```code
prompt=PromptTemplate(
template="You are a helpful assistant that translates {input_language} to {output_language}.",
input_variables=["input_language", "output_language"],
)
system_message_prompt_2 = SystemMessagePromptTemplate(prompt=prompt)

assert system_message_prompt == system_message_prompt_2

```

之后，您可以从一个或多个
```code
MessagePromptTemplates
```
构建一个
```code
ChatPromptTemplate
```
。
您可以使用
```code
ChatPromptTemplate
```
的
```code
format_prompt
```
方法 - 这将返回一个
```code
PromptValue
```
，您可以将其转换为字符串或Message对象，具体取决于您是否想将格式化值用作llm或chat模型的输入。

```code
chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])

# get a chat completion from the formatted messages
chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages()

```

```code
[SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}),
HumanMessage(content='I love programming.', additional_kwargs={})]

```

输出的格式 Format output#

```code
format_prompt
```
方法的输出可以作为字符串、消息列表和
```code
ChatPromptValue
```
使用。
作为字符串：

```code
output = chat_prompt.format(input_language="English", output_language="French", text="I love programming.")
output

```

```code
'System: You are a helpful assistant that translates English to French.\nHuman: I love programming.'

```

```code
# or alternatively
output_2 = chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_string()

assert output == output_2

```

作为
```code
ChatPromptValue
```

```code
chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.")

```

```code
ChatPromptValue(messages=[SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}), HumanMessage(content='I love programming.', additional_kwargs={})])

```

作为消息对象列表：

```code
chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages()

```

```code
[SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}),
HumanMessage(content='I love programming.', additional_kwargs={})]

```

不同类型的
```code
MessagePromptTemplate
```
#
LangChain 提供了不同类型的
```code
MessagePromptTemplate
```
。其中最常用的是
```code
AIMessagePromptTemplate
```
、
```code
SystemMessagePromptTemplate
```
和
```code
HumanMessagePromptTemplate
```
，分别用于创建 AI 消息、系统消息和人类消息。
但是，在聊天模型支持使用任意角色发送聊天消息的情况下，您可以使用
```code
ChatMessagePromptTemplate
```
，允许用户指定角色名称。

```code
from langchain.prompts import ChatMessagePromptTemplate

prompt = "May the {subject} be with you"

chat_message_prompt = ChatMessagePromptTemplate.from_template(role="Jedi", template=prompt)
chat_message_prompt.format(subject="force")

```

```code
ChatMessage(content='May the force be with you', additional_kwargs={}, role='Jedi')

```

LangChain 还提供了
```code
MessagesPlaceholder
```
，该占位符可以在格式化期间完全控制要呈现的消息。当您不确定应该使用哪个消息提示模板的角色或者希望在格式化期间插入消息列表时，这可能非常有用。

```code
from langchain.prompts import MessagesPlaceholder

human_prompt = "Summarize our conversation so far in {word_count} words."
human_message_template = HumanMessagePromptTemplate.from_template(human_prompt)

chat_prompt = ChatPromptTemplate.from_messages([MessagesPlaceholder(variable_name="conversation"), human_message_template])

```

```code
human_message = HumanMessage(content="What is the best way to learn programming?")
ai_message = AIMessage(content="""\
1. Choose a programming language: Decide on a programming language that you want to learn.

2. Start with the basics: Familiarize yourself with the basic programming concepts such as variables, data types and control structures.

3. Practice, practice, practice: The best way to learn programming is through hands-on experience\
""")

chat_prompt.format_prompt(conversation=[human_message, ai_message], word_count="10").to_messages()

```

```code
[HumanMessage(content='What is the best way to learn programming?', additional_kwargs={}),
AIMessage(content='1. Choose a programming language: Decide on a programming language that you want to learn.   2. Start with the basics: Familiarize yourself with the basic programming concepts such as variables, data types and control structures.  3. Practice, practice, practice: The best way to learn programming is through hands-on experience', additional_kwargs={}),
HumanMessage(content='Summarize our conversation so far in 10 words.', additional_kwargs={})]

```
提示工程（Prompts)示例选择器（Example Selectors）


Example Selectors – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）

Example Selectors
注意
概念指南 (opens in a new tab)
如果您有大量的示例，您可能需要选择包含在提示中的示例。ExampleSelector是负责执行此操作的类。
基本接口定义如下：

```code
class BaseExampleSelector(ABC):
"""Interface for selecting examples to include in prompts."""

@abstractmethod
def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:
"""Select which examples to use based on the inputs."""

```

ExampleSelector唯一需要公开的方法是
```code
select_examples
```
。该方法接受输入变量，然后返回一个示例列表。如何选择这些示例取决于每个具体实现。以下是一些示例。
请参见下面的示例选择器列表。

How to create a custom example selector

LengthBased ExampleSelector

最大边际相关性 ExampleSelector

NGram 重叠 ExampleSelector

相似度 ExampleSelector

聊天提示模板（Chat Prompt Template）自定义示例选择器（Custom Example Selector）


实现自定义示例选择器 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）示例自定义示例选择器（Custom Example Selector）

实现自定义示例选择器
在本教程中，我们将创建一个自定义示例选择器，该选择器从给定的示例列表中选择每个替代示例。

```code
ExampleSelector
```
必须实现两种方法：

```code
add_example
```
方法，它接受一个示例并将其添加到该ExampleSelector中。

```code
select_examples
```
方法，它接受输入变量（这些变量应为用户输入)，并返回要在few-shot提示中使用的示例列表。

让我们实现一个自定义
```code
ExampleSelector
```
，它仅随机选择两个示例。
注意
查看LangChain支持的当前示例选择器实现此处。
实现自定义示例选择器

```code
from langchain.prompts.example_selector.base import BaseExampleSelector
from typing import Dict, List
import numpy as np

class CustomExampleSelector(BaseExampleSelector):

def __init__(self, examples: List[Dict[str, str]]):
self.examples = examples

def add_example(self, example: Dict[str, str]) -> None:
"""Add new example to store for a key."""
self.examples.append(example)

def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:
"""Select which examples to use based on the inputs."""
return np.random.choice(self.examples, size=2, replace=False)

```

Use custom example selector#

```code

examples = [
{"foo": "1"},
{"foo": "2"},
{"foo": "3"}
]

# Initialize example selector.
example_selector = CustomExampleSelector(examples)

# Select examples
example_selector.select_examples({"foo": "foo"})
# -> array([{'foo': '2'}, {'foo': '3'}], dtype=object)

# Add new example to the set of examples
example_selector.add_example({"foo": "4"})
example_selector.examples
# -> [{'foo': '1'}, {'foo': '2'}, {'foo': '3'}, {'foo': '4'}]

# Select examples
example_selector.select_examples({"foo": "foo"})
# -> array([{'foo': '1'}, {'foo': '4'}], dtype=object)

```
示例选择器（Example Selectors）基于长度的（Length-based）


根据长度选择要使用的示例 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）示例基于长度的（Length-based）

根据长度选择要使用的示例
该ExampleSelector根据长度选择要使用的示例。当您担心构造的提示将超出上下文窗口的长度时，这很有用。对于较长的输入，它将选择包括较少的示例，而对于较短的输入，它将选择较多的示例。

```code
from langchain.prompts import PromptTemplate
from langchain.prompts import FewShotPromptTemplate
from langchain.prompts.example_selector import LengthBasedExampleSelector

```

```code
# These are a lot of examples of a pretend task of creating antonyms.
examples = [
{"input": "happy", "output": "sad"},
{"input": "tall", "output": "short"},
{"input": "energetic", "output": "lethargic"},
{"input": "sunny", "output": "gloomy"},
{"input": "windy", "output": "calm"},
]

```

```code
example_prompt = PromptTemplate(
input_variables=["input", "output"],
template="Input: {input}\nOutput: {output}",
)
example_selector = LengthBasedExampleSelector(
# These are the examples it has available to choose from.
examples=examples,
# This is the PromptTemplate being used to format the examples.
example_prompt=example_prompt,
# This is the maximum length that the formatted examples should be.
# Length is measured by the get_text_length function below.
max_length=25,
# This is the function used to get the length of a string, which is used
# to determine which examples to include. It is commented out because
# it is provided as a default value if none is specified.
# get_text_length: Callable[[str], int] = lambda x: len(re.split("\n| ", x))
)
dynamic_prompt = FewShotPromptTemplate(
# We provide an ExampleSelector instead of examples.
example_selector=example_selector,
example_prompt=example_prompt,
prefix="Give the antonym of every input",
suffix="Input: {adjective}\nOutput:",
input_variables=["adjective"],
)

```

```code
# An example with small input, so it selects all examples.
print(dynamic_prompt.format(adjective="big"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: tall
Output: short

Input: energetic
Output: lethargic

Input: sunny
Output: gloomy

Input: windy
Output: calm

Input: big
Output:

```

```code
# An example with long input, so it selects only one example.
long_string = "big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else"
print(dynamic_prompt.format(adjective=long_string))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else
Output:

```

```code
# You can add an example to an example selector as well.
new_example = {"input": "big", "output": "small"}
dynamic_prompt.example_selector.add_example(new_example)
print(dynamic_prompt.format(adjective="enthusiastic"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: tall
Output: short

Input: energetic
Output: lethargic

Input: sunny
Output: gloomy

Input: windy
Output: calm

Input: big
Output: small

Input: enthusiastic
Output:

```
自定义示例选择器（Custom Example Selector）最大边缘相关性（MMR）


最大边际相关性示例选择器# – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）示例最大边缘相关性（MMR）

最大边际相关性示例选择器#
MaxMarginalRelevanceExampleSelector基于哪些示例与输入最相似以及优化多样性的组合选择示例。
它通过找到嵌入与输入具有最大余弦相似度的示例，然后迭代地添加它们，同时惩罚它们与已选择示例的接近程度来实现这一目的。

```code
from langchain.prompts.example_selector import MaxMarginalRelevanceExampleSelector
from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.prompts import FewShotPromptTemplate, PromptTemplate

example_prompt = PromptTemplate(
input_variables=["input", "output"],
template="Input: {input}\nOutput: {output}",
)

# These are a lot of examples of a pretend task of creating antonyms.
examples = [
{"input": "happy", "output": "sad"},
{"input": "tall", "output": "short"},
{"input": "energetic", "output": "lethargic"},
{"input": "sunny", "output": "gloomy"},
{"input": "windy", "output": "calm"},
]

```

```code
example_selector = MaxMarginalRelevanceExampleSelector.from_examples(
# This is the list of examples available to select from.
examples,
# This is the embedding class used to produce embeddings which are used to measure semantic similarity.
OpenAIEmbeddings(),
# This is the VectorStore class that is used to store the embeddings and do a similarity search over.
FAISS,
# This is the number of examples to produce.
k=2
)
mmr_prompt = FewShotPromptTemplate(
# We provide an ExampleSelector instead of examples.
example_selector=example_selector,
example_prompt=example_prompt,
prefix="Give the antonym of every input",
suffix="Input: {adjective}\nOutput:",
input_variables=["adjective"],
)

```

```code
# Input is a feeling, so should select the happy/sad example as the first one
print(mmr_prompt.format(adjective="worried"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: windy
Output: calm

Input: worried
Output:

```

```code
# Let's compare this to what we would just get if we went solely off of similarity
similar_prompt = FewShotPromptTemplate(
# We provide an ExampleSelector instead of examples.
example_selector=example_selector,
example_prompt=example_prompt,
prefix="Give the antonym of every input",
suffix="Input: {adjective}\nOutput:",
input_variables=["adjective"],
)
similar_prompt.example_selector.k = 2
print(similar_prompt.format(adjective="worried"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: windy
Output: calm

Input: worried
Output:

```
基于长度的（Length-based）N-gram 重叠（Ngram Overlap）


ngram重叠 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）示例N-gram 重叠（Ngram Overlap）

ngram重叠
NGramOverlapExampleSelector根据ngram重叠得分选择和排序示例，该得分表示示例与输入的相似程度。 ngram重叠得分是一个介于0.0和1.0之间的浮点数。
选择器允许设置阈值得分。 ngram重叠得分小于或等于阈值的示例将被排除。默认情况下，阈值设置为-1.0，因此不会排除任何示例，只会对它们进行重新排序。将阈值设置为0.0将排除具有与输入无ngram重叠的示例。

```code
from langchain.prompts import PromptTemplate
from langchain.prompts.example_selector.ngram_overlap import NGramOverlapExampleSelector
from langchain.prompts import FewShotPromptTemplate, PromptTemplate

example_prompt = PromptTemplate(
input_variables=["input", "output"],
template="Input: {input}\nOutput: {output}",
)

# These are a lot of examples of a pretend task of creating antonyms.
examples = [
{"input": "happy", "output": "sad"},
{"input": "tall", "output": "short"},
{"input": "energetic", "output": "lethargic"},
{"input": "sunny", "output": "gloomy"},
{"input": "windy", "output": "calm"},
]

```

```code
# These are examples of a fictional translation task.
examples = [
{"input": "See Spot run.", "output": "Ver correr a Spot."},
{"input": "My dog barks.", "output": "Mi perro ladra."},
{"input": "Spot can run.", "output": "Spot puede correr."},
]

```

```code
example_prompt = PromptTemplate(
input_variables=["input", "output"],
template="Input: {input}\nOutput: {output}",
)
example_selector = NGramOverlapExampleSelector(
# These are the examples it has available to choose from.
examples=examples,
# This is the PromptTemplate being used to format the examples.
example_prompt=example_prompt,
# This is the threshold, at which selector stops.
# It is set to -1.0 by default.
threshold=-1.0,
# For negative threshold:
# Selector sorts examples by ngram overlap score, and excludes none.
# For threshold greater than 1.0:
# Selector excludes all examples, and returns an empty list.
# For threshold equal to 0.0:
# Selector sorts examples by ngram overlap score,
# and excludes those with no ngram overlap with input.
)
dynamic_prompt = FewShotPromptTemplate(
# We provide an ExampleSelector instead of examples.
example_selector=example_selector,
example_prompt=example_prompt,
prefix="Give the Spanish translation of every input",
suffix="Input: {sentence}\nOutput:",
input_variables=["sentence"],
)

```

```code
# An example input with large ngram overlap with "Spot can run."
# and no overlap with "My dog barks."
print(dynamic_prompt.format(sentence="Spot can run fast."))

```

```code
Give the Spanish translation of every input

Input: Spot can run.
Output: Spot puede correr.

Input: See Spot run.
Output: Ver correr a Spot.

Input: My dog barks.
Output: Mi perro ladra.

Input: Spot can run fast.
Output:

```

```code
# You can add examples to NGramOverlapExampleSelector as well.
new_example = {"input": "Spot plays fetch.", "output": "Spot juega a buscar."}

example_selector.add_example(new_example)
print(dynamic_prompt.format(sentence="Spot can run fast."))

```

```code
Give the Spanish translation of every input

Input: Spot can run.
Output: Spot puede correr.

Input: See Spot run.
Output: Ver correr a Spot.

Input: Spot plays fetch.
Output: Spot juega a buscar.

Input: My dog barks.
Output: Mi perro ladra.

Input: Spot can run fast.
Output:

```

```code
# You can set a threshold at which examples are excluded.
# For example, setting threshold equal to 0.0
# excludes examples with no ngram overlaps with input.
# Since "My dog barks." has no ngram overlaps with "Spot can run fast."
# it is excluded.
example_selector.threshold=0.0
print(dynamic_prompt.format(sentence="Spot can run fast."))

```

```code
Give the Spanish translation of every input

Input: Spot can run.
Output: Spot puede correr.

Input: See Spot run.
Output: Ver correr a Spot.

Input: Spot plays fetch.
Output: Spot juega a buscar.

Input: Spot can run fast.
Output:

```

```code
# Setting small nonzero threshold
example_selector.threshold=0.09
print(dynamic_prompt.format(sentence="Spot can play fetch."))

```

```code
Give the Spanish translation of every input

Input: Spot can run.
Output: Spot puede correr.

Input: Spot plays fetch.
Output: Spot juega a buscar.

Input: Spot can play fetch.
Output:

```

```code
# Setting threshold greater than 1.0
example_selector.threshold=1.0+1e-9
print(dynamic_prompt.format(sentence="Spot can play fetch."))

```

```code
Give the Spanish translation of every input

Input: Spot can play fetch.
Output:

```
最大边缘相关性（MMR）相似度（Similarity）


相似度 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)示例选择器（Example Selectors）示例相似度（Similarity）

相似度
SemanticSimilarityExampleSelector根据示例与输入的相似度选择示例。它通过查找嵌入与输入的余弦相似度最大的示例来实现此目的。

```code
from langchain.prompts.example_selector import SemanticSimilarityExampleSelector
from langchain.vectorstores import Chroma
from langchain.embeddings import OpenAIEmbeddings
from langchain.prompts import FewShotPromptTemplate, PromptTemplate

example_prompt = PromptTemplate(
input_variables=["input", "output"],
template="Input: {input}\nOutput: {output}",
)

# These are a lot of examples of a pretend task of creating antonyms.
examples = [
{"input": "happy", "output": "sad"},
{"input": "tall", "output": "short"},
{"input": "energetic", "output": "lethargic"},
{"input": "sunny", "output": "gloomy"},
{"input": "windy", "output": "calm"},
]

```

```code
example_selector = SemanticSimilarityExampleSelector.from_examples(
# This is the list of examples available to select from.
examples,
# This is the embedding class used to produce embeddings which are used to measure semantic similarity.
OpenAIEmbeddings(),
# This is the VectorStore class that is used to store the embeddings and do a similarity search over.
Chroma,
# This is the number of examples to produce.
k=1
)
similar_prompt = FewShotPromptTemplate(
# We provide an ExampleSelector instead of examples.
example_selector=example_selector,
example_prompt=example_prompt,
prefix="Give the antonym of every input",
suffix="Input: {adjective}\nOutput:",
input_variables=["adjective"],
)

```

```code
Running Chroma using direct local API.
Using DuckDB in-memory for database. Data will be transient.

```

```code
# Input is a feeling, so should select the happy/sad example
print(similar_prompt.format(adjective="worried"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: worried
Output:

```

```code
# Input is a measurement, so should select the tall/short example
print(similar_prompt.format(adjective="fat"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: fat
Output:

```

```code
# You can add new examples to the SemanticSimilarityExampleSelector as well
similar_prompt.example_selector.add_example({"input": "enthusiastic", "output": "apathetic"})
print(similar_prompt.format(adjective="joyful"))

```

```code
Give the antonym of every input

Input: happy
Output: sad

Input: joyful
Output:

```
N-gram 重叠（Ngram Overlap）输出解析器（Output Parsers）


输出解析器 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）

输出解析器

概念指南 (opens in a new tab)

语言模型输出文本。但是很多时候，你可能想要获得比文本更结构化的信息。这就是输出解析器的作用。
输出解析器是帮助结构化语言模型响应的类。有两种主要的方法，一个输出解析器必须实现：

```code
get_format_instructions() -> str
```
：一个方法，返回一个包含有关如何格式化语言模型输出的字符串。

```code
parse(str) -> Any
```
：一个方法，接受一个字符串（假定为语言模型的响应)并将其解析为某个结构。

然后是一个可选的：

```code
parse_with_prompt(str) -> Any
```
：一个方法，它接受一个字符串（假设是语言模型的响应）和一个提示（假设是生成这样的响应的提示），并将其解析为某种结构。提示在此大多数情况下是为了提供信息以便OutputParser重新尝试或以某种方式修复输出。

首先，我们建议您熟悉入门部分

输出解析器

之后，我们提供了有关所有不同类型的输出解析器的深入探讨。

逗号分隔列表输出解析器

输出修复解析器

Pydantic输出解析器

重试输出解析器

结构化输出解析器

相似度（Similarity）逗号分隔（Comma Separated）


逗号分隔列表输出解析器 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）示例（Examples）逗号分隔（Comma Separated）

逗号分隔列表输出解析器
这是另一个比Pydantic / JSON解析功能要弱的解析器。

```code
from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

```

```code
output_parser = CommaSeparatedListOutputParser()

```

```code
format_instructions = output_parser.get_format_instructions()
prompt = PromptTemplate(
template="List five {subject}.\n{format_instructions}",
input_variables=["subject"],
partial_variables={"format_instructions": format_instructions}
)

```

```code
model = OpenAI(temperature=0)

```

```code
_input = prompt.format(subject="ice cream flavors")
output = model(_input)

```

```code
output_parser.parse(output)

```

```code
['Vanilla',
'Chocolate',
'Strawberry',
'Mint Chocolate Chip',
'Cookies and Cream']

```
输出解析器（Output Parsers）输出修复解析器（Output Fixing Parser）


输出解析器封装 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）示例（Examples）输出修复解析器（Output Fixing Parser）

输出解析器封装
该输出解析器封装另一个输出解析器，并尝试修复任何错误
Pydantic防护栏只是尝试解析LLM响应。如果它无法正确解析，则会出现错误。
但是，我们除了抛出错误之外还可以做其他事情。具体而言，我们可以将格式不正确的输出与格式说明一起传递给模型，并要求它进行修复。
对于这个示例，我们将使用上面的OutputParser。如果我们将不符合模式的结果传递给它，将会发生什么呢：

```code
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field, validator
from typing import List

```

```code
class Actor(BaseModel):
name: str = Field(description="name of an actor")
film_names: List[str] = Field(description="list of names of films they starred in")

actor_query = "Generate the filmography for a random actor."

parser = PydanticOutputParser(pydantic_object=Actor)

```

```code
misformatted = "{'name': 'Tom Hanks', 'film_names': ['Forrest Gump']}"

```

```code
parser.parse(misformatted)

```

```code
---------------------------------------------------------------------------
JSONDecodeError Traceback (most recent call last)
File ~/workplace/langchain/langchain/output_parsers/pydantic.py:23, in PydanticOutputParser.parse(self, text)
22     json_str = match.group()
---> 23 json_object = json.loads(json_str)
24 return self.pydantic_object.parse_obj(json_object)

File ~/.pyenv/versions/3.9.1/lib/python3.9/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
343 if (cls is None and object_hook is None and
344         parse_int is None and parse_float is None and
345         parse_constant is None and object_pairs_hook is None and not kw):
--> 346     return _default_decoder.decode(s)
347 if cls is None:

File ~/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
333 """Return the Python representation of ``s`` (a ``str`` instance
334 containing a JSON document).
335
336 """
--> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
338 end = _w(s, end).end()

File ~/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py:353, in JSONDecoder.raw_decode(self, s, idx)
352 try:
--> 353     obj, end = self.scan_once(s, idx)
354 except StopIteration as err:

JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

During handling of the above exception, another exception occurred:

OutputParserException Traceback (most recent call last)
Cell In[6], line 1
----> 1 parser.parse(misformatted)

File ~/workplace/langchain/langchain/output_parsers/pydantic.py:29, in PydanticOutputParser.parse(self, text)
27 name = self.pydantic_object.__name__
28 msg = f"Failed to parse {name} from completion {text}. Got: {e}"
---> 29 raise OutputParserException(msg)

OutputParserException: Failed to parse Actor from completion {'name': 'Tom Hanks', 'film_names': ['Forrest Gump']}. Got: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

```

现在我们可以构建和使用
```code
OutputFixingParser
```
。
该输出解析器接受另一个输出解析器作为参数，还有一个LLM，用于尝试纠正任何格式错误。

```code
from langchain.output_parsers import OutputFixingParser

new_parser = OutputFixingParser.from_llm(parser=parser, llm=ChatOpenAI())

```

```code
new_parser.parse(misformatted)

```

```code
Actor(name='Tom Hanks', film_names=['Forrest Gump'])

```
逗号分隔（Comma Separated）Pydantic


Pydantic – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）示例（Examples）Pydantic

Pydantic
此输出解析器允许用户指定任意JSON架构，并查询符合该架构的JSON输出的LLMs。
请记住，大型语言模型是渗透的抽象！您将不得不使用足够容量的LLM生成格式良好的JSON。在OpenAI系列中，DaVinci可以可靠地完成此任务，但Curie的能力已经大幅下降。
使用Pydantic声明您的数据模型。Pydantic的BaseModel类似于Python数据类，但具有实际的类型检查+强制。

```code
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

```

```code
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field, validator
from typing import List

```

```code
model_name = 'text-davinci-003'
temperature = 0.0
model = OpenAI(model_name=model_name, temperature=temperature)

```

```code
# Define your desired data structure.
class Joke(BaseModel):
setup: str = Field(description="question to set up a joke")
punchline: str = Field(description="answer to resolve the joke")

# You can add custom validation logic easily with Pydantic.
@validator('setup')
def question_ends_with_question_mark(cls, field):
if field[-1] != '?':
raise ValueError("Badly formed question!")
return field

# And a query intented to prompt a language model to populate the data structure.
joke_query = "Tell me a joke."

# Set up a parser + inject instructions into the prompt template.
parser = PydanticOutputParser(pydantic_object=Joke)

prompt = PromptTemplate(
template="Answer the user query.\n{format_instructions}\n{query}\n",
input_variables=["query"],
partial_variables={"format_instructions": parser.get_format_instructions()}
)

_input = prompt.format_prompt(query=joke_query)

output = model(_input.to_string())

parser.parse(output)

```

```code
Joke(setup='Why did the chicken cross the road?', punchline='To get to the other side!')

```

```code
# Here's another example, but with a compound typed field.
class Actor(BaseModel):
name: str = Field(description="name of an actor")
film_names: List[str] = Field(description="list of names of films they starred in")

actor_query = "Generate the filmography for a random actor."

parser = PydanticOutputParser(pydantic_object=Actor)

prompt = PromptTemplate(
template="Answer the user query.\n{format_instructions}\n{query}\n",
input_variables=["query"],
partial_variables={"format_instructions": parser.get_format_instructions()}
)

_input = prompt.format_prompt(query=actor_query)

output = model(_input.to_string())

parser.parse(output)

```

```code
Actor(name='Tom Hanks', film_names=['Forrest Gump', 'Saving Private Ryan', 'The Green Mile', 'Cast Away', 'Toy Story'])

```
输出修复解析器（Output Fixing Parser）重试（Retry）


重试输出解析器# – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）示例（Examples）重试（Retry）

重试输出解析器#
在某些情况下，只查看输出就可以修复任何解析错误，但在其他情况下则不行。例如，当输出不仅格式不正确，而且部分不完整时，就是这种情况。请考虑下面的例子。

```code
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.output_parsers import PydanticOutputParser, OutputFixingParser, RetryOutputParser
from pydantic import BaseModel, Field, validator
from typing import List

```

```code
template = """Based on the user question, provide an Action and Action Input for what step should be taken.
{format_instructions}
Question: {query}
Response:"""
class Action(BaseModel):
action: str = Field(description="action to take")
action_input: str = Field(description="input to the action")

parser = PydanticOutputParser(pydantic_object=Action)

```

```code
prompt = PromptTemplate(
template="Answer the user query.\n{format_instructions}\n{query}\n",
input_variables=["query"],
partial_variables={"format_instructions": parser.get_format_instructions()}
)

```

```code
prompt_value = prompt.format_prompt(query="who is leo di caprios gf?")

```

```code
bad_response = '{"action": "search"}'

```

如果我们尝试直接解析此响应，将会出现错误

```code
parser.parse(bad_response)

```

```code
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
File ~/workplace/langchain/langchain/output_parsers/pydantic.py:24, in PydanticOutputParser.parse(self, text)
23     json_object = json.loads(json_str)
---> 24     return self.pydantic_object.parse_obj(json_object)
26 except (json.JSONDecodeError, ValidationError) as e:

File ~/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pydantic/main.py:527, in pydantic.main.BaseModel.parse_obj()

File ~/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pydantic/main.py:342, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for Action
action_input
field required (type=value_error.missing)

During handling of the above exception, another exception occurred:

OutputParserException Traceback (most recent call last)
Cell In[6], line 1
----> 1 parser.parse(bad_response)

File ~/workplace/langchain/langchain/output_parsers/pydantic.py:29, in PydanticOutputParser.parse(self, text)
27 name = self.pydantic_object.__name__
28 msg = f"Failed to parse {name} from completion {text}. Got: {e}"
---> 29 raise OutputParserException(msg)

OutputParserException: Failed to parse Action from completion {"action": "search"}. Got: 1 validation error for Action
action_input
field required (type=value_error.missing)

```

如果我们尝试使用
```code
OutputFixingParser
```
来修复此错误，它会感到困惑 - 也就是说，它不知道该为操作输入实际上放什么。

```code
fix_parser = OutputFixingParser.from_llm(parser=parser, llm=ChatOpenAI())

```

```code
fix_parser.parse(bad_response)

```

```code
Action(action='search', action_input='')

```

相反，我们可以使用RetryOutputParser，它将提示（以及原始输出)传递以尝试再次获取更好的响应。

```code
from langchain.output_parsers import RetryWithErrorOutputParser

```

```code
retry_parser = RetryWithErrorOutputParser.from_llm(parser=parser, llm=OpenAI(temperature=0))

```

```code
retry_parser.parse_with_prompt(bad_response, prompt_value)

```

```code
Action(action='search', action_input='who is leo di caprios gf?')

```
Pydantic结构化（Structured）


结构 StructuredOutputParser – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）示例（Examples）结构化（Structured）

结构 StructuredOutputParser
虽然Pydantic / JSON解析器更加强大，但我们最初尝试的数据结构仅具有文本字段。

```code
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

```

这里我们定义了我们想要接收的响应模式。

```code
response_schemas = [
ResponseSchema(name="answer", description="answer to the user's question"),
ResponseSchema(name="source", description="source used to answer the user's question, should be a website.")
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)

```

我们现在得到了一个字符串，其中包含响应应如何格式化的指令，然后我们将其插入到我们的提示中。

```code
format_instructions = output_parser.get_format_instructions()
prompt = PromptTemplate(
template="answer the users question as best as possible.\n{format_instructions}\n{question}",
input_variables=["question"],
partial_variables={"format_instructions": format_instructions}
)

```

我们现在可以使用它来格式化一个提示，以发送给语言模型，然后解析返回的结果。

```code
model = OpenAI(temperature=0)

```

```code
_input = prompt.format_prompt(question="what's the capital of france")
output = model(_input.to_string())

```

```code
output_parser.parse(output)

```

```code
{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}

```

下面是使用该方法在聊天模型中的示例：

```code
chat_model = ChatOpenAI(temperature=0)

```

```code
prompt = ChatPromptTemplate(
messages=[
HumanMessagePromptTemplate.from_template("answer the users question as best as possible.\n{format_instructions}\n{question}")
],
input_variables=["question"],
partial_variables={"format_instructions": format_instructions}
)

```

```code
_input = prompt.format_prompt(question="what's the capital of france")
output = chat_model(_input.to_messages())

```

```code
output_parser.parse(output.content)

```

```code
{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}

```
重试（Retry）入门（Getting Started）


入门 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)输出解析器（Output Parsers）入门（Getting Started）

入门
语言模型输出的是文本。但是很多时候，您可能想要获得的信息比仅仅是文本。这就是输出解析器的用处。
输出解析器是帮助结构化语言模型响应的类。输出解析器必须实现两种主要方法：

```code
get_format_instructions() -> str
```
：该方法返回一个包含语言模型输出格式说明的字符串。

```code
parse(str) -> Any
```
：该方法接受一个字符串（假定为语言模型的响应)，并将其解析成某种结构。

还有一个可选方法：

```code
parse_with_prompt(str, PromptValue) -> Any
```
：该方法接受一个字符串（假定为语言模型的响应)和一个提示（假定为生成此类响应的提示)，然后将其解析成某种结构。提示在很大程度上是提供的，以防OutputParser希望以某种方式重试或修复输出，并需要提示信息来执行此操作。

下面我们介绍主要类型的输出解析器——
```code
PydanticOutputParser
```
。其他选项请参见
```code
examples
```
文件夹。

```code
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI

from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field, validator
from typing import List

```

```code
model_name = 'text-davinci-003'
temperature = 0.0
model = OpenAI(model_name=model_name, temperature=temperature)

```

```code
# Define your desired data structure.
class Joke(BaseModel):
setup: str = Field(description="question to set up a joke")
punchline: str = Field(description="answer to resolve the joke")

# You can add custom validation logic easily with Pydantic.
@validator('setup')
def question_ends_with_question_mark(cls, field):
if field[-1] != '?':
raise ValueError("Badly formed question!")
return field

```

```code
# Set up a parser + inject instructions into the prompt template.
parser = PydanticOutputParser(pydantic_object=Joke)

```

```code
prompt = PromptTemplate(
template="Answer the user query.\n{format_instructions}\n{query}\n",
input_variables=["query"],
partial_variables={"format_instructions": parser.get_format_instructions()}
)

```

```code
# And a query intented to prompt a language model to populate the data structure.
joke_query = "Tell me a joke."
_input = prompt.format_prompt(query=joke_query)

```

```code
output = model(_input.to_string())

```

```code
parser.parse(output)

```

```code
Joke(setup='Why did the chicken cross the road?', punchline='To get to the other side!')

```
结构化（Structured）提示模板（Prompt Templates）


提示模板# – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)提示模板（Prompt Templates）

提示模板#

概念指南 (opens in a new tab)

语言模型以文本为输入 - 这个文本通常称为提示。
通常这不仅仅是一个硬编码的字符串，而是一个模板、一些例子和用户输入的组合。
LangChain 提供了几个类和函数，使构建和处理提示变得容易。
以下是提供的文档部分:

入门指南：LangChain 提供的用于处理和构建提示的所有功能的概述。

操作指南：一个操作指南集合。这些指南突出了如何使用我们的提示类完成各种目标。

参考文献：所有提示类的API参考文档。

入门（Getting Started）连接特征存储（Connecting to a Feature Store）


提示模板连接到特征存储 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)提示模板（Prompt Templates）示例（Examples）连接特征存储（Connecting to a Feature Store）

提示模板连接到特征存储
特征存储是传统机器学习中的一个概念，确保馈入模型的数据是最新和相关的。有关详细信息，请参见此处 (opens in a new tab)。
在考虑将 LLM 应用程序投入生产时，此概念非常相关。为了个性化 LLM 应用程序，您可能希望将 LLM 与特定用户的最新信息相结合。特征存储可以是保持数据新鲜的好方式，LangChain 提供了一种将该数据与 LLM 组合的简单方式。
在本教程中，我们将展示如何将提示模板连接到特征存储。基本思路是从提示模板内部调用特征存储以检索值，然后将其格式化到提示中。
Feast#
首先，我们将使用流行的开源特征存储框架Feast (opens in a new tab)。
这假定您已经运行了有关入门的步骤。我们将在入门示例的基础上构建，创建 LLMChain，以向特定司机写入有关其最新统计数据的注释。
加载Feast存储#
同样，这应该按照Feast README中的说明设置

```code
from feast import FeatureStore

# You may need to update the path depending on where you stored it
feast_repo_path = "../../../../../my_feature_repo/feature_repo/"
store = FeatureStore(repo_path=feast_repo_path)

```

提示#
在这里，我们将设置一个自定义的FeastPromptTemplate。这个提示模板将接受一个驱动程序ID，查找他们的统计数据，并将这些统计数据格式化为提示。
请注意，这个提示模板的输入只是
```code
driver_id
```
，因为这是唯一的用户定义部分（所有其他变量都在提示模板内查找)。

```code
from langchain.prompts import PromptTemplate, StringPromptTemplate

```

```code
template = """Given the driver's up to date stats, write them note relaying those stats to them.
If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel better

Here are the drivers stats:
Conversation rate: {conv_rate}
Acceptance rate: {acc_rate}
Average Daily Trips: {avg_daily_trips}

Your response:"""
prompt = PromptTemplate.from_template(template)

```

```code
class FeastPromptTemplate(StringPromptTemplate):

def format(self, **kwargs) -> str:
driver_id = kwargs.pop("driver_id")
feature_vector = store.get_online_features(
features=[
'driver_hourly_stats:conv_rate',
'driver_hourly_stats:acc_rate',
'driver_hourly_stats:avg_daily_trips'
],
entity_rows=[{"driver_id": driver_id}]
).to_dict()
kwargs["conv_rate"] = feature_vector["conv_rate"][0]
kwargs["acc_rate"] = feature_vector["acc_rate"][0]
kwargs["avg_daily_trips"] = feature_vector["avg_daily_trips"][0]
return prompt.format(**kwargs)

```

```code
prompt_template = FeastPromptTemplate(input_variables=["driver_id"])

```

```code
print(prompt_template.format(driver_id=1001))

```

```code
Given the driver's up to date stats, write them note relaying those stats to them.
If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel better

Here are the drivers stats:
Conversation rate: 0.4745151400566101
Acceptance rate: 0.055561766028404236
Average Daily Trips: 936

Your response:

```

在链中使用#
现在我们可以在链中使用这个，成功创建一个支持特征存储的个性化链

```code
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain

```

```code
chain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)

```

```code
chain.run(1001)

```

```code
"Hi there! I wanted to update you on your current stats. Your acceptance rate is 0.055561766028404236 and your average daily trips are 936. While your conversation rate is currently 0.4745151400566101, I have no doubt that with a little extra effort, you'll be able to exceed that .5 mark! Keep up the great work! And remember, even chickens can't always cross the road, but they still give it their best shot."

```

Tecton#
上面，我们展示了如何使用Feast，一个流行的开源和自管理特征存储，与LangChain一起使用。下面的示例将展示使用Tecton进行类似集成。Tecton是一个完全管理的特征平台，用于编排完整的ML特征生命周期，从转换到在线服务，具有企业级SLA。
先决条件#

Tecton部署（在https://tecton.ai (opens in a new tab)注册）

```code
TECTON_API_KEY
```
环境变量设置为有效的Service Account key

定义和加载特征#
我们将使用Tecton教程 (opens in a new tab)中的
```code
user_transaction_counts
```
特征视图作为特征服务的一部分。
为了简单起见，我们只使用一个特征视图；但是，更复杂的应用可能需要更多的特征视图来检索其提示所需的特征。

```code
user_transaction_metrics = FeatureService(
name = "user_transaction_metrics",
features = [user_transaction_counts]
)

```

以上特征服务的预期实现是： 应用到实时工作区 (opens in a new tab). For this example, we will be using the “prod” workspace.

```code
import tecton

workspace = tecton.get_workspace("prod")
feature_service = workspace.get_feature_service("user_transaction_metrics")

```

提示#
在这里，我们将设置一个自定义的TectonPromptTemplate。这个提示模板将接受一个user_id，查找他们的统计信息，并将这些统计信息格式化成一个提示。
请注意，此提示模板的输入只是
```code
user_id
```
，因为这是唯一的用户定义部分（所有其他变量都在提示模板内查找)。

```code
from langchain.prompts import PromptTemplate, StringPromptTemplate

```

```code
template = """Given the vendor's up to date transaction stats, write them a note based on the following rules:

1. If they had a transaction in the last day, write a short congratulations message on their recent sales
2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.
3. Always add a silly joke about chickens at the end

Here are the vendor's stats:
Number of Transactions Last Day: {transaction_count_1d}
Number of Transactions Last 30 Days: {transaction_count_30d}

Your response:"""
prompt = PromptTemplate.from_template(template)

```

```code
class TectonPromptTemplate(StringPromptTemplate):

def format(self, **kwargs) -> str:
user_id = kwargs.pop("user_id")
feature_vector = feature_service.get_online_features(join_keys={"user_id": user_id}).to_dict()
kwargs["transaction_count_1d"] = feature_vector["user_transaction_counts.transaction_count_1d_1d"]
kwargs["transaction_count_30d"] = feature_vector["user_transaction_counts.transaction_count_30d_1d"]
return prompt.format(**kwargs)

```

```code
prompt_template = TectonPromptTemplate(input_variables=["user_id"])

```

```code
print(prompt_template.format(user_id="user_469998441571"))

```

```code
Given the vendor's up to date transaction stats, write them a note based on the following rules:

1. If they had a transaction in the last day, write a short congratulations message on their recent sales
2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.
3. Always add a silly joke about chickens at the end

Here are the vendor's stats:
Number of Transactions Last Day: 657
Number of Transactions Last 30 Days: 20326

Your response:

```

在链中使用#
现在，我们可以在一个链中使用它，成功创建一个由Tecton Feature Platform支持的个性化链。

```code
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain

```

```code
chain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)

```

```code
chain.run("user_469998441571")

```

```code
'Wow, congratulations on your recent sales! Your business is really soaring like a chicken on a hot air balloon! Keep up the great work!'

```
提示模板（Prompt Templates）自定义提示模板（Custom Prompt Template）


自定义提示模板 – LangChain中文网

6大核心模块（Modules)提示工程（Prompts)提示模板（Prompt Templates）示例（Examples）自定义提示模板（Custom Prompt Template）

自定义提示模板
假设我们希望LLM根据函数的名称生成英语语言的解释。为了完成这个任务，我们将创建一个自定义提示模板，它以函数名称作为输入，并格式化提示模板以提供函数的源代码。
为什么需要自定义提示模板？
LangChain提供了一组默认提示模板，可用于生成各种任务的提示。
但是，可能存在默认提示模板不符合您需求的情况。例如，您可能希望创建具有特定动态说明的提示模板以供语言模型使用。
在这种情况下，可以创建自定义提示模板。
查看当前默认提示模板集合此处。
创建自定义提示模板
基本上有两种不同的提示模板可用——字符串提示模板和聊天提示模板。字符串提示模板以字符串格式提供简单提示，而聊天提示模板生成可用于聊天API的更结构化的提示。
在本指南中，我们将使用字符串提示模板创建自定义提示。
要创建自定义字符串提示模板，需要两个要求：

它具有input_variables属性，以公开提示模板期望的输入变量。
它公开format方法，该方法接受与预期的input_variables对应的关键字参数，并返回格式化的提示。

我们将创建一个自定义提示模板，它以函数名称作为输入，并格式化提示来提供函数的源代码。为此，让我们首先创建一个函数，该函数将根据其名称返回函数的源代码。

```code
import inspect

def get_source_code(function_name):
# Get the source code of the function
return inspect.getsource(function_name)

```

接下来，我们将创建一个自定义提示模板，它以函数名称作为输入，并格式化提示模板，以提供该函数的源代码。

```code
from langchain.prompts import StringPromptTemplate
from pydantic import BaseModel, validator

class FunctionExplainerPromptTemplate(StringPromptTemplate, BaseModel):
""" A custom prompt template that takes in the function name as input, and formats the prompt template to provide the source code of the function. """

@validator("input_variables")
def validate_input_variables(cls, v):
""" Validate that the input variables are correct. """
if len(v) != 1 or "function_name" not in v:
raise ValueError("function_name must be the only input_variable.")
return v

def format(self, **kwargs) -> str:
# Get the source code of the function
source_code = get_source_code(kwargs["function_name"])

# Generate the prompt to be sent to the language model
prompt = f"""
Given the function name and source code, generate an English language explanation of the function.
Function Name: {kwargs["function_name"].__name__}
Source Code:
{source_code}
Explanation:
"""
return prompt

def _prompt_type(self):
return "function-explainer"

```

使用 custom prompt template#
现在我们已经创建了自定义提示模板，可以使用它来为我们的任务生成提示。

```code
fn_explainer = FunctionExplainerPromptTemplate(input_variables=["function_name"])

# Generate a prompt for the function "get_source_code"
prompt = fn_explainer.format(function_name=get_source_code)
print(prompt)

```

```code
Given the function name and source code, generate an English language explanation of the function.
Function Name: get_source_code
Source Code:
def get_source_code(function_name):
# Get the source code of the function
return inspect.getsource(function_name)

Explanation:

```
连接特征存储（Connecting to a Feature Store）少量样例（Few-Shot Examples）