Overview of ChatGPT and LangChain and their use

Model of generative AI and ChatGPT

Generative AI, also discussed in “Automatic Generation by Machine Learning,” involves a computer learning patterns and regularities in data and generating new data based on them. There are several different approaches to automatic generation. In this article, we will discuss the most successful of them, the Large Language Model (LLM) such as GPT.

GPT, as described in “Overview of GPT, Algorithms and Examples of Implementations,” is a similar model to the Transformer model described in “Overview of Transformer Model, Algorithms and Examples of Implementations,” and to the BERT model described in “Overview of BERT, Algorithms and Examples of Implementations. The GPT is an advanced version of the BERT model described in “Overview, Algorithms, and Examples of BERT” and is a model that demonstrates high functionality by applying the attention mechanism described in “Attention in Deep Learning” to deep learning.

Open source versions of these models are available from Huggingface as described in “Overview of Automatic Sentence Generation with Huggingface” and can be used codelessly with text-generation-webui and AUTOMATIC1111, etc. as described in “Codeless Generation with text-generation-webui and AUTOMATIC1111

The most successful system using these models is ChatGPT, which was developed by OpenAI based on GPT3.5 and other LLMs, and is specialized for dialogue, allowing users to chat with AI and receive natural text. ChatGPT is used not only for text generation, but also for summarization, translation, Q&A, programming, and various content generation.

By using these functions, prompt engineering (a technique or method of devising a given text prompt to elicit the best response for a specific task or purpose in the development of natural language processing or machine learning models), as described in “Overview of Prompt Engineering and Its Uses,” becomes possible. This enables functions that could previously only be realized by combining many libraries, such as natural language processing, to be realized by simply providing appropriate questions or instructions to the model.

ChatGPT itself can be used free of charge, but a paid version of the service via API allows users to build their own systems with a variety of functions.

To use the paid API service, first access OpenAI, click [sign up], and create an account by entering your email address or using an account with Google or other services.

Once you have created an account, select “OpenAI API”, go to the “Welcome to the OpenAI developer platform” page, click on the top left mark, and select “API Keys” from the several options. You can go to the API KEY page.

In that page, click the “create new secret key” button to open a window, give it an arbitrary name, and click “create secret key” to generate an API KEY starting with “sk-“.

In case of general system construction, the OpenAI API is used by storing the key information in an environment variable to prevent leakage of the key information. SetEnviromentVariable command using PowerShell. Specifically, select “PowerShell” from the [Start] menu and set as follows.

[System.Enviroment]::SetEnviromentVariable('OPENAI_API_KEY','Obtained API Key","User')

Just executing the above command does not reflect the environment variable, so quit PowerShell once and execute the following command to display the API key you have set, and you are done.

echo &env:OPENAI_API_KEY

The zsh environment variables commonly used on macOS are set in the .zshrc file. (If you do not have a .zshrc file, use the command “touch ~/.zshrc” to create one.)

echo 'export OPENAI_API_KEY="Obtained API Key"' >> ~/.zshrc　← Writing to .zshrc file

source ~/.zshrc    ←Commands for reading .zshrc file settings

If the API key is displayed with the following command, the setting is complete.

echo $OPENAI_API_KEY

To manipulate OpenAI’s ChatGPT with python, install the Python library provided by OpenAI and

pip install openai

It can be implemented by implementing the following code.

import os
import openai

openai.api.key = os.environ["OPENAI_API_KEY"]  #API key set in environment variable

response = openai.ChatCompletion.create{
    model="gpt-3.5-turbo",
    message=[
         {"role": "user", "content": "Tell us about Python"},
    ],
}

print(response.choices[0]["message"]["content"]}

By using the above python code, it is possible to connect to ChatGPT and handle them.

Using OPENAI_API in this way, it is possible to build a solution in python, but if you try to build a complex system, the system of codes and variables will become complicated, and the productivity will decrease. LangChain is one of the ways to improve these problems.

Overview of LangChain

LangChain is a library that helps develop applications using language models and provides a platform on which various applications using ChatGPT and other generative models can be built.

One of the goals of LangChain is to allow for tasks that language models such as ChatGPT cannot perform, such as answering questions about information outside the scope of knowledge learned by the language model, or tasks that are logically complex or require computation, etc. Another goal is to provide a framework The other is to develop the framework as a framework to enable efficient development of complex applications such as the RAG described in “Overview of RAG (Retrieval-Augmented Generation) and its implementation examples” and the ReAct described in “Overview of ReAct (Reasoning and Acting) and its implementation examples“. The purpose of LangChain is to enable efficient development of complex applications such as RAG and ReAct (Reasoning and Acting).

Using LangChain, it is possible to efficiently perform tasks that have been difficult with conventional procedural programming, such as understanding natural language, generating answers to specific questions, and developing applications that operate under specific environments.

LangChain includes the following six modules. Each module can be used by itself, but by combining multiple modules, complex LLM applications can be created efficiently.

Model I/O (to make language models easier to handle): To develop an application using language models, it is necessary to invoke the language models. To invoke a language model, three steps are required: “prepare the prompt” as input, “invoke the language model,” and “receive the result.

Retrieval (making unknown data visible): The Retrieval module allows access to unknown information that is not in the language model. Using this module, you can create a chatbot that can ask questions about a PDF file at hand or provide customer support based on a CSV file with millions of Q&As stored.” The RAG described in “Overview of RAG (Retrieval-Augmented Generation) and examples of its implementation” can be realized with this module.

Memory (short- and long-term memory of past dialogues): In order for a language model to respond in the form of contextualized dialogues, it is necessary to send back all previous dialogues to the API. To achieve this, it is necessary to store past dialogues in a database and load them when calling the language model, and the Memory module provides functions to achieve this.

Chains (combining multiple processes): Provides the ability to combine a number of modules or different functions into a single application.

Agent (autonomously interferes with the outside world and exceeds the limits of the language model): The Agent model mainly provides the ReAct functionality described in “Overview of ReAct (Reasoning and Acting) and its implementation examples” and the OpenAI Function Calling technique, For details on the Agent module, see “Agents and Tools in LangChain“.

Callback (processing when various events occur): The Callback module provides the ability to perform processing when an event occurs in an application created using LamhChain. This is mainly used for log output and integration with external libraries.

Example of LangChain implementation

An example implementation using the Model I/O module is described below. to implement LangChain, the aforementioned opensi and langchain libraries must be installed.

pip install openai
pip install langchain
pip install -U langchain-community 

まず最もシンプルなコードは以下のようになる。

from langchain_community.chat_models import ChatOpenAI

from langchain.schema import HumanMessage #Import HumanMessage, messages from users

chat = ChatOpenAI(
    model="gpt-3.5-turbo",  #Specify the model to call
)

result = chat(
    [
        HumanMessage(content="Hello!"),
    ]
)
print(result.content)

When this code is executed, it accesses ChatGPT and returns the following answer.

Hello! How are you? Is there anything I can help you with?

In Model I/O, AIMessage can be used to represent a response from the language model in order to represent an interactive exchange. This is used to receive a message from the user first, and then use that response to create another response. The following code represents only the RESULT portion of the above code.

result = chat(
    [
        HumanMessage(content="Teach me how to make chawanmushi."),
　　　　　AIMessages(content="{How to make chawanmushi, a reply from ChatModel}"),
        HumanMessages(content="Translate to English."),
    ]
)

SystemMessage, which further customizes the dialogue function, is also available.

result = chat(
    [
        SystemMessage(content="You are a close friend. Please do not use honorifics in your replies, but speak frankly."),
        HumanMessages(content="Hello"),
    ]
)

It can also be used in conjunction with the Language model, combining input from the prepared Prompt and python.

from langchain import PromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage

chat = ChatOpenAI(  
    model="gpt-3.5-turbo",  
)

prompt = PromptTemplate(  #← Create a PromptTemplate
    template="What company developed the {product}?",  #← Create a prompt containing the variable {product}.
    input_variables=[
        "product"  #← Specify variables to be entered into product
    ]
)

result = chat( 
    [
        HumanMessage(content=prompt.format(product="iPhone")),
    ]
)
print(result.content) 

In addition, the output format can be specified.

from langchain.chat_models import ChatOpenAI
from langchain.output_parsers import 
    CommaSeparatedListOutputParser  #← Output ParserであるCommaSeparatedListOutputParserをインポート
from langchain.schema import HumanMessage

output_parser = CommaSeparatedListOutputParser() #←Initialize CommercialSeparatedListOutputParser

chat = ChatOpenAI(model="gpt-3.5-turbo", )

result = chat(
    [
        HumanMessage(content="What are three major products developed by Apple?"),
        HumanMessage(content=output_parser.get_format_instructions()),  #← Execute output_parser.get_format_instructions() to add instructions to the language model
    ]
)

output = output_parser.parse(result.content) #← Parse output results and convert to list format

for item in output: #← Retrieve lists one by one
    print("Typical Products => " + item)

One use of Template is to have the language model perform the desired task while presenting examples. (Few-shot-prompt)

from langchain.llms import OpenAI
from langchain.prompts import FewShotPromptTemplate, PromptTemplate

examples = [
    {
        "input": "LangChain is a set of tools for more flexible and simplified use of ChatGPT and Large Language Model (LLM).",  #← Input Example
        "output": "LangChain is a set of tools for more flexible and simplified practical use of ChatGPT and Large Language Model (LLM)."  #← Output Example
    }
]

prompt = PromptTemplate(  #← Prepare PromptTemplate
    input_variables=["input", "output"],  #← Set input and output as input variables
    template="Input: {input}n出力: {output}",  #← template
)

few_shot_prompt = FewShotPromptTemplate(  #← Preparation of FewShotPromptTemplate
    examples=examples,  #← Define input and output examples
    example_prompt=prompt,  #← Pass PromptTemplate to FewShotPromptTemplate
    prefix="Please add punctuation to the following missing input. The only punctuation allowed is "," ". Only "" and "." are allowed. Do not add any other punctuation.",  #← Add instructions
    suffix="input: {input_string}noutput:",  #← Define input variables for example output
    input_variables=["input_string"],  #← Set input variables for FewShotPromptTemplate
)
llm = OpenAI()
formatted_prompt = few_shot_prompt.format( #← Create prompts using FewShotPromptTemplate
    input_string="I am developing an application using LangChain, which provides various functions as modules."
)
result = llm.predict(formatted_prompt)
print("formatted_prompt: ", formatted_prompt)
print("result: ", result)

One way to interact with this response on the chatbot is to use the simple Javascript library described in “Implementing a Chatbot System Using Node.js and React” and “Implementing a Chatbot and Integrating AI Features Using Clojure and Javascript

In the next article, I will discuss Agents and Tools, frameworks for more general use of this LangChain.

Reference Information and Reference Books

“LangChain Crash Course: Build OpenAI LLM powered Apps: Fast track to building OpenAI LLM powered Apps using Python (English Edition)“

“Mastering LangChain: From Beginner to Expert“