docker
/
docs
mirror of https://github.com/docker/docs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

get-started: add genai guide (#18944) 

* add genai guide

Signed-off-by: Craig Osterhout <craig.osterhout@docker.com>
This commit is contained in:
Craig Osterhout 2024-01-18 19:02:08 -08:00 committed by GitHub
parent c6eaf89447
commit 3345dd20af
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 408 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
content/guides/use-case/genai-pdf-bot/_index.md Normal file
View File

@ -0,0 +1,16 @@
---
description: Containerize generative AI (GenAI) apps using Docker
keywords: python, generative ai, genai, llm, neo4j, ollama, langchain
title: Generative AI guide
toc_min: 1
toc_max: 2
---
The generative AI (GenAI) guide teaches you how to containerize an existing GenAI application using Docker. In this guide, you’ll learn how to:
* Containerize and run a Python-based GenAI application
* Set up a local environment to run the complete GenAI stack locally for development
Start by containerizing an existing GenAI application.
{{< button text="Containerize a GenAI app" url="containerize.md" >}}

133
content/guides/use-case/genai-pdf-bot/containerize.md Normal file
View File

@ -0,0 +1,133 @@
---
title: Containerize a generative AI application
keywords: python, generative ai, genai, llm, neo4j, ollama, containerize, intitialize, langchain, openai
description: Learn how to containerize a generative AI (GenAI) application.
---
## Prerequisites
* You have installed the latest version of [Docker Desktop](../../../get-docker.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop.
* You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client.
## Overview
This section walks you through containerizing a generative AI (GenAI) application using Docker Desktop.
> **Note**
>
> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications.
## Get the sample application
The sample application used in this guide is a modified version of the PDF Reader application from the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. The application is a full stack Python application that lets you ask questions about a PDF file.
The application uses [LangChain](https://www.langchain.com/) for orchestration, [Streamlit](https://streamlit.io/) for the UI, [Ollama](https://ollama.ai/) to run the LLM, and [Neo4j](https://neo4j.com/) to store vectors.
Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository:
```console
$ git clone https://github.com/docker/docker-genai-sample
```
You should now have the following files in your `docker-genai-sample` directory.
```text
├── docker-genai-sample/
│ ├── .gitignore
│ ├── app.py
│ ├── chains.py
│ ├── env.example
│ ├── requirements.txt
│ ├── util.py
│ ├── LICENSE
│ └── README.md
```
## Initialize Docker assets
Now that you have an application, you can use `docker init` to create the necessary Docker assets to containerize your application. Inside the `docker-genai-sample` directory, run the `docker init` command. `docker init` provides some default configuration, but you'll need to answer a few questions about your application. For example, this application uses Streamlit to run. Refer to the following `docker init` example and use the same answers for your prompts.
```console
$ docker init
Welcome to the Docker Init CLI!
This utility will walk you through creating the following files with sensible defaults for your project:
- .dockerignore
- Dockerfile
- compose.yaml
- README.Docker.md
Let's get started!
? What application platform does your project use? Python
? What version of Python do you want to use? 3.11.4
? What port do you want your app to listen on? 8000
? What is the command to run your app? streamlit run app.py --server.address=0.0.0.0 --server.port=8000
```
You should now have the following contents in your `docker-genai-sample`
directory.
```text
├── docker-genai-sample/
│ ├── .dockerignore
│ ├── .gitignore
│ ├── app.py
│ ├── chains.py
│ ├── compose.yaml
│ ├── env.example
│ ├── requirements.txt
│ ├── util.py
│ ├── Dockerfile
│ ├── LICENSE
│ ├── README.Docker.md
│ └── README.md
```
To learn more about the files that `docker init` added, see the following:
- [Dockerfile](../../../engine/reference/builder.md)
- [.dockerignore](../../../engine/reference/builder.md#dockerignore-file)
- [compose.yaml](../../../compose/compose-file/_index.md)
## Run the application
Inside the `docker-genai-sample` directory, run the following command in a
terminal.
```console
$ docker compose up --build
```
Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running.
```console
server-1 | You can now view your Streamlit app in your browser.
server-1 |
server-1 | URL: http://0.0.0.0:8000
server-1 |
```
Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. The application may take a few minutes to download the embedding model. While the download is in progress, **Running** appears in the top-right corner.
The application requires a Neo4j database service and an LLM service to
function. If you have access to services that you ran outside of Docker, specify
the connection information and try it out. If you don't have the services
running, continue with this guide to learn how you can run some or all of these
services with Docker.
In the terminal, press `ctrl`+`c` to stop the application.
## Summary
In this section, you learned how you can containerize and run your GenAI
application using Docker.
Related information:
- [docker init CLI reference](../../../engine/reference/commandline/init.md)
## Next steps
In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker.
{{< button text="Develop your application" url="develop.md" >}}

247
content/guides/use-case/genai-pdf-bot/develop.md Normal file
View File

@ -0,0 +1,247 @@
---
title: Use containers for generative AI development
keywords: python, local, development, generative ai, genai, llm, neo4j, ollama, langchain, openai
description: Learn how to develop your generative AI (GenAI) application locally.
---
## Prerequisites
Complete [Containerize a generative AI application](containerize.md).
## Overview
In this section, you'll learn how to set up a development environment to access all the services that your generative AI (GenAI) application needs. This includes:
- Adding a local database
- Adding a local or remote LLM service
> **Note**
>
> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications.
## Add a local database
You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time.
To run the database service:
1. In the cloned repository's directory, rename `env.example` file to `.env`.
This file contains the environment variables that the containers will use.
2. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor.
3. In the `compose.yaml` file, add the following:
- Add instructions to run a Neo4j database
- Specify the environment file under the server service in order to pass in the environment variables for the connection
The following is the updated `compose.yaml` file. All comments have been removed.
```yaml{hl_lines=["7-23"]}
services:
server:
build:
context: .
ports:
- 8000:8000
env_file:
- .env
depends_on:
database:
condition: service_healthy
database:
image: neo4j:5.11
ports:
- "7474:7474"
- "7687:7687"
environment:
- NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"]
interval: 5s
timeout: 3s
retries: 5
```
> **Note**
>
> To learn more about Neo4j, see the [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j).
4. Run the application. Inside the `docker-genai-sample` directory,
run the following command in a terminal.
```console
$ docker compose up --build
```
5. Access the application. Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. Note that asking questions to a PDF will cause the application to fail because the LLM service specified in the `.env` file isn't running yet.
6. Stop the application. In the terminal, press `ctrl`+`c` to stop the application.
## Add a local or remote LLM service
The sample application supports both [Ollama](https://ollama.ai/) and [OpenAI](https://openai.com/). This guide provides instructions for the following scenarios:
- Run Ollama in a container
- Run Ollama outside of a container
- Use OpenAI
While all platforms can use any of the previous scenarios, the performance and
GPU support may vary. You can use the following guidelines to help you choose the appropriate option:
- Run Ollama in a container if you're on Linux or Windows 11, you
have a CUDA-supported GPU, and your system has at least 8 GB of RAM.
- Run Ollama outside of a container if you're on an Apple silicon Mac.
- Use OpenAI if the previous two scenarios don't apply to you.
Choose one of the following options for your LLM service.
{{< tabs >}}
{{< tab name="Run Ollama in a container" >}}
When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers.
To run Ollama in a container and provide GPU access:
1. Install the prerequisites.
- For Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit).
- For Windows 11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx).
2. Add the Ollama service and a volume in your `compose.yaml`. The following is
the updated `compose.yaml`:
```yaml {hl_lines=["24-38"]}
services:
server:
build:
context: .
ports:
- 8000:8000
env_file:
- .env
depends_on:
database:
condition: service_healthy
database:
image: neo4j:5.11
ports:
- "7474:7474"
- "7687:7687"
environment:
- NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"]
interval: 5s
timeout: 3s
retries: 5
ollama:
image: ollama/ollama:latest
ports:
- "11434:11434"
volumes:
- ollama_volume:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
ollama_volume:
```
> **Note**
>
> For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](../../../compose/gpu-support.md).
3. Add the ollama-pull service to your `compose.yaml` file. This service uses
the `docker/genai:ollama-pull` image, based on the GenAI Stack's
[pull_model.Dockerfile](https://github.com/docker/genai-stack/blob/main/pull_model.Dockerfile).
The service will automatically pull the model for your Ollama
container. The following is the updated section of the `compose.yaml` file:
```yaml {hl_lines=["12-17"]}
services:
server:
build:
context: .
ports:
- 8000:8000
env_file:
- .env
depends_on:
database:
condition: service_healthy
ollama-pull:
condition: service_completed_successfully
ollama-pull:
image: docker/genai:ollama-pull
env_file:
- .env
# ...
```
{{< /tab >}}
{{< tab name="Run Ollama outside of a container" >}}
To run Ollama outside of a container:
1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host
machine.
2. Update the `OLLAMA_BASE_URL` value in your `.env` file to
`http://host.docker.internal:11434`.
3. Pull the model to Ollama using the following command.
```console
$ ollama pull llama2
```
{{< /tab >}}
{{< tab name="Use OpenAI" >}}
> **Important**
>
> Using OpenAI requires an [OpenAI account](https://platform.openai.com/login). OpenAI is a third-party hosted service and charges may apply.
{ .important }
1. Update the `LLM` value in your `.env` file to
`gpt-3.5`.
2. Uncomment and update the `OPENAI_API_KEY` value in your `.env` file to
your [OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-api-key).
{{< /tab >}}
{{< /tabs >}}
## Run your GenAI application
At this point, you have the following services in your Compose file:
- Server service for your main GenAI application
- Database service to store vectors in a Neo4j database
- (optional) Ollama service to run the LLM
- (optional) Ollama-pull service to automatically pull the model for the Ollama
service
To run all the services, run the following command in your `docker-genai-sample`
directory:
```console
$ docker compose up --build
```
If your Compose file has the ollama-pull service, it may take several minutes for the ollama-pull service to pull the model. The ollama-pull service will continuously update the console with its status. After pulling the model, the ollama-pull service container will stop and you can access the application.
Once the application is running, open a browser and access the application at [http://localhost:8000](http://localhost:8000).
Upload a PDF file, for example the [Docker CLI Cheat Sheet](https://docs.docker.com/get-started/docker_cheatsheet.pdf), and ask a question about the PDF.
Depending on your system and the LLM service that you chose, it may take several
minutes to answer. If you are using Ollama and the performance isn't
acceptable, try using OpenAI.
## Summary
In this section, you learned how to set up a development environment to provide
access all the services that your GenAI application needs.
Related information:
- [Dockerfile reference](../../../engine/reference/builder.md)
- [Compose file reference](../../../compose/compose-file/_index.md)
- [Ollama Docker image](https://hub.docker.com/r/ollama/ollama)
- [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j)
- [GenAI Stack demo applications](https://github.com/docker/genai-stack)
## Next steps
See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack).

12
data/toc.yaml
View File

@ -152,6 +152,18 @@ Guides:
path: /language/php/configure-ci-cd/
- title: "Test your deployment"
path: /language/php/deploy/
- sectiontitle: Use-case guides
section:
- sectiontitle: Generative AI
section:
- path: /guides/use-case/genai-pdf-bot/
title: Overview
- path: /guides/use-case/genai-pdf-bot/containerize/
title: Containerize your app
- path: /guides/use-case/genai-pdf-bot/develop/
title: Develop your app
- sectiontitle: Develop with Docker
section:
- path: /develop/