diff --git a/content/guides/use-case/genai-pdf-bot/_index.md b/content/guides/use-case/genai-pdf-bot/_index.md new file mode 100644 index 0000000000..bb4a09326a --- /dev/null +++ b/content/guides/use-case/genai-pdf-bot/_index.md @@ -0,0 +1,16 @@ +--- +description: Containerize generative AI (GenAI) apps using Docker +keywords: python, generative ai, genai, llm, neo4j, ollama, langchain +title: Generative AI guide +toc_min: 1 +toc_max: 2 +--- + +The generative AI (GenAI) guide teaches you how to containerize an existing GenAI application using Docker. In this guide, you’ll learn how to: + +* Containerize and run a Python-based GenAI application +* Set up a local environment to run the complete GenAI stack locally for development + +Start by containerizing an existing GenAI application. + +{{< button text="Containerize a GenAI app" url="containerize.md" >}} diff --git a/content/guides/use-case/genai-pdf-bot/containerize.md b/content/guides/use-case/genai-pdf-bot/containerize.md new file mode 100644 index 0000000000..1298ddf439 --- /dev/null +++ b/content/guides/use-case/genai-pdf-bot/containerize.md @@ -0,0 +1,133 @@ +--- +title: Containerize a generative AI application +keywords: python, generative ai, genai, llm, neo4j, ollama, containerize, intitialize, langchain, openai +description: Learn how to containerize a generative AI (GenAI) application. +--- + +## Prerequisites + +* You have installed the latest version of [Docker Desktop](../../../get-docker.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop. +* You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. + +## Overview + +This section walks you through containerizing a generative AI (GenAI) application using Docker Desktop. + +> **Note** +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +## Get the sample application + +The sample application used in this guide is a modified version of the PDF Reader application from the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. The application is a full stack Python application that lets you ask questions about a PDF file. + +The application uses [LangChain](https://www.langchain.com/) for orchestration, [Streamlit](https://streamlit.io/) for the UI, [Ollama](https://ollama.ai/) to run the LLM, and [Neo4j](https://neo4j.com/) to store vectors. + +Clone the sample application. Open a terminal, change directory to a directory that you want to work in, and run the following command to clone the repository: + +```console +$ git clone https://github.com/docker/docker-genai-sample +``` + +You should now have the following files in your `docker-genai-sample` directory. + +```text +├── docker-genai-sample/ +│ ├── .gitignore +│ ├── app.py +│ ├── chains.py +│ ├── env.example +│ ├── requirements.txt +│ ├── util.py +│ ├── LICENSE +│ └── README.md +``` + +## Initialize Docker assets + +Now that you have an application, you can use `docker init` to create the necessary Docker assets to containerize your application. Inside the `docker-genai-sample` directory, run the `docker init` command. `docker init` provides some default configuration, but you'll need to answer a few questions about your application. For example, this application uses Streamlit to run. Refer to the following `docker init` example and use the same answers for your prompts. + +```console +$ docker init +Welcome to the Docker Init CLI! + +This utility will walk you through creating the following files with sensible defaults for your project: + - .dockerignore + - Dockerfile + - compose.yaml + - README.Docker.md + +Let's get started! + +? What application platform does your project use? Python +? What version of Python do you want to use? 3.11.4 +? What port do you want your app to listen on? 8000 +? What is the command to run your app? streamlit run app.py --server.address=0.0.0.0 --server.port=8000 +``` + +You should now have the following contents in your `docker-genai-sample` +directory. + +```text +├── docker-genai-sample/ +│ ├── .dockerignore +│ ├── .gitignore +│ ├── app.py +│ ├── chains.py +│ ├── compose.yaml +│ ├── env.example +│ ├── requirements.txt +│ ├── util.py +│ ├── Dockerfile +│ ├── LICENSE +│ ├── README.Docker.md +│ └── README.md +``` + +To learn more about the files that `docker init` added, see the following: + - [Dockerfile](../../../engine/reference/builder.md) + - [.dockerignore](../../../engine/reference/builder.md#dockerignore-file) + - [compose.yaml](../../../compose/compose-file/_index.md) + + +## Run the application + +Inside the `docker-genai-sample` directory, run the following command in a +terminal. + +```console +$ docker compose up --build +``` + +Docker builds and runs your application. Depending on your network connection, it may take several minutes to download all the dependencies. You'll see a message like the following in the terminal when the application is running. + +```console +server-1 | You can now view your Streamlit app in your browser. +server-1 | +server-1 | URL: http://0.0.0.0:8000 +server-1 | +``` + +Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. The application may take a few minutes to download the embedding model. While the download is in progress, **Running** appears in the top-right corner. + +The application requires a Neo4j database service and an LLM service to +function. If you have access to services that you ran outside of Docker, specify +the connection information and try it out. If you don't have the services +running, continue with this guide to learn how you can run some or all of these +services with Docker. + +In the terminal, press `ctrl`+`c` to stop the application. + +## Summary + +In this section, you learned how you can containerize and run your GenAI +application using Docker. + +Related information: + - [docker init CLI reference](../../../engine/reference/commandline/init.md) + +## Next steps + +In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. + +{{< button text="Develop your application" url="develop.md" >}} \ No newline at end of file diff --git a/content/guides/use-case/genai-pdf-bot/develop.md b/content/guides/use-case/genai-pdf-bot/develop.md new file mode 100644 index 0000000000..acac48053e --- /dev/null +++ b/content/guides/use-case/genai-pdf-bot/develop.md @@ -0,0 +1,247 @@ +--- +title: Use containers for generative AI development +keywords: python, local, development, generative ai, genai, llm, neo4j, ollama, langchain, openai +description: Learn how to develop your generative AI (GenAI) application locally. +--- + +## Prerequisites + +Complete [Containerize a generative AI application](containerize.md). + +## Overview + +In this section, you'll learn how to set up a development environment to access all the services that your generative AI (GenAI) application needs. This includes: + +- Adding a local database +- Adding a local or remote LLM service + +> **Note** +> +> You can see more samples of containerized GenAI applications in the [GenAI Stack](https://github.com/docker/genai-stack) demo applications. + +## Add a local database + +You can use containers to set up local services, like a database. In this section, you'll update the `compose.yaml` file to define a database service. In addition, you'll specify an environment variables file to load the database connection information rather than manually entering the information every time. + +To run the database service: + +1. In the cloned repository's directory, rename `env.example` file to `.env`. + This file contains the environment variables that the containers will use. +2. In the cloned repository's directory, open the `compose.yaml` file in an IDE or text editor. +3. In the `compose.yaml` file, add the following: + - Add instructions to run a Neo4j database + - Specify the environment file under the server service in order to pass in the environment variables for the connection + + The following is the updated `compose.yaml` file. All comments have been removed. + + ```yaml{hl_lines=["7-23"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + database: + image: neo4j:5.11 + ports: + - "7474:7474" + - "7687:7687" + environment: + - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} + healthcheck: + test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"] + interval: 5s + timeout: 3s + retries: 5 + ``` + + > **Note** + > + > To learn more about Neo4j, see the [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j). + +4. Run the application. Inside the `docker-genai-sample` directory, +run the following command in a terminal. + + ```console + $ docker compose up --build + ``` + +5. Access the application. Open a browser and view the application at [http://localhost:8000](http://localhost:8000). You should see a simple Streamlit application. Note that asking questions to a PDF will cause the application to fail because the LLM service specified in the `.env` file isn't running yet. + +6. Stop the application. In the terminal, press `ctrl`+`c` to stop the application. + +## Add a local or remote LLM service + +The sample application supports both [Ollama](https://ollama.ai/) and [OpenAI](https://openai.com/). This guide provides instructions for the following scenarios: +- Run Ollama in a container +- Run Ollama outside of a container +- Use OpenAI + +While all platforms can use any of the previous scenarios, the performance and +GPU support may vary. You can use the following guidelines to help you choose the appropriate option: +- Run Ollama in a container if you're on Linux or Windows 11, you + have a CUDA-supported GPU, and your system has at least 8 GB of RAM. +- Run Ollama outside of a container if you're on an Apple silicon Mac. +- Use OpenAI if the previous two scenarios don't apply to you. + +Choose one of the following options for your LLM service. + +{{< tabs >}} +{{< tab name="Run Ollama in a container" >}} + +When running Ollama in a container, you should have a CUDA-supported GPU. While you can run Ollama in a container without a supported GPU, the performance may not be acceptable. Only Linux and Windows 11 support GPU access to containers. + +To run Ollama in a container and provide GPU access: +1. Install the prerequisites. + - For Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). + - For Windows 11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx). +2. Add the Ollama service and a volume in your `compose.yaml`. The following is + the updated `compose.yaml`: + + ```yaml {hl_lines=["24-38"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + database: + image: neo4j:5.11 + ports: + - "7474:7474" + - "7687:7687" + environment: + - NEO4J_AUTH=${NEO4J_USERNAME}/${NEO4J_PASSWORD} + healthcheck: + test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"] + interval: 5s + timeout: 3s + retries: 5 + ollama: + image: ollama/ollama:latest + ports: + - "11434:11434" + volumes: + - ollama_volume:/root/.ollama + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] + volumes: + ollama_volume: + ``` + + > **Note** + > + > For more details about the Compose instructions, see [Turn on GPU access with Docker Compose](../../../compose/gpu-support.md). + +3. Add the ollama-pull service to your `compose.yaml` file. This service uses + the `docker/genai:ollama-pull` image, based on the GenAI Stack's + [pull_model.Dockerfile](https://github.com/docker/genai-stack/blob/main/pull_model.Dockerfile). + The service will automatically pull the model for your Ollama + container. The following is the updated section of the `compose.yaml` file: + + ```yaml {hl_lines=["12-17"]} + services: + server: + build: + context: . + ports: + - 8000:8000 + env_file: + - .env + depends_on: + database: + condition: service_healthy + ollama-pull: + condition: service_completed_successfully + ollama-pull: + image: docker/genai:ollama-pull + env_file: + - .env + # ... + ``` + +{{< /tab >}} +{{< tab name="Run Ollama outside of a container" >}} + +To run Ollama outside of a container: +1. [Install](https://github.com/jmorganca/ollama) and run Ollama on your host + machine. +2. Update the `OLLAMA_BASE_URL` value in your `.env` file to + `http://host.docker.internal:11434`. +3. Pull the model to Ollama using the following command. + ```console + $ ollama pull llama2 + ``` + +{{< /tab >}} +{{< tab name="Use OpenAI" >}} + +> **Important** +> +> Using OpenAI requires an [OpenAI account](https://platform.openai.com/login). OpenAI is a third-party hosted service and charges may apply. +{ .important } + +1. Update the `LLM` value in your `.env` file to + `gpt-3.5`. +2. Uncomment and update the `OPENAI_API_KEY` value in your `.env` file to + your [OpenAI API key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-api-key). + +{{< /tab >}} +{{< /tabs >}} + +## Run your GenAI application + +At this point, you have the following services in your Compose file: +- Server service for your main GenAI application +- Database service to store vectors in a Neo4j database +- (optional) Ollama service to run the LLM +- (optional) Ollama-pull service to automatically pull the model for the Ollama + service + +To run all the services, run the following command in your `docker-genai-sample` +directory: + +```console +$ docker compose up --build +``` + +If your Compose file has the ollama-pull service, it may take several minutes for the ollama-pull service to pull the model. The ollama-pull service will continuously update the console with its status. After pulling the model, the ollama-pull service container will stop and you can access the application. + +Once the application is running, open a browser and access the application at [http://localhost:8000](http://localhost:8000). + +Upload a PDF file, for example the [Docker CLI Cheat Sheet](https://docs.docker.com/get-started/docker_cheatsheet.pdf), and ask a question about the PDF. + +Depending on your system and the LLM service that you chose, it may take several +minutes to answer. If you are using Ollama and the performance isn't +acceptable, try using OpenAI. + +## Summary + +In this section, you learned how to set up a development environment to provide +access all the services that your GenAI application needs. + +Related information: + - [Dockerfile reference](../../../engine/reference/builder.md) + - [Compose file reference](../../../compose/compose-file/_index.md) + - [Ollama Docker image](https://hub.docker.com/r/ollama/ollama) + - [Neo4j Official Docker Image](https://hub.docker.com/_/neo4j) + - [GenAI Stack demo applications](https://github.com/docker/genai-stack) + +## Next steps + +See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). \ No newline at end of file diff --git a/data/toc.yaml b/data/toc.yaml index 568570726d..2f42c0fab8 100644 --- a/data/toc.yaml +++ b/data/toc.yaml @@ -152,6 +152,18 @@ Guides: path: /language/php/configure-ci-cd/ - title: "Test your deployment" path: /language/php/deploy/ + +- sectiontitle: Use-case guides + section: + - sectiontitle: Generative AI + section: + - path: /guides/use-case/genai-pdf-bot/ + title: Overview + - path: /guides/use-case/genai-pdf-bot/containerize/ + title: Containerize your app + - path: /guides/use-case/genai-pdf-bot/develop/ + title: Develop your app + - sectiontitle: Develop with Docker section: - path: /develop/