From 728f70cc855a3e11b70f1edd198c0135ac56e25c Mon Sep 17 00:00:00 2001 From: Piotr Stankiewicz Date: Fri, 9 Feb 2024 15:07:01 +0100 Subject: [PATCH] Fix the GenAI usecase guide Attempt to make it clearer where users can expect to be able to use GPU acceleration. Currently that is (1) docker-ce on Linux; (2) docker desktop on windows wsl2. Signed-off-by: Piotr Stankiewicz --- content/guides/use-case/genai-pdf-bot/containerize.md | 8 ++++++-- content/guides/use-case/genai-pdf-bot/develop.md | 8 ++++---- 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/content/guides/use-case/genai-pdf-bot/containerize.md b/content/guides/use-case/genai-pdf-bot/containerize.md index b533b8bdf8..9e6c8ca3f4 100644 --- a/content/guides/use-case/genai-pdf-bot/containerize.md +++ b/content/guides/use-case/genai-pdf-bot/containerize.md @@ -6,7 +6,11 @@ description: Learn how to containerize a generative AI (GenAI) application. ## Prerequisites -* You have installed the latest version of [Docker Desktop](../../../get-docker.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop. +> **Note** +> +> GenAI applications can often benefit from GPU acceleration. Currently Docker Desktop supports GPU acceleration only on [Windows with the WSL2 backend](../../../desktop/gpu.md#using-nvidia-gpus-with-wsl2). Linux users can also access GPU acceleration using a native installation of the [Docker Engine](../../../engine/install/_index.md). + +* You have installed the latest version of [Docker Desktop](../../../get-docker.md) or, if you are a Linux user and are planning to use GPU acceleration, [Docker Engine](../../../engine/install/_index.md). Docker adds new features regularly and some parts of this guide may work only with the latest version of Docker Desktop. * You have a [git client](https://git-scm.com/downloads). The examples in this section use a command-line based git client, but you can use any client. ## Overview @@ -130,4 +134,4 @@ Related information: In the next section, you'll learn how you can run your application, database, and LLM service all locally using Docker. -{{< button text="Develop your application" url="develop.md" >}} \ No newline at end of file +{{< button text="Develop your application" url="develop.md" >}} diff --git a/content/guides/use-case/genai-pdf-bot/develop.md b/content/guides/use-case/genai-pdf-bot/develop.md index 6c0dcadc82..adaf300674 100644 --- a/content/guides/use-case/genai-pdf-bot/develop.md +++ b/content/guides/use-case/genai-pdf-bot/develop.md @@ -84,7 +84,7 @@ The sample application supports both [Ollama](https://ollama.ai/) and [OpenAI](h While all platforms can use any of the previous scenarios, the performance and GPU support may vary. You can use the following guidelines to help you choose the appropriate option: -- Run Ollama in a container if you're on Linux or Windows 11, you +- Run Ollama in a container if you're on Linux, and using a native installation of the Docker Engine, or Windows 10/11, and using Docker Desktop, you have a CUDA-supported GPU, and your system has at least 8 GB of RAM. - Run Ollama outside of a container if you're on an Apple silicon Mac. - Use OpenAI if the previous two scenarios don't apply to you. @@ -98,8 +98,8 @@ When running Ollama in a container, you should have a CUDA-supported GPU. While To run Ollama in a container and provide GPU access: 1. Install the prerequisites. - - For Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). - - For Windows 11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx). + - For Docker Engine on Linux, install the [NVIDIA Container Toolkilt](https://github.com/NVIDIA/nvidia-container-toolkit). + - For Docker Desktop on Windows 10/11, install the latest [NVIDIA driver](https://www.nvidia.com/Download/index.aspx) and make sure you are using the [WSL2 backend](../../../desktop/wsl/index.md/#turn-on-docker-desktop-wsl-2) 2. Add the Ollama service and a volume in your `compose.yaml`. The following is the updated `compose.yaml`: @@ -244,4 +244,4 @@ Related information: ## Next steps -See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack). \ No newline at end of file +See samples of more GenAI applications in the [GenAI Stack demo applications](https://github.com/docker/genai-stack).