containers
/
podman-ai-lab-stack
mirror of https://github.com/containers/podman-ai-lab-stack.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Podman AI Lab provider for Llama Stack
25 Commits 4 Branches 0 Tags 54 KiB
Python 96.8%
Dockerfile 3.2%
Go to file
Philippe Martin 8d6a4a9a7c
Merge pull request #10 from feloy/fix/add-missing-method 
fix: openai_embeddings method added in llama-stack 0.2.10
 		2025-06-06 10:08:45 +02:00
.github/workflows ci: use github actions 2025-06-06 07:22:10 +02:00
build v0.12.0 2025-06-04 15:45:54 +02:00
src/podman_ai_lab_stack fix: openai_embeddings method added in llama-stack 0.2.10 2025-06-06 10:08:15 +02:00
.gitignore v0.11.0 2025-06-04 15:00:40 +02:00
LICENSE License 2025-06-05 08:39:33 +02:00
README.md doc: update README with new repo 2025-06-05 12:01:45 +02:00
pyproject.toml v0.12.0 2025-06-04 15:45:54 +02:00
requirements.txt v0.9.0 2025-06-04 12:32:51 +02:00

README.md

podman-ai-lab-stack

An external provider for Llama Stack allowing for the use of Podman AI Lab for inference.

Usage

  1. Ensure Podman Desktop is running and AI Lab extension is loaded

  2. Run the Podman AI Lab external provider inside of a container via Podman:

podman run -p 8321:8321 ghcr.io/containers/podman-ai-lab-stack:nightly

This will start a Llama Stack server which will use port 8321 by default. You can test this works by using the Llama Stack Client:

llama-stack-client models list

llama-stack-client models register <id-of-model-loaded-in-podman-ai-lab>

llama-stack-client inference chat-completion --message "tell me a joke" --stream

Configuration

By default, Podman AI Lab listens on port 10434, and the Podman AI Lab external provider is configured to access this port by default. If you want to provide another address/port, you can pass the PODMAN_AI_LAB_URL environment variable to the provider, for example:

podman run -p 8321:8321 --env PODMAN_AI_LAB_URL=http://host.containers.internal:10435 ghcr.io/containers/podman-ai-lab-stack:nightly