docker
/
model-runner
mirror of https://github.com/docker/model-runner.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Prepare Docker file ahead of multi-backend builds of llama-server 

llama.cpp can be built to support multiple variants of the CPU backend,
and choose the best one at runtime. This requires a dynamically linked
build of the llama-server. So modify the model-runner Docker file to
play nice with such builds of the llama-server.

Signed-off-by: Piotr Stankiewicz <piotr.stankiewicz@docker.com>
This commit is contained in:
Piotr Stankiewicz 2025-06-12 15:30:21 +02:00 committed by Piotr
parent 5047eede95
commit 0130eb6cf6
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Dockerfile
View File

@ -53,7 +53,7 @@ COPY --from=builder /app/model-runner /app/model-runner
# Copy the llama.cpp binary from the llama-server stage
ARG LLAMA_BINARY_PATH
COPY --from=llama-server ${LLAMA_BINARY_PATH}/bin/com.docker.llama-server /app/bin/com.docker.llama-server
COPY --from=llama-server ${LLAMA_BINARY_PATH}/ /app/.
RUN chmod +x /app/bin/com.docker.llama-server
USER modelrunner
@ -64,6 +64,7 @@ ENV MODEL_RUNNER_PORT=12434
ENV LLAMA_SERVER_PATH=/app/bin
ENV HOME=/home/modelrunner
ENV MODELS_PATH=/models
ENV LD_LIBRARY_PATH=/app/lib
# Label the image so that it's hidden on cloud engines.
LABEL com.docker.desktop.service="model-runner"