Adds makefile (#20)

* Adds Makefile for local development

* Fix chat completions example request

* Added delete example
This commit is contained in:
Ignasi 2025-04-25 09:38:54 +02:00 committed by GitHub
parent b02cdd9568
commit 7545f180cd
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 130 additions and 0 deletions

2
.gitignore vendored
View File

@ -1 +1,3 @@
.idea/
model-runner
model-runner.sock

30
Makefile Normal file
View File

@ -0,0 +1,30 @@
# Project variables
APP_NAME := model-runner
GO_VERSION := 1.23.7
# Main targets
.PHONY: build run clean help
# Default target
.DEFAULT_GOAL := help
# Build the Go application
build:
CGO_ENABLED=0 go build -ldflags="-s -w" -o $(APP_NAME) ./main.go
# Run the application locally
run: build
./$(APP_NAME)
# Clean build artifacts
clean:
rm -f $(APP_NAME)
rm -f model-runner.sock
# Show help
help:
@echo "Available targets:"
@echo " build - Build the Go application"
@echo " run - Run the application locally"
@echo " clean - Clean build artifacts"
@echo " help - Show this help message"

View File

@ -14,3 +14,101 @@ with [Model Distribution](https://github.com/docker/model-distribution) and the
[Model CLI](https://github.com/docker/model-cli)). It includes a `main.go` that
mimics its integration with Docker Desktop and allows the package to be run in a
standalone mode.
## Using the Makefile
This project includes a Makefile to simplify common development tasks. It requires Docker Desktop >= 4.41.0
The Makefile provides the following targets:
- `build` - Build the Go application
- `run` - Run the application locally
- `clean` - Clean build artifacts
- `help` - Show available targets
### Examples
```sh
# Build the application
make build
# Run the application locally
make run
# Show all available targets
make help
```
## API Examples
The Model Runner exposes a REST API over a Unix socket. You can interact with it using curl commands with the `--unix-socket` option.
### Listing Models
To list all available models:
```sh
curl --unix-socket model-runner.sock localhost/models
```
### Creating a Model
To create a new model:
```sh
curl --unix-socket model-runner.sock localhost/models/create -X POST -d '{"from": "ai/smollm2"}'
```
### Getting Model Information
To get information about a specific model:
```sh
curl --unix-socket model-runner.sock localhost/models/ai/smollm2
```
### Chatting with a Model
To chat with a model, you can send a POST request to the model's chat endpoint:
```sh
curl --unix-socket model-runner.sock localhost/engines/llama.cpp/v1/chat/completions -X POST -d '{
"model": "ai/smollm2",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello, how are you?"}
]
}'
```
The response will contain the model's reply:
```json
{
"id": "chat-12345",
"object": "chat.completion",
"created": 1682456789,
"model": "ai/smollm2",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "I'm doing well, thank you for asking! How can I assist you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 16,
"total_tokens": 40
}
}
```
### Deleting a model
To delete a model from the server, send a DELETE request to the model's endpoint:
```sh
curl --unix-socket model-runner.sock localhost/models/ai/smollm2 -X DELETE
```