crun-vm/docs/2-podman-docker.md

2. Using crun-vm as a Podman or Docker runtime

Here we overview some of the major features provided by crun-vm. The commands below use podman, but unless otherwise stated you can simply replace it with docker.

Booting VMs

From regular VM image files

First, obtain a QEMU-compatible VM image and place it in a directory by itself:

$ mkdir my-vm-image
$ curl -LO --output-dir my-vm-image https://download.fedoraproject.org/pub/fedora/linux/releases/40/Cloud/x86_64/images/Fedora-Cloud-Base-Generic.x86_64-40-1.14.qcow2

Then run:

This example does not work with Docker, as docker-run does not support the --rootfs flag; see the next section for a Docker-compatible way of running VM images.

$ podman run \
    --runtime crun-vm \
    -it --rm \
    --rootfs my-vm-image \
    ""  # unused, but must specify command

The VM console should take over your terminal. At this point, the qcow2 image does not have any ssh keys, root password, or alternative users installed, so although you can interact with the VM's login screen, you will be unable to access a command prompt until more options are used in later sections. To abort the VM, press ctrl-].

You can also detach from the VM without terminating it by pressing ctrl-p, ctrl-q. Afterwards, reattach by running:

$ podman attach --latest

This command also works when you start the VM in detached mode using podman-run's -d/--detach flag.

It is also possible to omit flags -i/--interactive and -t/--tty to podman-run, in which case you won't be able to interact with the VM but can still observe its console. Note that pressing ctrl-] will have no effect, but you can always use the following command to terminate the VM:

For this command to work with Docker, you must replace the --latest flag with the container's name or ID.

$ podman stop --latest

Changes made by the VM to its image are by default not persisted in the original image file. This can be changed by passing in the non-standard option --persistent after the --rootfs option:

$ podman run \
    --runtime crun-vm \
    -it --rm \
    --rootfs my-vm-image \
    --persistent

[!WARNING]

When using --persistent, make sure that the image file is never simultaneously used by another process or VM, otherwise data corruption may occur.

From VM image files packaged into container images

crun-vm also works with container images that contain a VM image file with any name under / or under /disk/. No other files may exist in those directories. Containers built for use as KubeVirt containerDisks follow this convention, so you can use those here:

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/containerdisks/fedora:40

You can also use util/package-vm-image.sh to easily package a VM image into a container image, and util/extract-vm-image.sh to extract a VM image contained in a container image.

Note that flag --persistent has no effect when running VMs from container images.

From bootable container images

crun-vm can also work with bootable container images, which are containers that package a full operating system:

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/fedora/fedora-bootc:40

Internally, crun-vm generates a VM image from the bootable container and then boots it.

By default, the VM image is given a disk size roughly double the size of the bootc container image. To change this, use the --bootc-disk-size <size>[KMGT] option.

First-boot customization

cloud-init

In the examples above, you were able to boot the VM but not to log in. To fix this and do other first-boot customization, you can provide a cloud-init NoCloud configuration to the VM by passing in the non-standard option --cloud-init after the image specification:

$ ls examples/cloud-init/config/
meta-data  user-data  vendor-data

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/containerdisks/fedora:40 \
    --cloud-init ~/examples/cloud-init/config  # path must be absolute

You should now be able to log in with the default fedora username and password pass.

Alternatively, you can set the default user's password with the --password option:

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/containerdisks/fedora:40 \
    --password pass

Ignition

Similarly, you can provide an Ignition configuration to the VM by passing in the --ignition option:

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/crun-vm/example-fedora-coreos:40 \
    --ignition ~/examples/ignition/config.ign  # path must be absolute

You should now be able to log in with the default core username and password pass.

Note that the --password option requires cloud-init support and doesn't work if the VM uses Ignition.

SSH'ing into the VM

Assuming the VM supports cloud-init or Ignition and exposes an SSH server on port 22, you can ssh into it as root using podman-exec:

For this command to work with Docker, you must replace the --latest flag with the container's name or ID.

$ podman run \
    --runtime crun-vm \
    --detach --rm \
    quay.io/containerdisks/fedora:40
8068a2c180e0f4bf494f5e0baa37d9f13a9810f76b361c0771b73666e47ec383

$ podman exec --latest whoami
Please login as the user "fedora" rather than the user "root".

This particular VM image does not allow logging in as root. To ssh into the VM as a different user, specify its username using the --as option immediately before the command (if any). You may need to pass in -- before this option to prevent podman-exec from trying to interpret it:

$ podman exec --latest -- --as fedora whoami
fedora

If you just want a login shell, pass in an empty string as the command. The following would be the output if this VM image allowed logging in as root:

$ podman exec -it --latest ""
[root@8068a2c180e0 ~]$

You can also log in as a specific user:

$ podman exec -it --latest -- --as fedora
[fedora@8068a2c180e0 ~]$

When the VM supports cloud-init, authorized_keys is automatically set up to allow SSH access by podman-exec for users root and the default user as set in the image's cloud-init configuration. With Ignition, this is set up for users root and core.

If you want to exec into the container in which the VM is running (probably to debug some problem with crun-vm itself), pass in the --container flag immediately before the command (if any).

Port forwarding

You can use podman-run's standard -p/--publish option to set up TCP and/or UDP port forwarding:

$ podman run \
    --runtime crun-vm \
    --detach --rm \
    -p 8000:80 \
    quay.io/crun-vm/example-http-server:latest
36c8705482589cfc4336a03d3802e7699f5fb228123d18e693488ac7b80116d1

$ curl localhost:8000
<!DOCTYPE HTML>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Directory listing for /</title>
</head>
<body>
[...]

Passing things through to the VM

Directories

Bind mounting directories into the VM is supported:

[!WARNING]

This example recursively modifies the SELinux context of all files under the path being mounted, in this case ./util, which in the worst case may cause you to lose access to your files. This is due to the :z volume mount modifier, which instructs Podman to relabel the volume so that the VM can access it.

Alternatively, you may remove this modifier from the command below and add --security-opt label=disable instead to disable SELinux enforcement.

$ podman run \
    --runtime crun-vm \
    -it --rm \
    -v ./util:/home/fedora/util:z \
    quay.io/containerdisks/fedora:40 \
    --password pass

If the VM supports cloud-init or Ignition, the volume will automatically be mounted at the given destination path. Otherwise, you can mount it manually with the following command, where <index> must be the 0-based index of the volume according to the order the -v/--volume or --mount flags where given in:

$ mount -t virtiofs virtiofs-<index> /home/fedora/util

Regular files

Similarly to directories, you can bind mount regular files into the VM:

[!WARNING]

The warning about SELinux relabeling on the command above also applies here.

$ podman run \
    --runtime crun-vm \
    -it --rm \
    -v ./README.md:/home/fedora/README.md:z \
    quay.io/containerdisks/fedora:40 \
    --password pass

Regular files currently appear as block devices in the VM, but this is subject to change.

Block devices

If cloud-init or Ignition are supported by the VM, it is possible to pass block devices through to it at a specific path using podman-run's --device flag (this example assumes /dev/ram0 to exist and to be accessible by the current user):

$ podman run \
    --runtime crun-vm \
    -it --rm \
    --device /dev/ram0:/home/fedora/my-disk \
    quay.io/containerdisks/fedora:40 \
    --password pass

You can also use the more powerful --blockdev source=<path>,target=<path>,format=<fmt> custom option to this effect. This option also allows you specify a regular file as the source, and the source may be in any disk format known to QEMU (e.g., raw, qcow2; when using --device, raw format is assumed):

$ podman run \
    --runtime crun-vm \
    -it --rm \
    quay.io/containerdisks/fedora:40 \
    --password pass \
    --blockdev source=~/my-disk.qcow2,target=/home/fedora/my-disk,format=qcow2  # paths must be absolute

Advanced options

System emulation

To use system emulation instead of hardware-assisted virtualization, specify the --emulated flag. Without this flag, attempting to create a VM from a guest with a different architecture from the host's or on a host that doesn't support KVM will fail.

It's not currently possible to use this flag when the container image is a bootc bootable container.

Inspecting and customizing the libvirt domain XML

crun-vm internally uses libvirt to launch a VM, generating a domain XML definition from the options provided to podman-run. This XML definition can be printed by adding the non-standard --print-libvirt-xml flag to your podman-run invocation.

The generated XML definition can also be customized by specifying an XML file to be merged with it using the non-standard option --merge-libvirt-xml <file>.

[!NOTE]

While --merge-libvirt-xml gives you maximum flexibility, it thwarts crun-vm's premise of isolating the user from such details as libvirt domain definitions, and you have instead to take care that your XML is valid and that the customized definition is compatible with what crun-vm expects.

Before using this flag, consider if you would be better served using libvirt directly to manage your VM.