containers
/
podman
mirror of https://github.com/containers/podman.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
Podman: A tool for managing OCI containers and pods.
13,248 Commits 52 Branches 247 Tags 253 MiB
Go 81.4%
Shell 13.5%
Perl 2%
Python 1.3%
PowerShell 0.8%
Other 0.9%
Go to file
Paul Holzinger 0136a66a83
libpod: deduplicate ports in db 
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
 		2021-10-27 18:59:56 +02:00
.copr Switch all references to github.com/containers/libpod -> podman 2020-07-28 08:23:45 -04:00
.github Implement PR template to assist review & release 2021-09-28 12:20:03 -04:00
cmd libpod: deduplicate ports in db 2021-10-27 18:59:56 +02:00
cni Use correct extension for example network config 2021-05-20 12:02:04 +02:00
completions update shell completion scripts 2021-07-02 17:49:12 +02:00
contrib add kubernetes pause 2021-10-26 13:51:45 +02:00
dependencies [NO TESTS NEEDED] Update linter 2021-02-10 13:25:40 +01:00
docs pod create: remove need for pause image 2021-10-26 13:51:45 +02:00
hack System tests: tighten 'is' operator 2021-09-30 13:32:51 -06:00
libpod libpod: deduplicate ports in db 2021-10-27 18:59:56 +02:00
logo logo: correct light source reflection 2019-11-03 09:24:12 +00:00
nix rootlessport: reduce memory usage of the process 2021-10-12 21:43:11 +02:00
pause add kubernetes pause 2021-10-26 13:51:45 +02:00
pkg libpod: deduplicate ports in db 2021-10-27 18:59:56 +02:00
test libpod: deduplicate ports in db 2021-10-27 18:59:56 +02:00
utils utils: do not overwrite the err variable 2021-10-27 09:31:32 +02:00
vendor container create: fix --tls-verify parsing 2021-10-27 14:36:25 +02:00
version Bump to v4.0.0-dev 2021-08-02 13:27:49 -04:00
.cirrus.yml Cirrus: Add gitlab podman runner test 2021-09-27 10:37:31 -04:00
.dockerignore Cirrus: Fix gate image & false-positive exits 2020-03-02 08:50:54 -05:00
.gitignore Fix Error, empty output for info: 'VERSION' 2021-09-21 12:29:26 -04:00
.golangci.yml Enable more golangci-lint linters 2021-02-12 18:20:28 +01:00
.pre-commit-config.yaml buildah bud tests under podman-remote 2021-07-28 14:14:42 -06:00
.ubuntu_prepare.sh Globally replace http:// with https:// 2021-08-27 06:14:46 -04:00
CODE-OF-CONDUCT.md Fix docs links due to branch rename 2021-06-10 11:30:41 -04:00
CONTRIBUTING.md Globally replace http:// with https:// 2021-08-27 06:14:46 -04:00
Containerfile-testvol Add tests for volume plugins 2021-01-14 16:43:23 -05:00
LICENSE Globally replace http:// with https:// 2021-08-27 06:14:46 -04:00
Makefile add kubernetes pause 2021-10-26 13:51:45 +02:00
OWNERS add @mtrmac to OWNERS 2021-06-24 14:04:15 +02:00
README.md README.md: Point to Podman's channels 2021-10-04 15:33:30 -04:00
RELEASE_NOTES.md Fix some typos in documentation and comments (found by codespell) 2021-10-25 07:41:14 +02:00
RELEASE_PROCESS.md codespell code 2021-10-12 16:44:25 -04:00
SECURITY.md Fix docs links due to branch rename 2021-06-10 11:30:41 -04:00
Vagrantfile Enhancements to papr tests 2017-11-17 02:07:18 +00:00
build_osx.md Globally replace http:// with https:// 2021-08-27 06:14:46 -04:00
commands-demo.md Implement Secrets 2021-02-09 09:13:21 -05:00
commands.md commands: rename file and add likns to readthedocs 2020-03-09 19:11:52 +01:00
docker docker: Double quote array expansions to avoid re-splitting elements 2018-10-05 11:00:27 +02:00
go.mod container create: fix --tls-verify parsing 2021-10-27 14:36:25 +02:00
go.sum container create: fix --tls-verify parsing 2021-10-27 14:36:25 +02:00
install.md update website link for install instructions 2020-12-06 21:38:10 -06:00
rootless.md Fix link error 2021-06-01 21:49:35 +02:00
transfer.md Update transfer.md 2021-07-29 12:50:47 +08:00
troubleshooting.md [CI:DOCS] Fix typo keep_id -> keep-id 2021-10-24 09:54:23 +03:00

README.md

PODMAN logo

Podman: A tool for managing OCI containers and pods

Podman (the POD MANager) is a tool for managing containers and images, volumes mounted into those containers, and pods made from groups of containers. Podman is based on libpod, a library for container lifecycle management that is also contained in this repository. The libpod library provides APIs for managing containers, pods, container images, and volumes.

  • Latest Version: 3.4.0

    • Latest Remote client for Windows
    • Latest Remote client for macOS
    • Latest Static Remote client for Linux

  • Continuous Integration: Build Status

  • GoDoc: GoDoc

Overview and scope

At a high level, the scope of Podman and libpod is the following:

  • Support for multiple container image formats, including OCI and Docker images.
  • Full management of those images, including pulling from various sources (including trust and verification), creating (built via Containerfile or Dockerfile or committed from a container), and pushing to registries and other storage backends.
  • Full management of container lifecycle, including creation (both from an image and from an exploded root filesystem), running, checkpointing and restoring (via CRIU), and removal.
  • Support for pods, groups of containers that share resources and are managed together.
  • Support for running containers and pods without root or other elevated privileges.
  • Resource isolation of containers and pods.
  • Support for a Docker-compatible CLI interface.
  • No manager daemon, for improved security and lower resource utilization at idle.
  • Support for a REST API providing both a Docker-compatible interface and an improved interface exposing advanced Podman functionality.
  • In the future, integration with CRI-O to share containers and backend code.

Podman presently only supports running containers on Linux. However, we are building a remote client which can run on Windows and macOS and manage Podman containers on a Linux system via the REST API using SSH tunneling.

Roadmap

  1. Further improvements to the REST API, with a focus on bugfixes and implementing missing functionality
  2. Integrate libpod into CRI-O to replace its existing container management backend
  3. Improvements on rootless containers, with a focus on improving the user experience and exposing presently-unavailable features when possible

Communications

If you think you've identified a security issue in the project, please DO NOT report the issue publicly via the GitHub issue tracker, mailing list, or IRC. Instead, send an email with as many details as possible to security@lists.podman.io. This is a private mailing list for the core maintainers.

For general questions and discussion, please use Podman's channels.

For discussions around issues/bugs and features, you can use the GitHub issues and PRs tracking system.

There is also a mailing list at lists.podman.io. You can subscribe by sending a message to podman-join@lists.podman.io with the subject subscribe.

Rootless

Podman can be easily run as a normal user, without requiring a setuid binary. When run without root, Podman containers use user namespaces to set root in the container to the user running Podman. Rootless Podman runs locked-down containers with no privileges that the user running the container does not have. Some of these restrictions can be lifted (via --privileged, for example), but rootless containers will never have more privileges than the user that launched them. If you run Podman as your user and mount in /etc/passwd from the host, you still won't be able to change it, since your user doesn't have permission to do so.

Almost all normal Podman functionality is available, though there are some shortcomings. Any recent Podman release should be able to run rootless without any additional configuration, though your operating system may require some additional configuration detailed in the install guide.

A little configuration by an administrator is required before rootless Podman can be used, the necessary setup is documented here.

Out of scope

  • Specialized signing and pushing of images to various storage backends. See Skopeo for those tasks.
  • Support for the Kubernetes CRI interface for container management. The CRI-O daemon specializes in that.

OCI Projects Plans

The plan is to use OCI projects and best of breed libraries for different aspects:

  • Runtime: We use the OCI runtime tools to generate OCI runtime configurations that can be used with any OCI-compliant runtime, like crun and runc.
  • Images: Image management uses the containers/image library.
  • Storage: Container and image storage is managed by containers/storage.
  • Networking: Networking support through use of CNI.
  • Builds: Builds are supported via Buildah.
  • Conmon: Conmon is a tool for monitoring OCI runtimes, used by both Podman and CRI-O.
  • Seccomp: A unified Seccomp policy for Podman, Buildah, and CRI-O.

Podman Information for Developers

For blogs, release announcements and more, please checkout the podman.io website!

Installation notes Information on how to install Podman in your environment.

OCI Hooks Support Information on how Podman configures OCI Hooks to run when launching a container.

Podman API Documentation on the Podman REST API.

Podman Commands A list of the Podman commands with links to their man pages and in many cases videos showing the commands in use.

Podman Troubleshooting Guide A list of common issues and solutions for Podman.

Podman Usage Transfer Useful information for ops and dev transfer as it relates to infrastructure that utilizes Podman. This page includes tables showing Docker commands and their Podman equivalent commands.

Tutorials Tutorials on using Podman.

Remote Client A brief how-to on using the Podman remote-client.

Basic Setup and Use of Podman in a Rootless environment A tutorial showing the setup and configuration necessary to run Rootless Podman.

Release Notes Release notes for recent Podman versions.

Contributing Information about contributing to this project.

Buildah and Podman relationship

Buildah and Podman are two complementary open-source projects that are available on most Linux platforms and both projects reside at GitHub.com with Buildah here and Podman here. Both, Buildah and Podman are command line tools that work on Open Container Initiative (OCI) images and containers. The two projects differentiate in their specialization.

Buildah specializes in building OCI images. Buildah's commands replicate all of the commands that are found in a Dockerfile. This allows building images with and without Dockerfiles while not requiring any root privileges. Buildah’s ultimate goal is to provide a lower-level coreutils interface to build images. The flexibility of building images without Dockerfiles allows for the integration of other scripting languages into the build process. Buildah follows a simple fork-exec model and does not run as a daemon but it is based on a comprehensive API in golang, which can be vendored into other tools.

Podman specializes in all of the commands and functions that help you to maintain and modify OCI images, such as pulling and tagging. It also allows you to create, run, and maintain those containers created from those images. For building container images via Dockerfiles, Podman uses Buildah's golang API and can be installed independently from Buildah.

A major difference between Podman and Buildah is their concept of a container. Podman allows users to create "traditional containers" where the intent of these containers is to be long lived. While Buildah containers are really just created to allow content to be added back to the container image. An easy way to think of it is the buildah run command emulates the RUN command in a Dockerfile while the podman run command emulates the docker run command in functionality. Because of this and their underlying storage differences, you can not see Podman containers from within Buildah or vice versa.

In short, Buildah is an efficient way to create OCI images while Podman allows you to manage and maintain those images and containers in a production environment using familiar container cli commands. For more details, see the Container Tools Guide.

Podman Former API (Varlink)

Podman formerly offered a Varlink-based API for remote management of containers. However, this API was replaced by the REST API. Varlink support has been removed as of the 3.0 release. For more details, you can see this blog.

Static Binary Builds

The Cirrus CI integration within this repository contains a static_build job which produces a static Podman binary for testing purposes. Please note that this binary is not officially supported with respect to feature-completeness and functionality and should be only used for testing.