This commit is courtesy of
```
for f in $(git ls-files *.go | grep -v ^vendor/); do \
sed -i 's/\(errors\..*\)"Error /\1"error /' $f;
done
for f in $(git ls-files *.go | grep -v ^vendor/); do \
sed -i 's/\(errors\..*\)"Failed to /\1"failed to /' $f;
done
```
etc.
Self-reviewed using `git diff --word-diff`, found no issues.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
- Fixes issue where remote attach and exec only signaled the IdleTracker
on errors. Needs to done anytime after connection has been hijacked
- Fixes trying to send multiple http status codes to client
- Changes pprof and API server shutdowns to run in parallel
- Changes shutdown to run in sync.Once block
Signed-off-by: Jhon Honce <jhonce@redhat.com>
The `tag` parameter of the compat `images/create` endpoint can be both,
a tag and a digest. Fix parsing of the parameter to detect digests and
use the appropriate `@` separator.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Fix misspelled parameter
* add http-proxy support for builds
http_proxy must be set in the podman.service unit file, for example
Environment=http_proxy=<value>
Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Remove stutter naming for package and types
* Stop treating StateIdle the same as StateClosed, rather transitions to
StateIdle will keep API timeout window open
* Remove redundate code
Fixes#7826
Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Refactor auth pkg to support X-Registry-Config
* Refactor build endpoint to support X-Registry-Config. Supports:
* --creds
* --authfile
* Added X-Reference-Id Header to http.Request to support log event
correlation
* Log headers from http.Request
Signed-off-by: Jhon Honce <jhonce@redhat.com>
in the compatibility layer, creating a volume with a name that already does not result in an error. instead a 201 response with the existing volume's information is returned. while it seems like a bug on the part of docker and they agree, no attempt has been made to fix it in five years. See https://github.com/moby/moby/issues/16068Fixes: #7740
Signed-off-by: baude <bbaude@redhat.com>
Currently the --pull missing|always|never is ignored
This PR implements this for local API. For remote we
need to default to pullpolicy specified in the containers.conf
file.
Also fixed an issue when images were matching other images names
based on prefix, causing images to always be pulled.
I had named an image myfedora and when ever I pulled fedora, the system
thought that it there were two images named fedora since it was checking
for the name fedora as well as the prefix fedora. I changed it to check
for fedora and the prefix /fedora, to prefent failures like I had.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
when creating a container using the compat endpoint, the interactive bool was being hard set to false and ignoring the user's input.
Signed-off-by: baude <bbaude@redhat.com>
Enables podman create, pull, run, import to use --signature-policy option. Set it as hidden flag to be consistent with other commands.
Signed-off-by: Qi Wang <qiwan@redhat.com>
* API-Version header now Major.Minor to support tools parsing this
header
* Libpod Version updated to 2.0.0 to reflect changes in API field
values
* API-Version and Libpod-API-Version headers are now included in all
results
Fixes#7327
* Header support tested against goland 2020.2 and
https://www.jetbrains.com/help/idea/docker.html plugin
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Do not wait for events to occur before writing the OK header.
Events can take an unknown amount of time to occur and clients
do not need to wait until then to know if the connection is
good.
Fixes: #7263
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* Refactor/Rename channel.WriteCloser() to encapsulate the channel
* Refactor build endpoint to "live" stream buildah output channels
over API rather then buffering output
* Refactor bindings/tunnel build because endpoint changes
* building tar file now in bindings rather then depending on
caller
* Cleanup initiating extra image engine
* Remove setting fields to zero values (less noise in code)
* Update tests to support remote builds
Fixes#7136Fixes#7137
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Invert the branch logic to match the comment. Docker seems to wait for
the container while Podman does not.
Enable the remote-disabled system test as well.
Fixes: #7135
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Support loading and saving tarballs with more than one image.
Add a new `/libpod/images/export` endpoint to the rest API to
allow for exporting/saving multiple images into an archive.
Note that a non-release version of containers/image is vendored.
A release version must be vendored before cutting a new Podman
release. We force the containers/image version via a replace in
the go.mod file; this way go won't try to match the versions.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add the filter option to the libpod endpoint.
Add support for the name filter on the docker endpoint.
Add apiv2 tests for the network list endpoints.
Enable podman network integration tests for remote.
Signed-off-by: Paul Holzinger <paul.holzinger@web.de>
Our previous flow was to perform a hijack before passing a
connection into Libpod, and then Libpod would attach to the
container's attach socket and begin forwarding traffic.
A problem emerges: we write the attach header as soon as the
attach complete. As soon as we write the header, the client
assumes that all is ready, and sends a Start request. This Start
may be processed *before* we successfully finish attaching,
causing us to lose output.
The solution is to handle hijacking inside Libpod. Unfortunately,
this requires a downright extensive refactor of the Attach and
HTTP Exec StartAndAttach code. I think the result is an
improvement in some places (a lot more errors will be handled
with a proper HTTP error code, before the hijack occurs) but
other parts, like the relocation of printing container logs, are
just *bad*. Still, we need this fixed now to get CI back into
good shape...
Fixes#7195
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
even though the official documentation suggests that application/x-tar should be used for tar files, it seems docker-compose uses application/tar. we now accept them and issue a warning.
Fixes: #7185
Signed-off-by: Brent Baude <bbaude@redhat.com>
The versions Docker that the compat endpoints currently support are
using another type for the `filters` parameter than later versions
of Docker, which the libpod/events endpoint is also using.
To prevent existing deplopyments from breaking while still achieving
backward compat, we now support both types for the filters parameter.
Tested manually.
Fixes: #6899
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix a potential panic in the events endpoint when parsing the filters
parameter. Values of the filters map might be empty, so we need to
account for that instead of uncondtitionally accessing the first item.
Also apply a similar for race conditions as done in commit f4a2d25c0fca:
Fix a race that could cause read errors to be masked. Masking
such errors is likely to report red herrings since users don't
see that reading failed for some reasons but that a given event
could not be found.
Another race was the handler closing event channel, which could lead to
two kinds of panics: double close, send to close channel. The backend
takes care of that. However, make sure that the backend stops working
in case the context has been cancelled.
Fixes: #6899
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This matches Docker behavior, and will make the Docker frontend
work with `podman system service` (Docker tries to create, then
if that fails with 404 sends a request to pull the image).
Fixes#6960
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This patch fixes connection counters for v2 endpoints
Idletracker was moved to a new package to prevent package cycle.
Hijacking code still remains in wrong place and should be moved
later to isolated package
Signed-off-by: Sami Korhonen <skorhone@gmail.com>
After this patch v2 hijacking endpoints, exec/start and
containers/attach follow rfc 7230 specification.
Connection will only be upgraded, if client specifies upgrade
headers:
For tcp connections:
Connection: Upgrade
Upgrade: tcp
For unix socket connections:
Connection: Upgrade
Upgrade: sock
There are currently no checks if upgrade type actually matches with
available protocols. Implementation just protocol that client
requested
Signed-off-by: Sami Korhonen <skorhone@gmail.com>
Add a `context.Context` to the log APIs to allow for cancelling
streaming (e.g., via `podman logs -f`). This fixes issues for
the remote API where some go routines of the server will continue
writing and produce nothing but heat and waste CPU cycles.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This change ensures that we pull volume bind specification strings from
the correct spot in the POSTed data when creating containers. We should
probably canity check that the mapping keys in `Volumes` are a superset
of the binds listed in `HostConfig.Binds` but this cheap change removes
an annoying behaviour where named volumes or host mount would be
silently replaced with newly created anonymous volumes.
Signed-off-by: Matt Brindley <58414429+maybe-sybr@users.noreply.github.com>
With the advent of Podman 2.0.0 we crossed the magical barrier of go
modules. While we were able to continue importing all packages inside
of the project, the project could not be vendored anymore from the
outside.
Move the go module to new major version and change all imports to
`github.com/containers/libpod/v2`. The renaming of the imports
was done via `gomove` [1].
[1] https://github.com/KSubedi/gomove
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We weren't actually halting the goroutine that sent events, so it
would continue sending even when the channel closed (the most
notable cause being early hangup - e.g. Control-c on a curl
session). Use a context to cancel the events goroutine and stop
sending events.
Fixes#6805
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
In response to input regarding the semantic difference for the `force`
parameter for volume removal between Docker and us, this change ensures
that we emulate the Dockr behaviour correctly when this parameter is
specified.
Signed-off-by: Matt Brindley <58414429+maybe-sybr@users.noreply.github.com>
This change implements docker compatibile endpoint for interacting with
volumes. The code is mostly lifted from the `libpod` API handlers but
decodes and constructs data using types defined in the docker API
package.
Some notable support caveats with the current implementation:
* we don't return the nullable `Status` or `UsageData` keys when
returning volume information for inspect and create endpoints
* we don't support filters when pruning
* we return a fixed `0` for the `SpaceReclaimed` key when pruning
since we have no insight into how much space was freed from runtime
Signed-off-by: Matt Brindley <58414429+maybe-sybr@users.noreply.github.com>
We have a flag, --syslog, for telling logrus to log to syslog as
well as to the terminal. Previously, this flag also set the exit
command for containers to use `--syslog` (otherwise all output
from exit commands is lost). I attempted to replicate this with
Podman v2.0, but quickly ran into circular import hell (the flag
is defined in cmd/podman, I needed it in cmd/podman/containers,
cmd/podman imports cmd/podman/containers already, etc). Instead,
let's just set the syslog flag automatically on
`--log-level=debug` so we log exit commands automatically when
debug-level logs are requested. This is consistent with Conmon
and seems to make sense.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
As part of APIv2 Attach, we need to be able to attach to freshly
created containers (in ContainerStateConfigured). This isn't
something Libpod is interested in supporting, so we use Init() to
get the container into ContainerStateCreated, in which attach is
possible. Problem: Init() will fail if dependencies are not
started, so a fresh container in a fresh pod will fail. The
simplest solution is to extend the existing recursive start code
from Start() to Init(), allowing dependency containers to be
started when we initialize the container (optionally, controlled
via bool).
Also, update some comments in container_api.go to make it more
clear how some of our major API calls work.
Fixes#6646
Signed-off-by: Matthew Heon <mheon@redhat.com>
This simply moves the function for the log handler for
APIv2 to a separate file to be consistent with other parts
of the code base.
Signed-off-by: jgallucci32 <john.gallucci.iv@gmail.com>
address problem when multiple -t were sent. and rework remote build's tarball if a context dir is given other than ".".
Fixes: #6578Fixes: #6577
Signed-off-by: Brent Baude <bbaude@redhat.com>
Fix a lint error of an used parameter. The error must have sneaked in
with a PR that was merged after the recent linter enablement.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The biggest obstacle here was cleanup - we needed a way to remove
detached exec sessions after they exited, but there's no way to
tell if an exec session will be attached or detached when it's
created, and that's when we must add the exit command that would
do the removal. The solution was adding a delay to the exit
command (5 minutes), which gives sufficient time for attached
exec sessions to retrieve the exit code of the session after it
exits, but still guarantees that they will be removed, even for
detached sessions. This requires Conmon 2.0.17, which has the new
`--exit-delay` flag.
As part of the exit command rework, we can drop the hack we were
using to clean up exec sessions (remove them as part of inspect).
This is a lot cleaner, and I'm a lot happier about it.
Otherwise, this is just plumbing - we need a bindings call for
detached exec, and that needed to be added to the tunnel mode
backend for entities.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This adds bindings for starting exec sessions, and then uses them
to wire up detached exec. Code is heavily based on Attach code
for containers, slightly modified to handle exec sessions.
Bindings are presently attached-only, detached is pending on a
Conmon update landing in CI. I'll probably get to that next.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Support the `X-Registry-Auth` http-request header.
* The content of the header is a base64 encoded JSON payload which can
either be a single auth config or a map of auth configs (user+pw or
token) with the corresponding registries being the keys. Vanilla
Docker, projectatomic Docker and the bindings are transparantly
supported.
* Add a hidden `--registries-conf` flag. Buildah exposes the same
flag, mostly for testing purposes.
* Do all credential parsing in the client (i.e., `cmd/podman`) pass
the username and password in the backend instead of unparsed
credentials.
* Add a `pkg/auth` which handles most of the heavy lifting.
* Go through the authentication-handling code of most commands, bindings
and endpoints. Migrate them to the new code and fix issues as seen.
A final evaluation and more tests is still required *after* this
change.
* The manifest-push endpoint is missing certain parameters and should
use the ABI function instead. Adding auth-support isn't really
possible without these parts working.
* The container commands and endpoints (i.e., create and run) have not
been changed yet. The APIs don't yet account for the authfile.
* Add authentication tests to `pkg/bindings`.
Fixes: #6384
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
add copy endpoint inputs and outputs. these endpoints are not implemented yet, nor are any bindings. this allows us to update this later without having to change our api version.
Signed-off-by: Brent Baude <bbaude@redhat.com>
* wire up bindings and handler for obtaining logs remotely
* enable debug logging from podman in e2e test using DEBUG and
DEBUG_SERVICE env variables
* Fix error in streaming log frames
* enable remote logs test
Signed-off-by: Jhon Honce <jhonce@redhat.com>
this is a very basic implementation of build. some of the more advanced options need to be included still as well. i think the endpoints for compat and libpod will have to split given buildahs more advanced set of options. that should probably be done by someone more experienced with build internals.
Signed-off-by: Brent Baude <bbaude@redhat.com>
* Update blang/semver to allow ParseTolerant() support
* Provide helper functions for API handlers to obtain client's 'version'
path variable focused on API endpoint tree: libpod vs. compat
* Introduce new errors:
* version not given in path, endpoints may determine if this is a hard
error (ErrVersionNotGiven)
* given version not supported (ErrVersionNotSupported), only a soft
error if the handler is going to hijack the connection
* Added unit tests for version parsing
* bindings check version on connect:
* client <= Server API version connection is continued
* client >= Server API version connection fails
Signed-off-by: Jhon Honce <jhonce@redhat.com>
* Add support for /exec/{id}/resize
* Add support for ErrSessionNotFound
* Resize container TTY as stdin changes size
* Refactor all resize functions into one handler
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Jhon is working on an alternative version that will combine
container and exec session resize, so we'll wait for that.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
for the remote client, add the ability to attach to a container, start a container, and run a container.
Signed-off-by: Brent Baude <bbaude@redhat.com>
The usual flow for exec is going to be:
- Create exec session
- Start and attach to exec session
- Exec session exits, attach session terminates
- Client does an exec inspect to pick up exit code
The safest point to remove the exec session, without doing any
database changes to track stale sessions, is to remove during the
last part of this - the single inspect after the exec session
exits.
This is definitely different from Docker (which would retain the
exec session for up to 10 minutes after it exits, where we will
immediately discard) but should be close enough to be not
noticeable in regular usage.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
This is still very early not not well tested, and missing resize
capability, but it does provide the first bits of exec.
Signed-off-by: Matthew Heon <mheon@redhat.com>
* Add ErrLostSync to report lost of sync when de-mux'ing stream
* Add logus.SetLevel(logrus.DebugLevel) when `go test -v` given
* Add context to debugging messages
Signed-off-by: Jhon Honce <jhonce@redhat.com>
Register system prune route, handler to support system prune,
Adds testcase to validate the system prune flow.
Signed-off-by: Sujil02 <sushah@redhat.com>
While this commit was initially meant to fix#5847, it has turned into a
bigger refactoring which I did not manage to break into smaller pieces:
* Fix#5847 by refactoring the image-removal logic.
* Make the api handler for image-removal use the ABI code. This way,
both (i.e., ABI and Tunnel) end up using the same code. Achieving
this code share required to move some code around to prevent circular
dependencies.
* Everything in pkg/api (excluding pkg/api/types) must now only be
accessed from code using `ABISupport`.
* Avoid imports from entities on handlers to prevent circular
dependencies.
* Move `podman system service` logic into `cmd` to prevent circular
dependencies - it depends on pkg/api.
* Also remove the build header from infra/abi files. It will otherwise
confuse swagger and other tools; errors we cannot fix as go doesn't
expose a build-tag env variable.
Fixes: #5847
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add more default options parsing
Switch to using --time as opposed to --timeout to better match Docker.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
this is second phase of removing unneeded bloat in the remote client. this is important to be able to reduce the client size as well as possible native compilation for windows/mac.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Adds ability to prune containers for v2.
Adds client side prompt with force flag and filters options to prune.
Signed-off-by: Sujil02 <sushah@redhat.com>
A few major fixes here:
- Support for attaching to Configured containers, to match Docker
behavior.
- Support for stream parameter has been improved (we now properly
handle cases where it is not set).
- Initial support for logs parameter has been added.
- Setting attach streams when the container has a terminal is now
supported.
- Errors are properly reported once the hijack has begun.
Signed-off-by: Matthew Heon <mheon@redhat.com>
Also implement a new libpod endpoint to add more parameters and to
prevent us from converting between slices and maps and make use of
the filter parsing in the image backend.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Implement the `podman {container} logs` for the v2 client. The remote
client does not yet support it. There's some more work needed for the
rest api; some options are missing (e.g., printing names) while others
are broken (e.g., the until http parameter).
The remote parts will be tackled in a future change.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
the current implementation of info, while typed, is very loosely done so. we need stronger types for our apiv2 implmentation and bindings.
Signed-off-by: Brent Baude <bbaude@redhat.com>
* Implement `podman-push` and `podman-image-push` for the podmanV2
client.
* Tests for `pkg/bindings` are not possible at the time of writing as we
don't have a local registry running.
* Implement `/images/{name}/push` compat endpoint. Tests are not
implemented for this v2 endpoint. It has been tested manually.
General note: The auth config extraction from the http header is not
implement for push. Since it's not yet supported for other endpoints
either, I deferred it to future work.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Also adds some basic tests for these two. More tests are needed
but will have to wait for state to be finished.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
* Add support for rm and rmi commands
* Support for registry.ExitCode
* Support for N-errors from domain layer
* Add log-level support
* Add syslog support
Signed-off-by: Jhon Honce <jhonce@redhat.com>
honor -1 in in list containers for compatibility mode. it is commonly used to indicate no limit.
change the json id parameter to Id in container create.
Fixes: #5553
Signed-off-by: Brent Baude <bbaude@redhat.com>
zhangguanzhang
Kir Kolyshkin
Jhon Honce
Valentin Rothberg
OpenShift Merge Robot
baude
Daniel J Walsh
Ashley Cui
Qi Wang
Paul Holzinger
Ed Santiago
Matthew Heon
Akihiro Suda
Korhonen Sami (Samlink)
maybe-sybr
Maximilian Müller
Matthew Heon
jgallucci32
Sujil02