Initial checkin from CRI-O repo

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>

.gitignore

@@ -0,0 +1,13 @@
+/.artifacts/
+/_output/
+/conmon/conmon.o
+/docs/*.[158]
+/docs/*.[158].gz
+/crio.conf
+*.o
+*.orig
+/pause/pause.o
+/bin/
+/test/bin2img/bin2img
+/test/checkseccomp/checkseccomp
+/test/copyimg/copyimg

.travis.yml

@@ -0,0 +1,62 @@
+language: go
+
+sudo: required
+
+services:
+  - docker
+
+before_install:
+  - sudo apt-get -qq update
+  - sudo apt-get -qq install btrfs-tools libdevmapper-dev libgpgme11-dev libapparmor-dev libseccomp-dev
+  - sudo apt-get -qq install autoconf automake bison e2fslibs-dev libfuse-dev libtool liblzma-dev gettext
+
+install:
+  - make install.tools
+  - OSTREE_VERSION=v2017.9
+  - git clone https://github.com/ostreedev/ostree ${TRAVIS_BUILD_DIR}/ostree
+  - pushd ${TRAVIS_BUILD_DIR}/ostree
+  -   git checkout $OSTREE_VERSION
+  -   ./autogen.sh --prefix=/usr/local
+  -   make all
+  -   sudo make install
+  - popd
+
+before_script:
+  - export PATH=$HOME/gopath/bin:$PATH
+  - export LD_LIBRARY_PATH=/usr/local/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
+
+jobs:
+  include:
+    - stage: Build and Verify
+      script:
+        - make .gitvalidation
+        - make gofmt
+        - make lint
+        - make testunit
+        - make docs
+        - make
+      go: 1.8.x
+    - stage: Build and Verify
+      script:
+        - make .gitvalidation
+        - make gofmt
+        - make lint
+        - make testunit
+        - make docs
+        - make
+      go: 1.9.x
+    - script:
+        - make .gitvalidation
+        - make gofmt
+        - make lint
+        - make testunit
+        - make docs
+        - make
+      go: tip
+    - stage: Integration Test
+      script:
+        - make integration
+      go: 1.8.x
+
+notifications:
+  irc: "chat.freenode.net#cri-o"

CONTRIBUTING.md

@@ -0,0 +1,142 @@
+# Contributing to CRI-O
+
+We'd love to have you join the community! Below summarizes the processes
+that we follow.
+
+## Topics
+
+* [Reporting Issues](#reporting-issues)
+* [Submitting Pull Requests](#submitting-pull-requests)
+* [Communications](#communications)
+* [Becoming a Maintainer](#becoming-a-maintainer)
+
+## Reporting Issues
+
+Before reporting an issue, check our backlog of
+[open issues](https://github.com/kubernetes-incubator/cri-o/issues)
+to see if someone else has already reported it. If so, feel free to add
+your scenario, or additional information, to the discussion. Or simply
+"subscribe" to it to be notified when it is updated.
+
+If you find a new issue with the project we'd love to hear about it! The most
+important aspect of a bug report is that it includes enough information for
+us to reproduce it. So, please include as much detail as possible and try
+to remove the extra stuff that doesn't really relate to the issue itself.
+The easier it is for us to reproduce it, the faster it'll be fixed!
+
+Please don't include any private/sensitive information in your issue!
+
+## Submitting Pull Requests
+
+No Pull Request (PR) is too small! Typos, additional comments in the code,
+new testcases, bug fixes, new features, more documentation, ... it's all
+welcome!
+
+While bug fixes can first be identified via an "issue", that is not required.
+It's ok to just open up a PR with the fix, but make sure you include the same
+information you would have included in an issue - like how to reproduce it.
+
+PRs for new features should include some background on what use cases the
+new code is trying to address. When possible and when it makes sense, try to break-up
+larger PRs into smaller ones - it's easier to review smaller
+code changes. But only if those smaller ones make sense as stand-alone PRs.
+
+Regardless of the type of PR, all PRs should include:
+* well documented code changes
+* additional testcases. Ideally, they should fail w/o your code change applied
+* documentation changes
+
+Squash your commits into logical pieces of work that might want to be reviewed
+separate from the rest of the PRs. But, squashing down to just one commit is ok
+too since in the end the entire PR will be reviewed anyway. When in doubt,
+squash.
+
+PRs that fix issues should include a reference like `Closes #XXXX` in the
+commit message so that github will automatically close the referenced issue
+when the PR is merged.
+
+<!--
+All PRs require at least two LGTMs (Looks Good To Me) from maintainers.
+-->
+
+### Sign your PRs
+
+The sign-off is a line at the end of the explanation for the patch. Your
+signature certifies that you wrote the patch or otherwise have the right to pass
+it on as an open-source patch. The rules are simple: if you can certify
+the below (from [developercertificate.org](http://developercertificate.org/)):
+
+```
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+660 York Street, Suite 102,
+San Francisco, CA 94110 USA
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.
+```
+
+Then you just add a line to every git commit message:
+
+    Signed-off-by: Joe Smith <joe.smith@email.com>
+
+Use your real name (sorry, no pseudonyms or anonymous contributions.)
+
+If you set your `user.name` and `user.email` git configs, you can sign your
+commit automatically with `git commit -s`.
+
+## Communications
+
+For general questions, or discussions, please use the
+IRC group on `irc.freenode.net` called `cri-o`
+that has been setup.
+
+For discussions around issues/bugs and features, you can use the github
+[issues](https://github.com/kubernetes-incubator/cri-o/issues)
+and
+[PRs](https://github.com/kubernetes-incubator/cri-o/pulls)
+tracking system.
+
+<!--
+## Becoming a Maintainer
+
+To become a maintainer you must first be nominated by an existing maintainer.
+If a majority (>50%) of maintainers agree then the proposal is adopted and
+you will be added to the list.
+
+Removing a maintainer requires at least 75% of the remaining maintainers
+approval, or if the person requests to be removed then it is automatic.
+Normally, a maintainer will only be removed if they are considered to be
+inactive for a long period of time or are viewed as disruptive to the community.
+
+The current list of maintainers can be found in the
+[MAINTAINERS](MAINTAINERS) file.
+-->

Dockerfile

@@ -0,0 +1,117 @@
+FROM golang:1.8
+
+# libseccomp in jessie is not _quite_ new enough -- need backports version
+RUN echo 'deb http://httpredir.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/backports.list
+
+RUN apt-get update && apt-get install -y \
+    apparmor \
+    autoconf \
+    automake \
+    bison \
+    build-essential \
+    curl \
+    e2fslibs-dev \
+    gawk \
+    gettext \
+    iptables \
+    pkg-config \
+    libaio-dev \
+    libcap-dev \
+    libfuse-dev \
+    libostree-dev \
+    libprotobuf-dev \
+    libprotobuf-c0-dev \
+    libseccomp2/jessie-backports \
+    libseccomp-dev/jessie-backports \
+    libtool \
+    libudev-dev \
+    protobuf-c-compiler \
+    protobuf-compiler \
+    python-minimal \
+    libglib2.0-dev \
+    libapparmor-dev \
+    btrfs-tools \
+    libdevmapper1.02.1 \
+    libdevmapper-dev \
+    libgpgme11-dev \
+    liblzma-dev \
+    netcat \
+    socat \
+    --no-install-recommends \
+    && apt-get clean
+
+# install bats
+RUN cd /tmp \
+    && git clone https://github.com/sstephenson/bats.git \
+    && cd bats \
+    && git reset --hard 03608115df2071fff4eaaff1605768c275e5f81f \
+    && ./install.sh /usr/local
+
+# install criu
+ENV CRIU_VERSION 1.7
+RUN mkdir -p /usr/src/criu \
+    && curl -sSL https://github.com/xemul/criu/archive/v${CRIU_VERSION}.tar.gz | tar -v -C /usr/src/criu/ -xz --strip-components=1 \
+    && cd /usr/src/criu \
+    && make install-criu \
+    && rm -rf /usr/src/criu
+
+# Install runc
+ENV RUNC_COMMIT 84a082bfef6f932de921437815355186db37aeb1
+RUN set -x \
+	&& export GOPATH="$(mktemp -d)" \
+	&& git clone https://github.com/opencontainers/runc.git "$GOPATH/src/github.com/opencontainers/runc" \
+	&& cd "$GOPATH/src/github.com/opencontainers/runc" \
+	&& git fetch origin --tags \
+	&& git checkout -q "$RUNC_COMMIT" \
+	&& make static BUILDTAGS="seccomp selinux" \
+	&& cp runc /usr/bin/runc \
+	&& rm -rf "$GOPATH"
+
+# Install CNI plugins
+ENV CNI_COMMIT dcf7368eeab15e2affc6256f0bb1e84dd46a34de
+RUN set -x \
+       && export GOPATH="$(mktemp -d)" \
+       && git clone https://github.com/containernetworking/plugins.git "$GOPATH/src/github.com/containernetworking/plugins" \
+       && cd "$GOPATH/src/github.com/containernetworking/plugins" \
+       && git checkout -q "$CNI_COMMIT" \
+       && ./build.sh \
+       && mkdir -p /opt/cni/bin \
+       && cp bin/* /opt/cni/bin/ \
+       && rm -rf "$GOPATH"
+
+# Install custom CNI bridge test plugin
+# XXX: this plugin is meant to be a replacement for the old "test_plugin_args.bash"
+# we need this in testing because sandbox_run now gather IP address and the mock
+# plugin wasn't able to properly setup the net ns.
+# The bridge is based on the same commit as the one above.
+#ENV CNI_COMMIT 6bfe036c38c8e1410f1acaa4b2ee16f1851472e4
+ENV CNI_TEST_BRANCH custom-bridge
+RUN set -x \
+       && export GOPATH="$(mktemp -d)" \
+       && git clone https://github.com/runcom/plugins.git "$GOPATH/src/github.com/containernetworking/plugins" \
+       && cd "$GOPATH/src/github.com/containernetworking/plugins" \
+       && git checkout -q "$CNI_TEST_BRANCH" \
+       && ./build.sh \
+       && mkdir -p /opt/cni/bin \
+       && cp bin/bridge /opt/cni/bin/bridge-custom \
+       && rm -rf "$GOPATH"
+
+# Install crictl
+ENV CRICTL_COMMIT 16e6fe4d7199c5689db4630a9330e6a8a12cecd1
+RUN set -x \
+       && export GOPATH="$(mktemp -d)" \
+       && git clone https://github.com/kubernetes-incubator/cri-tools.git "$GOPATH/src/github.com/kubernetes-incubator/cri-tools" \
+       && cd "$GOPATH/src/github.com/kubernetes-incubator/cri-tools" \
+       && git checkout -q "$CRICTL_COMMIT" \
+       && go install github.com/kubernetes-incubator/cri-tools/cmd/crictl \
+       && cp "$GOPATH"/bin/crictl /usr/bin/ \
+       && rm -rf "$GOPATH"
+
+# Make sure we have some policy for pulling images
+RUN mkdir -p /etc/containers
+COPY test/policy.json /etc/containers/policy.json
+COPY test/redhat_sigstore.yaml /etc/containers/registries.d/registry.access.redhat.com.yaml
+
+WORKDIR /go/src/github.com/kubernetes-incubator/cri-o
+
+ADD . /go/src/github.com/kubernetes-incubator/cri-o

KPOD_VERSION

@@ -0,0 +1 @@
+0.1

Makefile

@@ -0,0 +1,244 @@
+GO ?= go
+EPOCH_TEST_COMMIT ?= 1cc5a27
+PROJECT := github.com/kubernetes-incubator/cri-o
+GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
+GIT_BRANCH_CLEAN := $(shell echo $(GIT_BRANCH) | sed -e "s/[^[:alnum:]]/-/g")
+CRIO_IMAGE := crio_dev$(if $(GIT_BRANCH_CLEAN),:$(GIT_BRANCH_CLEAN))
+CRIO_INSTANCE := crio_dev
+PREFIX ?= ${DESTDIR}/usr/local
+BINDIR ?= ${PREFIX}/bin
+LIBEXECDIR ?= ${PREFIX}/libexec
+MANDIR ?= ${PREFIX}/share/man
+ETCDIR ?= ${DESTDIR}/etc
+ETCDIR_CRIO ?= ${ETCDIR}/crio
+BUILDTAGS ?= seccomp $(shell hack/btrfs_tag.sh) $(shell hack/libdm_tag.sh) $(shell hack/btrfs_installed_tag.sh) $(shell hack/ostree_tag.sh) $(shell hack/selinux_tag.sh)
+
+BASHINSTALLDIR=${PREFIX}/share/bash-completion/completions
+OCIUMOUNTINSTALLDIR=$(PREFIX)/share/oci-umount/oci-umount.d
+
+SELINUXOPT ?= $(shell test -x /usr/sbin/selinuxenabled && selinuxenabled && echo -Z)
+PACKAGES ?= $(shell go list -tags "${BUILDTAGS}" ./... | grep -v github.com/kubernetes-incubator/cri-o/vendor)
+
+COMMIT_NO := $(shell git rev-parse HEAD 2> /dev/null || true)
+GIT_COMMIT := $(if $(shell git status --porcelain --untracked-files=no),"${COMMIT_NO}-dirty","${COMMIT_NO}")
+BUILD_INFO := $(shell date +%s)
+
+KPOD_VERSION := ${shell cat ./KPOD_VERSION}
+
+# If GOPATH not specified, use one in the local directory
+ifeq ($(GOPATH),)
+export GOPATH := $(CURDIR)/_output
+unexport GOBIN
+endif
+GOPKGDIR := $(GOPATH)/src/$(PROJECT)
+GOPKGBASEDIR := $(shell dirname "$(GOPKGDIR)")
+
+# Update VPATH so make finds .gopathok
+VPATH := $(VPATH):$(GOPATH)
+SHRINKFLAGS := -s -w
+BASE_LDFLAGS := ${SHRINKFLAGS} -X main.gitCommit=${GIT_COMMIT} -X main.buildInfo=${BUILD_INFO}
+KPOD_LDFLAGS := -X main.kpodVersion=${KPOD_VERSION}
+LDFLAGS := -ldflags '${BASE_LDFLAGS}'
+LDFLAGS_KPOD := -ldflags '${BASE_LDFLAGS} ${KPOD_LDFLAGS}'
+
+all: binaries crio.conf docs
+
+default: help
+
+help:
+	@echo "Usage: make <target>"
+	@echo
+	@echo " * 'install' - Install binaries to system locations"
+	@echo " * 'binaries' - Build crio, conmon, pause, crioctl and kpod"
+	@echo " * 'integration' - Execute integration tests"
+	@echo " * 'clean' - Clean artifacts"
+	@echo " * 'lint' - Execute the source code linter"
+	@echo " * 'gofmt' - Verify the source code gofmt"
+
+.gopathok:
+ifeq ("$(wildcard $(GOPKGDIR))","")
+	mkdir -p "$(GOPKGBASEDIR)"
+	ln -s "$(CURDIR)" "$(GOPKGBASEDIR)"
+endif
+	touch "$(GOPATH)/.gopathok"
+
+lint: .gopathok
+	@echo "checking lint"
+	@./.tool/lint
+
+gofmt:
+	@./hack/verify-gofmt.sh
+
+conmon:
+	$(MAKE) -C $@
+
+pause:
+	$(MAKE) -C $@
+
+test/bin2img/bin2img: .gopathok $(wildcard test/bin2img/*.go)
+	$(GO) build $(LDFLAGS) -tags "$(BUILDTAGS) containers_image_ostree_stub" -o $@ $(PROJECT)/test/bin2img
+
+test/copyimg/copyimg: .gopathok $(wildcard test/copyimg/*.go)
+	$(GO) build $(LDFLAGS) -tags "$(BUILDTAGS) containers_image_ostree_stub" -o $@ $(PROJECT)/test/copyimg
+
+test/checkseccomp/checkseccomp: .gopathok $(wildcard test/checkseccomp/*.go)
+	$(GO) build $(LDFLAGS) -tags "$(BUILDTAGS) containers_image_ostree_stub" -o $@ $(PROJECT)/test/checkseccomp
+
+crio: .gopathok $(shell hack/find-godeps.sh $(GOPKGDIR) cmd/crio $(PROJECT))
+	$(GO) build $(LDFLAGS) -tags "$(BUILDTAGS) containers_image_ostree_stub" -o bin/$@ $(PROJECT)/cmd/crio
+
+crioctl: .gopathok $(shell hack/find-godeps.sh $(GOPKGDIR) cmd/crioctl $(PROJECT))
+	$(GO) build $(LDFLAGS) -tags "$(BUILDTAGS) containers_image_ostree_stub" -o bin/$@ $(PROJECT)/cmd/crioctl
+
+kpod: .gopathok $(shell hack/find-godeps.sh $(GOPKGDIR) cmd/kpod $(PROJECT))
+	$(GO) build $(LDFLAGS_KPOD) -tags "$(BUILDTAGS)" -o bin/$@ $(PROJECT)/cmd/kpod
+
+crio.conf: crio
+	./bin/crio --config="" config --default > crio.conf
+
+clean:
+ifneq ($(GOPATH),)
+	rm -f "$(GOPATH)/.gopathok"
+endif
+	rm -rf _output
+	rm -f docs/*.1 docs/*.5 docs/*.8
+	rm -fr test/testdata/redis-image
+	find . -name \*~ -delete
+	find . -name \#\* -delete
+	rm -f bin/crioctl bin/crio bin/kpod
+	make -C conmon clean
+	make -C pause clean
+	rm -f test/bin2img/bin2img
+	rm -f test/copyimg/copyimg
+	rm -f test/checkseccomp/checkseccomp
+
+crioimage:
+	docker build -t ${CRIO_IMAGE} .
+
+dbuild: crioimage
+	docker run --name=${CRIO_INSTANCE} --privileged ${CRIO_IMAGE} -v ${PWD}:/go/src/${PROJECT} --rm make binaries
+
+integration: crioimage
+	docker run -e STORAGE_OPTIONS="--storage-driver=vfs" -e TESTFLAGS -e TRAVIS -t --privileged --rm -v ${CURDIR}:/go/src/${PROJECT} ${CRIO_IMAGE} make localintegration
+
+testunit:
+	$(GO) test -tags "$(BUILDTAGS)" -cover $(PACKAGES)
+
+localintegration: clean binaries test-binaries
+	./test/test_runner.sh ${TESTFLAGS}
+
+binaries: crio conmon pause kpod crioctl
+test-binaries: test/bin2img/bin2img test/copyimg/copyimg test/checkseccomp/checkseccomp
+
+MANPAGES_MD := $(wildcard docs/*.md)
+MANPAGES    := $(MANPAGES_MD:%.md=%)
+
+docs/%.1: docs/%.1.md .gopathok
+	(go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@) || ($(GOPATH)/bin/go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@)
+
+docs/%.5: docs/%.5.md .gopathok
+	(go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@) || ($(GOPATH)/bin/go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@)
+
+docs/%.8: docs/%.8.md .gopathok
+	(go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@) || ($(GOPATH)/bin/go-md2man -in $< -out $@.tmp && touch $@.tmp && mv $@.tmp $@)
+
+docs: $(MANPAGES)
+
+install: .gopathok install.bin install.man
+
+install.bin:
+	install ${SELINUXOPT} -D -m 755 bin/crio $(BINDIR)/crio
+	install ${SELINUXOPT} -D -m 755 bin/crioctl $(BINDIR)/crioctl
+	install ${SELINUXOPT} -D -m 755 bin/kpod $(BINDIR)/kpod
+	install ${SELINUXOPT} -D -m 755 bin/conmon $(LIBEXECDIR)/crio/conmon
+	install ${SELINUXOPT} -D -m 755 bin/pause $(LIBEXECDIR)/crio/pause
+
+install.man:
+	install ${SELINUXOPT} -d -m 755 $(MANDIR)/man1
+	install ${SELINUXOPT} -d -m 755 $(MANDIR)/man5
+	install ${SELINUXOPT} -d -m 755 $(MANDIR)/man8
+	install ${SELINUXOPT} -m 644 $(filter %.1,$(MANPAGES)) -t $(MANDIR)/man1
+	install ${SELINUXOPT} -m 644 $(filter %.5,$(MANPAGES)) -t $(MANDIR)/man5
+	install ${SELINUXOPT} -m 644 $(filter %.8,$(MANPAGES)) -t $(MANDIR)/man8
+
+install.config:
+	install ${SELINUXOPT} -D -m 644 crio.conf $(ETCDIR_CRIO)/crio.conf
+	install ${SELINUXOPT} -D -m 644 seccomp.json $(ETCDIR_CRIO)/seccomp.json
+	install ${SELINUXOPT} -D -m 644 crio-umount.conf $(OCIUMOUNTINSTALLDIR)/crio-umount.conf
+
+install.completions:
+	install ${SELINUXOPT} -d -m 755 ${BASHINSTALLDIR}
+	install ${SELINUXOPT} -m 644 -D completions/bash/kpod ${BASHINSTALLDIR}
+
+install.systemd:
+	install ${SELINUXOPT} -D -m 644 contrib/systemd/crio.service $(PREFIX)/lib/systemd/system/crio.service
+	ln -sf crio.service $(PREFIX)/lib/systemd/system/cri-o.service
+	install ${SELINUXOPT} -D -m 644 contrib/systemd/crio-shutdown.service $(PREFIX)/lib/systemd/system/crio-shutdown.service
+
+uninstall:
+	rm -f $(BINDIR)/crio
+	rm -f $(BINDIR)/crioctl
+	rm -f $(LIBEXECDIR)/crio/conmon
+	rm -f $(LIBEXECDIR)/crio/pause
+	for i in $(filter %.1,$(MANPAGES)); do \
+		rm -f $(MANDIR)/man8/$$(basename $${i}); \
+	done
+	for i in $(filter %.5,$(MANPAGES)); do \
+		rm -f $(MANDIR)/man5/$$(basename $${i}); \
+	done
+	for i in $(filter %.8,$(MANPAGES)); do \
+		rm -f $(MANDIR)/man8/$$(basename $${i}); \
+	done
+
+.PHONY: .gitvalidation
+# When this is running in travis, it will only check the travis commit range
+.gitvalidation: .gopathok
+ifeq ($(TRAVIS),true)
+	GIT_CHECK_EXCLUDE="./vendor" $(GOPATH)/bin/git-validation -q -run DCO,short-subject,dangling-whitespace
+else
+	GIT_CHECK_EXCLUDE="./vendor" $(GOPATH)/bin/git-validation -v -run DCO,short-subject,dangling-whitespace -range $(EPOCH_TEST_COMMIT)..HEAD
+endif
+
+.PHONY: install.tools
+
+install.tools: .install.gitvalidation .install.gometalinter .install.md2man
+
+.install.gitvalidation: .gopathok
+	if [ ! -x "$(GOPATH)/bin/git-validation" ]; then \
+		go get -u github.com/vbatts/git-validation; \
+	fi
+
+.install.gometalinter: .gopathok
+	if [ ! -x "$(GOPATH)/bin/gometalinter" ]; then \
+		go get -u github.com/alecthomas/gometalinter; \
+		cd $(GOPATH)/src/github.com/alecthomas/gometalinter; \
+		git checkout 23261fa046586808612c61da7a81d75a658e0814; \
+		go install github.com/alecthomas/gometalinter; \
+		$(GOPATH)/bin/gometalinter --install; \
+	fi
+
+.install.md2man: .gopathok
+	if [ ! -x "$(GOPATH)/bin/go-md2man" ]; then \
+		go get -u github.com/cpuguy83/go-md2man; \
+	fi
+
+.install.ostree: .gopathok
+	if ! pkg-config ostree-1 2> /dev/null ; then \
+		git clone https://github.com/ostreedev/ostree $(GOPATH)/src/github.com/ostreedev/ostree ; \
+		cd $(GOPATH)/src/github.com/ostreedev/ostree ; \
+		./autogen.sh --prefix=/usr/local; \
+		make all install; \
+	fi
+
+.PHONY: \
+	binaries \
+	clean \
+	conmon \
+	default \
+	docs \
+	gofmt \
+	help \
+	install \
+	lint \
+	pause \
+	uninstall

OWNERS

@@ -0,0 +1,9 @@
+assignees:
+  - mrunalp
+  - runcom
+  - cyphar
+  - mikebrow
+  - feiskyer
+  - sameo
+  - rhatdan
+  - nalind

README.md

@@ -0,0 +1,260 @@
+![CRI-O logo](https://cdn.rawgit.com/kubernetes-incubator/cri-o/master/logo/crio-logo.svg)
+# CRI-O - OCI-based implementation of Kubernetes Container Runtime Interface
+
+[![Build Status](https://img.shields.io/travis/kubernetes-incubator/cri-o.svg?maxAge=2592000&style=flat-square)](https://travis-ci.org/kubernetes-incubator/cri-o)
+[![Go Report Card](https://goreportcard.com/badge/github.com/kubernetes-incubator/cri-o?style=flat-square)](https://goreportcard.com/report/github.com/kubernetes-incubator/cri-o)
+
+### Status: Stable
+
+## What is the scope of this project?
+
+CRI-O is meant to provide an integration path between OCI conformant runtimes and the kubelet.
+Specifically, it implements the Kubelet [Container Runtime Interface (CRI)](https://github.com/kubernetes/community/blob/master/contributors/devel/container-runtime-interface.md) using OCI conformant runtimes.
+The scope of CRI-O is tied to the scope of the CRI.
+
+At a high level, we expect the scope of CRI-O to be restricted to the following functionalities:
+
+* Support multiple image formats including the existing Docker image format
+* Support for multiple means to download images including trust & image verification
+* Container image management (managing image layers, overlay filesystems, etc)
+* Container process lifecycle management
+* Monitoring and logging required to satisfy the CRI
+* Resource isolation as required by the CRI
+
+## What is not in scope for this project?
+
+* Building, signing and pushing images to various image storages
+* A CLI utility for interacting with CRI-O. Any CLIs built as part of this project are only meant for testing this project and there will be no guarantees on the backward compatibility with it.
+
+This is an implementation of the Kubernetes Container Runtime Interface (CRI) that will allow Kubernetes to directly launch and manage Open Container Initiative (OCI) containers.
+
+The plan is to use OCI projects and best of breed libraries for different aspects:
+- Runtime: [runc](https://github.com/opencontainers/runc) (or any OCI runtime-spec implementation) and [oci runtime tools](https://github.com/opencontainers/runtime-tools)
+- Images: Image management using [containers/image](https://github.com/containers/image)
+- Storage: Storage and management of image layers using [containers/storage](https://github.com/containers/storage)
+- Networking: Networking support through use of [CNI](https://github.com/containernetworking/cni)
+
+It is currently in active development in the Kubernetes community through the [design proposal](https://github.com/kubernetes/kubernetes/pull/26788).  Questions and issues should be raised in the Kubernetes [sig-node Slack channel](https://kubernetes.slack.com/archives/sig-node).
+
+## Commands
+| Command                                              | Description                                                               | Demo|
+| ---------------------------------------------------- | --------------------------------------------------------------------------|-----|
+| [crio(8)](/docs/crio.8.md)                           | OCI Kubernetes Container Runtime daemon                                   ||
+| [kpod(1)](/docs/kpod.1.md)                           | Simple management tool for pods and images                                ||
+| [kpod-attach(1)](/docs/kpod-attach.1.md)             | Instead of providing a `kpod attach` command, the man page `kpod-attach` describes how to use the `kpod logs` and `kpod exec` commands to achieve the same goals as `kpod attach`.||
+| [kpod-cp(1)](/docs/kpod-cp.1.md)                     | Instead of providing a `kpod cp` command, the man page `kpod-cp` describes how to use the `kpod mount` command to have even more flexibility and functionality.||
+| [kpod-diff(1)](/docs/kpod-diff.1.md)                 | Inspect changes on a container or image's filesystem                      |[![...](/docs/play.png)](https://asciinema.org/a/FXfWB9CKYFwYM4EfqW3NSZy1G)|
+| [kpod-export(1)](/docs/kpod-export.1.md)             | Export container's filesystem contents as a tar archive                   |[![...](/docs/play.png)](https://asciinema.org/a/913lBIRAg5hK8asyIhhkQVLtV)|
+| [kpod-history(1)](/docs/kpod-history.1.md)           | Shows the history of an image                                             |[![...](/docs/play.png)](https://asciinema.org/a/bCvUQJ6DkxInMELZdc5DinNSx)|
+| [kpod-images(1)](/docs/kpod-images.1.md)             | List images in local storage                                              |[![...](/docs/play.png)](https://asciinema.org/a/133649)|
+| [kpod-info(1)](/docs/kpod-info.1.md)                 | Display system information                                                ||
+| [kpod-inspect(1)](/docs/kpod-inspect.1.md)           | Display the configuration of a container or image                         |[![...](/docs/play.png)](https://asciinema.org/a/133418)|
+| [kpod-kill(1)](/docs/kpod-kill.1.md)                 | Kill the main process in one or more running containers                   |[![...](/docs/play.png)](https://asciinema.org/a/3jNos0A5yzO4hChu7ddKkUPw7)|
+| [kpod-load(1)](/docs/kpod-load.1.md)                 | Load an image from docker archive or oci                                  |[![...](/docs/play.png)](https://asciinema.org/a/kp8kOaexEhEa20P1KLZ3L5X4g)|
+| [kpod-login(1)](/docs/kpod-login.1.md)               | Login to a container registry	                                           |[![...](/docs/play.png)](https://asciinema.org/a/oNiPgmfo1FjV2YdesiLpvihtV)|
+| [kpod-logout(1)](/docs/kpod-logout.1.md)             | Logout of a container registry                                            |[![...](/docs/play.png)](https://asciinema.org/a/oNiPgmfo1FjV2YdesiLpvihtV)|
+| [kpod-logs(1)](/docs/kpod-logs.1.md)                 | Display the logs of a container                                           ||
+| [kpod-mount(1)](/docs/kpod-mount.1.md)               | Mount a working container's root filesystem                               ||
+| [kpod-pause(1)](/docs/kpod-pause.1.md)               | Pause one or more running containers                                      |[![...](/docs/play.png)](https://asciinema.org/a/141292)|
+| [kpod-ps(1)](/docs/kpod-ps.1.md)                     | Prints out information about containers                                   |[![...](/docs/play.png)](https://asciinema.org/a/bbT41kac6CwZ5giESmZLIaTLR)|
+| [kpod-pull(1)](/docs/kpod-pull.1.md)                 | Pull an image from a registry                                             |[![...](/docs/play.png)](https://asciinema.org/a/lr4zfoynHJOUNu1KaXa1dwG2X)|
+| [kpod-push(1)](/docs/kpod-push.1.md)                 | Push an image to a specified destination                                  |[![...](/docs/play.png)](https://asciinema.org/a/133276)|
+| [kpod-rename(1)](/docs/kpod-rename.1.md)             | Rename a container                                                        ||
+| [kpod-rm(1)](/docs/kpod-rm.1.md)                     | Removes one or more containers                                            |[![...](/docs/play.png)](https://asciinema.org/a/7EMk22WrfGtKWmgHJX9Nze1Qp)|
+| [kpod-rmi(1)](/docs/kpod-rmi.1.md)                   | Removes one or more images                                                |[![...](/docs/play.png)](https://asciinema.org/a/133799)|
+| [kpod-save(1)](/docs/kpod-save.1.md)                 | Saves an image to an archive                                              |[![...](/docs/play.png)](https://asciinema.org/a/kp8kOaexEhEa20P1KLZ3L5X4g)|
+| [kpod-stats(1)](/docs/kpod-stats.1.md)               | Display a live stream of one or more containers' resource usage statistics||
+| [kpod-stop(1)](/docs/kpod-stop.1.md)                 | Stops one or more running containers                                      ||
+| [kpod-tag(1)](/docs/kpod-tag.1.md)                   | Add an additional name to a local image                                   |[![...](/docs/play.png)](https://asciinema.org/a/133803)|
+| [kpod-umount(1)](/docs/kpod-umount.1.md)             | Unmount a working container's root filesystem                             ||
+| [kpod-unpause(1)](/docs/kpod-unpause.1.md)           | Unpause one or more running containers                                    |[![...](/docs/play.png)](https://asciinema.org/a/141292)|
+| [kpod-version(1)](/docs/kpod-version.1.md)           | Display the version information                                           |[![...](/docs/play.png)](https://asciinema.org/a/mfrn61pjZT9Fc8L4NbfdSqfgu)|
+| [kpod-wait(1)](/docs/kpod-wait.1.md)                 | Wait on one or more containers to stop and print their exit codes||
+
+## Configuration
+| File                                       | Description                                                                                          |
+| ---------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
+| [crio.conf(5)](/docs/crio.conf.5.md)       | CRI-O Configuation file |
+
+## OCI Hooks Support
+
+[CRI-O configures OCI Hooks to run when launching a container](./hooks.md)
+
+## CRI-O Usage Transfer
+
+[Useful information for ops and dev transfer as it relates to infrastructure that utilizes CRI-O](/transfer.md)
+
+## Communication
+
+For async communication and long running discussions please use issues and pull requests on the github repo. This will be the best place to discuss design and implementation.
+
+For sync communication we have an IRC channel #CRI-O, on chat.freenode.net, that everyone is welcome to join and chat about development.
+
+## Getting started
+
+### Prerequisites
+
+Latest version of `runc` is expected to be installed on the system. It is picked up as the default runtime by CRI-O.
+
+### Build and Run Dependencies
+
+**Required**
+
+Fedora, CentOS, RHEL, and related distributions:
+
+```bash
+yum install -y \
+  btrfs-progs-devel \
+  device-mapper-devel \
+  git \
+  glib2-devel \
+  glibc-devel \
+  glibc-static \
+  go \
+  golang-github-cpuguy83-go-md2man \
+  gpgme-devel \
+  libassuan-devel \
+  libgpg-error-devel \
+  libseccomp-devel \
+  libselinux-devel \
+  ostree-devel \
+  pkgconfig \
+  runc \
+  skopeo-containers
+```
+
+Debian, Ubuntu, and related distributions:
+
+```bash
+apt-get install -y \
+  btrfs-tools \
+  git \
+  golang-go \
+  libassuan-dev \
+  libdevmapper-dev \
+  libglib2.0-dev \
+  libc6-dev \
+  libgpgme11-dev \
+  libgpg-error-dev \
+  libseccomp-dev \
+  libselinux1-dev \
+  pkg-config \
+  runc \
+  skopeo-containers
+```
+
+Debian, Ubuntu, and related distributions will also need a copy of the development libraries for `ostree`, either in the form of the `libostree-dev` package from the [flatpak](https://launchpad.net/~alexlarsson/+archive/ubuntu/flatpak) PPA, or built [from source](https://github.com/ostreedev/ostree) (more on that [here](https://ostree.readthedocs.io/en/latest/#building)).
+
+If using an older release or a long-term support release, be careful to double-check that the version of `runc` is new enough (running `runc --version` should produce `spec: 1.0.0`), or else build your own.
+
+**NOTE**
+
+Be careful to double-check that the version of golang is new enough, version 1.8.x or higher is required.  If needed, golang kits are avaliable at https://golang.org/dl/
+
+**Optional**
+
+Fedora, CentOS, RHEL, and related distributions:
+
+(no optional packages)
+
+Debian, Ubuntu, and related distributions:
+
+```bash
+apt-get install -y \
+  libapparmor-dev
+```
+
+### Get Source Code
+
+As with other Go projects, CRI-O must be cloned into a directory structure like:
+
+```
+GOPATH
+└── src
+    └── github.com
+        └── kubernetes-incubator
+            └── cri-o
+```
+
+First, configure a `GOPATH` (if you are using go1.8 or later, this defaults to `~/go`).
+
+```bash
+export GOPATH=~/go
+mkdir -p $GOPATH
+```
+
+Next, clone the source code using:
+
+```bash
+mkdir -p $GOPATH/src/github.com/kubernetes-incubator
+cd $_ # or cd $GOPATH/src/github.com/kubernetes-incubator
+git clone https://github.com/kubernetes-incubator/cri-o # or your fork
+cd cri-o
+```
+
+### Build
+
+```bash
+make install.tools
+make
+sudo make install
+```
+
+Otherwise, if you do not want to build `CRI-O` with seccomp support you can add `BUILDTAGS=""` when running make.
+
+```bash
+make BUILDTAGS=""
+sudo make install
+```
+
+#### Build Tags
+
+`CRI-O` supports optional build tags for compiling support of various features.
+To add build tags to the make option the `BUILDTAGS` variable must be set.
+
+```bash
+make BUILDTAGS='seccomp apparmor'
+```
+
+| Build Tag | Feature                            | Dependency  |
+|-----------|------------------------------------|-------------|
+| seccomp   | syscall filtering                  | libseccomp  |
+| selinux   | selinux process and mount labeling | libselinux  |
+| apparmor  | apparmor profile support           | libapparmor |
+
+### Running pods and containers
+
+Follow this [tutorial](tutorial.md) to get started with CRI-O.
+
+### Setup CNI networking
+
+A proper description of setting up CNI networking is given in the
+[`contrib/cni` README](contrib/cni/README.md). But the gist is that you need to
+have some basic network configurations enabled and CNI plugins installed on
+your system.
+
+### Running with kubernetes
+
+You can run a local version of kubernetes with CRI-O using `local-up-cluster.sh`:
+
+1. Clone the [kubernetes repository](https://github.com/kubernetes/kubernetes)
+1. Start the CRI-O daemon (`crio`)
+1. From the kubernetes project directory, run:
+```shell
+CGROUP_DRIVER=systemd \
+CONTAINER_RUNTIME=remote \
+CONTAINER_RUNTIME_ENDPOINT='/var/run/crio.sock  --runtime-request-timeout=15m' \
+./hack/local-up-cluster.sh
+```
+
+To run a full cluster, see [the instructions](kubernetes.md).
+
+### Current Roadmap
+
+1. Basic pod/container lifecycle, basic image pull (done)
+1. Support for tty handling and state management (done)
+1. Basic integration with kubelet once client side changes are ready (done)
+1. Support for log management, networking integration using CNI, pluggable image/storage management (done)
+1. Support for exec/attach (done)
+1. Target fully automated kubernetes testing without failures [e2e status](https://github.com/kubernetes-incubator/cri-o/issues/533)
+1. Track upstream k8s releases

client/client.go

@@ -0,0 +1,103 @@
+package client
+
+import (
+	"encoding/json"
+	"fmt"
+	"net"
+	"net/http"
+	"syscall"
+	"time"
+
+	"github.com/kubernetes-incubator/cri-o/types"
+)
+
+const (
+	maxUnixSocketPathSize = len(syscall.RawSockaddrUnix{}.Path)
+)
+
+// CrioClient is an interface to get information from crio daemon endpoint.
+type CrioClient interface {
+	DaemonInfo() (types.CrioInfo, error)
+	ContainerInfo(string) (*types.ContainerInfo, error)
+}
+
+type crioClientImpl struct {
+	client         *http.Client
+	crioSocketPath string
+}
+
+func configureUnixTransport(tr *http.Transport, proto, addr string) error {
+	if len(addr) > maxUnixSocketPathSize {
+		return fmt.Errorf("Unix socket path %q is too long", addr)
+	}
+	// No need for compression in local communications.
+	tr.DisableCompression = true
+	tr.Dial = func(_, _ string) (net.Conn, error) {
+		return net.DialTimeout(proto, addr, 32*time.Second)
+	}
+	return nil
+}
+
+// New returns a crio client
+func New(crioSocketPath string) (CrioClient, error) {
+	tr := new(http.Transport)
+	configureUnixTransport(tr, "unix", crioSocketPath)
+	c := &http.Client{
+		Transport: tr,
+	}
+	return &crioClientImpl{
+		client:         c,
+		crioSocketPath: crioSocketPath,
+	}, nil
+}
+
+func (c *crioClientImpl) getRequest(path string) (*http.Request, error) {
+	req, err := http.NewRequest("GET", path, nil)
+	if err != nil {
+		return nil, err
+	}
+	// For local communications over a unix socket, it doesn't matter what
+	// the host is. We just need a valid and meaningful host name.
+	req.Host = "crio"
+	req.URL.Host = c.crioSocketPath
+	req.URL.Scheme = "http"
+	return req, nil
+}
+
+// DaemonInfo return cri-o daemon info from the cri-o
+// info endpoint.
+func (c *crioClientImpl) DaemonInfo() (types.CrioInfo, error) {
+	info := types.CrioInfo{}
+	req, err := c.getRequest("/info")
+	if err != nil {
+		return info, err
+	}
+	resp, err := c.client.Do(req)
+	if err != nil {
+		return info, err
+	}
+	defer resp.Body.Close()
+	if err := json.NewDecoder(resp.Body).Decode(&info); err != nil {
+		return info, err
+	}
+	return info, nil
+}
+
+// ContainerInfo returns container info by querying
+// the cri-o container endpoint.
+func (c *crioClientImpl) ContainerInfo(id string) (*types.ContainerInfo, error) {
+	req, err := c.getRequest("/containers/" + id)
+	if err != nil {
+		return nil, err
+	}
+	resp, err := c.client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	defer resp.Body.Close()
+	cInfo := types.ContainerInfo{}
+	if err := json.NewDecoder(resp.Body).Decode(&cInfo); err != nil {
+		return nil, err
+	}
+	return &cInfo, nil
+}

cmd/crio/config.go

@@ -0,0 +1,192 @@
+package main
+
+import (
+	"os"
+	"text/template"
+
+	"github.com/kubernetes-incubator/cri-o/server"
+	"github.com/urfave/cli"
+)
+
+var commentedConfigTemplate = template.Must(template.New("config").Parse(`
+# The "crio" table contains all of the server options.
+[crio]
+
+# root is a path to the "root directory". CRIO stores all of its data,
+# including container images, in this directory.
+root = "{{ .Root }}"
+
+# run is a path to the "run directory". CRIO stores all of its state
+# in this directory.
+runroot = "{{ .RunRoot }}"
+
+# storage_driver select which storage driver is used to manage storage
+# of images and containers.
+storage_driver = "{{ .Storage }}"
+
+# storage_option is used to pass an option to the storage driver.
+storage_option = [
+{{ range $opt := .StorageOptions }}{{ printf "\t%q,\n" $opt }}{{ end }}]
+
+# The "crio.api" table contains settings for the kubelet/gRPC
+# interface (which is also used by crioctl).
+[crio.api]
+
+# listen is the path to the AF_LOCAL socket on which crio will listen.
+listen = "{{ .Listen }}"
+
+# stream_address is the IP address on which the stream server will listen
+stream_address = "{{ .StreamAddress }}"
+
+# stream_port is the port on which the stream server will listen
+stream_port = "{{ .StreamPort }}"
+
+# file_locking is whether file-based locking will be used instead of
+# in-memory locking
+file_locking = {{ .FileLocking }}
+
+# The "crio.runtime" table contains settings pertaining to the OCI
+# runtime used and options for how to set up and manage the OCI runtime.
+[crio.runtime]
+
+# runtime is the OCI compatible runtime used for trusted container workloads.
+# This is a mandatory setting as this runtime will be the default one
+# and will also be used for untrusted container workloads if
+# runtime_untrusted_workload is not set.
+runtime = "{{ .Runtime }}"
+
+# runtime_untrusted_workload is the OCI compatible runtime used for untrusted
+# container workloads. This is an optional setting, except if
+# default_container_trust is set to "untrusted".
+runtime_untrusted_workload = "{{ .RuntimeUntrustedWorkload }}"
+
+# default_workload_trust is the default level of trust crio puts in container
+# workloads. It can either be "trusted" or "untrusted", and the default
+# is "trusted".
+# Containers can be run through different container runtimes, depending on
+# the trust hints we receive from kubelet:
+# - If kubelet tags a container workload as untrusted, crio will try first to
+# run it through the untrusted container workload runtime. If it is not set,
+# crio will use the trusted runtime.
+# - If kubelet does not provide any information about the container workload trust
+# level, the selected runtime will depend on the default_container_trust setting.
+# If it is set to "untrusted", then all containers except for the host privileged
+# ones, will be run by the runtime_untrusted_workload runtime. Host privileged
+# containers are by definition trusted and will always use the trusted container
+# runtime. If default_container_trust is set to "trusted", crio will use the trusted
+# container runtime for all containers.
+default_workload_trust = "{{ .DefaultWorkloadTrust }}"
+
+# no_pivot instructs the runtime to not use pivot_root, but instead use MS_MOVE
+no_pivot = {{ .NoPivot }}
+
+# conmon is the path to conmon binary, used for managing the runtime.
+conmon = "{{ .Conmon }}"
+
+# conmon_env is the environment variable list for conmon process,
+# used for passing necessary environment variable to conmon or runtime.
+conmon_env = [
+{{ range $env := .ConmonEnv }}{{ printf "\t%q,\n" $env }}{{ end }}]
+
+# selinux indicates whether or not SELinux will be used for pod
+# separation on the host. If you enable this flag, SELinux must be running
+# on the host.
+selinux = {{ .SELinux }}
+
+# seccomp_profile is the seccomp json profile path which is used as the
+# default for the runtime.
+seccomp_profile = "{{ .SeccompProfile }}"
+
+# apparmor_profile is the apparmor profile name which is used as the
+# default for the runtime.
+apparmor_profile = "{{ .ApparmorProfile }}"
+
+# cgroup_manager is the cgroup management implementation to be used
+# for the runtime.
+cgroup_manager = "{{ .CgroupManager }}"
+
+# hooks_dir_path is the oci hooks directory for automatically executed hooks
+hooks_dir_path = "{{ .HooksDirPath }}"
+
+# default_mounts is the mounts list to be mounted for the container when created
+default_mounts = [
+{{ range $mount := .DefaultMounts }}{{ printf "\t%q, \n" $mount }}{{ end }}]
+
+# pids_limit is the number of processes allowed in a container
+pids_limit = {{ .PidsLimit }}
+
+# log_size_max is the max limit for the container log size in bytes.
+# Negative values indicate that no limit is imposed.
+log_size_max = {{ .LogSizeMax }}
+
+# The "crio.image" table contains settings pertaining to the
+# management of OCI images.
+[crio.image]
+
+# default_transport is the prefix we try prepending to an image name if the
+# image name as we receive it can't be parsed as a valid source reference
+default_transport = "{{ .DefaultTransport }}"
+
+# pause_image is the image which we use to instantiate infra containers.
+pause_image = "{{ .PauseImage }}"
+
+# pause_command is the command to run in a pause_image to have a container just
+# sit there.  If the image contains the necessary information, this value need
+# not be specified.
+pause_command = "{{ .PauseCommand }}"
+
+# signature_policy is the name of the file which decides what sort of policy we
+# use when deciding whether or not to trust an image that we've pulled.
+# Outside of testing situations, it is strongly advised that this be left
+# unspecified so that the default system-wide policy will be used.
+signature_policy = "{{ .SignaturePolicyPath }}"
+
+# image_volumes controls how image volumes are handled.
+# The valid values are mkdir and ignore.
+image_volumes = "{{ .ImageVolumes }}"
+
+# insecure_registries is used to skip TLS verification when pulling images.
+insecure_registries = [
+{{ range $opt := .InsecureRegistries }}{{ printf "\t%q,\n" $opt }}{{ end }}]
+
+# registries is used to specify a comma separated list of registries to be used
+# when pulling an unqualified image (e.g. fedora:rawhide).
+registries = [
+{{ range $opt := .Registries }}{{ printf "\t%q,\n" $opt }}{{ end }}]
+
+# The "crio.network" table contains settings pertaining to the
+# management of CNI plugins.
+[crio.network]
+
+# network_dir is is where CNI network configuration
+# files are stored.
+network_dir = "{{ .NetworkDir }}"
+
+# plugin_dir is is where CNI plugin binaries are stored.
+plugin_dir = "{{ .PluginDir }}"
+`))
+
+// TODO: Currently ImageDir isn't really used, so we haven't added it to this
+//       template. Add it once the storage code has been merged.
+
+var configCommand = cli.Command{
+	Name:  "config",
+	Usage: "generate crio configuration files",
+	Flags: []cli.Flag{
+		cli.BoolFlag{
+			Name:  "default",
+			Usage: "output the default configuration",
+		},
+	},
+	Action: func(c *cli.Context) error {
+		// At this point, app.Before has already parsed the user's chosen
+		// config file. So no need to handle that here.
+		config := c.App.Metadata["config"].(*server.Config)
+		if c.Bool("default") {
+			config = server.DefaultConfig()
+		}
+
+		// Output the commented config.
+		return commentedConfigTemplate.ExecuteTemplate(os.Stdout, "config", config)
+	},
+}

cmd/crio/daemon_linux.go

@@ -0,0 +1,20 @@
+// +build linux
+
+package main
+
+import (
+	systemdDaemon "github.com/coreos/go-systemd/daemon"
+	"github.com/sirupsen/logrus"
+)
+
+func sdNotify() {
+	if _, err := systemdDaemon.SdNotify(true, "READY=1"); err != nil {
+		logrus.Warnf("Failed to sd_notify systemd: %v", err)
+	}
+}
+
+// notifySystem sends a message to the host when the server is ready to be used
+func notifySystem() {
+	// Tell the init daemon we are accepting requests
+	go sdNotify()
+}

cmd/crio/main.go

@@ -0,0 +1,532 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"net"
+	"net/http"
+	_ "net/http/pprof"
+	"os"
+	"os/signal"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/containers/storage/pkg/reexec"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/kubernetes-incubator/cri-o/server"
+	"github.com/kubernetes-incubator/cri-o/version"
+	"github.com/opencontainers/selinux/go-selinux"
+	"github.com/sirupsen/logrus"
+	"github.com/soheilhy/cmux"
+	"github.com/urfave/cli"
+	"golang.org/x/sys/unix"
+	"google.golang.org/grpc"
+	"k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+// gitCommit is the commit that the binary is being built from.
+// It will be populated by the Makefile.
+var gitCommit = ""
+
+func validateConfig(config *server.Config) error {
+	switch config.ImageVolumes {
+	case libkpod.ImageVolumesMkdir:
+	case libkpod.ImageVolumesIgnore:
+	case libkpod.ImageVolumesBind:
+	default:
+		return fmt.Errorf("Unrecognized image volume type specified")
+
+	}
+
+	// This needs to match the read buffer size in conmon
+	if config.LogSizeMax >= 0 && config.LogSizeMax < 8192 {
+		return fmt.Errorf("log size max should be negative or >= 8192")
+	}
+	return nil
+}
+
+func mergeConfig(config *server.Config, ctx *cli.Context) error {
+	// Don't parse the config if the user explicitly set it to "".
+	if path := ctx.GlobalString("config"); path != "" {
+		if err := config.UpdateFromFile(path); err != nil {
+			if ctx.GlobalIsSet("config") || !os.IsNotExist(err) {
+				return err
+			}
+
+			// We don't error out if --config wasn't explicitly set and the
+			// default doesn't exist. But we will log a warning about it, so
+			// the user doesn't miss it.
+			logrus.Warnf("default configuration file does not exist: %s", server.CrioConfigPath)
+		}
+	}
+
+	// Override options set with the CLI.
+	if ctx.GlobalIsSet("conmon") {
+		config.Conmon = ctx.GlobalString("conmon")
+	}
+	if ctx.GlobalIsSet("pause-command") {
+		config.PauseCommand = ctx.GlobalString("pause-command")
+	}
+	if ctx.GlobalIsSet("pause-image") {
+		config.PauseImage = ctx.GlobalString("pause-image")
+	}
+	if ctx.GlobalIsSet("signature-policy") {
+		config.SignaturePolicyPath = ctx.GlobalString("signature-policy")
+	}
+	if ctx.GlobalIsSet("root") {
+		config.Root = ctx.GlobalString("root")
+	}
+	if ctx.GlobalIsSet("runroot") {
+		config.RunRoot = ctx.GlobalString("runroot")
+	}
+	if ctx.GlobalIsSet("storage-driver") {
+		config.Storage = ctx.GlobalString("storage-driver")
+	}
+	if ctx.GlobalIsSet("storage-opt") {
+		config.StorageOptions = ctx.GlobalStringSlice("storage-opt")
+	}
+	if ctx.GlobalIsSet("file-locking") {
+		config.FileLocking = ctx.GlobalBool("file-locking")
+	}
+	if ctx.GlobalIsSet("insecure-registry") {
+		config.InsecureRegistries = ctx.GlobalStringSlice("insecure-registry")
+	}
+	if ctx.GlobalIsSet("registry") {
+		config.Registries = ctx.GlobalStringSlice("registry")
+	}
+	if ctx.GlobalIsSet("default-transport") {
+		config.DefaultTransport = ctx.GlobalString("default-transport")
+	}
+	if ctx.GlobalIsSet("listen") {
+		config.Listen = ctx.GlobalString("listen")
+	}
+	if ctx.GlobalIsSet("stream-address") {
+		config.StreamAddress = ctx.GlobalString("stream-address")
+	}
+	if ctx.GlobalIsSet("stream-port") {
+		config.StreamPort = ctx.GlobalString("stream-port")
+	}
+	if ctx.GlobalIsSet("runtime") {
+		config.Runtime = ctx.GlobalString("runtime")
+	}
+	if ctx.GlobalIsSet("selinux") {
+		config.SELinux = ctx.GlobalBool("selinux")
+	}
+	if ctx.GlobalIsSet("seccomp-profile") {
+		config.SeccompProfile = ctx.GlobalString("seccomp-profile")
+	}
+	if ctx.GlobalIsSet("apparmor-profile") {
+		config.ApparmorProfile = ctx.GlobalString("apparmor-profile")
+	}
+	if ctx.GlobalIsSet("cgroup-manager") {
+		config.CgroupManager = ctx.GlobalString("cgroup-manager")
+	}
+	if ctx.GlobalIsSet("hooks-dir-path") {
+		config.HooksDirPath = ctx.GlobalString("hooks-dir-path")
+	}
+	if ctx.GlobalIsSet("default-mounts") {
+		config.DefaultMounts = ctx.GlobalStringSlice("default-mounts")
+	}
+	if ctx.GlobalIsSet("pids-limit") {
+		config.PidsLimit = ctx.GlobalInt64("pids-limit")
+	}
+	if ctx.GlobalIsSet("log-size-max") {
+		config.LogSizeMax = ctx.GlobalInt64("log-size-max")
+	}
+	if ctx.GlobalIsSet("cni-config-dir") {
+		config.NetworkDir = ctx.GlobalString("cni-config-dir")
+	}
+	if ctx.GlobalIsSet("cni-plugin-dir") {
+		config.PluginDir = ctx.GlobalString("cni-plugin-dir")
+	}
+	if ctx.GlobalIsSet("image-volumes") {
+		config.ImageVolumes = libkpod.ImageVolumesType(ctx.GlobalString("image-volumes"))
+	}
+	return nil
+}
+
+func catchShutdown(gserver *grpc.Server, sserver *server.Server, hserver *http.Server, signalled *bool) {
+	sig := make(chan os.Signal, 10)
+	signal.Notify(sig, unix.SIGINT, unix.SIGTERM)
+	go func() {
+		for s := range sig {
+			switch s {
+			case unix.SIGINT:
+				logrus.Debugf("Caught SIGINT")
+			case unix.SIGTERM:
+				logrus.Debugf("Caught SIGTERM")
+			default:
+				continue
+			}
+			*signalled = true
+			gserver.GracefulStop()
+			hserver.Shutdown(context.Background())
+			// TODO(runcom): enable this after https://github.com/kubernetes/kubernetes/pull/51377
+			//sserver.StopStreamServer()
+			sserver.StopExitMonitor()
+			if err := sserver.Shutdown(); err != nil {
+				logrus.Warnf("error shutting down main service %v", err)
+			}
+			return
+		}
+	}()
+}
+
+func main() {
+	if reexec.Init() {
+		return
+	}
+	app := cli.NewApp()
+
+	var v []string
+	v = append(v, version.Version)
+	if gitCommit != "" {
+		v = append(v, fmt.Sprintf("commit: %s", gitCommit))
+	}
+	app.Name = "crio"
+	app.Usage = "crio server"
+	app.Version = strings.Join(v, "\n")
+	app.Metadata = map[string]interface{}{
+		"config": server.DefaultConfig(),
+	}
+
+	app.Flags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "config",
+			Value: server.CrioConfigPath,
+			Usage: "path to configuration file",
+		},
+		cli.StringFlag{
+			Name:  "conmon",
+			Usage: "path to the conmon executable",
+		},
+		cli.StringFlag{
+			Name:  "listen",
+			Usage: "path to crio socket",
+		},
+		cli.StringFlag{
+			Name:  "stream-address",
+			Usage: "bind address for streaming socket",
+		},
+		cli.StringFlag{
+			Name:  "stream-port",
+			Usage: "bind port for streaming socket (default: \"10010\")",
+		},
+		cli.StringFlag{
+			Name:  "log",
+			Value: "",
+			Usage: "set the log file path where internal debug information is written",
+		},
+		cli.StringFlag{
+			Name:  "log-format",
+			Value: "text",
+			Usage: "set the format used by logs ('text' (default), or 'json')",
+		},
+		cli.StringFlag{
+			Name:  "log-level",
+			Usage: "log messages above specified level: debug, info (default), warn, error, fatal or panic",
+		},
+
+		cli.StringFlag{
+			Name:  "pause-command",
+			Usage: "name of the pause command in the pause image",
+		},
+		cli.StringFlag{
+			Name:  "pause-image",
+			Usage: "name of the pause image",
+		},
+		cli.StringFlag{
+			Name:  "signature-policy",
+			Usage: "path to signature policy file",
+		},
+		cli.StringFlag{
+			Name:  "root",
+			Usage: "crio root dir",
+		},
+		cli.StringFlag{
+			Name:  "runroot",
+			Usage: "crio state dir",
+		},
+		cli.StringFlag{
+			Name:  "storage-driver",
+			Usage: "storage driver",
+		},
+		cli.StringSliceFlag{
+			Name:  "storage-opt",
+			Usage: "storage driver option",
+		},
+		cli.BoolFlag{
+			Name:  "file-locking",
+			Usage: "enable or disable file-based locking",
+		},
+		cli.StringSliceFlag{
+			Name:  "insecure-registry",
+			Usage: "whether to disable TLS verification for the given registry",
+		},
+		cli.StringSliceFlag{
+			Name:  "registry",
+			Usage: "registry to be prepended when pulling unqualified images, can be specified multiple times",
+		},
+		cli.StringFlag{
+			Name:  "default-transport",
+			Usage: "default transport",
+		},
+		cli.StringFlag{
+			Name:  "runtime",
+			Usage: "OCI runtime path",
+		},
+		cli.StringFlag{
+			Name:  "seccomp-profile",
+			Usage: "default seccomp profile path",
+		},
+		cli.StringFlag{
+			Name:  "apparmor-profile",
+			Usage: "default apparmor profile name (default: \"crio-default\")",
+		},
+		cli.BoolFlag{
+			Name:  "selinux",
+			Usage: "enable selinux support",
+		},
+		cli.StringFlag{
+			Name:  "cgroup-manager",
+			Usage: "cgroup manager (cgroupfs or systemd)",
+		},
+		cli.Int64Flag{
+			Name:  "pids-limit",
+			Value: libkpod.DefaultPidsLimit,
+			Usage: "maximum number of processes allowed in a container",
+		},
+		cli.Int64Flag{
+			Name:  "log-size-max",
+			Value: libkpod.DefaultLogSizeMax,
+			Usage: "maximum log size in bytes for a container",
+		},
+		cli.StringFlag{
+			Name:  "cni-config-dir",
+			Usage: "CNI configuration files directory",
+		},
+		cli.StringFlag{
+			Name:  "cni-plugin-dir",
+			Usage: "CNI plugin binaries directory",
+		},
+		cli.StringFlag{
+			Name:  "image-volumes",
+			Value: string(libkpod.ImageVolumesMkdir),
+			Usage: "image volume handling ('mkdir', 'bind', or 'ignore')",
+		},
+		cli.StringFlag{
+			Name:   "hooks-dir-path",
+			Usage:  "set the OCI hooks directory path",
+			Value:  libkpod.DefaultHooksDirPath,
+			Hidden: true,
+		},
+		cli.StringSliceFlag{
+			Name:   "default-mounts",
+			Usage:  "add one or more default mount paths in the form host:container",
+			Hidden: true,
+		},
+		cli.BoolFlag{
+			Name:  "profile",
+			Usage: "enable pprof remote profiler on localhost:6060",
+		},
+		cli.IntFlag{
+			Name:  "profile-port",
+			Value: 6060,
+			Usage: "port for the pprof profiler",
+		},
+		cli.BoolFlag{
+			Name:  "enable-metrics",
+			Usage: "enable metrics endpoint for the servier on localhost:9090",
+		},
+		cli.IntFlag{
+			Name:  "metrics-port",
+			Value: 9090,
+			Usage: "port for the metrics endpoint",
+		},
+	}
+
+	sort.Sort(cli.FlagsByName(app.Flags))
+	sort.Sort(cli.FlagsByName(configCommand.Flags))
+
+	app.Commands = []cli.Command{
+		configCommand,
+	}
+
+	app.Before = func(c *cli.Context) error {
+		// Load the configuration file.
+		config := c.App.Metadata["config"].(*server.Config)
+		if err := mergeConfig(config, c); err != nil {
+			return err
+		}
+
+		if err := validateConfig(config); err != nil {
+			return err
+		}
+
+		cf := &logrus.TextFormatter{
+			TimestampFormat: "2006-01-02 15:04:05.000000000Z07:00",
+			FullTimestamp:   true,
+		}
+
+		logrus.SetFormatter(cf)
+
+		if loglevel := c.GlobalString("log-level"); loglevel != "" {
+			level, err := logrus.ParseLevel(loglevel)
+			if err != nil {
+				return err
+			}
+
+			logrus.SetLevel(level)
+		}
+
+		if path := c.GlobalString("log"); path != "" {
+			f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND|os.O_SYNC, 0666)
+			if err != nil {
+				return err
+			}
+			logrus.SetOutput(f)
+		}
+
+		switch c.GlobalString("log-format") {
+		case "text":
+			// retain logrus's default.
+		case "json":
+			logrus.SetFormatter(new(logrus.JSONFormatter))
+		default:
+			return fmt.Errorf("unknown log-format %q", c.GlobalString("log-format"))
+		}
+
+		return nil
+	}
+
+	app.Action = func(c *cli.Context) error {
+		if c.GlobalBool("profile") {
+			profilePort := c.GlobalInt("profile-port")
+			profileEndpoint := fmt.Sprintf("localhost:%v", profilePort)
+			go func() {
+				http.ListenAndServe(profileEndpoint, nil)
+			}()
+		}
+
+		args := c.Args()
+		if len(args) > 0 {
+			for _, command := range app.Commands {
+				if args[0] == command.Name {
+					break
+				}
+			}
+			return fmt.Errorf("command %q not supported", args[0])
+		}
+
+		config := c.App.Metadata["config"].(*server.Config)
+
+		if !config.SELinux {
+			selinux.SetDisabled()
+		}
+
+		if _, err := os.Stat(config.Runtime); os.IsNotExist(err) {
+			// path to runtime does not exist
+			return fmt.Errorf("invalid --runtime value %q", err)
+		}
+
+		// Remove the socket if it already exists
+		if _, err := os.Stat(config.Listen); err == nil {
+			if err := os.Remove(config.Listen); err != nil {
+				logrus.Fatal(err)
+			}
+		}
+		lis, err := net.Listen("unix", config.Listen)
+		if err != nil {
+			logrus.Fatalf("failed to listen: %v", err)
+		}
+
+		s := grpc.NewServer()
+
+		service, err := server.New(config)
+		if err != nil {
+			logrus.Fatal(err)
+		}
+
+		if c.GlobalBool("enable-metrics") {
+			metricsPort := c.GlobalInt("metrics-port")
+			me, err := service.CreateMetricsEndpoint()
+			if err != nil {
+				logrus.Fatalf("Failed to create metrics endpoint: %v", err)
+			}
+			l, err := net.Listen("tcp", fmt.Sprintf(":%v", metricsPort))
+			if err != nil {
+				logrus.Fatalf("Failed to create listener for metrics: %v", err)
+			}
+			go func() {
+				if err := http.Serve(l, me); err != nil {
+					logrus.Fatalf("Failed to serve metrics endpoint: %v", err)
+				}
+			}()
+		}
+
+		runtime.RegisterRuntimeServiceServer(s, service)
+		runtime.RegisterImageServiceServer(s, service)
+
+		// after the daemon is done setting up we can notify systemd api
+		notifySystem()
+
+		go func() {
+			service.StartExitMonitor()
+		}()
+
+		m := cmux.New(lis)
+		grpcL := m.Match(cmux.HTTP2HeaderField("content-type", "application/grpc"))
+		httpL := m.Match(cmux.HTTP1Fast())
+
+		infoMux := service.GetInfoMux()
+		srv := &http.Server{
+			Handler:     infoMux,
+			ReadTimeout: 5 * time.Second,
+		}
+
+		graceful := false
+		catchShutdown(s, service, srv, &graceful)
+
+		go s.Serve(grpcL)
+		go srv.Serve(httpL)
+
+		serverCloseCh := make(chan struct{})
+		go func() {
+			defer close(serverCloseCh)
+			if err := m.Serve(); err != nil {
+				if graceful && strings.Contains(strings.ToLower(err.Error()), "use of closed network connection") {
+					err = nil
+				} else {
+					logrus.Errorf("Failed to serve grpc grpc request: %v", err)
+				}
+			}
+		}()
+
+		// TODO(runcom): enable this after https://github.com/kubernetes/kubernetes/pull/51377
+		//streamServerCloseCh := service.StreamingServerCloseChan()
+		serverExitMonitorCh := service.ExitMonitorCloseChan()
+		select {
+		// TODO(runcom): enable this after https://github.com/kubernetes/kubernetes/pull/51377
+		//case <-streamServerCloseCh:
+		case <-serverExitMonitorCh:
+		case <-serverCloseCh:
+		}
+
+		service.Shutdown()
+
+		// TODO(runcom): enable this after https://github.com/kubernetes/kubernetes/pull/51377
+		//<-streamServerCloseCh
+		//logrus.Debug("closed stream server")
+		<-serverExitMonitorCh
+		logrus.Debug("closed exit monitor")
+		<-serverCloseCh
+		logrus.Debug("closed main server")
+
+		return nil
+	}
+
+	if err := app.Run(os.Args); err != nil {
+		logrus.Fatal(err)
+	}
+}

cmd/crioctl/container.go

@@ -0,0 +1,653 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"log"
+	"net/url"
+	"os"
+	"strings"
+	"time"
+
+	"github.com/kubernetes-incubator/cri-o/client"
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+	remocommandconsts "k8s.io/apimachinery/pkg/util/remotecommand"
+	restclient "k8s.io/client-go/rest"
+	"k8s.io/client-go/tools/remotecommand"
+	pb "k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+var containerCommand = cli.Command{
+	Name:    "container",
+	Aliases: []string{"ctr"},
+	Subcommands: []cli.Command{
+		createContainerCommand,
+		inspectContainerCommand,
+		startContainerCommand,
+		stopContainerCommand,
+		removeContainerCommand,
+		containerStatusCommand,
+		listContainersCommand,
+		execSyncCommand,
+		execCommand,
+	},
+}
+
+type createOptions struct {
+	// configPath is path to the config for container
+	configPath string
+	// name sets the container name
+	name string
+	// podID of the container
+	podID string
+	// labels for the container
+	labels map[string]string
+}
+
+var createContainerCommand = cli.Command{
+	Name:  "create",
+	Usage: "create a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "pod",
+			Usage: "the id of the pod sandbox to which the container belongs",
+		},
+		cli.StringFlag{
+			Name:  "config",
+			Value: "config.json",
+			Usage: "the path of a container config file",
+		},
+		cli.StringFlag{
+			Name:  "name",
+			Value: "",
+			Usage: "the name of the container",
+		},
+		cli.StringSliceFlag{
+			Name:  "label",
+			Usage: "add key=value labels to the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		if !context.IsSet("pod") {
+			return fmt.Errorf("Please specify the id of the pod sandbox to which the container belongs via the --pod option")
+		}
+
+		opts := createOptions{
+			configPath: context.String("config"),
+			name:       context.String("name"),
+			podID:      context.String("pod"),
+			labels:     make(map[string]string),
+		}
+
+		for _, l := range context.StringSlice("label") {
+			pair := strings.Split(l, "=")
+			if len(pair) != 2 {
+				return fmt.Errorf("incorrectly specified label: %v", l)
+			}
+			opts.labels[pair[0]] = pair[1]
+		}
+
+		// Test RuntimeServiceClient.CreateContainer
+		err = CreateContainer(client, opts)
+		if err != nil {
+			return fmt.Errorf("Creating container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var startContainerCommand = cli.Command{
+	Name:  "start",
+	Usage: "start a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = StartContainer(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("Starting the container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var stopContainerCommand = cli.Command{
+	Name:  "stop",
+	Usage: "stop a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+		cli.Int64Flag{
+			Name:  "timeout",
+			Value: 10,
+			Usage: "seconds to wait to kill the container after a graceful stop is requested",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = StopContainer(client, context.String("id"), context.Int64("timeout"))
+		if err != nil {
+			return fmt.Errorf("Stopping the container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var removeContainerCommand = cli.Command{
+	Name:  "remove",
+	Usage: "remove a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = RemoveContainer(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("Removing the container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var containerStatusCommand = cli.Command{
+	Name:  "status",
+	Usage: "get the status of a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = ContainerStatus(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("Getting the status of the container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var execSyncCommand = cli.Command{
+	Name:  "execsync",
+	Usage: "exec a command synchronously in a container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+		cli.Int64Flag{
+			Name:  "timeout",
+			Value: 0,
+			Usage: "timeout for the command",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = ExecSync(client, context.String("id"), context.Args(), context.Int64("timeout"))
+		if err != nil {
+			return fmt.Errorf("execing command in container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var execCommand = cli.Command{
+	Name:  "exec",
+	Usage: "prepare a streaming endpoint to execute a command in the container",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+		cli.BoolFlag{
+			Name:  "tty",
+			Usage: "whether to use tty",
+		},
+		cli.BoolFlag{
+			Name:  "stdin",
+			Usage: "whether to stream to stdin",
+		},
+		cli.BoolFlag{
+			Name:  "url",
+			Usage: "do not exec command, just prepare streaming endpoint",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = Exec(client, context.String("id"), context.Bool("tty"), context.Bool("stdin"), context.Bool("url"), context.Args())
+		if err != nil {
+			return fmt.Errorf("execing command in container failed: %v", err)
+		}
+		return nil
+	},
+}
+
+type listOptions struct {
+	// id of the container
+	id string
+	// podID of the container
+	podID string
+	// state of the container
+	state string
+	// quiet is for listing just container IDs
+	quiet bool
+	// labels are selectors for the container
+	labels map[string]string
+}
+
+var listContainersCommand = cli.Command{
+	Name:  "list",
+	Usage: "list containers",
+	Flags: []cli.Flag{
+		cli.BoolFlag{
+			Name:  "quiet",
+			Usage: "list only container IDs",
+		},
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "filter by container id",
+		},
+		cli.StringFlag{
+			Name:  "pod",
+			Value: "",
+			Usage: "filter by container pod id",
+		},
+		cli.StringFlag{
+			Name:  "state",
+			Value: "",
+			Usage: "filter by container state",
+		},
+		cli.StringSliceFlag{
+			Name:  "label",
+			Usage: "filter by key=value label",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+		opts := listOptions{
+			id:     context.String("id"),
+			podID:  context.String("pod"),
+			state:  context.String("state"),
+			quiet:  context.Bool("quiet"),
+			labels: make(map[string]string),
+		}
+
+		for _, l := range context.StringSlice("label") {
+			pair := strings.Split(l, "=")
+			if len(pair) != 2 {
+				return fmt.Errorf("incorrectly specified label: %v", l)
+			}
+			opts.labels[pair[0]] = pair[1]
+		}
+
+		err = ListContainers(client, opts)
+		if err != nil {
+			return fmt.Errorf("listing containers failed: %v", err)
+		}
+		return nil
+	},
+}
+
+// CreateContainer sends a CreateContainerRequest to the server, and parses
+// the returned CreateContainerResponse.
+func CreateContainer(client pb.RuntimeServiceClient, opts createOptions) error {
+	config, err := loadContainerConfig(opts.configPath)
+	if err != nil {
+		return err
+	}
+
+	// Override the name by the one specified through CLI
+	if opts.name != "" {
+		config.Metadata.Name = opts.name
+	}
+
+	for k, v := range opts.labels {
+		config.Labels[k] = v
+	}
+
+	r, err := client.CreateContainer(context.Background(), &pb.CreateContainerRequest{
+		PodSandboxId: opts.podID,
+		Config:       config,
+		// TODO(runcom): this is missing PodSandboxConfig!!!
+		// we should/could find a way to retrieve it from the fs and set it here
+	})
+	if err != nil {
+		return err
+	}
+	fmt.Println(r.ContainerId)
+	return nil
+}
+
+// StartContainer sends a StartContainerRequest to the server, and parses
+// the returned StartContainerResponse.
+func StartContainer(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	_, err := client.StartContainer(context.Background(), &pb.StartContainerRequest{
+		ContainerId: ID,
+	})
+	if err != nil {
+		return err
+	}
+	fmt.Println(ID)
+	return nil
+}
+
+// StopContainer sends a StopContainerRequest to the server, and parses
+// the returned StopContainerResponse.
+func StopContainer(client pb.RuntimeServiceClient, ID string, timeout int64) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	_, err := client.StopContainer(context.Background(), &pb.StopContainerRequest{
+		ContainerId: ID,
+		Timeout:     timeout,
+	})
+	if err != nil {
+		return err
+	}
+	fmt.Println(ID)
+	return nil
+}
+
+// RemoveContainer sends a RemoveContainerRequest to the server, and parses
+// the returned RemoveContainerResponse.
+func RemoveContainer(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	_, err := client.RemoveContainer(context.Background(), &pb.RemoveContainerRequest{
+		ContainerId: ID,
+	})
+	if err != nil {
+		return err
+	}
+	fmt.Println(ID)
+	return nil
+}
+
+// ContainerStatus sends a ContainerStatusRequest to the server, and parses
+// the returned ContainerStatusResponse.
+func ContainerStatus(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	r, err := client.ContainerStatus(context.Background(), &pb.ContainerStatusRequest{
+		ContainerId: ID})
+	if err != nil {
+		return err
+	}
+	fmt.Printf("ID: %s\n", r.Status.Id)
+	if r.Status.Metadata != nil {
+		if r.Status.Metadata.Name != "" {
+			fmt.Printf("Name: %s\n", r.Status.Metadata.Name)
+		}
+		fmt.Printf("Attempt: %v\n", r.Status.Metadata.Attempt)
+	}
+	// TODO(mzylowski): print it prettier
+	fmt.Printf("Status: %s\n", r.Status.State)
+	ctm := time.Unix(0, r.Status.CreatedAt)
+	fmt.Printf("Created: %v\n", ctm)
+	stm := time.Unix(0, r.Status.StartedAt)
+	fmt.Printf("Started: %v\n", stm)
+	ftm := time.Unix(0, r.Status.FinishedAt)
+	fmt.Printf("Finished: %v\n", ftm)
+	fmt.Printf("Exit Code: %v\n", r.Status.ExitCode)
+	fmt.Printf("Reason: %v\n", r.Status.Reason)
+	if r.Status.Image != nil {
+		fmt.Printf("Image: %v\n", r.Status.Image.Image)
+	}
+	fmt.Printf("ImageRef: %v\n", r.Status.ImageRef)
+
+	return nil
+}
+
+// ExecSync sends an ExecSyncRequest to the server, and parses
+// the returned ExecSyncResponse.
+func ExecSync(client pb.RuntimeServiceClient, ID string, cmd []string, timeout int64) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	r, err := client.ExecSync(context.Background(), &pb.ExecSyncRequest{
+		ContainerId: ID,
+		Cmd:         cmd,
+		Timeout:     timeout,
+	})
+	if err != nil {
+		return err
+	}
+	fmt.Println("Stdout:")
+	fmt.Println(string(r.Stdout))
+	fmt.Println("Stderr:")
+	fmt.Println(string(r.Stderr))
+	fmt.Printf("Exit code: %v\n", r.ExitCode)
+
+	return nil
+}
+
+// Exec sends an ExecRequest to the server, and parses
+// the returned ExecResponse.
+func Exec(client pb.RuntimeServiceClient, ID string, tty bool, stdin bool, urlOnly bool, cmd []string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	r, err := client.Exec(context.Background(), &pb.ExecRequest{
+		ContainerId: ID,
+		Cmd:         cmd,
+		Tty:         tty,
+		Stdin:       stdin,
+	})
+	if err != nil {
+		return err
+	}
+
+	if urlOnly {
+		fmt.Println("URL:")
+		fmt.Println(r.Url)
+		return nil
+	}
+
+	execURL, err := url.Parse(r.Url)
+	if err != nil {
+		return err
+	}
+
+	streamExec, err := remotecommand.NewExecutor(&restclient.Config{}, "GET", execURL)
+	if err != nil {
+		return err
+	}
+
+	options := remotecommand.StreamOptions{
+		SupportedProtocols: remocommandconsts.SupportedStreamingProtocols,
+		Stdout:             os.Stdout,
+		Stderr:             os.Stderr,
+		Tty:                tty,
+	}
+
+	if stdin {
+		options.Stdin = os.Stdin
+	}
+
+	return streamExec.Stream(options)
+}
+
+// ListContainers sends a ListContainerRequest to the server, and parses
+// the returned ListContainerResponse.
+func ListContainers(client pb.RuntimeServiceClient, opts listOptions) error {
+	filter := &pb.ContainerFilter{}
+	if opts.id != "" {
+		filter.Id = opts.id
+	}
+	if opts.podID != "" {
+		filter.PodSandboxId = opts.podID
+	}
+	if opts.state != "" {
+		st := &pb.ContainerStateValue{}
+		st.State = pb.ContainerState_CONTAINER_UNKNOWN
+		switch opts.state {
+		case "created":
+			st.State = pb.ContainerState_CONTAINER_CREATED
+			filter.State = st
+		case "running":
+			st.State = pb.ContainerState_CONTAINER_RUNNING
+			filter.State = st
+		case "stopped":
+			st.State = pb.ContainerState_CONTAINER_EXITED
+			filter.State = st
+		default:
+			log.Fatalf("--state should be one of created, running or stopped")
+		}
+	}
+	if opts.labels != nil {
+		filter.LabelSelector = opts.labels
+	}
+	r, err := client.ListContainers(context.Background(), &pb.ListContainersRequest{
+		Filter: filter,
+	})
+	if err != nil {
+		return err
+	}
+	for _, c := range r.GetContainers() {
+		if opts.quiet {
+			fmt.Println(c.Id)
+			continue
+		}
+		fmt.Printf("ID: %s\n", c.Id)
+		fmt.Printf("Pod: %s\n", c.PodSandboxId)
+		if c.Metadata != nil {
+			if c.Metadata.Name != "" {
+				fmt.Printf("Name: %s\n", c.Metadata.Name)
+			}
+			fmt.Printf("Attempt: %v\n", c.Metadata.Attempt)
+		}
+		fmt.Printf("Status: %s\n", c.State)
+		if c.Image != nil {
+			fmt.Printf("Image: %s\n", c.Image.Image)
+		}
+		ctm := time.Unix(0, c.CreatedAt)
+		fmt.Printf("Created: %v\n", ctm)
+		if c.Labels != nil {
+			fmt.Println("Labels:")
+			for _, k := range getSortedKeys(c.Labels) {
+				fmt.Printf("\t%s -> %s\n", k, c.Labels[k])
+			}
+		}
+		if c.Annotations != nil {
+			fmt.Println("Annotations:")
+			for _, k := range getSortedKeys(c.Annotations) {
+				fmt.Printf("\t%s -> %s\n", k, c.Annotations[k])
+			}
+		}
+		fmt.Println()
+	}
+	return nil
+}
+
+var inspectContainerCommand = cli.Command{
+	Name:  "inspect",
+	Usage: "get container info from crio daemon",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		ID := context.String("id")
+		if ID == "" {
+			return fmt.Errorf("ID cannot be empty")
+		}
+		c, err := client.New(context.GlobalString("connect"))
+		if err != nil {
+			return err
+		}
+
+		cInfo, err := c.ContainerInfo(ID)
+		if err != nil {
+			return err
+		}
+
+		jsonBytes, err := json.MarshalIndent(cInfo, "", "    ")
+		if err != nil {
+			return err
+		}
+		fmt.Println(string(jsonBytes))
+		return nil
+	},
+}

cmd/crioctl/image.go

@@ -0,0 +1,173 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+	pb "k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+var imageCommand = cli.Command{
+	Name: "image",
+	Subcommands: []cli.Command{
+		pullImageCommand,
+		listImageCommand,
+		imageStatusCommand,
+		removeImageCommand,
+	},
+}
+
+var pullImageCommand = cli.Command{
+	Name:  "pull",
+	Usage: "pull an image",
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewImageServiceClient(conn)
+
+		_, err = PullImage(client, context.Args().Get(0))
+		if err != nil {
+			return fmt.Errorf("pulling image failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var listImageCommand = cli.Command{
+	Name:  "list",
+	Usage: "list images",
+	Flags: []cli.Flag{
+		cli.BoolFlag{
+			Name:  "quiet",
+			Usage: "list only image IDs",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewImageServiceClient(conn)
+
+		r, err := ListImages(client, context.Args().Get(0))
+		if err != nil {
+			return fmt.Errorf("listing images failed: %v", err)
+		}
+		quiet := context.Bool("quiet")
+		for _, image := range r.Images {
+			if quiet {
+				fmt.Printf("%s\n", image.Id)
+				continue
+			}
+			fmt.Printf("ID: %s\n", image.Id)
+			for _, tag := range image.RepoTags {
+				fmt.Printf("Tag: %s\n", tag)
+			}
+			for _, digest := range image.RepoDigests {
+				fmt.Printf("Digest: %s\n", digest)
+			}
+			if image.Size_ != 0 {
+				fmt.Printf("Size: %d\n", image.Size_)
+			}
+		}
+		return nil
+	},
+}
+
+var imageStatusCommand = cli.Command{
+	Name:  "status",
+	Usage: "return the status of an image",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Usage: "id of the image",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewImageServiceClient(conn)
+
+		r, err := ImageStatus(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("image status request failed: %v", err)
+		}
+		image := r.Image
+		if image == nil {
+			return fmt.Errorf("no such image present")
+		}
+		fmt.Printf("ID: %s\n", image.Id)
+		for _, tag := range image.RepoTags {
+			fmt.Printf("Tag: %s\n", tag)
+		}
+		for _, digest := range image.RepoDigests {
+			fmt.Printf("Digest: %s\n", digest)
+		}
+		fmt.Printf("Size: %d\n", image.Size_)
+		return nil
+	},
+}
+var removeImageCommand = cli.Command{
+	Name:  "remove",
+	Usage: "remove an image",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the image",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewImageServiceClient(conn)
+
+		_, err = RemoveImage(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("removing the image failed: %v", err)
+		}
+		return nil
+	},
+}
+
+// PullImage sends a PullImageRequest to the server, and parses
+// the returned PullImageResponse.
+func PullImage(client pb.ImageServiceClient, image string) (*pb.PullImageResponse, error) {
+	return client.PullImage(context.Background(), &pb.PullImageRequest{Image: &pb.ImageSpec{Image: image}})
+}
+
+// ListImages sends a ListImagesRequest to the server, and parses
+// the returned ListImagesResponse.
+func ListImages(client pb.ImageServiceClient, image string) (*pb.ListImagesResponse, error) {
+	return client.ListImages(context.Background(), &pb.ListImagesRequest{Filter: &pb.ImageFilter{Image: &pb.ImageSpec{Image: image}}})
+}
+
+// ImageStatus sends an ImageStatusRequest to the server, and parses
+// the returned ImageStatusResponse.
+func ImageStatus(client pb.ImageServiceClient, image string) (*pb.ImageStatusResponse, error) {
+	return client.ImageStatus(context.Background(), &pb.ImageStatusRequest{Image: &pb.ImageSpec{Image: image}})
+}
+
+// RemoveImage sends a RemoveImageRequest to the server, and parses
+// the returned RemoveImageResponse.
+func RemoveImage(client pb.ImageServiceClient, image string) (*pb.RemoveImageResponse, error) {
+	if image == "" {
+		return nil, fmt.Errorf("ID cannot be empty")
+	}
+	return client.RemoveImage(context.Background(), &pb.RemoveImageRequest{Image: &pb.ImageSpec{Image: image}})
+}

cmd/crioctl/info.go

@@ -0,0 +1,31 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+
+	"github.com/kubernetes-incubator/cri-o/client"
+	"github.com/urfave/cli"
+)
+
+var infoCommand = cli.Command{
+	Name:  "info",
+	Usage: "get crio daemon info",
+	Action: func(context *cli.Context) error {
+		c, err := client.New(context.GlobalString("connect"))
+		if err != nil {
+			return err
+		}
+		di, err := c.DaemonInfo()
+		if err != nil {
+			return err
+		}
+
+		jsonBytes, err := json.MarshalIndent(di, "", "    ")
+		if err != nil {
+			return err
+		}
+		fmt.Println(string(jsonBytes))
+		return nil
+	},
+}

cmd/crioctl/main.go

@@ -0,0 +1,113 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"net"
+	"os"
+	"strings"
+	"time"
+
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli"
+	"google.golang.org/grpc"
+	pb "k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+// This is populated by the Makefile from the VERSION file
+// in the repository
+var version = ""
+
+// gitCommit is the commit that the binary is being built from.
+// It will be populated by the Makefile.
+var gitCommit = ""
+
+func getClientConnection(context *cli.Context) (*grpc.ClientConn, error) {
+	conn, err := grpc.Dial(context.GlobalString("connect"), grpc.WithInsecure(), grpc.WithTimeout(context.GlobalDuration("timeout")),
+		grpc.WithDialer(func(addr string, timeout time.Duration) (net.Conn, error) {
+			return net.DialTimeout("unix", addr, timeout)
+		}))
+	if err != nil {
+		return nil, fmt.Errorf("failed to connect: %v", err)
+	}
+	return conn, nil
+}
+
+func openFile(path string) (*os.File, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, fmt.Errorf("config at %s not found", path)
+		}
+		return nil, err
+	}
+	return f, nil
+}
+
+func loadPodSandboxConfig(path string) (*pb.PodSandboxConfig, error) {
+	f, err := openFile(path)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	var config pb.PodSandboxConfig
+	if err := json.NewDecoder(f).Decode(&config); err != nil {
+		return nil, err
+	}
+	return &config, nil
+}
+
+func loadContainerConfig(path string) (*pb.ContainerConfig, error) {
+	f, err := openFile(path)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	var config pb.ContainerConfig
+	if err := json.NewDecoder(f).Decode(&config); err != nil {
+		return nil, err
+	}
+	return &config, nil
+}
+
+func main() {
+	app := cli.NewApp()
+	var v []string
+	if version != "" {
+		v = append(v, version)
+	}
+	if gitCommit != "" {
+		v = append(v, fmt.Sprintf("commit: %s", gitCommit))
+	}
+
+	app.Name = "crioctl"
+	app.Usage = "client for crio"
+	app.Version = strings.Join(v, "\n")
+
+	app.Commands = []cli.Command{
+		podSandboxCommand,
+		containerCommand,
+		runtimeVersionCommand,
+		imageCommand,
+		infoCommand,
+	}
+
+	app.Flags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "connect",
+			Value: "/var/run/crio.sock",
+			Usage: "Socket to connect to",
+		},
+		cli.DurationFlag{
+			Name:  "timeout",
+			Value: 10 * time.Second,
+			Usage: "Timeout of connecting to server",
+		},
+	}
+
+	if err := app.Run(os.Args); err != nil {
+		logrus.Fatal(err)
+	}
+}

cmd/crioctl/sandbox.go

@@ -0,0 +1,386 @@
+package main
+
+import (
+	"fmt"
+	"log"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+	pb "k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+var podSandboxCommand = cli.Command{
+	Name: "pod",
+	Subcommands: []cli.Command{
+		runPodSandboxCommand,
+		stopPodSandboxCommand,
+		removePodSandboxCommand,
+		podSandboxStatusCommand,
+		listPodSandboxCommand,
+	},
+}
+
+var runPodSandboxCommand = cli.Command{
+	Name:  "run",
+	Usage: "run a pod",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "config",
+			Value: "",
+			Usage: "the path of a pod sandbox config file",
+		},
+		cli.StringFlag{
+			Name:  "name",
+			Value: "",
+			Usage: "the name of the pod sandbox",
+		},
+		cli.StringSliceFlag{
+			Name:  "label",
+			Usage: "add key=value labels to the container",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		opts := createOptions{
+			configPath: context.String("config"),
+			name:       context.String("name"),
+			labels:     make(map[string]string),
+		}
+
+		for _, l := range context.StringSlice("label") {
+			pair := strings.Split(l, "=")
+			if len(pair) != 2 {
+				return fmt.Errorf("incorrectly specified label: %v", l)
+			}
+			opts.labels[pair[0]] = pair[1]
+		}
+
+		// Test RuntimeServiceClient.RunPodSandbox
+		err = RunPodSandbox(client, opts)
+		if err != nil {
+			return fmt.Errorf("Creating the pod sandbox failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var stopPodSandboxCommand = cli.Command{
+	Name:  "stop",
+	Usage: "stop a pod sandbox",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the pod sandbox",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = StopPodSandbox(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("stopping the pod sandbox failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var removePodSandboxCommand = cli.Command{
+	Name:  "remove",
+	Usage: "remove a pod sandbox",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the pod sandbox",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = RemovePodSandbox(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("removing the pod sandbox failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var podSandboxStatusCommand = cli.Command{
+	Name:  "status",
+	Usage: "return the status of a pod",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "id of the pod",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		err = PodSandboxStatus(client, context.String("id"))
+		if err != nil {
+			return fmt.Errorf("getting the pod sandbox status failed: %v", err)
+		}
+		return nil
+	},
+}
+
+var listPodSandboxCommand = cli.Command{
+	Name:  "list",
+	Usage: "list pod sandboxes",
+	Flags: []cli.Flag{
+		cli.StringFlag{
+			Name:  "id",
+			Value: "",
+			Usage: "filter by pod sandbox id",
+		},
+		cli.StringFlag{
+			Name:  "state",
+			Value: "",
+			Usage: "filter by pod sandbox state",
+		},
+		cli.StringSliceFlag{
+			Name:  "label",
+			Usage: "filter by key=value label",
+		},
+		cli.BoolFlag{
+			Name:  "quiet",
+			Usage: "list only pod IDs",
+		},
+	},
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		opts := listOptions{
+			id:     context.String("id"),
+			state:  context.String("state"),
+			quiet:  context.Bool("quiet"),
+			labels: make(map[string]string),
+		}
+
+		for _, l := range context.StringSlice("label") {
+			pair := strings.Split(l, "=")
+			if len(pair) != 2 {
+				return fmt.Errorf("incorrectly specified label: %v", l)
+			}
+			opts.labels[pair[0]] = pair[1]
+		}
+
+		err = ListPodSandboxes(client, opts)
+		if err != nil {
+			return fmt.Errorf("listing pod sandboxes failed: %v", err)
+		}
+		return nil
+	},
+}
+
+// RunPodSandbox sends a RunPodSandboxRequest to the server, and parses
+// the returned RunPodSandboxResponse.
+func RunPodSandbox(client pb.RuntimeServiceClient, opts createOptions) error {
+	config, err := loadPodSandboxConfig(opts.configPath)
+	if err != nil {
+		return err
+	}
+
+	// Override the name by the one specified through CLI
+	if opts.name != "" {
+		config.Metadata.Name = opts.name
+	}
+
+	for k, v := range opts.labels {
+		config.Labels[k] = v
+	}
+
+	r, err := client.RunPodSandbox(context.Background(), &pb.RunPodSandboxRequest{Config: config})
+	if err != nil {
+		return err
+	}
+	fmt.Println(r.PodSandboxId)
+	return nil
+}
+
+// StopPodSandbox sends a StopPodSandboxRequest to the server, and parses
+// the returned StopPodSandboxResponse.
+func StopPodSandbox(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	_, err := client.StopPodSandbox(context.Background(), &pb.StopPodSandboxRequest{PodSandboxId: ID})
+	if err != nil {
+		return err
+	}
+	fmt.Println(ID)
+	return nil
+}
+
+// RemovePodSandbox sends a RemovePodSandboxRequest to the server, and parses
+// the returned RemovePodSandboxResponse.
+func RemovePodSandbox(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	_, err := client.RemovePodSandbox(context.Background(), &pb.RemovePodSandboxRequest{PodSandboxId: ID})
+	if err != nil {
+		return err
+	}
+	fmt.Println(ID)
+	return nil
+}
+
+// PodSandboxStatus sends a PodSandboxStatusRequest to the server, and parses
+// the returned PodSandboxStatusResponse.
+func PodSandboxStatus(client pb.RuntimeServiceClient, ID string) error {
+	if ID == "" {
+		return fmt.Errorf("ID cannot be empty")
+	}
+	r, err := client.PodSandboxStatus(context.Background(), &pb.PodSandboxStatusRequest{PodSandboxId: ID})
+	if err != nil {
+		return err
+	}
+	fmt.Printf("ID: %s\n", r.Status.Id)
+	if r.Status.Metadata != nil {
+		if r.Status.Metadata.Name != "" {
+			fmt.Printf("Name: %s\n", r.Status.Metadata.Name)
+		}
+		if r.Status.Metadata.Uid != "" {
+			fmt.Printf("UID: %s\n", r.Status.Metadata.Uid)
+		}
+		if r.Status.Metadata.Namespace != "" {
+			fmt.Printf("Namespace: %s\n", r.Status.Metadata.Namespace)
+		}
+		fmt.Printf("Attempt: %v\n", r.Status.Metadata.Attempt)
+	}
+	fmt.Printf("Status: %s\n", r.Status.State)
+	ctm := time.Unix(0, r.Status.CreatedAt)
+	fmt.Printf("Created: %v\n", ctm)
+	if r.Status.Network != nil {
+		fmt.Printf("IP Address: %v\n", r.Status.Network.Ip)
+	}
+	if r.Status.Labels != nil {
+		fmt.Println("Labels:")
+		for _, k := range getSortedKeys(r.Status.Labels) {
+			fmt.Printf("\t%s -> %s\n", k, r.Status.Labels[k])
+		}
+	}
+	if r.Status.Annotations != nil {
+		fmt.Println("Annotations:")
+		for _, k := range getSortedKeys(r.Status.Annotations) {
+			fmt.Printf("\t%s -> %s\n", k, r.Status.Annotations[k])
+		}
+	}
+	return nil
+}
+
+// ListPodSandboxes sends a ListPodSandboxRequest to the server, and parses
+// the returned ListPodSandboxResponse.
+func ListPodSandboxes(client pb.RuntimeServiceClient, opts listOptions) error {
+	filter := &pb.PodSandboxFilter{}
+	if opts.id != "" {
+		filter.Id = opts.id
+	}
+	if opts.state != "" {
+		st := &pb.PodSandboxStateValue{}
+		st.State = pb.PodSandboxState_SANDBOX_NOTREADY
+		switch opts.state {
+		case "ready":
+			st.State = pb.PodSandboxState_SANDBOX_READY
+			filter.State = st
+		case "notready":
+			st.State = pb.PodSandboxState_SANDBOX_NOTREADY
+			filter.State = st
+		default:
+			log.Fatalf("--state should be ready or notready")
+		}
+	}
+	if opts.labels != nil {
+		filter.LabelSelector = opts.labels
+	}
+	r, err := client.ListPodSandbox(context.Background(), &pb.ListPodSandboxRequest{
+		Filter: filter,
+	})
+	if err != nil {
+		return err
+	}
+	for _, pod := range r.Items {
+		if opts.quiet {
+			fmt.Println(pod.Id)
+			continue
+		}
+		fmt.Printf("ID: %s\n", pod.Id)
+		if pod.Metadata != nil {
+			if pod.Metadata.Name != "" {
+				fmt.Printf("Name: %s\n", pod.Metadata.Name)
+			}
+			if pod.Metadata.Uid != "" {
+				fmt.Printf("UID: %s\n", pod.Metadata.Uid)
+			}
+			if pod.Metadata.Namespace != "" {
+				fmt.Printf("Namespace: %s\n", pod.Metadata.Namespace)
+			}
+			fmt.Printf("Attempt: %v\n", pod.Metadata.Attempt)
+		}
+		fmt.Printf("Status: %s\n", pod.State)
+		ctm := time.Unix(0, pod.CreatedAt)
+		fmt.Printf("Created: %v\n", ctm)
+		if pod.Labels != nil {
+			fmt.Println("Labels:")
+			for _, k := range getSortedKeys(pod.Labels) {
+				fmt.Printf("\t%s -> %s\n", k, pod.Labels[k])
+			}
+		}
+		if pod.Annotations != nil {
+			fmt.Println("Annotations:")
+			for _, k := range getSortedKeys(pod.Annotations) {
+				fmt.Printf("\t%s -> %s\n", k, pod.Annotations[k])
+			}
+		}
+		fmt.Println()
+	}
+	return nil
+}
+
+func getSortedKeys(m map[string]string) []string {
+	var keys []string
+	for k := range m {
+		keys = append(keys, k)
+	}
+	sort.Strings(keys)
+
+	return keys
+}

cmd/crioctl/system.go

@@ -0,0 +1,41 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+	pb "k8s.io/kubernetes/pkg/kubelet/apis/cri/v1alpha1/runtime"
+)
+
+var runtimeVersionCommand = cli.Command{
+	Name:  "runtimeversion",
+	Usage: "get runtime version information",
+	Action: func(context *cli.Context) error {
+		// Set up a connection to the server.
+		conn, err := getClientConnection(context)
+		if err != nil {
+			return fmt.Errorf("failed to connect: %v", err)
+		}
+		defer conn.Close()
+		client := pb.NewRuntimeServiceClient(conn)
+
+		// Test RuntimeServiceClient.Version
+		version := "v1alpha1"
+		err = Version(client, version)
+		if err != nil {
+			return fmt.Errorf("Getting the runtime version failed: %v", err)
+		}
+		return nil
+	},
+}
+
+// Version sends a VersionRequest to the server, and parses the returned VersionResponse.
+func Version(client pb.RuntimeServiceClient, version string) error {
+	r, err := client.Version(context.Background(), &pb.VersionRequest{Version: version})
+	if err != nil {
+		return err
+	}
+	fmt.Printf("VersionResponse: Version: %s, RuntimeName: %s, RuntimeVersion: %s, RuntimeApiVersion: %s\n", r.Version, r.RuntimeName, r.RuntimeVersion, r.RuntimeApiVersion)
+	return nil
+}

cmd/kpod/README.md

@@ -0,0 +1,16 @@
+# kpod - Simple debugging tool for pods and images
+kpod is a simple client only tool to help with debugging issues when daemons such as CRI runtime and the kubelet are not responding or
+failing. A shared API layer could be created to share code between the daemon and kpod. kpod does not require any daemon running. kpod
+utilizes the same underlying components that crio uses i.e. containers/image, container/storage, oci-runtime-tool/generate, runc or
+any other OCI compatible runtime. kpod shares state with crio and so has the capability to debug pods/images created by crio.
+
+## Use cases
+1. List pods.
+2. Launch simple pods (that require no daemon support).
+3. Exec commands in a container in a pod.
+4. Launch additional containers in a pod.
+5. List images.
+6. Remove images not in use.
+7. Pull images.
+8. Check image size.
+9. Report pod disk resource usage.

cmd/kpod/common.go

@@ -0,0 +1,135 @@
+package main
+
+import (
+	"os"
+	"reflect"
+	"regexp"
+	"strings"
+
+	is "github.com/containers/image/storage"
+	"github.com/containers/storage"
+	"github.com/fatih/camelcase"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/kubernetes-incubator/cri-o/server"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	stores = make(map[storage.Store]struct{})
+)
+
+func getStore(c *libkpod.Config) (storage.Store, error) {
+	options := storage.DefaultStoreOptions
+	options.GraphRoot = c.Root
+	options.RunRoot = c.RunRoot
+	options.GraphDriverName = c.Storage
+	options.GraphDriverOptions = c.StorageOptions
+
+	store, err := storage.GetStore(options)
+	if err != nil {
+		return nil, err
+	}
+	is.Transport.SetStore(store)
+	stores[store] = struct{}{}
+	return store, nil
+}
+
+func getRuntime(c *cli.Context) (*libpod.Runtime, error) {
+
+	config, err := getConfig(c)
+	if err != nil {
+		return nil, errors.Wrapf(err, "could not get config")
+	}
+
+	options := storage.DefaultStoreOptions
+	options.GraphRoot = config.Root
+	options.RunRoot = config.RunRoot
+	options.GraphDriverName = config.Storage
+	options.GraphDriverOptions = config.StorageOptions
+
+	return libpod.NewRuntime(libpod.WithStorageConfig(options))
+}
+
+func shutdownStores() {
+	for store := range stores {
+		if _, err := store.Shutdown(false); err != nil {
+			break
+		}
+	}
+}
+
+func getConfig(c *cli.Context) (*libkpod.Config, error) {
+	config := libkpod.DefaultConfig()
+	var configFile string
+	if c.GlobalIsSet("config") {
+		configFile = c.GlobalString("config")
+	} else if _, err := os.Stat(server.CrioConfigPath); err == nil {
+		configFile = server.CrioConfigPath
+	}
+	// load and merge the configfile from the commandline or use
+	// the default crio config file
+	if configFile != "" {
+		err := config.UpdateFromFile(configFile)
+		if err != nil {
+			return config, err
+		}
+	}
+	if c.GlobalIsSet("root") {
+		config.Root = c.GlobalString("root")
+	}
+	if c.GlobalIsSet("runroot") {
+		config.RunRoot = c.GlobalString("runroot")
+	}
+
+	if c.GlobalIsSet("storage-driver") {
+		config.Storage = c.GlobalString("storage-driver")
+	}
+	if c.GlobalIsSet("storage-opt") {
+		opts := c.GlobalStringSlice("storage-opt")
+		if len(opts) > 0 {
+			config.StorageOptions = opts
+		}
+	}
+	if c.GlobalIsSet("runtime") {
+		config.Runtime = c.GlobalString("runtime")
+	}
+	return config, nil
+}
+
+func splitCamelCase(src string) string {
+	entries := camelcase.Split(src)
+	return strings.Join(entries, " ")
+}
+
+// validateFlags searches for StringFlags or StringSlice flags that never had
+// a value set.  This commonly occurs when the CLI mistakenly takes the next
+// option and uses it as a value.
+func validateFlags(c *cli.Context, flags []cli.Flag) error {
+	for _, flag := range flags {
+		switch reflect.TypeOf(flag).String() {
+		case "cli.StringSliceFlag":
+			{
+				f := flag.(cli.StringSliceFlag)
+				name := strings.Split(f.Name, ",")
+				val := c.StringSlice(name[0])
+				for _, v := range val {
+					if ok, _ := regexp.MatchString("^-.+", v); ok {
+						return errors.Errorf("option --%s requires a value", name[0])
+					}
+				}
+			}
+		case "cli.StringFlag":
+			{
+				f := flag.(cli.StringFlag)
+				name := strings.Split(f.Name, ",")
+				val := c.String(name[0])
+				if ok, _ := regexp.MatchString("^-.+", val); ok {
+					return errors.Errorf("option --%s requires a value", name[0])
+				}
+			}
+		}
+	}
+	return nil
+}

cmd/kpod/common_test.go

@@ -0,0 +1,51 @@
+package main
+
+import (
+	"os/exec"
+	"os/user"
+	"testing"
+
+	"flag"
+
+	"github.com/urfave/cli"
+)
+
+func TestGetStore(t *testing.T) {
+	t.Skip("FIX THIS!")
+
+	//cmd/kpod/common_test.go:27: cannot use c (type *cli.Context) as type *libkpod.Config in argument to getStore
+
+	// Make sure the tests are running as root
+	skipTestIfNotRoot(t)
+
+	set := flag.NewFlagSet("test", 0)
+	globalSet := flag.NewFlagSet("test", 0)
+	globalSet.String("root", "", "path to the root directory in which data, including images,  is stored")
+	globalCtx := cli.NewContext(nil, globalSet, nil)
+	command := cli.Command{Name: "imagesCommand"}
+	c := cli.NewContext(nil, set, globalCtx)
+	c.Command = command
+
+	//_, err := getStore(c)
+	//if err != nil {
+	//t.Error(err)
+	//}
+}
+
+func skipTestIfNotRoot(t *testing.T) {
+	u, err := user.Current()
+	if err != nil {
+		t.Skip("Could not determine user.  Running without root may cause tests to fail")
+	} else if u.Uid != "0" {
+		t.Skip("tests will fail unless run as root")
+	}
+}
+
+func pullTestImage(name string) error {
+	cmd := exec.Command("crioctl", "image", "pull", name)
+	err := cmd.Run()
+	if err != nil {
+		return err
+	}
+	return nil
+}

cmd/kpod/diff.go

@@ -0,0 +1,128 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/containers/storage/pkg/archive"
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+type diffJSONOutput struct {
+	Changed []string `json:"changed,omitempty"`
+	Added   []string `json:"added,omitempty"`
+	Deleted []string `json:"deleted,omitempty"`
+}
+
+type diffOutputParams struct {
+	Change archive.ChangeType
+	Path   string
+}
+
+type stdoutStruct struct {
+	output []diffOutputParams
+}
+
+func (so stdoutStruct) Out() error {
+	for _, d := range so.output {
+		fmt.Printf("%s %s\n", d.Change, d.Path)
+	}
+	return nil
+}
+
+var (
+	diffFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:   "archive",
+			Usage:  "Save the diff as a tar archive",
+			Hidden: true,
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Change the output format.",
+		},
+	}
+	diffDescription = fmt.Sprint(`Displays changes on a container or image's filesystem.  The
+	container or image will be compared to its parent layer`)
+
+	diffCommand = cli.Command{
+		Name:        "diff",
+		Usage:       "Inspect changes on container's file systems",
+		Description: diffDescription,
+		Flags:       diffFlags,
+		Action:      diffCmd,
+		ArgsUsage:   "ID-NAME",
+	}
+)
+
+func formatJSON(output []diffOutputParams) (diffJSONOutput, error) {
+	jsonStruct := diffJSONOutput{}
+	for _, output := range output {
+		switch output.Change {
+		case archive.ChangeModify:
+			jsonStruct.Changed = append(jsonStruct.Changed, output.Path)
+		case archive.ChangeAdd:
+			jsonStruct.Added = append(jsonStruct.Added, output.Path)
+		case archive.ChangeDelete:
+			jsonStruct.Deleted = append(jsonStruct.Deleted, output.Path)
+		default:
+			return jsonStruct, errors.Errorf("output kind %q not recognized", output.Change.String())
+		}
+	}
+	return jsonStruct, nil
+}
+
+func diffCmd(c *cli.Context) error {
+	if err := validateFlags(c, diffFlags); err != nil {
+		return err
+	}
+
+	if len(c.Args()) != 1 {
+		return errors.Errorf("container, image, or layer name must be specified: kpod diff [options [...]] ID-NAME")
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	to := c.Args().Get(0)
+	changes, err := runtime.GetDiff("", to)
+	if err != nil {
+		return errors.Wrapf(err, "could not get changes for %q", to)
+	}
+
+	diffOutput := []diffOutputParams{}
+	outputFormat := c.String("format")
+
+	for _, change := range changes {
+
+		params := diffOutputParams{
+			Change: change.Kind,
+			Path:   change.Path,
+		}
+		diffOutput = append(diffOutput, params)
+	}
+
+	var out formats.Writer
+
+	if outputFormat != "" {
+		switch outputFormat {
+		case formats.JSONString:
+			data, err := formatJSON(diffOutput)
+			if err != nil {
+				return err
+			}
+			out = formats.JSONStruct{Output: data}
+		default:
+			return errors.New("only valid format for diff is 'json'")
+		}
+	} else {
+		out = stdoutStruct{output: diffOutput}
+	}
+	formats.Writer(out).Out()
+
+	return nil
+}

cmd/kpod/docker/types.go

@@ -0,0 +1,271 @@
+package docker
+
+//
+//  Types extracted from Docker
+//
+
+import (
+	"time"
+
+	"github.com/containers/image/pkg/strslice"
+	"github.com/opencontainers/go-digest"
+)
+
+// TypeLayers github.com/docker/docker/image/rootfs.go
+const TypeLayers = "layers"
+
+// V2S2MediaTypeManifest github.com/docker/distribution/manifest/schema2/manifest.go
+const V2S2MediaTypeManifest = "application/vnd.docker.distribution.manifest.v2+json"
+
+// V2S2MediaTypeImageConfig github.com/docker/distribution/manifest/schema2/manifest.go
+const V2S2MediaTypeImageConfig = "application/vnd.docker.container.image.v1+json"
+
+// V2S2MediaTypeLayer github.com/docker/distribution/manifest/schema2/manifest.go
+const V2S2MediaTypeLayer = "application/vnd.docker.image.rootfs.diff.tar.gzip"
+
+// V2S2MediaTypeUncompressedLayer github.com/docker/distribution/manifest/schema2/manifest.go
+const V2S2MediaTypeUncompressedLayer = "application/vnd.docker.image.rootfs.diff.tar"
+
+// V2S2RootFS describes images root filesystem
+// This is currently a placeholder that only supports layers. In the future
+// this can be made into an interface that supports different implementations.
+// github.com/docker/docker/image/rootfs.go
+type V2S2RootFS struct {
+	Type    string          `json:"type"`
+	DiffIDs []digest.Digest `json:"diff_ids,omitempty"`
+}
+
+// V2S2History stores build commands that were used to create an image
+// github.com/docker/docker/image/image.go
+type V2S2History struct {
+	// Created is the timestamp at which the image was created
+	Created time.Time `json:"created"`
+	// Author is the name of the author that was specified when committing the image
+	Author string `json:"author,omitempty"`
+	// CreatedBy keeps the Dockerfile command used while building the image
+	CreatedBy string `json:"created_by,omitempty"`
+	// Comment is the commit message that was set when committing the image
+	Comment string `json:"comment,omitempty"`
+	// EmptyLayer is set to true if this history item did not generate a
+	// layer. Otherwise, the history item is associated with the next
+	// layer in the RootFS section.
+	EmptyLayer bool `json:"empty_layer,omitempty"`
+}
+
+// ID is the content-addressable ID of an image.
+// github.com/docker/docker/image/image.go
+type ID digest.Digest
+
+// HealthConfig holds configuration settings for the HEALTHCHECK feature.
+// github.com/docker/docker/api/types/container/config.go
+type HealthConfig struct {
+	// Test is the test to perform to check that the container is healthy.
+	// An empty slice means to inherit the default.
+	// The options are:
+	// {} : inherit healthcheck
+	// {"NONE"} : disable healthcheck
+	// {"CMD", args...} : exec arguments directly
+	// {"CMD-SHELL", command} : run command with system's default shell
+	Test []string `json:",omitempty"`
+
+	// Zero means to inherit. Durations are expressed as integer nanoseconds.
+	Interval time.Duration `json:",omitempty"` // Interval is the time to wait between checks.
+	Timeout  time.Duration `json:",omitempty"` // Timeout is the time to wait before considering the check to have hung.
+
+	// Retries is the number of consecutive failures needed to consider a container as unhealthy.
+	// Zero means inherit.
+	Retries int `json:",omitempty"`
+}
+
+// PortSet is a collection of structs indexed by Port
+// github.com/docker/go-connections/nat/nat.go
+type PortSet map[Port]struct{}
+
+// Port is a string containing port number and protocol in the format "80/tcp"
+// github.com/docker/go-connections/nat/nat.go
+type Port string
+
+// Config contains the configuration data about a container.
+// It should hold only portable information about the container.
+// Here, "portable" means "independent from the host we are running on".
+// Non-portable information *should* appear in HostConfig.
+// All fields added to this struct must be marked `omitempty` to keep getting
+// predictable hashes from the old `v1Compatibility` configuration.
+// github.com/docker/docker/api/types/container/config.go
+type Config struct {
+	Hostname        string              // Hostname
+	Domainname      string              // Domainname
+	User            string              // User that will run the command(s) inside the container, also support user:group
+	AttachStdin     bool                // Attach the standard input, makes possible user interaction
+	AttachStdout    bool                // Attach the standard output
+	AttachStderr    bool                // Attach the standard error
+	ExposedPorts    PortSet             `json:",omitempty"` // List of exposed ports
+	Tty             bool                // Attach standard streams to a tty, including stdin if it is not closed.
+	OpenStdin       bool                // Open stdin
+	StdinOnce       bool                // If true, close stdin after the 1 attached client disconnects.
+	Env             []string            // List of environment variable to set in the container
+	Cmd             strslice.StrSlice   // Command to run when starting the container
+	Healthcheck     *HealthConfig       `json:",omitempty"` // Healthcheck describes how to check the container is healthy
+	ArgsEscaped     bool                `json:",omitempty"` // True if command is already escaped (Windows specific)
+	Image           string              // Name of the image as it was passed by the operator (e.g. could be symbolic)
+	Volumes         map[string]struct{} // List of volumes (mounts) used for the container
+	WorkingDir      string              // Current directory (PWD) in the command will be launched
+	Entrypoint      strslice.StrSlice   // Entrypoint to run when starting the container
+	NetworkDisabled bool                `json:",omitempty"` // Is network disabled
+	MacAddress      string              `json:",omitempty"` // Mac Address of the container
+	OnBuild         []string            // ONBUILD metadata that were defined on the image Dockerfile
+	Labels          map[string]string   // List of labels set to this container
+	StopSignal      string              `json:",omitempty"` // Signal to stop a container
+	StopTimeout     *int                `json:",omitempty"` // Timeout (in seconds) to stop a container
+	Shell           strslice.StrSlice   `json:",omitempty"` // Shell for shell-form of RUN, CMD, ENTRYPOINT
+}
+
+// V1Compatibility - For non-top-level layers, create fake V1Compatibility
+// strings that fit the format and don't collide with anything else, but
+// don't result in runnable images on their own.
+// github.com/docker/distribution/manifest/schema1/config_builder.go
+type V1Compatibility struct {
+	ID              string    `json:"id"`
+	Parent          string    `json:"parent,omitempty"`
+	Comment         string    `json:"comment,omitempty"`
+	Created         time.Time `json:"created"`
+	ContainerConfig struct {
+		Cmd []string
+	} `json:"container_config,omitempty"`
+	Author    string `json:"author,omitempty"`
+	ThrowAway bool   `json:"throwaway,omitempty"`
+}
+
+// V1Image stores the V1 image configuration.
+// github.com/docker/docker/image/image.go
+type V1Image struct {
+	// ID is a unique 64 character identifier of the image
+	ID string `json:"id,omitempty"`
+	// Parent is the ID of the parent image
+	Parent string `json:"parent,omitempty"`
+	// Comment is the commit message that was set when committing the image
+	Comment string `json:"comment,omitempty"`
+	// Created is the timestamp at which the image was created
+	Created time.Time `json:"created"`
+	// Container is the id of the container used to commit
+	Container string `json:"container,omitempty"`
+	// ContainerConfig is the configuration of the container that is committed into the image
+	ContainerConfig Config `json:"container_config,omitempty"`
+	// DockerVersion specifies the version of Docker that was used to build the image
+	DockerVersion string `json:"docker_version,omitempty"`
+	// Author is the name of the author that was specified when committing the image
+	Author string `json:"author,omitempty"`
+	// Config is the configuration of the container received from the client
+	Config *Config `json:"config,omitempty"`
+	// Architecture is the hardware that the image is build and runs on
+	Architecture string `json:"architecture,omitempty"`
+	// OS is the operating system used to build and run the image
+	OS string `json:"os,omitempty"`
+	// Size is the total size of the image including all layers it is composed of
+	Size int64 `json:",omitempty"`
+}
+
+// V2Image stores the image configuration
+// github.com/docker/docker/image/image.go
+type V2Image struct {
+	V1Image
+	Parent     ID            `json:"parent,omitempty"`
+	RootFS     *V2S2RootFS   `json:"rootfs,omitempty"`
+	History    []V2S2History `json:"history,omitempty"`
+	OSVersion  string        `json:"os.version,omitempty"`
+	OSFeatures []string      `json:"os.features,omitempty"`
+
+	// rawJSON caches the immutable JSON associated with this image.
+	//rawJSON []byte
+
+	// computedID is the ID computed from the hash of the image config.
+	// Not to be confused with the legacy V1 ID in V1Image.
+	//computedID ID
+}
+
+// V2Versioned provides a struct with the manifest schemaVersion and mediaType.
+// Incoming content with unknown schema version can be decoded against this
+// struct to check the version.
+// github.com/docker/distribution/manifest/versioned.go
+type V2Versioned struct {
+	// SchemaVersion is the image manifest schema that this image follows
+	SchemaVersion int `json:"schemaVersion"`
+
+	// MediaType is the media type of this schema.
+	MediaType string `json:"mediaType,omitempty"`
+}
+
+// V2S1FSLayer is a container struct for BlobSums defined in an image manifest
+// github.com/docker/distribution/manifest/schema1/manifest.go
+type V2S1FSLayer struct {
+	// BlobSum is the tarsum of the referenced filesystem image layer
+	BlobSum digest.Digest `json:"blobSum"`
+}
+
+// V2S1History stores unstructured v1 compatibility information
+// github.com/docker/distribution/manifest/schema1/manifest.go
+type V2S1History struct {
+	// V1Compatibility is the raw v1 compatibility information
+	V1Compatibility string `json:"v1Compatibility"`
+}
+
+// V2S1Manifest provides the base accessible fields for working with V2 image
+// format in the registry.
+// github.com/docker/distribution/manifest/schema1/manifest.go
+type V2S1Manifest struct {
+	V2Versioned
+
+	// Name is the name of the image's repository
+	Name string `json:"name"`
+
+	// Tag is the tag of the image specified by this manifest
+	Tag string `json:"tag"`
+
+	// Architecture is the host architecture on which this image is intended to
+	// run
+	Architecture string `json:"architecture"`
+
+	// FSLayers is a list of filesystem layer blobSums contained in this image
+	FSLayers []V2S1FSLayer `json:"fsLayers"`
+
+	// History is a list of unstructured historical data for v1 compatibility
+	History []V2S1History `json:"history"`
+}
+
+// V2S2Descriptor describes targeted content. Used in conjunction with a blob
+// store, a descriptor can be used to fetch, store and target any kind of
+// blob. The struct also describes the wire protocol format. Fields should
+// only be added but never changed.
+// github.com/docker/distribution/blobs.go
+type V2S2Descriptor struct {
+	// MediaType describe the type of the content. All text based formats are
+	// encoded as utf-8.
+	MediaType string `json:"mediaType,omitempty"`
+
+	// Size in bytes of content.
+	Size int64 `json:"size,omitempty"`
+
+	// Digest uniquely identifies the content. A byte stream can be verified
+	// against against this digest.
+	Digest digest.Digest `json:"digest,omitempty"`
+
+	// URLs contains the source URLs of this content.
+	URLs []string `json:"urls,omitempty"`
+
+	// NOTE: Before adding a field here, please ensure that all
+	// other options have been exhausted. Much of the type relationships
+	// depend on the simplicity of this type.
+}
+
+// V2S2Manifest defines a schema2 manifest.
+// github.com/docker/distribution/manifest/schema2/manifest.go
+type V2S2Manifest struct {
+	V2Versioned
+
+	// Config references the image configuration as a blob.
+	Config V2S2Descriptor `json:"config"`
+
+	// Layers lists descriptors for the layers referenced by the
+	// configuration.
+	Layers []V2S2Descriptor `json:"layers"`
+}

cmd/kpod/export.go

@@ -0,0 +1,106 @@
+package main
+
+import (
+	"io"
+	"os"
+
+	"fmt"
+
+	"github.com/containers/storage"
+	"github.com/containers/storage/pkg/archive"
+	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli"
+)
+
+type exportOptions struct {
+	output    string
+	container string
+}
+
+var (
+	exportFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "output, o",
+			Usage: "Write to a file, default is STDOUT",
+			Value: "/dev/stdout",
+		},
+	}
+	exportDescription = "Exports container's filesystem contents as a tar archive" +
+		" and saves it on the local machine."
+	exportCommand = cli.Command{
+		Name:        "export",
+		Usage:       "Export container's filesystem contents as a tar archive",
+		Description: exportDescription,
+		Flags:       exportFlags,
+		Action:      exportCmd,
+		ArgsUsage:   "CONTAINER",
+	}
+)
+
+// exportCmd saves a container to a tarball on disk
+func exportCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("container id must be specified")
+	}
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments given, need 1 at most.")
+	}
+	container := args[0]
+	if err := validateFlags(c, exportFlags); err != nil {
+		return err
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	store, err := getStore(config)
+	if err != nil {
+		return err
+	}
+
+	output := c.String("output")
+	if output == "/dev/stdout" {
+		file := os.Stdout
+		if logrus.IsTerminal(file) {
+			return errors.Errorf("refusing to export to terminal. Use -o flag or redirect")
+		}
+	}
+
+	opts := exportOptions{
+		output:    output,
+		container: container,
+	}
+
+	return exportContainer(store, opts)
+}
+
+// exportContainer exports the contents of a container and saves it as
+// a tarball on disk
+func exportContainer(store storage.Store, opts exportOptions) error {
+	mountPoint, err := store.Mount(opts.container, "")
+	if err != nil {
+		return errors.Wrapf(err, "error finding container %q", opts.container)
+	}
+	defer func() {
+		if err := store.Unmount(opts.container); err != nil {
+			fmt.Printf("error unmounting container %q: %v\n", opts.container, err)
+		}
+	}()
+
+	input, err := archive.Tar(mountPoint, archive.Uncompressed)
+	if err != nil {
+		return errors.Wrapf(err, "error reading container directory %q", opts.container)
+	}
+
+	outFile, err := os.Create(opts.output)
+	if err != nil {
+		return errors.Wrapf(err, "error creating file %q", opts.output)
+	}
+	defer outFile.Close()
+
+	_, err = io.Copy(outFile, input)
+	return err
+}

cmd/kpod/formats/formats.go

@@ -0,0 +1,143 @@
+package formats
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"strings"
+	"text/tabwriter"
+	"text/template"
+
+	"bytes"
+	"github.com/ghodss/yaml"
+	"github.com/pkg/errors"
+)
+
+const (
+	// JSONString const to save on duplicate variable names
+	JSONString = "json"
+	// IDString const to save on duplicates for Go templates
+	IDString = "{{.ID}}"
+)
+
+// Writer interface for outputs
+type Writer interface {
+	Out() error
+}
+
+// JSONStructArray for JSON output
+type JSONStructArray struct {
+	Output []interface{}
+}
+
+// StdoutTemplateArray for Go template output
+type StdoutTemplateArray struct {
+	Output   []interface{}
+	Template string
+	Fields   map[string]string
+}
+
+// JSONStruct for JSON output
+type JSONStruct struct {
+	Output interface{}
+}
+
+// StdoutTemplate for Go template output
+type StdoutTemplate struct {
+	Output   interface{}
+	Template string
+	Fields   map[string]string
+}
+
+// YAMLStruct for YAML output
+type YAMLStruct struct {
+	Output interface{}
+}
+
+// Out method for JSON Arrays
+func (j JSONStructArray) Out() error {
+	data, err := json.MarshalIndent(j.Output, "", "    ")
+	if err != nil {
+		return err
+	}
+
+	// JSON returns a byte array with a literal null [110 117 108 108] in it
+	// if it is passed empty data.  We used bytes.Compare to see if that is
+	// the case.
+	if diff := bytes.Compare(data, []byte("null")); diff == 0 {
+		data = []byte("[]")
+	}
+
+	// If the we did get NULL back, we should spit out {} which is
+	// at least valid JSON for the consumer.
+	fmt.Printf("%s\n", data)
+	return nil
+}
+
+// Out method for Go templates
+func (t StdoutTemplateArray) Out() error {
+	w := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
+	if strings.HasPrefix(t.Template, "table") {
+		// replace any spaces with tabs in template so that tabwriter can align it
+		t.Template = strings.Replace(strings.TrimSpace(t.Template[5:]), " ", "\t", -1)
+		headerTmpl, err := template.New("header").Funcs(headerFunctions).Parse(t.Template)
+		if err != nil {
+			return errors.Wrapf(err, "Template parsing error")
+		}
+		err = headerTmpl.Execute(w, t.Fields)
+		if err != nil {
+			return err
+		}
+		fmt.Fprintln(w, "")
+	}
+	t.Template = strings.Replace(t.Template, " ", "\t", -1)
+	tmpl, err := template.New("image").Funcs(basicFunctions).Parse(t.Template)
+	if err != nil {
+		return errors.Wrapf(err, "Template parsing error")
+	}
+	for _, img := range t.Output {
+		basicTmpl := tmpl.Funcs(basicFunctions)
+		err = basicTmpl.Execute(w, img)
+		if err != nil {
+			return err
+		}
+		fmt.Fprintln(w, "")
+	}
+	return w.Flush()
+}
+
+// Out method for JSON struct
+func (j JSONStruct) Out() error {
+	data, err := json.MarshalIndent(j.Output, "", "    ")
+	if err != nil {
+		return err
+	}
+	fmt.Printf("%s\n", data)
+	return nil
+}
+
+//Out method for Go templates
+func (t StdoutTemplate) Out() error {
+	tmpl, err := template.New("image").Parse(t.Template)
+	if err != nil {
+		return errors.Wrapf(err, "template parsing error")
+	}
+	err = tmpl.Execute(os.Stdout, t.Output)
+	if err != nil {
+		return err
+	}
+	fmt.Println()
+	return nil
+}
+
+// Out method for YAML
+func (y YAMLStruct) Out() error {
+	var buf []byte
+	var err error
+	buf, err = yaml.Marshal(y.Output)
+	if err != nil {
+		return err
+	}
+	fmt.Println(string(buf))
+	return nil
+}

cmd/kpod/formats/templates.go

@@ -0,0 +1,78 @@
+package formats
+
+import (
+	"bytes"
+	"encoding/json"
+	"strings"
+	"text/template"
+)
+
+// basicFunctions are the set of initial
+// functions provided to every template.
+var basicFunctions = template.FuncMap{
+	"json": func(v interface{}) string {
+		buf := &bytes.Buffer{}
+		enc := json.NewEncoder(buf)
+		enc.SetEscapeHTML(false)
+		_ = enc.Encode(v)
+		// Remove the trailing new line added by the encoder
+		return strings.TrimSpace(buf.String())
+	},
+	"split":    strings.Split,
+	"join":     strings.Join,
+	"title":    strings.Title,
+	"lower":    strings.ToLower,
+	"upper":    strings.ToUpper,
+	"pad":      padWithSpace,
+	"truncate": truncateWithLength,
+}
+
+// HeaderFunctions are used to created headers of a table.
+// This is a replacement of basicFunctions for header generation
+// because we want the header to remain intact.
+// Some functions like `split` are irrelevant so not added.
+var headerFunctions = template.FuncMap{
+	"json": func(v string) string {
+		return v
+	},
+	"title": func(v string) string {
+		return v
+	},
+	"lower": func(v string) string {
+		return v
+	},
+	"upper": func(v string) string {
+		return v
+	},
+	"truncate": func(v string, l int) string {
+		return v
+	},
+}
+
+// Parse creates a new anonymous template with the basic functions
+// and parses the given format.
+func Parse(format string) (*template.Template, error) {
+	return NewParse("", format)
+}
+
+// NewParse creates a new tagged template with the basic functions
+// and parses the given format.
+func NewParse(tag, format string) (*template.Template, error) {
+	return template.New(tag).Funcs(basicFunctions).Parse(format)
+}
+
+// padWithSpace adds whitespace to the input if the input is non-empty
+func padWithSpace(source string, prefix, suffix int) string {
+	if source == "" {
+		return source
+	}
+	return strings.Repeat(" ", prefix) + source + strings.Repeat(" ", suffix)
+}
+
+// truncateWithLength truncates the source string up to the length provided by the input
+func truncateWithLength(source string, length int) string {
+	if len(source) < length {
+		return source
+	}
+	return source[:length]
+}

cmd/kpod/history.go

@@ -0,0 +1,243 @@
+package main
+
+import (
+	"reflect"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/containers/image/types"
+	units "github.com/docker/go-units"
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/opencontainers/image-spec/specs-go/v1"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+const (
+	createdByTruncLength = 45
+	idTruncLength        = 13
+)
+
+// historyTemplateParams stores info about each layer
+type historyTemplateParams struct {
+	ID        string
+	Created   string
+	CreatedBy string
+	Size      string
+	Comment   string
+}
+
+// historyJSONParams is only used when the JSON format is specified,
+// and is better for data processing from JSON.
+// historyJSONParams will be populated by data from v1.History and types.BlobInfo,
+// the members of the struct are the sama data types as their sources.
+type historyJSONParams struct {
+	ID        string     `json:"id"`
+	Created   *time.Time `json:"created"`
+	CreatedBy string     `json:"createdBy"`
+	Size      int64      `json:"size"`
+	Comment   string     `json:"comment"`
+}
+
+// historyOptions stores cli flag values
+type historyOptions struct {
+	human   bool
+	noTrunc bool
+	quiet   bool
+	format  string
+}
+
+var (
+	historyFlags = []cli.Flag{
+		cli.BoolTFlag{
+			Name:  "human, H",
+			Usage: "Display sizes and dates in human readable format",
+		},
+		cli.BoolFlag{
+			Name:  "no-trunc, notruncate",
+			Usage: "Do not truncate the output",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "Display the numeric IDs only",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Change the output to JSON or a Go template",
+		},
+	}
+
+	historyDescription = "Displays the history of an image. The information can be printed out in an easy to read, " +
+		"or user specified format, and can be truncated."
+	historyCommand = cli.Command{
+		Name:        "history",
+		Usage:       "Show history of a specified image",
+		Description: historyDescription,
+		Flags:       historyFlags,
+		Action:      historyCmd,
+		ArgsUsage:   "",
+	}
+)
+
+func historyCmd(c *cli.Context) error {
+	if err := validateFlags(c, historyFlags); err != nil {
+		return err
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get config")
+	}
+	defer runtime.Shutdown(false)
+
+	format := genHistoryFormat(c.Bool("quiet"))
+	if c.IsSet("format") {
+		format = c.String("format")
+	}
+
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("an image name must be specified")
+	}
+	if len(args) > 1 {
+		return errors.Errorf("Kpod history takes at most 1 argument")
+	}
+	imgName := args[0]
+
+	opts := historyOptions{
+		human:   c.BoolT("human"),
+		noTrunc: c.Bool("no-trunc"),
+		quiet:   c.Bool("quiet"),
+		format:  format,
+	}
+
+	history, layers, imageID, err := runtime.GetHistory(imgName)
+	if err != nil {
+		return errors.Wrapf(err, "error getting history of image %q", imgName)
+	}
+
+	return generateHistoryOutput(history, layers, imageID, opts)
+}
+
+func genHistoryFormat(quiet bool) (format string) {
+	if quiet {
+		return formats.IDString
+	}
+	return "table {{.ID}}\t{{.Created}}\t{{.CreatedBy}}\t{{.Size}}\t{{.Comment}}\t"
+}
+
+// historyToGeneric makes an empty array of interfaces for output
+func historyToGeneric(templParams []historyTemplateParams, JSONParams []historyJSONParams) (genericParams []interface{}) {
+	if len(templParams) > 0 {
+		for _, v := range templParams {
+			genericParams = append(genericParams, interface{}(v))
+		}
+		return
+	}
+	for _, v := range JSONParams {
+		genericParams = append(genericParams, interface{}(v))
+	}
+	return
+}
+
+// generate the header based on the template provided
+func (h *historyTemplateParams) headerMap() map[string]string {
+	v := reflect.Indirect(reflect.ValueOf(h))
+	values := make(map[string]string)
+	for h := 0; h < v.NumField(); h++ {
+		key := v.Type().Field(h).Name
+		value := key
+		values[key] = strings.ToUpper(splitCamelCase(value))
+	}
+	return values
+}
+
+// getHistorytemplateOutput gets the modified history information to be printed in human readable format
+func getHistoryTemplateOutput(history []v1.History, layers []types.BlobInfo, imageID string, opts historyOptions) (historyOutput []historyTemplateParams) {
+	var (
+		outputSize  string
+		createdTime string
+		createdBy   string
+		count       = 1
+	)
+	for i := len(history) - 1; i >= 0; i-- {
+		if i != len(history)-1 {
+			imageID = "<missing>"
+		}
+		if !opts.noTrunc && i == len(history)-1 {
+			imageID = imageID[:idTruncLength]
+		}
+
+		var size int64
+		if !history[i].EmptyLayer {
+			size = layers[len(layers)-count].Size
+			count++
+		}
+
+		if opts.human {
+			createdTime = units.HumanDuration(time.Since((*history[i].Created))) + " ago"
+			outputSize = units.HumanSize(float64(size))
+		} else {
+			createdTime = (history[i].Created).Format(time.RFC3339)
+			outputSize = strconv.FormatInt(size, 10)
+		}
+
+		createdBy = strings.Join(strings.Fields(history[i].CreatedBy), " ")
+		if !opts.noTrunc && len(createdBy) > createdByTruncLength {
+			createdBy = createdBy[:createdByTruncLength-3] + "..."
+		}
+
+		params := historyTemplateParams{
+			ID:        imageID,
+			Created:   createdTime,
+			CreatedBy: createdBy,
+			Size:      outputSize,
+			Comment:   history[i].Comment,
+		}
+		historyOutput = append(historyOutput, params)
+	}
+	return
+}
+
+// getHistoryJSONOutput returns the history information in its raw form
+func getHistoryJSONOutput(history []v1.History, layers []types.BlobInfo, imageID string) (historyOutput []historyJSONParams) {
+	count := 1
+	for i := len(history) - 1; i >= 0; i-- {
+		var size int64
+		if !history[i].EmptyLayer {
+			size = layers[len(layers)-count].Size
+			count++
+		}
+
+		params := historyJSONParams{
+			ID:        imageID,
+			Created:   history[i].Created,
+			CreatedBy: history[i].CreatedBy,
+			Size:      size,
+			Comment:   history[i].Comment,
+		}
+		historyOutput = append(historyOutput, params)
+	}
+	return
+}
+
+// generateHistoryOutput generates the history based on the format given
+func generateHistoryOutput(history []v1.History, layers []types.BlobInfo, imageID string, opts historyOptions) error {
+	if len(history) == 0 {
+		return nil
+	}
+
+	var out formats.Writer
+
+	switch opts.format {
+	case formats.JSONString:
+		historyOutput := getHistoryJSONOutput(history, layers, imageID)
+		out = formats.JSONStructArray{Output: historyToGeneric([]historyTemplateParams{}, historyOutput)}
+	default:
+		historyOutput := getHistoryTemplateOutput(history, layers, imageID, opts)
+		out = formats.StdoutTemplateArray{Output: historyToGeneric(historyOutput, []historyJSONParams{}), Template: opts.format, Fields: historyOutput[0].headerMap()}
+	}
+
+	return formats.Writer(out).Out()
+}

cmd/kpod/images.go

@@ -0,0 +1,330 @@
+package main
+
+import (
+	"fmt"
+	"reflect"
+	"strings"
+	"time"
+
+	"github.com/containers/image/types"
+	"github.com/containers/storage"
+	"github.com/docker/go-units"
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/kubernetes-incubator/cri-o/libpod/common"
+	digest "github.com/opencontainers/go-digest"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+type imagesTemplateParams struct {
+	ID        string
+	Name      string
+	Digest    digest.Digest
+	CreatedAt string
+	Size      string
+}
+
+type imagesJSONParams struct {
+	ID        string        `json:"id"`
+	Name      []string      `json:"names"`
+	Digest    digest.Digest `json:"digest"`
+	CreatedAt time.Time     `json:"created"`
+	Size      int64         `json:"size"`
+}
+
+type imagesOptions struct {
+	quiet     bool
+	noHeading bool
+	noTrunc   bool
+	digests   bool
+	format    string
+}
+
+var (
+	imagesFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "display only image IDs",
+		},
+		cli.BoolFlag{
+			Name:  "noheading, n",
+			Usage: "do not print column headings",
+		},
+		cli.BoolFlag{
+			Name:  "no-trunc, notruncate",
+			Usage: "do not truncate output",
+		},
+		cli.BoolFlag{
+			Name:  "digests",
+			Usage: "show digests",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Change the output format to JSON or a Go template",
+		},
+		cli.StringFlag{
+			Name:  "filter, f",
+			Usage: "filter output based on conditions provided (default [])",
+		},
+	}
+
+	imagesDescription = "lists locally stored images."
+	imagesCommand     = cli.Command{
+		Name:        "images",
+		Usage:       "list images in local storage",
+		Description: imagesDescription,
+		Flags:       imagesFlags,
+		Action:      imagesCmd,
+		ArgsUsage:   "",
+	}
+)
+
+func imagesCmd(c *cli.Context) error {
+	if err := validateFlags(c, imagesFlags); err != nil {
+		return err
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	var format string
+	if c.IsSet("format") {
+		format = c.String("format")
+	} else {
+		format = genImagesFormat(c.Bool("quiet"), c.Bool("noheading"), c.Bool("digests"))
+	}
+
+	opts := imagesOptions{
+		quiet:     c.Bool("quiet"),
+		noHeading: c.Bool("noheading"),
+		noTrunc:   c.Bool("no-trunc"),
+		digests:   c.Bool("digests"),
+		format:    format,
+	}
+
+	var imageInput string
+	if len(c.Args()) == 1 {
+		imageInput = c.Args().Get(0)
+	}
+	if len(c.Args()) > 1 {
+		return errors.New("'kpod images' requires at most 1 argument")
+	}
+
+	params, err := runtime.ParseImageFilter(imageInput, c.String("filter"))
+	if err != nil {
+		return errors.Wrapf(err, "error parsing filter")
+	}
+
+	// generate the different filters
+	labelFilter := generateImagesFilter(params, "label")
+	beforeImageFilter := generateImagesFilter(params, "before-image")
+	sinceImageFilter := generateImagesFilter(params, "since-image")
+	danglingFilter := generateImagesFilter(params, "dangling")
+	referenceFilter := generateImagesFilter(params, "reference")
+	imageInputFilter := generateImagesFilter(params, "image-input")
+
+	images, err := runtime.GetImages(params, labelFilter, beforeImageFilter, sinceImageFilter, danglingFilter, referenceFilter, imageInputFilter)
+	if err != nil {
+		return errors.Wrapf(err, "could not get list of images matching filter")
+	}
+
+	return generateImagesOutput(runtime, images, opts)
+}
+
+func genImagesFormat(quiet, noHeading, digests bool) (format string) {
+	if quiet {
+		return formats.IDString
+	}
+	format = "table {{.ID}}\t{{.Name}}\t"
+	if noHeading {
+		format = "{{.ID}}\t{{.Name}}\t"
+	}
+	if digests {
+		format += "{{.Digest}}\t"
+	}
+	format += "{{.CreatedAt}}\t{{.Size}}\t"
+	return
+}
+
+// imagesToGeneric creates an empty array of interfaces for output
+func imagesToGeneric(templParams []imagesTemplateParams, JSONParams []imagesJSONParams) (genericParams []interface{}) {
+	if len(templParams) > 0 {
+		for _, v := range templParams {
+			genericParams = append(genericParams, interface{}(v))
+		}
+		return
+	}
+	for _, v := range JSONParams {
+		genericParams = append(genericParams, interface{}(v))
+	}
+	return
+}
+
+// generate the header based on the template provided
+func (i *imagesTemplateParams) headerMap() map[string]string {
+	v := reflect.Indirect(reflect.ValueOf(i))
+	values := make(map[string]string)
+
+	for i := 0; i < v.NumField(); i++ {
+		key := v.Type().Field(i).Name
+		value := key
+		if value == "ID" || value == "Name" {
+			value = "Image" + value
+		}
+		values[key] = strings.ToUpper(splitCamelCase(value))
+	}
+	return values
+}
+
+// getImagesTemplateOutput returns the images information to be printed in human readable format
+func getImagesTemplateOutput(runtime *libpod.Runtime, images []*storage.Image, opts imagesOptions) (imagesOutput []imagesTemplateParams) {
+	var (
+		lastID string
+	)
+	for _, img := range images {
+		if opts.quiet && lastID == img.ID {
+			continue // quiet should not show the same ID multiple times
+		}
+		createdTime := img.Created
+
+		imageID := img.ID
+		if !opts.noTrunc {
+			imageID = imageID[:idTruncLength]
+		}
+
+		imageName := "<none>"
+		if len(img.Names) > 0 {
+			imageName = img.Names[0]
+		}
+
+		info, imageDigest, size, _ := runtime.InfoAndDigestAndSize(*img)
+		if info != nil {
+			createdTime = info.Created
+		}
+
+		params := imagesTemplateParams{
+			ID:        imageID,
+			Name:      imageName,
+			Digest:    imageDigest,
+			CreatedAt: units.HumanDuration(time.Since((createdTime))) + " ago",
+			Size:      units.HumanSize(float64(size)),
+		}
+		imagesOutput = append(imagesOutput, params)
+	}
+	return
+}
+
+// getImagesJSONOutput returns the images information in its raw form
+func getImagesJSONOutput(runtime *libpod.Runtime, images []*storage.Image) (imagesOutput []imagesJSONParams) {
+	for _, img := range images {
+		createdTime := img.Created
+
+		info, imageDigest, size, _ := runtime.InfoAndDigestAndSize(*img)
+		if info != nil {
+			createdTime = info.Created
+		}
+
+		params := imagesJSONParams{
+			ID:        img.ID,
+			Name:      img.Names,
+			Digest:    imageDigest,
+			CreatedAt: createdTime,
+			Size:      size,
+		}
+		imagesOutput = append(imagesOutput, params)
+	}
+	return
+}
+
+// generateImagesOutput generates the images based on the format provided
+func generateImagesOutput(runtime *libpod.Runtime, images []*storage.Image, opts imagesOptions) error {
+	if len(images) == 0 {
+		return nil
+	}
+
+	var out formats.Writer
+
+	switch opts.format {
+	case formats.JSONString:
+		imagesOutput := getImagesJSONOutput(runtime, images)
+		out = formats.JSONStructArray{Output: imagesToGeneric([]imagesTemplateParams{}, imagesOutput)}
+	default:
+		imagesOutput := getImagesTemplateOutput(runtime, images, opts)
+		out = formats.StdoutTemplateArray{Output: imagesToGeneric(imagesOutput, []imagesJSONParams{}), Template: opts.format, Fields: imagesOutput[0].headerMap()}
+
+	}
+
+	return formats.Writer(out).Out()
+}
+
+// generateImagesFilter returns an ImageFilter based on filterType
+// to add more filters, define a new case and write what the ImageFilter function should do
+func generateImagesFilter(params *libpod.ImageFilterParams, filterType string) libpod.ImageFilter {
+	switch filterType {
+	case "label":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.Label == "" {
+				return true
+			}
+
+			pair := strings.SplitN(params.Label, "=", 2)
+			if val, ok := info.Labels[pair[0]]; ok {
+				if len(pair) == 2 && val == pair[1] {
+					return true
+				}
+				if len(pair) == 1 {
+					return true
+				}
+			}
+			return false
+		}
+	case "before-image":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.BeforeImage.IsZero() {
+				return true
+			}
+			return info.Created.Before(params.BeforeImage)
+		}
+	case "since-image":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.SinceImage.IsZero() {
+				return true
+			}
+			return info.Created.After(params.SinceImage)
+		}
+	case "dangling":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.Dangling == "" {
+				return true
+			}
+			if common.IsFalse(params.Dangling) && params.ImageName != "<none>" {
+				return true
+			}
+			if common.IsTrue(params.Dangling) && params.ImageName == "<none>" {
+				return true
+			}
+			return false
+		}
+	case "reference":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.ReferencePattern == "" {
+				return true
+			}
+			return libpod.MatchesReference(params.ImageName, params.ReferencePattern)
+		}
+	case "image-input":
+		return func(image *storage.Image, info *types.ImageInspectInfo) bool {
+			if params == nil || params.ImageInput == "" {
+				return true
+			}
+			return libpod.MatchesReference(params.ImageName, params.ImageInput)
+		}
+	default:
+		fmt.Println("invalid filter type", filterType)
+		return nil
+	}
+}

cmd/kpod/info.go

@@ -0,0 +1,200 @@
+package main
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"runtime"
+
+	"github.com/docker/docker/pkg/system"
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	infoDescription = "display system information"
+	infoCommand     = cli.Command{
+		Name:        "info",
+		Usage:       infoDescription,
+		Description: `Information display here pertain to the host, current storage stats, and build of kpod. Useful for the user and when reporting issues.`,
+		Flags:       infoFlags,
+		Action:      infoCmd,
+		ArgsUsage:   "",
+	}
+	infoFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "debug, D",
+			Usage: "display additional debug information",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Change the output format to JSON or a Go template",
+		},
+	}
+)
+
+func infoCmd(c *cli.Context) error {
+	if err := validateFlags(c, infoFlags); err != nil {
+		return err
+	}
+	info := map[string]interface{}{}
+
+	infoGivers := []infoGiverFunc{
+		storeInfo,
+		hostInfo,
+	}
+
+	if c.Bool("debug") {
+		infoGivers = append(infoGivers, debugInfo)
+	}
+
+	for _, giver := range infoGivers {
+		thisName, thisInfo, err := giver(c)
+		if err != nil {
+			info[thisName] = infoErr(err)
+			continue
+		}
+		info[thisName] = thisInfo
+	}
+
+	var out formats.Writer
+	infoOutputFormat := c.String("format")
+	switch infoOutputFormat {
+	case formats.JSONString:
+		out = formats.JSONStruct{Output: info}
+	case "":
+		out = formats.YAMLStruct{Output: info}
+	default:
+		out = formats.StdoutTemplate{Output: info, Template: infoOutputFormat}
+	}
+
+	formats.Writer(out).Out()
+
+	return nil
+}
+
+func infoErr(err error) map[string]interface{} {
+	return map[string]interface{}{
+		"error": err.Error(),
+	}
+}
+
+type infoGiverFunc func(c *cli.Context) (name string, info map[string]interface{}, err error)
+
+// top-level "debug" info
+func debugInfo(c *cli.Context) (string, map[string]interface{}, error) {
+	info := map[string]interface{}{}
+	info["compiler"] = runtime.Compiler
+	info["go version"] = runtime.Version()
+	info["kpod version"] = c.App.Version
+	info["git commit"] = gitCommit
+	return "debug", info, nil
+}
+
+// top-level "host" info
+func hostInfo(c *cli.Context) (string, map[string]interface{}, error) {
+	// lets say OS, arch, number of cpus, amount of memory, maybe os distribution/version, hostname, kernel version, uptime
+	info := map[string]interface{}{}
+	info["os"] = runtime.GOOS
+	info["arch"] = runtime.GOARCH
+	info["cpus"] = runtime.NumCPU()
+	mi, err := system.ReadMemInfo()
+	if err != nil {
+		info["meminfo"] = infoErr(err)
+	} else {
+		// TODO this might be a place for github.com/dustin/go-humanize
+		info["MemTotal"] = mi.MemTotal
+		info["MemFree"] = mi.MemFree
+		info["SwapTotal"] = mi.SwapTotal
+		info["SwapFree"] = mi.SwapFree
+	}
+	if kv, err := readKernelVersion(); err != nil {
+		info["kernel"] = infoErr(err)
+	} else {
+		info["kernel"] = kv
+	}
+
+	if up, err := readUptime(); err != nil {
+		info["uptime"] = infoErr(err)
+	} else {
+		info["uptime"] = up
+	}
+	if host, err := os.Hostname(); err != nil {
+		info["hostname"] = infoErr(err)
+	} else {
+		info["hostname"] = host
+	}
+	return "host", info, nil
+}
+
+// top-level "store" info
+func storeInfo(c *cli.Context) (string, map[string]interface{}, error) {
+	storeStr := "store"
+	config, err := getConfig(c)
+	if err != nil {
+		return storeStr, nil, errors.Wrapf(err, "Could not get config")
+	}
+	store, err := getStore(config)
+	if err != nil {
+		return storeStr, nil, err
+	}
+
+	// lets say storage driver in use, number of images, number of containers
+	info := map[string]interface{}{}
+	info["GraphRoot"] = store.GraphRoot()
+	info["RunRoot"] = store.RunRoot()
+	info["GraphDriverName"] = store.GraphDriverName()
+	info["GraphOptions"] = store.GraphOptions()
+	statusPairs, err := store.Status()
+	if err != nil {
+		return storeStr, nil, err
+	}
+	status := map[string]string{}
+	for _, pair := range statusPairs {
+		status[pair[0]] = pair[1]
+	}
+	info["GraphStatus"] = status
+	images, err := store.Images()
+	if err != nil {
+		info["ImageStore"] = infoErr(err)
+	} else {
+		info["ImageStore"] = map[string]interface{}{
+			"number": len(images),
+		}
+	}
+	containers, err := store.Containers()
+	if err != nil {
+		info["ContainerStore"] = infoErr(err)
+	} else {
+		info["ContainerStore"] = map[string]interface{}{
+			"number": len(containers),
+		}
+	}
+	return storeStr, info, nil
+}
+
+func readKernelVersion() (string, error) {
+	buf, err := ioutil.ReadFile("/proc/version")
+	if err != nil {
+		return "", err
+	}
+	f := bytes.Fields(buf)
+	if len(f) < 2 {
+		return string(bytes.TrimSpace(buf)), nil
+	}
+	return string(f[2]), nil
+}
+
+func readUptime() (string, error) {
+	buf, err := ioutil.ReadFile("/proc/uptime")
+	if err != nil {
+		return "", err
+	}
+	f := bytes.Fields(buf)
+	if len(f) < 1 {
+		return "", fmt.Errorf("invalid uptime")
+	}
+	return string(f[0]), nil
+}

cmd/kpod/inspect.go

@@ -0,0 +1,120 @@
+package main
+
+import (
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/kubernetes-incubator/cri-o/libpod/images"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+const (
+	inspectTypeContainer = "container"
+	inspectTypeImage     = "image"
+	inspectAll           = "all"
+)
+
+var (
+	inspectFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "type, t",
+			Value: inspectAll,
+			Usage: "Return JSON for specified type, (e.g image, container or task)",
+		},
+		cli.StringFlag{
+			Name:  "format, f",
+			Usage: "Change the output format to a Go template",
+		},
+		cli.BoolFlag{
+			Name:  "size",
+			Usage: "Display total file size if the type is container",
+		},
+	}
+	inspectDescription = "This displays the low-level information on containers and images identified by name or ID. By default, this will render all results in a JSON array. If the container and image have the same name, this will return container JSON for unspecified type."
+	inspectCommand     = cli.Command{
+		Name:        "inspect",
+		Usage:       "Displays the configuration of a container or image",
+		Description: inspectDescription,
+		Flags:       inspectFlags,
+		Action:      inspectCmd,
+		ArgsUsage:   "CONTAINER-OR-IMAGE",
+	}
+)
+
+func inspectCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("container or image name must be specified: kpod inspect [options [...]] name")
+	}
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments specified")
+	}
+	if err := validateFlags(c, inspectFlags); err != nil {
+		return err
+	}
+
+	itemType := c.String("type")
+	size := c.Bool("size")
+
+	switch itemType {
+	case inspectTypeContainer:
+	case inspectTypeImage:
+	case inspectAll:
+	default:
+		return errors.Errorf("the only recognized types are %q, %q, and %q", inspectTypeContainer, inspectTypeImage, inspectAll)
+	}
+
+	name := args[0]
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	if err = server.Update(); err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+
+	outputFormat := c.String("format")
+	var data interface{}
+	switch itemType {
+	case inspectTypeContainer:
+		data, err = server.GetContainerData(name, size)
+		if err != nil {
+			return errors.Wrapf(err, "error parsing container data")
+		}
+	case inspectTypeImage:
+		data, err = images.GetData(server.Store(), name)
+		if err != nil {
+			return errors.Wrapf(err, "error parsing image data")
+		}
+	case inspectAll:
+		ctrData, err := server.GetContainerData(name, size)
+		if err != nil {
+			imgData, err := images.GetData(server.Store(), name)
+			if err != nil {
+				return errors.Wrapf(err, "error parsing container or image data")
+			}
+			data = imgData
+
+		} else {
+			data = ctrData
+		}
+	}
+
+	var out formats.Writer
+	if outputFormat != "" && outputFormat != formats.JSONString {
+		//template
+		out = formats.StdoutTemplate{Output: data, Template: outputFormat}
+	} else {
+		// default is json output
+		out = formats.JSONStruct{Output: data}
+	}
+
+	formats.Writer(out).Out()
+	return nil
+}

cmd/kpod/kill.go

@@ -0,0 +1,74 @@
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/docker/docker/pkg/signal"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	killFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "signal, s",
+			Usage: "Signal to send to the container",
+			Value: "KILL",
+		},
+	}
+	killDescription = "The main process inside each container specified will be sent SIGKILL, or any signal specified with option --signal."
+	killCommand     = cli.Command{
+		Name:        "kill",
+		Usage:       "Kill one or more running containers with a specific signal",
+		Description: killDescription,
+		Flags:       killFlags,
+		Action:      killCmd,
+		ArgsUsage:   "[CONTAINER_NAME_OR_ID]",
+	}
+)
+
+// killCmd kills one or more containers with a signal
+func killCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("specify one or more containers to kill")
+	}
+	if err := validateFlags(c, killFlags); err != nil {
+		return err
+	}
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	killSignal := c.String("signal")
+	// Check if the signalString provided by the user is valid
+	// Invalid signals will return err
+	sysSignal, err := signal.ParseSignal(killSignal)
+	if err != nil {
+		return err
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	var lastError error
+	for _, container := range c.Args() {
+		id, err := server.ContainerKill(container, sysSignal)
+		if err != nil {
+			if lastError != nil {
+				fmt.Fprintln(os.Stderr, lastError)
+			}
+			lastError = errors.Wrapf(err, "unable to kill %v", container)
+		} else {
+			fmt.Println(id)
+		}
+	}
+	return lastError
+}

cmd/kpod/load.go

@@ -0,0 +1,116 @@
+package main
+
+import (
+	"io"
+	"io/ioutil"
+	"os"
+
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	loadFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "input, i",
+			Usage: "Read from archive file, default is STDIN",
+			Value: "/dev/stdin",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "Suppress the output",
+		},
+		cli.StringFlag{
+			Name:  "signature-policy",
+			Usage: "`pathname` of signature policy file (not usually used)",
+		},
+	}
+	loadDescription = "Loads the image from docker-archive stored on the local machine."
+	loadCommand     = cli.Command{
+		Name:        "load",
+		Usage:       "load an image from docker archive",
+		Description: loadDescription,
+		Flags:       loadFlags,
+		Action:      loadCmd,
+		ArgsUsage:   "",
+	}
+)
+
+// loadCmd gets the image/file to be loaded from the command line
+// and calls loadImage to load the image to containers-storage
+func loadCmd(c *cli.Context) error {
+
+	args := c.Args()
+	var image string
+	if len(args) == 1 {
+		image = args[0]
+	}
+	if len(args) > 1 {
+		return errors.New("too many arguments. Requires exactly 1")
+	}
+	if err := validateFlags(c, loadFlags); err != nil {
+		return err
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	input := c.String("input")
+
+	if input == "/dev/stdin" {
+		fi, err := os.Stdin.Stat()
+		if err != nil {
+			return err
+		}
+		// checking if loading from pipe
+		if !fi.Mode().IsRegular() {
+			outFile, err := ioutil.TempFile("/var/tmp", "kpod")
+			if err != nil {
+				return errors.Errorf("error creating file %v", err)
+			}
+			defer outFile.Close()
+			defer os.Remove(outFile.Name())
+
+			inFile, err := os.OpenFile(input, 0, 0666)
+			if err != nil {
+				return errors.Errorf("error reading file %v", err)
+			}
+			defer inFile.Close()
+
+			_, err = io.Copy(outFile, inFile)
+			if err != nil {
+				return errors.Errorf("error copying file %v", err)
+			}
+
+			input = outFile.Name()
+		}
+	}
+
+	var writer io.Writer
+	if !c.Bool("quiet") {
+		writer = os.Stdout
+	}
+
+	options := libpod.CopyOptions{
+		SignaturePolicyPath: c.String("signature-policy"),
+		Writer:              writer,
+	}
+
+	src := libpod.DockerArchive + ":" + input
+	if err := runtime.PullImage(src, options); err != nil {
+		src = libpod.OCIArchive + ":" + input
+		// generate full src name with specified image:tag
+		if image != "" {
+			src = src + ":" + image
+		}
+		if err := runtime.PullImage(src, options); err != nil {
+			return errors.Wrapf(err, "error pulling %q", src)
+		}
+	}
+
+	return nil
+}

cmd/kpod/login.go

@@ -0,0 +1,110 @@
+package main
+
+import (
+	"bufio"
+	"context"
+	"fmt"
+	"os"
+	"strings"
+
+	"github.com/containers/image/docker"
+	"github.com/containers/image/pkg/docker/config"
+	"github.com/kubernetes-incubator/cri-o/libpod/common"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"golang.org/x/crypto/ssh/terminal"
+)
+
+var (
+	loginFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "password, p",
+			Usage: "Password for registry",
+		},
+		cli.StringFlag{
+			Name:  "username, u",
+			Usage: "Username for registry",
+		},
+		cli.StringFlag{
+			Name:  "authfile",
+			Usage: "Path of the authentication file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json",
+		},
+	}
+	loginDescription = "Login to a container registry on a specified server."
+	loginCommand     = cli.Command{
+		Name:        "login",
+		Usage:       "login to a container registry",
+		Description: loginDescription,
+		Flags:       loginFlags,
+		Action:      loginCmd,
+		ArgsUsage:   "REGISTRY",
+	}
+)
+
+// loginCmd uses the authentication package to store a user's authenticated credentials
+// in an auth.json file for future use
+func loginCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments, login takes only 1 argument")
+	}
+	if len(args) == 0 {
+		return errors.Errorf("registry must be given")
+	}
+	var server string
+	if len(args) == 1 {
+		server = args[0]
+	}
+
+	sc := common.GetSystemContext("", c.String("authfile"))
+
+	// username of user logged in to server (if one exists)
+	userFromAuthFile := config.GetUserLoggedIn(sc, server)
+	username, password, err := getUserAndPass(c.String("username"), c.String("password"), userFromAuthFile)
+	if err != nil {
+		return errors.Wrapf(err, "error getting username and password")
+	}
+
+	if err = docker.CheckAuth(context.TODO(), sc, username, password, server); err == nil {
+		if err := config.SetAuthentication(sc, server, username, password); err != nil {
+			return err
+		}
+	}
+	switch err {
+	case nil:
+		fmt.Println("Login Succeeded!")
+		return nil
+	case docker.ErrUnauthorizedForCredentials:
+		return errors.Errorf("error logging into %q: invalid username/password\n", server)
+	default:
+		return errors.Wrapf(err, "error authenticating creds for %q", server)
+	}
+}
+
+// getUserAndPass gets the username and password from STDIN if not given
+// using the -u and -p flags
+func getUserAndPass(username, password, userFromAuthFile string) (string, string, error) {
+	var err error
+	reader := bufio.NewReader(os.Stdin)
+	if username == "" {
+		if userFromAuthFile != "" {
+			fmt.Printf("Username (%s): ", userFromAuthFile)
+		} else {
+			fmt.Print("Username: ")
+		}
+		username, err = reader.ReadString('\n')
+		if err != nil {
+			return "", "", errors.Wrapf(err, "error reading username")
+		}
+	}
+	if password == "" {
+		fmt.Print("Password: ")
+		pass, err := terminal.ReadPassword(0)
+		if err != nil {
+			return "", "", errors.Wrapf(err, "error reading password")
+		}
+		password = string(pass)
+		fmt.Println()
+	}
+	return strings.TrimSpace(username), password, err
+}

cmd/kpod/logout.go

@@ -0,0 +1,69 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/containers/image/pkg/docker/config"
+	"github.com/kubernetes-incubator/cri-o/libpod/common"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	logoutFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "authfile",
+			Usage: "Path of the authentication file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json",
+		},
+		cli.BoolFlag{
+			Name:  "all, a",
+			Usage: "Remove the cached credentials for all registries in the auth file",
+		},
+	}
+	logoutDescription = "Remove the cached username and password for the registry."
+	logoutCommand     = cli.Command{
+		Name:        "logout",
+		Usage:       "logout of a container registry",
+		Description: logoutDescription,
+		Flags:       logoutFlags,
+		Action:      logoutCmd,
+		ArgsUsage:   "REGISTRY",
+	}
+)
+
+// logoutCmd uses the authentication package to remove the authenticated of a registry
+// stored in the auth.json file
+func logoutCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments, logout takes only 1 argument")
+	}
+	if len(args) == 0 {
+		return errors.Errorf("registry must be given")
+	}
+	var server string
+	if len(args) == 1 {
+		server = args[0]
+	}
+
+	sc := common.GetSystemContext("", c.String("authfile"))
+
+	if c.Bool("all") {
+		if err := config.RemoveAllAuthentication(sc); err != nil {
+			return err
+		}
+		fmt.Println("Remove login credentials for all registries")
+		return nil
+	}
+
+	err := config.RemoveAuthentication(sc, server)
+	switch err {
+	case nil:
+		fmt.Printf("Remove login credentials for %s\n", server)
+		return nil
+	case config.ErrNotLoggedIn:
+		return errors.Errorf("Not logged into %s\n", server)
+	default:
+		return errors.Wrapf(err, "error logging out of %q", server)
+	}
+}

cmd/kpod/logs.go

@@ -0,0 +1,92 @@
+package main
+
+import (
+	"fmt"
+	"time"
+
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	logsFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:   "details",
+			Usage:  "Show extra details provided to the logs",
+			Hidden: true,
+		},
+		cli.BoolFlag{
+			Name:  "follow, f",
+			Usage: "Follow log output.  The default is false",
+		},
+		cli.StringFlag{
+			Name:  "since",
+			Usage: "Show logs since TIMESTAMP",
+		},
+		cli.Uint64Flag{
+			Name:  "tail",
+			Usage: "Output the specified number of LINES at the end of the logs.  Defaults to 0, which prints all lines",
+		},
+	}
+	logsDescription = "The kpod logs command batch-retrieves whatever logs are present for a container at the time of execution.  This does not guarantee execution" +
+		"order when combined with kpod run (i.e. your run may not have generated any logs at the time you execute kpod logs"
+	logsCommand = cli.Command{
+		Name:        "logs",
+		Usage:       "Fetch the logs of a container",
+		Description: logsDescription,
+		Flags:       logsFlags,
+		Action:      logsCmd,
+		ArgsUsage:   "CONTAINER",
+	}
+)
+
+func logsCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) != 1 {
+		return errors.Errorf("'kpod logs' requires exactly one container name/ID")
+	}
+	if err := validateFlags(c, logsFlags); err != nil {
+		return err
+	}
+	container := c.Args().First()
+	var opts libkpod.LogOptions
+	opts.Details = c.Bool("details")
+	opts.Follow = c.Bool("follow")
+	opts.SinceTime = time.Time{}
+	if c.IsSet("since") {
+		// parse time, error out if something is wrong
+		since, err := time.Parse("2006-01-02T15:04:05.999999999-07:00", c.String("since"))
+		if err != nil {
+			return errors.Wrapf(err, "could not parse time: %q", c.String("since"))
+		}
+		opts.SinceTime = since
+	}
+	opts.Tail = c.Uint64("tail")
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not create container server")
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	logs := make(chan string)
+	go func() {
+		err = server.GetLogs(container, logs, opts)
+	}()
+	printLogs(logs)
+	return err
+}
+
+func printLogs(logs chan string) {
+	for line := range logs {
+		fmt.Println(line)
+	}
+}

cmd/kpod/main.go

@@ -0,0 +1,129 @@
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/containers/storage/pkg/reexec"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli"
+)
+
+// This is populated by the Makefile from the VERSION file
+// in the repository
+var kpodVersion = ""
+
+func main() {
+	debug := false
+
+	if reexec.Init() {
+		return
+	}
+
+	app := cli.NewApp()
+	app.Name = "kpod"
+	app.Usage = "manage pods and images"
+
+	var v string
+	if kpodVersion != "" {
+		v = kpodVersion
+	}
+	app.Version = v
+
+	app.Commands = []cli.Command{
+		diffCommand,
+		exportCommand,
+		historyCommand,
+		imagesCommand,
+		infoCommand,
+		inspectCommand,
+		killCommand,
+		loadCommand,
+		loginCommand,
+		logoutCommand,
+		logsCommand,
+		mountCommand,
+		pauseCommand,
+		psCommand,
+		pullCommand,
+		pushCommand,
+		renameCommand,
+		rmCommand,
+		rmiCommand,
+		saveCommand,
+		statsCommand,
+		stopCommand,
+		tagCommand,
+		umountCommand,
+		unpauseCommand,
+		versionCommand,
+		waitCommand,
+	}
+	app.Before = func(c *cli.Context) error {
+		logLevel := c.GlobalString("log-level")
+		if logLevel != "" {
+			level, err := logrus.ParseLevel(logLevel)
+			if err != nil {
+				return err
+			}
+
+			logrus.SetLevel(level)
+		}
+
+		if logLevel == "debug" {
+			debug = true
+
+		}
+
+		return nil
+	}
+	app.After = func(*cli.Context) error {
+		// called by Run() when the command handler succeeds
+		shutdownStores()
+		return nil
+	}
+	cli.OsExiter = func(code int) {
+		// called by Run() when the command fails, bypassing After()
+		shutdownStores()
+		os.Exit(code)
+	}
+	app.Flags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "config, c",
+			Usage: "path of a config file detailing container server configuration options",
+		},
+		cli.StringFlag{
+			Name:  "log-level",
+			Usage: "log messages above specified level: debug, info, warn, error (default), fatal or panic",
+			Value: "error",
+		},
+		cli.StringFlag{
+			Name:  "root",
+			Usage: "path to the root directory in which data, including images, is stored",
+		},
+		cli.StringFlag{
+			Name:  "runroot",
+			Usage: "path to the 'run directory' where all state information is stored",
+		},
+		cli.StringFlag{
+			Name:  "runtime",
+			Usage: "path to the OCI-compatible binary used to run containers, default is /usr/bin/runc",
+		},
+		cli.StringFlag{
+			Name:  "storage-driver, s",
+			Usage: "select which storage driver is used to manage storage of images and containers (default is overlay)",
+		},
+		cli.StringSliceFlag{
+			Name:  "storage-opt",
+			Usage: "used to pass an option to the storage driver",
+		},
+	}
+	if err := app.Run(os.Args); err != nil {
+		if debug {
+			logrus.Errorf(err.Error())
+		} else {
+			fmt.Fprintln(os.Stderr, err.Error())
+		}
+		cli.OsExiter(1)
+	}
+}

cmd/kpod/mount.go

@@ -0,0 +1,121 @@
+package main
+
+import (
+	js "encoding/json"
+	"fmt"
+
+	of "github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	mountDescription = `
+   kpod mount
+   Lists all mounted containers mount points
+
+   kpod mount CONTAINER-NAME-OR-ID
+   Mounts the specified container and outputs the mountpoint
+`
+
+	mountFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "notruncate",
+			Usage: "do not truncate output",
+		},
+		cli.StringFlag{
+			Name:  "label",
+			Usage: "SELinux label for the mount point",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Change the output format to Go template",
+		},
+	}
+	mountCommand = cli.Command{
+		Name:        "mount",
+		Usage:       "Mount a working container's root filesystem",
+		Description: mountDescription,
+		Action:      mountCmd,
+		ArgsUsage:   "[CONTAINER-NAME-OR-ID]",
+		Flags:       mountFlags,
+	}
+)
+
+// MountOutputParams stores info about each layer
+type jsonMountPoint struct {
+	ID         string   `json:"id"`
+	Names      []string `json:"names"`
+	MountPoint string   `json:"mountpoint"`
+}
+
+func mountCmd(c *cli.Context) error {
+	formats := map[string]bool{
+		"":            true,
+		of.JSONString: true,
+	}
+
+	args := c.Args()
+	json := c.String("format") == of.JSONString
+	if !formats[c.String("format")] {
+		return errors.Errorf("%q is not a supported format", c.String("format"))
+	}
+
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments specified")
+	}
+	if err := validateFlags(c, mountFlags); err != nil {
+		return err
+	}
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get config")
+	}
+	store, err := getStore(config)
+	if err != nil {
+		return errors.Wrapf(err, "error getting store")
+	}
+	if len(args) == 1 {
+		if json {
+			return errors.Wrapf(err, "json option can not be used with a container id")
+		}
+		mountPoint, err := store.Mount(args[0], c.String("label"))
+		if err != nil {
+			return errors.Wrapf(err, "error finding container %q", args[0])
+		}
+		fmt.Printf("%s\n", mountPoint)
+	} else {
+		jsonMountPoints := []jsonMountPoint{}
+		containers, err2 := store.Containers()
+		if err2 != nil {
+			return errors.Wrapf(err2, "error reading list of all containers")
+		}
+		for _, container := range containers {
+			layer, err := store.Layer(container.LayerID)
+			if err != nil {
+				return errors.Wrapf(err, "error finding layer %q for container %q", container.LayerID, container.ID)
+			}
+			if layer.MountPoint == "" {
+				continue
+			}
+			if json {
+				jsonMountPoints = append(jsonMountPoints, jsonMountPoint{ID: container.ID, Names: container.Names, MountPoint: layer.MountPoint})
+				continue
+			}
+
+			if c.Bool("notruncate") {
+				fmt.Printf("%-64s %s\n", container.ID, layer.MountPoint)
+			} else {
+				fmt.Printf("%-12.12s %s\n", container.ID, layer.MountPoint)
+			}
+		}
+		if json {
+			data, err := js.MarshalIndent(jsonMountPoints, "", "    ")
+			if err != nil {
+				return err
+			}
+			fmt.Printf("%s\n", data)
+		}
+	}
+	return nil
+}

cmd/kpod/pause.go

@@ -0,0 +1,58 @@
+package main
+
+import (
+	"fmt"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"os"
+)
+
+var (
+	pauseDescription = `
+   kpod pause
+
+   Pauses one or more running containers.  The container name or ID can be used.
+`
+	pauseCommand = cli.Command{
+		Name:        "pause",
+		Usage:       "Pauses all the processes in one or more containers",
+		Description: pauseDescription,
+		Action:      pauseCmd,
+		ArgsUsage:   "CONTAINER-NAME [CONTAINER-NAME ...]",
+	}
+)
+
+func pauseCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) < 1 {
+		return errors.Errorf("you must provide at least one container name or id")
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	if err := server.Update(); err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	var lastError error
+	for _, container := range c.Args() {
+		cid, err := server.ContainerPause(container)
+		if err != nil {
+			if lastError != nil {
+				fmt.Fprintln(os.Stderr, lastError)
+			}
+			lastError = errors.Wrapf(err, "failed to pause container %v", container)
+		} else {
+			fmt.Println(cid)
+		}
+	}
+
+	return lastError
+}

cmd/kpod/ps.go

@@ -0,0 +1,665 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"reflect"
+	"regexp"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/docker/go-units"
+	specs "github.com/opencontainers/runtime-spec/specs-go"
+	"github.com/sirupsen/logrus"
+
+	"k8s.io/apimachinery/pkg/fields"
+
+	"github.com/kubernetes-incubator/cri-o/cmd/kpod/formats"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/kubernetes-incubator/cri-o/oci"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+type psOptions struct {
+	all       bool
+	filter    string
+	format    string
+	last      int
+	latest    bool
+	noTrunc   bool
+	quiet     bool
+	size      bool
+	label     string
+	namespace bool
+}
+
+type psTemplateParams struct {
+	ID         string
+	Image      string
+	Command    string
+	CreatedAt  string
+	RunningFor string
+	Status     string
+	Ports      string
+	Size       string
+	Names      string
+	Labels     string
+	Mounts     string
+	PID        int
+	Cgroup     string
+	IPC        string
+	MNT        string
+	NET        string
+	PIDNS      string
+	User       string
+	UTS        string
+}
+
+// psJSONParams is only used when the JSON format is specified,
+// and is better for data processing from JSON.
+// psJSONParams will be populated by data from libkpod.ContainerData,
+// the members of the struct are the sama data types as their sources.
+type psJSONParams struct {
+	ID               string              `json:"id"`
+	Image            string              `json:"image"`
+	ImageID          string              `json:"image_id"`
+	Command          string              `json:"command"`
+	CreatedAt        time.Time           `json:"createdAt"`
+	RunningFor       time.Duration       `json:"runningFor"`
+	Status           string              `json:"status"`
+	Ports            map[string]struct{} `json:"ports"`
+	Size             uint                `json:"size"`
+	Names            string              `json:"names"`
+	Labels           fields.Set          `json:"labels"`
+	Mounts           []specs.Mount       `json:"mounts"`
+	ContainerRunning bool                `json:"ctrRunning"`
+	Namespaces       *namespace          `json:"namespace,omitempty"`
+}
+
+type namespace struct {
+	PID    string `json:"pid,omitempty"`
+	Cgroup string `json:"cgroup,omitempty"`
+	IPC    string `json:"ipc,omitempty"`
+	MNT    string `json:"mnt,omitempty"`
+	NET    string `json:"net,omitempty"`
+	PIDNS  string `json:"pidns,omitempty"`
+	User   string `json:"user,omitempty"`
+	UTS    string `json:"uts,omitempty"`
+}
+
+var (
+	psFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "all, a",
+			Usage: "Show all the containers, default is only running containers",
+		},
+		cli.StringFlag{
+			Name:  "filter, f",
+			Usage: "Filter output based on conditions given",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Pretty-print containers to JSON or using a Go template",
+		},
+		cli.IntFlag{
+			Name:  "last, n",
+			Usage: "Print the n last created containers (all states)",
+			Value: -1,
+		},
+		cli.BoolFlag{
+			Name:  "latest, l",
+			Usage: "Show the latest container created (all states)",
+		},
+		cli.BoolFlag{
+			Name:  "no-trunc",
+			Usage: "Display the extended information",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "Print the numeric IDs of the containers only",
+		},
+		cli.BoolFlag{
+			Name:  "size, s",
+			Usage: "Display the total file sizes",
+		},
+		cli.BoolFlag{
+			Name:  "namespace, ns",
+			Usage: "Display namespace information",
+		},
+	}
+	psDescription = "Prints out information about the containers"
+	psCommand     = cli.Command{
+		Name:        "ps",
+		Usage:       "List containers",
+		Description: psDescription,
+		Flags:       psFlags,
+		Action:      psCmd,
+		ArgsUsage:   "",
+	}
+)
+
+func psCmd(c *cli.Context) error {
+	if err := validateFlags(c, psFlags); err != nil {
+		return err
+	}
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "error creating server")
+	}
+	if err := server.Update(); err != nil {
+		return errors.Wrapf(err, "error updating list of containers")
+	}
+
+	if len(c.Args()) > 0 {
+		return errors.Errorf("too many arguments, ps takes no arguments")
+	}
+
+	format := genPsFormat(c.Bool("quiet"), c.Bool("size"), c.Bool("namespace"))
+	if c.IsSet("format") {
+		format = c.String("format")
+	}
+
+	opts := psOptions{
+		all:       c.Bool("all"),
+		filter:    c.String("filter"),
+		format:    format,
+		last:      c.Int("last"),
+		latest:    c.Bool("latest"),
+		noTrunc:   c.Bool("no-trunc"),
+		quiet:     c.Bool("quiet"),
+		size:      c.Bool("size"),
+		namespace: c.Bool("namespace"),
+	}
+
+	// all, latest, and last are mutually exclusive. Only one flag can be used at a time
+	exclusiveOpts := 0
+	if opts.last >= 0 {
+		exclusiveOpts++
+	}
+	if opts.latest {
+		exclusiveOpts++
+	}
+	if opts.all {
+		exclusiveOpts++
+	}
+	if exclusiveOpts > 1 {
+		return errors.Errorf("Last, latest and all are mutually exclusive")
+	}
+
+	containers, err := server.ListContainers()
+	if err != nil {
+		return errors.Wrapf(err, "error getting containers from server")
+	}
+	var params *FilterParamsPS
+	if opts.filter != "" {
+		params, err = parseFilter(opts.filter, containers)
+		if err != nil {
+			return errors.Wrapf(err, "error parsing filter")
+		}
+	} else {
+		params = nil
+	}
+
+	containerList := getContainersMatchingFilter(containers, params, server)
+
+	return generatePsOutput(containerList, server, opts)
+}
+
+// generate the template based on conditions given
+func genPsFormat(quiet, size, namespace bool) (format string) {
+	if quiet {
+		return formats.IDString
+	}
+	if namespace {
+		format = "table {{.ID}}\t{{.Names}}\t{{.PID}}\t{{.Cgroup}}\t{{.IPC}}\t{{.MNT}}\t{{.NET}}\t{{.PIDNS}}\t{{.User}}\t{{.UTS}}\t"
+		return
+	}
+	format = "table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.CreatedAt}}\t{{.Status}}\t{{.Ports}}\t{{.Names}}\t"
+	if size {
+		format += "{{.Size}}\t"
+	}
+	return
+}
+
+func psToGeneric(templParams []psTemplateParams, JSONParams []psJSONParams) (genericParams []interface{}) {
+	if len(templParams) > 0 {
+		for _, v := range templParams {
+			genericParams = append(genericParams, interface{}(v))
+		}
+		return
+	}
+	for _, v := range JSONParams {
+		genericParams = append(genericParams, interface{}(v))
+	}
+	return
+}
+
+// generate the accurate header based on template given
+func (p *psTemplateParams) headerMap() map[string]string {
+	v := reflect.Indirect(reflect.ValueOf(p))
+	values := make(map[string]string)
+
+	for i := 0; i < v.NumField(); i++ {
+		key := v.Type().Field(i).Name
+		value := key
+		if value == "ID" {
+			value = "Container" + value
+		}
+		values[key] = strings.ToUpper(splitCamelCase(value))
+	}
+	return values
+}
+
+// getContainers gets the containers that match the flags given
+func getContainers(containers []*libkpod.ContainerData, opts psOptions) []*libkpod.ContainerData {
+	var containersOutput []*libkpod.ContainerData
+	if opts.last >= 0 && opts.last < len(containers) {
+		for i := 0; i < opts.last; i++ {
+			containersOutput = append(containersOutput, containers[i])
+		}
+		return containersOutput
+	}
+	if opts.latest {
+		return []*libkpod.ContainerData{containers[0]}
+	}
+	if opts.all || opts.last >= len(containers) {
+		return containers
+	}
+	for _, ctr := range containers {
+		if ctr.State.Status == oci.ContainerStateRunning {
+			containersOutput = append(containersOutput, ctr)
+		}
+	}
+	return containersOutput
+}
+
+// getTemplateOutput returns the modified container information
+func getTemplateOutput(containers []*libkpod.ContainerData, opts psOptions) (psOutput []psTemplateParams) {
+	var status string
+	for _, ctr := range containers {
+		ctrID := ctr.ID
+		runningFor := units.HumanDuration(time.Since(ctr.State.Created))
+		createdAt := runningFor + " ago"
+		command := getStrFromSquareBrackets(ctr.ImageCreatedBy)
+		imageName := ctr.FromImage
+		mounts := getMounts(ctr.Mounts, opts.noTrunc)
+		ports := getPorts(ctr.Config.ExposedPorts)
+		size := units.HumanSize(float64(ctr.SizeRootFs))
+		labels := getLabels(ctr.Labels)
+
+		ns := getNamespaces(ctr.State.Pid)
+
+		switch ctr.State.Status {
+		case oci.ContainerStateStopped:
+			status = "Exited (" + strconv.FormatInt(int64(ctr.State.ExitCode), 10) + ") " + runningFor + " ago"
+		case oci.ContainerStateRunning:
+			status = "Up " + runningFor + " ago"
+		case oci.ContainerStatePaused:
+			status = "Paused"
+		default:
+			status = "Created"
+		}
+
+		if !opts.noTrunc {
+			ctrID = ctr.ID[:idTruncLength]
+			imageName = getImageName(ctr.FromImage)
+		}
+
+		params := psTemplateParams{
+			ID:         ctrID,
+			Image:      imageName,
+			Command:    command,
+			CreatedAt:  createdAt,
+			RunningFor: runningFor,
+			Status:     status,
+			Ports:      ports,
+			Size:       size,
+			Names:      ctr.Name,
+			Labels:     labels,
+			Mounts:     mounts,
+			PID:        ctr.State.Pid,
+			Cgroup:     ns.Cgroup,
+			IPC:        ns.IPC,
+			MNT:        ns.MNT,
+			NET:        ns.NET,
+			PIDNS:      ns.PID,
+			User:       ns.User,
+			UTS:        ns.UTS,
+		}
+		psOutput = append(psOutput, params)
+	}
+	return
+}
+
+func getNamespaces(pid int) *namespace {
+	ctrPID := strconv.Itoa(pid)
+	cgroup, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "cgroup"))
+	ipc, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "ipc"))
+	mnt, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "mnt"))
+	net, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "net"))
+	pidns, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "pid"))
+	user, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "user"))
+	uts, _ := getNamespaceInfo(filepath.Join("/proc", ctrPID, "ns", "uts"))
+
+	return &namespace{
+		PID:    ctrPID,
+		Cgroup: cgroup,
+		IPC:    ipc,
+		MNT:    mnt,
+		NET:    net,
+		PIDNS:  pidns,
+		User:   user,
+		UTS:    uts,
+	}
+}
+
+func getNamespaceInfo(path string) (string, error) {
+	val, err := os.Readlink(path)
+	if err != nil {
+		return "", errors.Wrapf(err, "error getting info from %q", path)
+	}
+	return getStrFromSquareBrackets(val), nil
+}
+
+// getJSONOutput returns the container info in its raw form
+func getJSONOutput(containers []*libkpod.ContainerData, nSpace bool) (psOutput []psJSONParams) {
+	var ns *namespace
+	for _, ctr := range containers {
+		if nSpace {
+			ns = getNamespaces(ctr.State.Pid)
+		}
+
+		params := psJSONParams{
+			ID:               ctr.ID,
+			Image:            ctr.FromImage,
+			ImageID:          ctr.FromImageID,
+			Command:          getStrFromSquareBrackets(ctr.ImageCreatedBy),
+			CreatedAt:        ctr.State.Created,
+			RunningFor:       time.Since(ctr.State.Created),
+			Status:           ctr.State.Status,
+			Ports:            ctr.Config.ExposedPorts,
+			Size:             ctr.SizeRootFs,
+			Names:            ctr.Name,
+			Labels:           ctr.Labels,
+			Mounts:           ctr.Mounts,
+			ContainerRunning: ctr.State.Status == oci.ContainerStateRunning,
+			Namespaces:       ns,
+		}
+		psOutput = append(psOutput, params)
+	}
+	return
+}
+
+func generatePsOutput(containers []*libkpod.ContainerData, server *libkpod.ContainerServer, opts psOptions) error {
+	containersOutput := getContainers(containers, opts)
+	// In the case of JSON, we want to continue so we at least pass
+	// {} --valid JSON-- to the consumer
+	if len(containersOutput) == 0 && opts.format != formats.JSONString {
+		return nil
+	}
+
+	var out formats.Writer
+
+	switch opts.format {
+	case formats.JSONString:
+		psOutput := getJSONOutput(containersOutput, opts.namespace)
+		out = formats.JSONStructArray{Output: psToGeneric([]psTemplateParams{}, psOutput)}
+	default:
+		psOutput := getTemplateOutput(containersOutput, opts)
+		out = formats.StdoutTemplateArray{Output: psToGeneric(psOutput, []psJSONParams{}), Template: opts.format, Fields: psOutput[0].headerMap()}
+	}
+
+	return formats.Writer(out).Out()
+}
+
+// getStrFromSquareBrackets gets the string inside [] from a string
+func getStrFromSquareBrackets(cmd string) string {
+	reg, err := regexp.Compile(".*\\[|\\].*")
+	if err != nil {
+		return ""
+	}
+	arr := strings.Split(reg.ReplaceAllLiteralString(cmd, ""), ",")
+	return strings.Join(arr, ",")
+}
+
+// getImageName shortens the image name
+func getImageName(img string) string {
+	arr := strings.Split(img, "/")
+	if arr[0] == "docker.io" && arr[1] == "library" {
+		img = strings.Join(arr[2:], "/")
+	} else if arr[0] == "docker.io" {
+		img = strings.Join(arr[1:], "/")
+	}
+	return img
+}
+
+// getLabels converts the labels to a string of the form "key=value, key2=value2"
+func getLabels(labels fields.Set) string {
+	var arr []string
+	if len(labels) > 0 {
+		for key, val := range labels {
+			temp := key + "=" + val
+			arr = append(arr, temp)
+		}
+		return strings.Join(arr, ",")
+	}
+	return ""
+}
+
+// getMounts converts the volumes mounted to a string of the form "mount1, mount2"
+// it truncates it if noTrunc is false
+func getMounts(mounts []specs.Mount, noTrunc bool) string {
+	var arr []string
+	if len(mounts) == 0 {
+		return ""
+	}
+	for _, mount := range mounts {
+		if noTrunc {
+			arr = append(arr, mount.Source)
+			continue
+		}
+		tempArr := strings.SplitAfter(mount.Source, "/")
+		if len(tempArr) >= 3 {
+			arr = append(arr, strings.Join(tempArr[:3], ""))
+		} else {
+			arr = append(arr, mount.Source)
+		}
+	}
+	return strings.Join(arr, ",")
+}
+
+// getPorts converts the ports used to a string of the from "port1, port2"
+func getPorts(ports map[string]struct{}) string {
+	var arr []string
+	if len(ports) == 0 {
+		return ""
+	}
+	for key := range ports {
+		arr = append(arr, key)
+	}
+	return strings.Join(arr, ",")
+}
+
+// FilterParamsPS contains the filter options for ps
+type FilterParamsPS struct {
+	id       string
+	label    string
+	name     string
+	exited   int32
+	status   string
+	ancestor string
+	before   time.Time
+	since    time.Time
+	volume   string
+}
+
+// parseFilter takes a filter string and a list of containers and filters it
+func parseFilter(filter string, containers []*oci.Container) (*FilterParamsPS, error) {
+	params := new(FilterParamsPS)
+	allFilters := strings.Split(filter, ",")
+
+	for _, param := range allFilters {
+		pair := strings.SplitN(param, "=", 2)
+		switch strings.TrimSpace(pair[0]) {
+		case "id":
+			params.id = pair[1]
+		case "label":
+			params.label = pair[1]
+		case "name":
+			params.name = pair[1]
+		case "exited":
+			exitedCode, err := strconv.ParseInt(pair[1], 10, 32)
+			if err != nil {
+				return nil, errors.Errorf("exited code out of range %q", pair[1])
+			}
+			params.exited = int32(exitedCode)
+		case "status":
+			params.status = pair[1]
+		case "ancestor":
+			params.ancestor = pair[1]
+		case "before":
+			if ctr, err := findContainer(containers, pair[1]); err == nil {
+				params.before = ctr.CreatedAt()
+			} else {
+				return nil, errors.Wrapf(err, "no such container %q", pair[1])
+			}
+		case "since":
+			if ctr, err := findContainer(containers, pair[1]); err == nil {
+				params.before = ctr.CreatedAt()
+			} else {
+				return nil, errors.Wrapf(err, "no such container %q", pair[1])
+			}
+		case "volume":
+			params.volume = pair[1]
+		default:
+			return nil, errors.Errorf("invalid filter %q", pair[0])
+		}
+	}
+	return params, nil
+}
+
+// findContainer finds a container with a specific name or id from a list of containers
+func findContainer(containers []*oci.Container, ref string) (*oci.Container, error) {
+	for _, ctr := range containers {
+		if strings.HasPrefix(ctr.ID(), ref) || ctr.Name() == ref {
+			return ctr, nil
+		}
+	}
+	return nil, errors.Errorf("could not find container")
+}
+
+// matchesFilter checks if a container matches all the filter parameters
+func matchesFilter(ctrData *libkpod.ContainerData, params *FilterParamsPS) bool {
+	if params == nil {
+		return true
+	}
+	if params.id != "" && !matchesID(ctrData, params.id) {
+		return false
+	}
+	if params.name != "" && !matchesName(ctrData, params.name) {
+		return false
+	}
+	if !params.before.IsZero() && !matchesBeforeContainer(ctrData, params.before) {
+		return false
+	}
+	if !params.since.IsZero() && !matchesSinceContainer(ctrData, params.since) {
+		return false
+	}
+	if params.exited > 0 && !matchesExited(ctrData, params.exited) {
+		return false
+	}
+	if params.status != "" && !matchesStatus(ctrData, params.status) {
+		return false
+	}
+	if params.ancestor != "" && !matchesAncestor(ctrData, params.ancestor) {
+		return false
+	}
+	if params.label != "" && !matchesLabel(ctrData, params.label) {
+		return false
+	}
+	if params.volume != "" && !matchesVolume(ctrData, params.volume) {
+		return false
+	}
+	return true
+}
+
+// GetContainersMatchingFilter returns a slice of all the containers that match the provided filter parameters
+func getContainersMatchingFilter(containers []*oci.Container, filter *FilterParamsPS, server *libkpod.ContainerServer) []*libkpod.ContainerData {
+	var filteredCtrs []*libkpod.ContainerData
+	for _, ctr := range containers {
+		ctrData, err := server.GetContainerData(ctr.ID(), true)
+		if err != nil {
+			logrus.Warn("unable to get container data for matched container")
+		}
+		if filter == nil || matchesFilter(ctrData, filter) {
+			filteredCtrs = append(filteredCtrs, ctrData)
+		}
+	}
+	return filteredCtrs
+}
+
+// matchesID returns true if the id's match
+func matchesID(ctrData *libkpod.ContainerData, id string) bool {
+	return strings.HasPrefix(ctrData.ID, id)
+}
+
+// matchesBeforeContainer returns true if the container was created before the filter image
+func matchesBeforeContainer(ctrData *libkpod.ContainerData, beforeTime time.Time) bool {
+	return ctrData.State.Created.Before(beforeTime)
+}
+
+// matchesSincecontainer returns true if the container was created since the filter image
+func matchesSinceContainer(ctrData *libkpod.ContainerData, sinceTime time.Time) bool {
+	return ctrData.State.Created.After(sinceTime)
+}
+
+// matchesLabel returns true if the container label matches that of the filter label
+func matchesLabel(ctrData *libkpod.ContainerData, label string) bool {
+	pair := strings.SplitN(label, "=", 2)
+	if val, ok := ctrData.Labels[pair[0]]; ok {
+		if len(pair) == 2 && val == pair[1] {
+			return true
+		}
+		if len(pair) == 1 {
+			return true
+		}
+		return false
+	}
+	return false
+}
+
+// matchesName returns true if the names are identical
+func matchesName(ctrData *libkpod.ContainerData, name string) bool {
+	return ctrData.Name == name
+}
+
+// matchesExited returns true if the exit codes are identical
+func matchesExited(ctrData *libkpod.ContainerData, exited int32) bool {
+	return ctrData.State.ExitCode == exited
+}
+
+// matchesStatus returns true if the container status matches that of filter status
+func matchesStatus(ctrData *libkpod.ContainerData, status string) bool {
+	return ctrData.State.Status == status
+}
+
+// matchesAncestor returns true if filter ancestor is in container image name
+func matchesAncestor(ctrData *libkpod.ContainerData, ancestor string) bool {
+	return strings.Contains(ctrData.FromImage, ancestor)
+}
+
+// matchesVolue returns true if the volume mounted or path to volue of the container matches that of filter volume
+func matchesVolume(ctrData *libkpod.ContainerData, volume string) bool {
+	for _, vol := range ctrData.Mounts {
+		if strings.Contains(vol.Source, volume) {
+			return true
+		}
+	}
+	return false
+}

cmd/kpod/pull.go

@@ -0,0 +1,118 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"os"
+
+	"golang.org/x/crypto/ssh/terminal"
+
+	"github.com/containers/image/types"
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/kubernetes-incubator/cri-o/libpod/common"
+	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli"
+)
+
+var (
+	pullFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "authfile",
+			Usage: "Path of the authentication file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json",
+		},
+		cli.StringFlag{
+			Name:  "cert-dir",
+			Usage: "`pathname` of a directory containing TLS certificates and keys",
+		},
+		cli.StringFlag{
+			Name:  "creds",
+			Usage: "`credentials` (USERNAME:PASSWORD) to use for authenticating to a registry",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "Suppress output information when pulling images",
+		},
+		cli.StringFlag{
+			Name:  "signature-policy",
+			Usage: "`pathname` of signature policy file (not usually used)",
+		},
+		cli.BoolTFlag{
+			Name:  "tls-verify",
+			Usage: "require HTTPS and verify certificates when contacting registries (default: true)",
+		},
+	}
+
+	pullDescription = "Pulls an image from a registry and stores it locally.\n" +
+		"An image can be pulled using its tag or digest. If a tag is not\n" +
+		"specified, the image with the 'latest' tag (if it exists) is pulled."
+	pullCommand = cli.Command{
+		Name:        "pull",
+		Usage:       "pull an image from a registry",
+		Description: pullDescription,
+		Flags:       pullFlags,
+		Action:      pullCmd,
+		ArgsUsage:   "",
+	}
+)
+
+// pullCmd gets the data from the command line and calls pullImage
+// to copy an image from a registry to a local machine
+func pullCmd(c *cli.Context) error {
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	args := c.Args()
+	if len(args) == 0 {
+		logrus.Errorf("an image name must be specified")
+		return nil
+	}
+	if len(args) > 1 {
+		logrus.Errorf("too many arguments. Requires exactly 1")
+		return nil
+	}
+	if err := validateFlags(c, pullFlags); err != nil {
+		return err
+	}
+	image := args[0]
+
+	var registryCreds *types.DockerAuthConfig
+	if c.String("creds") != "" {
+		creds, err := common.ParseRegistryCreds(c.String("creds"))
+		if err != nil {
+			if err == common.ErrNoPassword {
+				fmt.Print("Password: ")
+				password, err := terminal.ReadPassword(0)
+				if err != nil {
+					return errors.Wrapf(err, "could not read password from terminal")
+				}
+				creds.Password = string(password)
+			} else {
+				return err
+			}
+		}
+		registryCreds = creds
+	}
+
+	var writer io.Writer
+	if !c.Bool("quiet") {
+		writer = os.Stdout
+	}
+
+	options := libpod.CopyOptions{
+		SignaturePolicyPath: c.String("signature-policy"),
+		AuthFile:            c.String("authfile"),
+		DockerRegistryOptions: common.DockerRegistryOptions{
+			DockerRegistryCreds:         registryCreds,
+			DockerCertPath:              c.String("cert-dir"),
+			DockerInsecureSkipTLSVerify: !c.BoolT("tls-verify"),
+		},
+		Writer: writer,
+	}
+
+	return runtime.PullImage(image, options)
+
+}

cmd/kpod/push.go

@@ -0,0 +1,132 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"os"
+
+	"github.com/containers/image/types"
+	"github.com/containers/storage/pkg/archive"
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/kubernetes-incubator/cri-o/libpod/common"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"golang.org/x/crypto/ssh/terminal"
+)
+
+var (
+	pushFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:   "signature-policy",
+			Usage:  "`pathname` of signature policy file (not usually used)",
+			Hidden: true,
+		},
+		cli.StringFlag{
+			Name:  "creds",
+			Usage: "`credentials` (USERNAME:PASSWORD) to use for authenticating to a registry",
+		},
+		cli.StringFlag{
+			Name:  "cert-dir",
+			Usage: "`pathname` of a directory containing TLS certificates and keys",
+		},
+		cli.BoolTFlag{
+			Name:  "tls-verify",
+			Usage: "require HTTPS and verify certificates when contacting registries (default: true)",
+		},
+		cli.BoolFlag{
+			Name:  "remove-signatures",
+			Usage: "discard any pre-existing signatures in the image",
+		},
+		cli.StringFlag{
+			Name:  "sign-by",
+			Usage: "add a signature at the destination using the specified key",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "don't output progress information when pushing images",
+		},
+		cli.StringFlag{
+			Name:  "authfile",
+			Usage: "Path of the authentication file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json",
+		},
+	}
+	pushDescription = fmt.Sprintf(`
+   Pushes an image to a specified location.
+   The Image "DESTINATION" uses a "transport":"details" format.
+   See kpod-push(1) section "DESTINATION" for the expected format`)
+
+	pushCommand = cli.Command{
+		Name:        "push",
+		Usage:       "push an image to a specified destination",
+		Description: pushDescription,
+		Flags:       pushFlags,
+		Action:      pushCmd,
+		ArgsUsage:   "IMAGE DESTINATION",
+	}
+)
+
+func pushCmd(c *cli.Context) error {
+	var registryCreds *types.DockerAuthConfig
+
+	args := c.Args()
+	if len(args) < 2 {
+		return errors.New("kpod push requires exactly 2 arguments")
+	}
+	if err := validateFlags(c, pushFlags); err != nil {
+		return err
+	}
+	srcName := c.Args().Get(0)
+	destName := c.Args().Get(1)
+
+	registryCredsString := c.String("creds")
+	certPath := c.String("cert-dir")
+	skipVerify := !c.BoolT("tls-verify")
+	removeSignatures := c.Bool("remove-signatures")
+	signBy := c.String("sign-by")
+
+	if registryCredsString != "" {
+		creds, err := common.ParseRegistryCreds(registryCredsString)
+		if err != nil {
+			if err == common.ErrNoPassword {
+				fmt.Print("Password: ")
+				password, err := terminal.ReadPassword(0)
+				if err != nil {
+					return errors.Wrapf(err, "could not read password from terminal")
+				}
+				creds.Password = string(password)
+			} else {
+				return err
+			}
+		}
+		registryCreds = creds
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not create runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	var writer io.Writer
+	if !c.Bool("quiet") {
+		writer = os.Stdout
+	}
+
+	options := libpod.CopyOptions{
+		Compression:         archive.Uncompressed,
+		SignaturePolicyPath: c.String("signature-policy"),
+		DockerRegistryOptions: common.DockerRegistryOptions{
+			DockerRegistryCreds:         registryCreds,
+			DockerCertPath:              certPath,
+			DockerInsecureSkipTLSVerify: skipVerify,
+		},
+		SigningOptions: common.SigningOptions{
+			RemoveSignatures: removeSignatures,
+			SignBy:           signBy,
+		},
+		AuthFile: c.String("authfile"),
+		Writer:   writer,
+	}
+
+	return runtime.PushImage(srcName, destName, options)
+}

cmd/kpod/rename.go

@@ -0,0 +1,49 @@
+package main
+
+import (
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	renameDescription = "Rename a container.  Container may be created, running, paused, or stopped"
+	renameFlags       = []cli.Flag{}
+	renameCommand     = cli.Command{
+		Name:        "rename",
+		Usage:       "rename a container",
+		Description: renameDescription,
+		Action:      renameCmd,
+		ArgsUsage:   "CONTAINER NEW-NAME",
+		Flags:       renameFlags,
+	}
+)
+
+func renameCmd(c *cli.Context) error {
+	if len(c.Args()) != 2 {
+		return errors.Errorf("Rename requires a src container name/ID and a dest container name")
+	}
+	if err := validateFlags(c, renameFlags); err != nil {
+		return err
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+
+	err = server.ContainerRename(c.Args().Get(0), c.Args().Get(1))
+	if err != nil {
+		return errors.Wrapf(err, "could not rename container")
+	}
+	return nil
+}

cmd/kpod/rm.go

@@ -0,0 +1,69 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+)
+
+var (
+	rmFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "force, f",
+			Usage: "Force removal of a running container.  The default is false",
+		},
+	}
+	rmDescription = "Remove one or more containers"
+	rmCommand     = cli.Command{
+		Name: "rm",
+		Usage: fmt.Sprintf(`kpod rm will remove one or more containers from the host.  The container name or ID can be used.
+							This does not remove images.  Running containers will not be removed without the -f option.`),
+		Description: rmDescription,
+		Flags:       rmFlags,
+		Action:      rmCmd,
+		ArgsUsage:   "",
+	}
+)
+
+// saveCmd saves the image to either docker-archive or oci
+func rmCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("specify one or more containers to remove")
+	}
+	if err := validateFlags(c, rmFlags); err != nil {
+		return err
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	force := c.Bool("force")
+
+	for _, container := range c.Args() {
+		id, err2 := server.Remove(context.Background(), container, force)
+		if err2 != nil {
+			if err == nil {
+				err = err2
+			} else {
+				err = errors.Wrapf(err, "%v.  Stop the container before attempting removal or use -f\n", err2)
+			}
+		} else {
+			fmt.Println(id)
+		}
+	}
+	return err
+}

cmd/kpod/rmi.go

@@ -0,0 +1,56 @@
+package main
+
+import (
+	"fmt"
+
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	rmiDescription = "removes one or more locally stored images."
+	rmiFlags       = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "force, f",
+			Usage: "force removal of the image",
+		},
+	}
+	rmiCommand = cli.Command{
+		Name:        "rmi",
+		Usage:       "removes one or more images from local storage",
+		Description: rmiDescription,
+		Action:      rmiCmd,
+		ArgsUsage:   "IMAGE-NAME-OR-ID [...]",
+		Flags:       rmiFlags,
+	}
+)
+
+func rmiCmd(c *cli.Context) error {
+	if err := validateFlags(c, rmiFlags); err != nil {
+		return err
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("image name or ID must be specified")
+	}
+
+	for _, arg := range args {
+		image, err := runtime.GetImage(arg)
+		if err != nil {
+			return errors.Wrapf(err, "could not get image %q", arg)
+		}
+		id, err := runtime.RemoveImage(image, c.Bool("force"))
+		if err != nil {
+			return errors.Wrapf(err, "error removing image %q", id)
+		}
+		fmt.Printf("%s\n", id)
+	}
+	return nil
+}

cmd/kpod/save.go

@@ -0,0 +1,98 @@
+package main
+
+import (
+	"io"
+	"os"
+
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/pkg/errors"
+	"github.com/sirupsen/logrus"
+	"github.com/urfave/cli"
+)
+
+var (
+	saveFlags = []cli.Flag{
+		cli.StringFlag{
+			Name:  "output, o",
+			Usage: "Write to a file, default is STDOUT",
+			Value: "/dev/stdout",
+		},
+		cli.BoolFlag{
+			Name:  "quiet, q",
+			Usage: "Suppress the output",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "Save image to oci-archive",
+		},
+	}
+	saveDescription = `
+	Save an image to docker-archive or oci-archive on the local machine.
+	Default is docker-archive`
+
+	saveCommand = cli.Command{
+		Name:        "save",
+		Usage:       "Save image to an archive",
+		Description: saveDescription,
+		Flags:       saveFlags,
+		Action:      saveCmd,
+		ArgsUsage:   "",
+	}
+)
+
+// saveCmd saves the image to either docker-archive or oci
+func saveCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("need at least 1 argument")
+	}
+	if err := validateFlags(c, saveFlags); err != nil {
+		return err
+	}
+
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not create runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	var writer io.Writer
+	if !c.Bool("quiet") {
+		writer = os.Stdout
+	}
+
+	output := c.String("output")
+	if output == "/dev/stdout" {
+		fi := os.Stdout
+		if logrus.IsTerminal(fi) {
+			return errors.Errorf("refusing to save to terminal. Use -o flag or redirect")
+		}
+	}
+
+	var dst string
+	switch c.String("format") {
+	case libpod.OCIArchive:
+		dst = libpod.OCIArchive + ":" + output
+	case libpod.DockerArchive:
+		fallthrough
+	case "":
+		dst = libpod.DockerArchive + ":" + output
+	default:
+		return errors.Errorf("unknown format option %q", c.String("format"))
+	}
+
+	saveOpts := libpod.CopyOptions{
+		SignaturePolicyPath: "",
+		Writer:              writer,
+	}
+
+	// only one image is supported for now
+	// future pull requests will fix this
+	for _, image := range args {
+		dest := dst + ":" + image
+		if err := runtime.PushImage(image, dest, saveOpts); err != nil {
+			return errors.Wrapf(err, "unable to save %q", image)
+		}
+	}
+	return nil
+}

cmd/kpod/stats.go

@@ -0,0 +1,245 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"strings"
+	"text/template"
+	"time"
+
+	"github.com/docker/go-units"
+
+	tm "github.com/buger/goterm"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/kubernetes-incubator/cri-o/oci"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var printf func(format string, a ...interface{}) (n int, err error)
+var println func(a ...interface{}) (n int, err error)
+
+type statsOutputParams struct {
+	Container string
+	ID        string
+	CPUPerc   string
+	MemUsage  string
+	MemPerc   string
+	NetIO     string
+	BlockIO   string
+	PIDs      uint64
+}
+
+var (
+	statsFlags = []cli.Flag{
+		cli.BoolFlag{
+			Name:  "all, a",
+			Usage: "show all containers. Only running containers are shown by default. The default is false",
+		},
+		cli.BoolFlag{
+			Name:  "no-stream",
+			Usage: "disable streaming stats and only pull the first result, default setting is false",
+		},
+		cli.StringFlag{
+			Name:  "format",
+			Usage: "pretty-print container statistics using a Go template",
+		},
+		cli.BoolFlag{
+			Name:  "json",
+			Usage: "output container statistics in json format",
+		},
+	}
+
+	statsDescription = "display a live stream of one or more containers' resource usage statistics"
+	statsCommand     = cli.Command{
+		Name:        "stats",
+		Usage:       "Display percentage of CPU, memory, network I/O, block I/O and PIDs for one or more containers",
+		Description: statsDescription,
+		Flags:       statsFlags,
+		Action:      statsCmd,
+		ArgsUsage:   "",
+	}
+)
+
+func statsCmd(c *cli.Context) error {
+	if err := validateFlags(c, statsFlags); err != nil {
+		return err
+	}
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not read config")
+	}
+	containerServer, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not create container server")
+	}
+	defer containerServer.Shutdown()
+	err = containerServer.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	times := -1
+	if c.Bool("no-stream") {
+		times = 1
+	}
+	statsChan := make(chan []*libkpod.ContainerStats)
+	// iterate over the channel until it is closed
+	go func() {
+		// print using goterm
+		printf = tm.Printf
+		println = tm.Println
+		for stats := range statsChan {
+			// Continually refresh statistics
+			tm.Clear()
+			tm.MoveCursor(1, 1)
+			outputStats(stats, c.String("format"), c.Bool("json"))
+			tm.Flush()
+			time.Sleep(time.Second)
+		}
+	}()
+	return getStats(containerServer, c.Args(), c.Bool("all"), statsChan, times)
+}
+
+func getStats(server *libkpod.ContainerServer, args []string, all bool, statsChan chan []*libkpod.ContainerStats, times int) error {
+	ctrs, err := server.ListContainers(isRunning, ctrInList(args))
+	if err != nil {
+		return err
+	}
+	containerStats := map[string]*libkpod.ContainerStats{}
+	for _, ctr := range ctrs {
+		initialStats, err := server.GetContainerStats(ctr, &libkpod.ContainerStats{})
+		if err != nil {
+			return err
+		}
+		containerStats[ctr.ID()] = initialStats
+	}
+	step := 1
+	if times == -1 {
+		times = 1
+		step = 0
+	}
+	for i := 0; i < times; i += step {
+		reportStats := []*libkpod.ContainerStats{}
+		for _, ctr := range ctrs {
+			id := ctr.ID()
+			if _, ok := containerStats[ctr.ID()]; !ok {
+				initialStats, err := server.GetContainerStats(ctr, &libkpod.ContainerStats{})
+				if err != nil {
+					return err
+				}
+				containerStats[id] = initialStats
+			}
+			stats, err := server.GetContainerStats(ctr, containerStats[id])
+			if err != nil {
+				return err
+			}
+			// replace the previous measurement with the current one
+			containerStats[id] = stats
+			reportStats = append(reportStats, stats)
+		}
+		statsChan <- reportStats
+
+		err := server.Update()
+		if err != nil {
+			return err
+		}
+		ctrs, err = server.ListContainers(isRunning, ctrInList(args))
+		if err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func outputStats(stats []*libkpod.ContainerStats, format string, json bool) error {
+	if format == "" {
+		outputStatsHeader()
+	}
+	if json {
+		return outputStatsAsJSON(stats)
+	}
+	var err error
+	for _, s := range stats {
+		if format == "" {
+			outputStatsUsingFormatString(s)
+		} else {
+			params := getStatsOutputParams(s)
+			err2 := outputStatsUsingTemplate(format, params)
+			if err2 != nil {
+				err = errors.Wrapf(err, err2.Error())
+			}
+		}
+	}
+	return err
+}
+
+func outputStatsHeader() {
+	printf("%-64s %-16s %-32s %-16s %-24s %-24s %s\n", "CONTAINER", "CPU %", "MEM USAGE / MEM LIMIT", "MEM %", "NET I/O", "BLOCK I/O", "PIDS")
+}
+
+func outputStatsUsingFormatString(stats *libkpod.ContainerStats) {
+	printf("%-64s %-16s %-32s %-16s %-24s %-24s %d\n", stats.Container, floatToPercentString(stats.CPU), combineHumanValues(stats.MemUsage, stats.MemLimit), floatToPercentString(stats.MemPerc), combineHumanValues(stats.NetInput, stats.NetOutput), combineHumanValues(stats.BlockInput, stats.BlockOutput), stats.PIDs)
+}
+
+func combineHumanValues(a, b uint64) string {
+	return fmt.Sprintf("%s / %s", units.HumanSize(float64(a)), units.HumanSize(float64(b)))
+}
+
+func floatToPercentString(f float64) string {
+	return fmt.Sprintf("%.2f %s", f, "%")
+}
+
+func getStatsOutputParams(stats *libkpod.ContainerStats) statsOutputParams {
+	return statsOutputParams{
+		Container: stats.Container,
+		ID:        stats.Container,
+		CPUPerc:   floatToPercentString(stats.CPU),
+		MemUsage:  combineHumanValues(stats.MemUsage, stats.MemLimit),
+		MemPerc:   floatToPercentString(stats.MemPerc),
+		NetIO:     combineHumanValues(stats.NetInput, stats.NetOutput),
+		BlockIO:   combineHumanValues(stats.BlockInput, stats.BlockOutput),
+		PIDs:      stats.PIDs,
+	}
+}
+
+func outputStatsUsingTemplate(format string, params statsOutputParams) error {
+	tmpl, err := template.New("stats").Parse(format)
+	if err != nil {
+		return errors.Wrapf(err, "template parsing error")
+	}
+
+	err = tmpl.Execute(os.Stdout, params)
+	if err != nil {
+		return err
+	}
+	println()
+	return nil
+}
+
+func outputStatsAsJSON(stats []*libkpod.ContainerStats) error {
+	s, err := json.Marshal(stats)
+	if err != nil {
+		return err
+	}
+	println(s)
+	return nil
+}
+
+func isRunning(ctr *oci.Container) bool {
+	return ctr.State().Status == "running"
+}
+
+func ctrInList(idsOrNames []string) func(ctr *oci.Container) bool {
+	if len(idsOrNames) == 0 {
+		return func(*oci.Container) bool { return true }
+	}
+	return func(ctr *oci.Container) bool {
+		for _, idOrName := range idsOrNames {
+			if strings.HasPrefix(ctr.ID(), idOrName) || strings.HasSuffix(ctr.Name(), idOrName) {
+				return true
+			}
+		}
+		return false
+	}
+}

cmd/kpod/stop.go

@@ -0,0 +1,77 @@
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"golang.org/x/net/context"
+)
+
+var (
+	defaultTimeout int64 = 10
+	stopFlags            = []cli.Flag{
+		cli.Int64Flag{
+			Name:  "timeout, t",
+			Usage: "Seconds to wait for stop before killing the container",
+			Value: defaultTimeout,
+		},
+	}
+	stopDescription = `
+   kpod stop
+
+   Stops one or more running containers.  The container name or ID can be used.
+   A timeout to forcibly stop the container can also be set but defaults to 10
+   seconds otherwise.
+`
+
+	stopCommand = cli.Command{
+		Name:        "stop",
+		Usage:       "Stop one or more containers",
+		Description: stopDescription,
+		Flags:       stopFlags,
+		Action:      stopCmd,
+		ArgsUsage:   "CONTAINER-NAME [CONTAINER-NAME ...]",
+	}
+)
+
+func stopCmd(c *cli.Context) error {
+	args := c.Args()
+	stopTimeout := c.Int64("timeout")
+	if len(args) < 1 {
+		return errors.Errorf("you must provide at least one container name or id")
+	}
+	if err := validateFlags(c, stopFlags); err != nil {
+		return err
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	var lastError error
+	for _, container := range c.Args() {
+		cid, err := server.ContainerStop(context.Background(), container, stopTimeout)
+		if err != nil {
+			if lastError != nil {
+				fmt.Fprintln(os.Stderr, lastError)
+			}
+			lastError = errors.Wrapf(err, "failed to stop container %v", container)
+		} else {
+			fmt.Println(cid)
+		}
+	}
+
+	return lastError
+}

cmd/kpod/tag.go

@@ -0,0 +1,77 @@
+package main
+
+import (
+	"github.com/containers/image/docker/reference"
+	"github.com/containers/storage"
+	"github.com/kubernetes-incubator/cri-o/libpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	tagDescription = "Adds one or more additional names to locally-stored image"
+	tagCommand     = cli.Command{
+		Name:        "tag",
+		Usage:       "Add an additional name to a local image",
+		Description: tagDescription,
+		Action:      tagCmd,
+		ArgsUsage:   "IMAGE-NAME [IMAGE-NAME ...]",
+	}
+)
+
+func tagCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) < 2 {
+		return errors.Errorf("image name and at least one new name must be specified")
+	}
+	runtime, err := getRuntime(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not create runtime")
+	}
+	defer runtime.Shutdown(false)
+
+	img, err := runtime.GetImage(args[0])
+	if err != nil {
+		return err
+	}
+	if img == nil {
+		return errors.New("null image")
+	}
+	err = addImageNames(runtime, img, args[1:])
+	if err != nil {
+		return errors.Wrapf(err, "error adding names %v to image %q", args[1:], args[0])
+	}
+	return nil
+}
+
+func addImageNames(runtime *libpod.Runtime, image *storage.Image, addNames []string) error {
+	// Add tags to the names if applicable
+	names, err := expandedTags(addNames)
+	if err != nil {
+		return err
+	}
+	for _, name := range names {
+		if err := runtime.TagImage(image, name); err != nil {
+			return errors.Wrapf(err, "error adding name (%v) to image %q", name, image.ID)
+		}
+	}
+	return nil
+}
+
+func expandedTags(tags []string) ([]string, error) {
+	expandedNames := []string{}
+	for _, tag := range tags {
+		var labelName string
+		name, err := reference.Parse(tag)
+		if err != nil {
+			return nil, errors.Wrapf(err, "error parsing tag %q", name)
+		}
+		if _, ok := name.(reference.NamedTagged); ok {
+			labelName = name.String()
+		} else {
+			labelName = name.String() + ":latest"
+		}
+		expandedNames = append(expandedNames, labelName)
+	}
+	return expandedNames, nil
+}

cmd/kpod/umount.go

@@ -0,0 +1,41 @@
+package main
+
+import (
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	umountCommand = cli.Command{
+		Name:        "umount",
+		Aliases:     []string{"unmount"},
+		Usage:       "Unmount a working container's root filesystem",
+		Description: "Unmounts a working container's root filesystem",
+		Action:      umountCmd,
+		ArgsUsage:   "CONTAINER-NAME-OR-ID",
+	}
+)
+
+func umountCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) == 0 {
+		return errors.Errorf("container ID must be specified")
+	}
+	if len(args) > 1 {
+		return errors.Errorf("too many arguments specified")
+	}
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "Could not get config")
+	}
+	store, err := getStore(config)
+	if err != nil {
+		return err
+	}
+
+	err = store.Unmount(args[0])
+	if err != nil {
+		return errors.Wrapf(err, "error unmounting container %q", args[0])
+	}
+	return nil
+}

cmd/kpod/unpause.go

@@ -0,0 +1,58 @@
+package main
+
+import (
+	"fmt"
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+	"os"
+)
+
+var (
+	unpauseDescription = `
+   kpod unpause
+
+   Unpauses one or more running containers.  The container name or ID can be used.
+`
+	unpauseCommand = cli.Command{
+		Name:        "unpause",
+		Usage:       "Unpause the processes in one or more containers",
+		Description: unpauseDescription,
+		Action:      unpauseCmd,
+		ArgsUsage:   "CONTAINER-NAME [CONTAINER-NAME ...]",
+	}
+)
+
+func unpauseCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) < 1 {
+		return errors.Errorf("you must provide at least one container name or id")
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	if err := server.Update(); err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+	var lastError error
+	for _, container := range c.Args() {
+		cid, err := server.ContainerUnpause(container)
+		if err != nil {
+			if lastError != nil {
+				fmt.Fprintln(os.Stderr, lastError)
+			}
+			lastError = errors.Wrapf(err, "failed to unpause container %v", container)
+		} else {
+			fmt.Println(cid)
+		}
+	}
+
+	return lastError
+}

cmd/kpod/version.go

@@ -0,0 +1,48 @@
+package main
+
+import (
+	"fmt"
+	"runtime"
+	"strconv"
+	"time"
+
+	"github.com/urfave/cli"
+)
+
+// Overwritten at build time
+var (
+	// gitCommit is the commit that the binary is being built from.
+	// It will be populated by the Makefile.
+	gitCommit string
+	// buildInfo is the time at which the binary was built
+	// It will be populated by the Makefile.
+	buildInfo string
+)
+
+// versionCmd gets and prints version info for version command
+func versionCmd(c *cli.Context) error {
+	fmt.Println("Version:      ", c.App.Version)
+	fmt.Println("Go Version:   ", runtime.Version())
+	if gitCommit != "" {
+		fmt.Println("Git Commit:   ", gitCommit)
+	}
+	if buildInfo != "" {
+		// Converts unix time from string to int64
+		buildTime, err := strconv.ParseInt(buildInfo, 10, 64)
+		if err != nil {
+			return err
+		}
+		// Prints out the build time in readable format
+		fmt.Println("Built:        ", time.Unix(buildTime, 0).Format(time.ANSIC))
+	}
+	fmt.Println("OS/Arch:      ", runtime.GOOS+"/"+runtime.GOARCH)
+
+	return nil
+}
+
+// Cli command to print out the full version of kpod
+var versionCommand = cli.Command{
+	Name:   "version",
+	Usage:  "Display the KPOD Version Information",
+	Action: versionCmd,
+}

cmd/kpod/wait.go

@@ -0,0 +1,62 @@
+package main
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/kubernetes-incubator/cri-o/libkpod"
+	"github.com/pkg/errors"
+	"github.com/urfave/cli"
+)
+
+var (
+	waitDescription = `
+	kpod wait
+
+	Block until one or more containers stop and then print their exit codes
+`
+
+	waitCommand = cli.Command{
+		Name:        "wait",
+		Usage:       "Block on one or more containers",
+		Description: waitDescription,
+		Action:      waitCmd,
+		ArgsUsage:   "CONTAINER-NAME [CONTAINER-NAME ...]",
+	}
+)
+
+func waitCmd(c *cli.Context) error {
+	args := c.Args()
+	if len(args) < 1 {
+		return errors.Errorf("you must provide at least one container name or id")
+	}
+
+	config, err := getConfig(c)
+	if err != nil {
+		return errors.Wrapf(err, "could not get config")
+	}
+	server, err := libkpod.New(config)
+	if err != nil {
+		return errors.Wrapf(err, "could not get container server")
+	}
+	defer server.Shutdown()
+	err = server.Update()
+	if err != nil {
+		return errors.Wrapf(err, "could not update list of containers")
+	}
+
+	var lastError error
+	for _, container := range c.Args() {
+		returnCode, err := server.ContainerWait(container)
+		if err != nil {
+			if lastError != nil {
+				fmt.Fprintln(os.Stderr, lastError)
+			}
+			lastError = errors.Wrapf(err, "failed to wait for the container %v", container)
+		} else {
+			fmt.Println(returnCode)
+		}
+	}
+
+	return lastError
+}

code-of-conduct.md

@@ -0,0 +1,55 @@
+## Kubernetes Community Code of Conduct
+
+### Contributor Code of Conduct
+
+As contributors and maintainers of this project, and in the interest of fostering
+an open and welcoming community, we pledge to respect all people who contribute
+through reporting issues, posting feature requests, updating documentation,
+submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free experience for
+everyone, regardless of level of experience, gender, gender identity and expression,
+sexual orientation, disability, personal appearance, body size, race, ethnicity, age,
+religion, or nationality.
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery.
+* Personal attacks.
+* Trolling or insulting/derogatory comments.
+* Public or private harassment.
+* Publishing other's private information, such as physical or electronic addresses,
+ without explicit permission.
+* Other unethical or unprofessional conduct.
+
+Project maintainers have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are not
+aligned to this Code of Conduct.  By adopting this Code of Conduct, project maintainers
+commit themselves to fairly and consistently applying these principles to every aspect
+of managing this project.  Project maintainers who do not follow or enforce the Code of
+Conduct may be permanently removed from the project team.
+
+This code of conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting a Kubernetes maintainer, Sarah Novotny <sarahnovotny@google.com>, and/or Dan Kohn <dan@linuxfoundation.org>.
+
+This Code of Conduct is adapted from the Contributor Covenant
+(http://contributor-covenant.org), version 1.2.0, available at
+http://contributor-covenant.org/version/1/2/0/
+
+### Kubernetes Events Code of Conduct
+
+Kubernetes events are working conferences intended for professional networking and collaboration in the
+Kubernetes community.  Attendees are expected to behave according to professional standards and in accordance
+with their employer's policies on appropriate workplace behavior.
+
+While at Kubernetes events or related social networking opportunities, attendees should not engage in
+discriminatory or offensive speech or actions regarding gender, sexuality, race, or religion.  Speakers should
+be especially aware of these concerns.
+
+The Kubernetes team does not condone any statements by speakers contrary to these standards.  The Kubernetes
+team reserves the right to deny entrance and/or eject from an event (without refund) any individual found to
+be engaging in discriminatory or offensive speech or actions.
+
+Please bring any concerns to the immediate attention of the Kubernetes event staff.

completions/bash/kpod

@@ -0,0 +1,564 @@
+#! /bin/bash
+
+: ${PROG:=$(basename ${BASH_SOURCE})}
+
+__kpod_list_images() {
+    COMPREPLY=($(compgen -W "$(kpod images -q)" -- $cur))
+}
+
+__kpod_list_containers() {
+    COMPREPLY=($(compgen -W "$(kpod ps -aq)" -- $cur))
+}
+
+_kpod_diff() {
+    local options_with_args="
+     --format
+     "
+    local boolean_options="
+     "
+    _complete_ "$options_with_args" "$boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_images
+            ;;
+    esac
+}
+
+_kpod_export() {
+    local options_with_args="
+     --output
+     -o
+     "
+    local boolean_options="
+     "
+    _complete_ "$options_with_args" "$boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_images
+            ;;
+    esac
+}
+
+_kpod_history() {
+    local options_with_args="
+     --format
+     "
+    local boolean_options="
+     --human -H
+     --no-trunc
+     --quiet -q
+     "
+    _complete_ "$options_with_args" "$boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_images
+            ;;
+    esac
+}
+
+_kpod_info() {
+    local boolean_options="
+     --help
+     -h
+     --debug
+     "
+    local options_with_args="
+    --format
+  "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_images
+            ;;
+    esac
+}
+
+_kpod_images() {
+    local boolean_options="
+     --help
+     -h
+     --quiet
+     -q
+     --noheading
+     -n
+     --no-trunc
+     --digests
+     --filter
+     -f
+     "
+    local options_with_args="
+    --format
+  "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+    esac
+}
+
+_kpod_inspect() {
+    local boolean_options="
+     --help
+     -h
+     "
+    local options_with_args="
+    --format
+     -f
+     --type
+     -t
+     --size
+  "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+    esac
+}
+_kpod_kill() {
+     local options_with_args="
+     --signal -s
+     "
+     local boolean_options="
+     --help
+     -h"
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_logs() {
+     local options_with_args="
+     --since
+     --tail
+     "
+     local boolean_options="
+     --follow
+     -f
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_containers
+            ;;
+    esac
+}
+
+_kpod_pull() {
+    local options_with_args="
+    --authfile
+    --creds
+    --cert-dir
+    --signature-policy
+    "
+    local boolean_options="
+    --all-tags -a
+    --quiet
+    -q
+    --tls-verify
+    "
+    _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_unmount() {
+    _kpod_umount $@
+}
+
+_kpod_umount() {
+    local boolean_options="
+    --help
+    -h
+ "
+    local options_with_args="
+ "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+    esac
+}
+
+_kpod_mount() {
+    local boolean_options="
+    --help
+    -h
+    --notruncate
+ "
+
+    local options_with_args="
+    --label
+    --format
+ "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+    esac
+}
+
+_kpod_push() {
+    local boolean_options="
+    --disable-compression
+    -D
+    --quiet
+    -q
+    --remove-signatures
+    --tls-verify
+  "
+
+    local options_with_args="
+    --authfile
+    --cert-dir
+    --creds
+    --sign-by
+    --signature-policy
+  "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+    esac
+}
+
+_kpod_rename() {
+    local boolean_options="
+    --help
+    -h
+ "
+    local options_with_args="
+ "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_containers
+            ;;
+    esac
+}
+
+_kpod_rm() {
+    local boolean_options="
+    --force
+    -f
+    "
+
+    local options_with_args="
+    "
+
+    local all_options="$options_with_args $boolean_options"
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_containers
+            ;;
+    esac
+}
+
+_kpod_rmi() {
+    local boolean_options="
+     --help
+     -h
+     --force
+     -f
+  "
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_images
+            ;;
+    esac
+}
+
+_kpod_stats() {
+    local boolean_options="
+     --help
+     --all
+     -a
+     --no-stream
+     --format
+    "
+
+    case "$cur" in
+        -*)
+            COMPREPLY=($(compgen -W "$boolean_options $options_with_args" -- "$cur"))
+            ;;
+        *)
+            __kpod_list_containers
+            ;;
+    esac
+}
+
+kpod_tag() {
+    local options_with_args="
+    "
+    local boolean_options="
+    "
+    _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_version() {
+     local options_with_args="
+     "
+     local boolean_options="
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_save() {
+     local options_with_args="
+     --output -o
+     --format
+     "
+     local boolean_options="
+     --quiet -q
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_export() {
+     local options_with_args="
+     --output -o
+     "
+     local boolean_options="
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_pause() {
+     local options_with_args="
+     --help -h
+     "
+     local boolean_options=""
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_ps() {
+     local options_with_args="
+     --filter -f
+     --format
+     --last -n
+     "
+     local boolean_options="
+     --all -a
+     --latest -l
+     --no-trunc
+     --quiet -q
+     --size -s
+     --namespace --ns
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_stop() {
+     local options_with_args="
+     --timeout -t
+     "
+     local boolean_options=""
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_unpause() {
+     local options_with_args="
+     --help -h
+     "
+     local boolean_options=""
+     _complete_ "$options_with_args" "$boolean_options"
+
+_kpod_wait() {
+     local options_with_args=""
+     local boolean_options="--help -h"
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_complete_() {
+    local options_with_args=$1
+    local boolean_options="$2 -h --help"
+
+    case "$prev" in
+	$options_with_args)
+	    return
+	    ;;
+    esac
+
+    case "$cur" in
+	-*)
+	    COMPREPLY=( $( compgen -W "$boolean_options $options_with_args" -- "$cur" ) )
+	    ;;
+    esac
+}
+
+_kpod_load() {
+    local options_with_args="
+    --input -i
+    --signature-policy
+    "
+    local boolean_options="
+    --quiet -q
+    "
+    _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_login() {
+     local options_with_args="
+     --username
+     -u
+     --password
+     -p
+     --authfile
+     "
+     local boolean_options="
+     --help
+     -h
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_logout() {
+     local options_with_args="
+     --authfile
+     "
+     local boolean_options="
+     --all
+     -a
+     --help
+     -h
+     "
+     _complete_ "$options_with_args" "$boolean_options"
+}
+
+_kpod_kpod() {
+     local options_with_args="
+           --config -c
+           --root
+           --runroot
+           --storage-driver
+           --storage-opt
+           --log-level
+    "
+     local boolean_options="
+           --help -h
+           --version -v
+     "
+     commands="
+    diff
+    export
+    history
+    images
+    info
+    inspect
+    kill
+    load
+    login
+    logout
+    logs
+    mount
+    pause
+    ps
+    pull
+    push
+    rename
+    rm
+    rmi
+    save
+    stats
+    stop
+    tag
+    umount
+    unmount
+    unpause
+    version
+    wait
+     "
+
+     case "$prev" in
+	 $main_options_with_args_glob )
+	     return
+	     ;;
+     esac
+
+     case "$cur" in
+	 -*)
+	     COMPREPLY=( $( compgen -W "$boolean_options $options_with_args" -- "$cur" ) )
+	     ;;
+	 *)
+	     COMPREPLY=( $( compgen -W "${commands[*]} help" -- "$cur" ) )
+	     ;;
+     esac
+}
+
+_cli_bash_autocomplete() {
+     local cur opts base
+
+     COMPREPLY=()
+     cur="${COMP_WORDS[COMP_CWORD]}"
+     COMPREPLY=()
+     local cur prev words cword
+
+     _get_comp_words_by_ref -n : cur prev words cword
+
+     local command=${PROG} cpos=0
+     local counter=1
+     counter=1
+     while [ $counter -lt $cword ]; do
+	 case "!${words[$counter]}" in
+	     *)
+		 command=$(echo "${words[$counter]}" | sed 's/-/_/g')
+		 cpos=$counter
+		 (( cpos++ ))
+		 break
+		 ;;
+	 esac
+	 (( counter++ ))
+     done
+
+     local completions_func=_kpod_${command}
+     declare -F $completions_func >/dev/null && $completions_func
+
+     eval "$previous_extglob_setting"
+     return 0
+}
+
+complete -F _cli_bash_autocomplete $PROG

conmon/Makefile

@@ -0,0 +1,12 @@
+src = $(wildcard *.c)
+obj = $(src:.c=.o)
+
+override LIBS += $(shell pkg-config --libs glib-2.0)
+override CFLAGS += -std=c99 -Os -Wall -Wextra $(shell pkg-config --cflags glib-2.0)
+
+conmon: $(obj)
+	$(CC) -o ../bin/$@ $^ $(CFLAGS) $(LIBS)
+
+.PHONY: clean
+clean:
+	rm -f $(obj) ../bin/conmon

conmon/cmsg.c

@@ -0,0 +1,149 @@
+/*
+ * Copyright 2016 SUSE LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* NOTE: This code comes directly from runc/libcontainer/utils/cmsg.c. */
+
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/socket.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "cmsg.h"
+
+#define error(fmt, ...)							\
+	({								\
+		fprintf(stderr, "nsenter: " fmt ": %m\n", ##__VA_ARGS__); \
+		errno = ECOMM;						\
+		goto err; /* return value */				\
+	})
+
+/*
+ * Sends a file descriptor along the sockfd provided. Returns the return
+ * value of sendmsg(2). Any synchronisation and preparation of state
+ * should be done external to this (we expect the other side to be in
+ * recvfd() in the code).
+ */
+ssize_t sendfd(int sockfd, struct file_t file)
+{
+	struct msghdr msg = {0};
+	struct iovec iov[1] = {0};
+	struct cmsghdr *cmsg;
+	int *fdptr;
+
+	union {
+		char buf[CMSG_SPACE(sizeof(file.fd))];
+		struct cmsghdr align;
+	} u;
+
+	/*
+	 * We need to send some other data along with the ancillary data,
+	 * otherwise the other side won't recieve any data. This is very
+	 * well-hidden in the documentation (and only applies to
+	 * SOCK_STREAM). See the bottom part of unix(7).
+	 */
+	iov[0].iov_base = file.name;
+	iov[0].iov_len = strlen(file.name) + 1;
+
+	msg.msg_name = NULL;
+	msg.msg_namelen = 0;
+	msg.msg_iov = iov;
+	msg.msg_iovlen = 1;
+	msg.msg_control = u.buf;
+	msg.msg_controllen = sizeof(u.buf);
+
+	cmsg = CMSG_FIRSTHDR(&msg);
+	cmsg->cmsg_level = SOL_SOCKET;
+	cmsg->cmsg_type = SCM_RIGHTS;
+	cmsg->cmsg_len = CMSG_LEN(sizeof(int));
+
+	fdptr = (int *) CMSG_DATA(cmsg);
+	memcpy(fdptr, &file.fd, sizeof(int));
+
+	return sendmsg(sockfd, &msg, 0);
+}
+
+/*
+ * Receives a file descriptor from the sockfd provided. Returns the file
+ * descriptor as sent from sendfd(). It will return the file descriptor
+ * or die (literally) trying. Any synchronisation and preparation of
+ * state should be done external to this (we expect the other side to be
+ * in sendfd() in the code).
+ */
+struct file_t recvfd(int sockfd)
+{
+	struct msghdr msg = {0};
+	struct iovec iov[1] = {0};
+	struct cmsghdr *cmsg;
+	struct file_t file = {0};
+	int *fdptr;
+	int olderrno;
+
+	union {
+		char buf[CMSG_SPACE(sizeof(file.fd))];
+		struct cmsghdr align;
+	} u;
+
+	/* Allocate a buffer. */
+	/* TODO: Make this dynamic with MSG_PEEK. */
+	file.name = malloc(TAG_BUFFER);
+	if (!file.name)
+		error("recvfd: failed to allocate file.tag buffer\n");
+
+	/*
+	 * We need to "recieve" the non-ancillary data even though we don't
+	 * plan to use it at all. Otherwise, things won't work as expected.
+	 * See unix(7) and other well-hidden documentation.
+	 */
+	iov[0].iov_base = file.name;
+	iov[0].iov_len = TAG_BUFFER;
+
+	msg.msg_name = NULL;
+	msg.msg_namelen = 0;
+	msg.msg_iov = iov;
+	msg.msg_iovlen = 1;
+	msg.msg_control = u.buf;
+	msg.msg_controllen = sizeof(u.buf);
+
+	ssize_t ret = recvmsg(sockfd, &msg, 0);
+	if (ret < 0)
+		goto err;
+
+	cmsg = CMSG_FIRSTHDR(&msg);
+	if (!cmsg)
+		error("recvfd: got NULL from CMSG_FIRSTHDR");
+	if (cmsg->cmsg_level != SOL_SOCKET)
+		error("recvfd: expected SOL_SOCKET in cmsg: %d", cmsg->cmsg_level);
+	if (cmsg->cmsg_type != SCM_RIGHTS)
+		error("recvfd: expected SCM_RIGHTS in cmsg: %d", cmsg->cmsg_type);
+	if (cmsg->cmsg_len != CMSG_LEN(sizeof(int)))
+		error("recvfd: expected correct CMSG_LEN in cmsg: %lu", cmsg->cmsg_len);
+
+	fdptr = (int *) CMSG_DATA(cmsg);
+	if (!fdptr || *fdptr < 0)
+		error("recvfd: recieved invalid pointer");
+
+	file.fd = *fdptr;
+	return file;
+
+err:
+	olderrno = errno;
+	free(file.name);
+	errno = olderrno;
+	return (struct file_t){0};
+}

conmon/cmsg.h

@@ -0,0 +1,38 @@
+/*
+ * Copyright 2016 SUSE LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/* NOTE: This code comes directly from runc/libcontainer/utils/cmsg.h. */
+
+#pragma once
+
+#if !defined(CMSG_H)
+#define CMSG_H
+
+#include <sys/types.h>
+
+/* TODO: Implement this properly with MSG_PEEK. */
+#define TAG_BUFFER 4096
+
+/* This mirrors Go's (*os.File). */
+struct file_t {
+	char *name;
+	int fd;
+};
+
+struct file_t recvfd(int sockfd);
+ssize_t sendfd(int sockfd, struct file_t file);
+
+#endif /* !defined(CMSG_H) */

contrib/cni/10-crio-bridge.conf

@@ -0,0 +1,15 @@
+{
+    "cniVersion": "0.2.0",
+    "name": "crio-bridge",
+    "type": "bridge",
+    "bridge": "cni0",
+    "isGateway": true,
+    "ipMasq": true,
+    "ipam": {
+        "type": "host-local",
+        "subnet": "10.88.0.0/16",
+        "routes": [
+            { "dst": "0.0.0.0/0" }
+        ]
+    }
+}

contrib/cni/99-loopback.conf

@@ -0,0 +1,4 @@
+{
+    "cniVersion": "0.2.0",
+    "type": "loopback"
+}

contrib/cni/README.md

@@ -0,0 +1,16 @@
+## `contrib/cni` ##
+
+There are a wide variety of different [CNI][cni] network configurations. This
+directory just contains some example configurations that can be used as the
+basis for your own configurations (distributions should package these files in
+example directories).
+
+To use these configurations, place them in `/etc/cni/net.d` (or the directory
+specified by `crio.network.network_dir` in your `crio.conf`).
+
+In addition, you need to install the [CNI plugins][cni] necessary into
+`/opt/cni/bin` (or the directory specified by `crio.network.plugin_dir`). The
+two plugins necessary for the example CNI configurations are `loopback` and
+`bridge`.
+
+[cni]: https://github.com/containernetworking/plugins

contrib/rpm/Makefile

@@ -0,0 +1,14 @@
+.PHONY: dist
+dist: crio.spec
+	spectool -g crio.spec
+
+.PHONY: rpm
+rpm: dist
+	rpmbuild --define "_sourcedir `pwd`" --define "_specdir `pwd`" \
+	--define "_rpmdir `pwd`" --define "_srcrpmdir `pwd`" -ba crio.spec
+
+all: rpm
+
+clean:
+	rm -f *rpm *gz
+	rm -rf x86_64

contrib/rpm/crio.spec

@@ -0,0 +1,76 @@
+%define debug_package %{nil}
+%global provider        github
+%global provider_tld    com
+%global project         kubernetes-incubator
+%global repo            cri-o
+%global Name            crio
+# https://github.com/kubernetes-incubator/cri-o
+%global provider_prefix %{provider}.%{provider_tld}/%{project}/%{repo}
+%global import_path     %{provider_prefix}
+%global commit          8ba639952a95f2e24cc98987689138b67545576c
+%global shortcommit     %(c=%{commit}; echo ${c:0:7})
+
+Name:           %{Name}
+Version:        0.0.1
+Release:        1.git%{shortcommit}%{?dist}
+Summary:        Kubelet Container Runtime Interface (CRI) for OCI runtimes.
+Group:          Applications/Text
+License:        Apache 2.0
+URL:            https://%{provider_prefix}
+Source0:        https://%{provider_prefix}/archive/%{commit}/%{repo}-%{shortcommit}.tar.gz
+Provides:       %{repo}
+
+BuildRequires:  golang-github-cpuguy83-go-md2man
+
+%description
+The crio package provides an implementation of the
+Kubelet Container Runtime Interface (CRI) using OCI conformant runtimes.
+
+crio provides following functionalities:
+
+    Support multiple image formats including the existing Docker image format
+    Support for multiple means to download images including trust & image verification
+    Container image management (managing image layers, overlay filesystems, etc)
+    Container process lifecycle management
+    Monitoring and logging required to satisfy the CRI
+    Resource isolation as required by the CRI
+
+%prep
+%setup -q -n %{repo}-%{commit}
+
+%build
+make all
+
+%install
+%make_install
+%make_install install.systemd
+
+#define license tag if not already defined
+%{!?_licensedir:%global license %doc}
+%files
+%{_bindir}/crio
+%{_bindir}/crioctl
+%{_mandir}/man5/crio.conf.5*
+%{_mandir}/man8/crio.8*
+%{_sysconfdir}/crio.conf
+%{_sysconfdir}/seccomp.json
+%dir /%{_libexecdir}/crio
+/%{_libexecdir}/crio/conmon
+/%{_libexecdir}/crio/pause
+%{_unitdir}/crio.service
+%doc README.md
+%license LICENSE
+%dir /usr/share/oci-umount/oci-umount.d
+/usr/share/oci-umount/oci-umount.d/cri-umount.conf
+
+
+%preun
+%systemd_preun %{Name}
+
+%postun
+%systemd_postun_with_restart %{Name}
+
+%changelog
+* Mon Oct 31 2016 Dan Walsh <dwalsh@redhat.com> - 0.0.1
+- Initial RPM release
+

contrib/systemd/crio-shutdown.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=Shutdown CRIO containers before shutting down the system
+Wants=crio.service
+After=crio.service
+Documentation=man:crio(8)
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/true
+ExecStop=mkdir -p /var/lib/crio; touch /var/lib/crio/crio.shutdown
+RemainAfterExit=yes
+
+[Install]
+WantedBy=multi-user.target

contrib/systemd/crio.service

@@ -0,0 +1,24 @@
+[Unit]
+Description=Open Container Initiative Daemon
+Documentation=https://github.com/kubernetes-incubator/cri-o
+After=network-online.target
+
+[Service]
+Type=notify
+EnvironmentFile=-/etc/sysconfig/crio-storage
+EnvironmentFile=-/etc/sysconfig/crio-network
+Environment=GOTRACEBACK=crash
+ExecStart=/usr/local/bin/crio \
+          $CRIO_STORAGE_OPTIONS \
+          $CRIO_NETWORK_OPTIONS
+ExecReload=/bin/kill -s HUP $MAINPID
+TasksMax=infinity
+LimitNOFILE=1048576
+LimitNPROC=1048576
+LimitCORE=infinity
+OOMScoreAdjust=-999
+TimeoutStartSec=0
+Restart=on-abnormal
+
+[Install]
+WantedBy=multi-user.target

contrib/test/integration/README.md

@@ -0,0 +1,21 @@
+# Fedora and RHEL Integration and End-to-End Tests
+
+This directory contains playbooks to set up for and run the integration and
+end-to-end tests for CRI-O on RHEL and Fedora hosts. Two entrypoints exist:
+
+ - `main.yml`: sets up the machine and runs tests
+ - `results.yml`: gathers test output to `/tmp/artifacts`
+
+When running `main.yml`, three tags are present:
+
+ - `setup`: run all tasks to set up the system for testing
+ - `e2e`: build CRI-O from source and run Kubernetes node E2Es
+ - `integration`: build CRI-O from source and run the local integration suite
+
+The playbooks assume the following things about your system:
+
+ - on RHEL, the server and extras repos are configured and certs are present
+ - `ansible` is installed and the host is boot-strapped to allow `ansible` to run against it
+ - the `$GOPATH` is set and present for all shells (*e.g.* written in `/etc/environment`)
+ - CRI-O is checked out to the correct state at `${GOPATH}/src/github.com/kubernetes-incubator/cri-o`
+ - the user running the playbook has access to passwordless `sudo`

contrib/test/integration/ansible.cfg

@@ -0,0 +1,359 @@
+# config file for ansible -- http://ansible.com/
+# ==============================================
+
+# nearly all parameters can be overridden in ansible-playbook
+# or with command line flags. ansible will read ANSIBLE_CONFIG,
+# ansible.cfg in the current working directory, .ansible.cfg in
+# the home directory or /etc/ansible/ansible.cfg, whichever it
+# finds first
+
+[defaults]
+
+# some basic default values...
+
+#inventory       = inventory
+#library        = /usr/share/my_modules/
+#remote_tmp     = $HOME/.ansible/tmp
+#local_tmp      = .ansible/tmp
+#forks          = 5
+forks           = 10
+#poll_interval  = 15
+#sudo_user      = root
+#ask_sudo_pass = True
+ask_sudo_pass = False
+#ask_pass      = True
+ask_pass      = False
+#transport      = smart
+#remote_port    = 22
+#module_lang    = C
+#module_set_locale = True
+
+# plays will gather facts by default, which contain information about
+# the remote system.
+#
+# smart - gather by default, but don't regather if already gathered
+# implicit - gather by default, turn off with gather_facts: False
+# explicit - do not gather by default, must say gather_facts: True
+#gathering = implicit
+gathering = smart
+
+# by default retrieve all facts subsets
+# all - gather all subsets
+# network - gather min and network facts
+# hardware - gather hardware facts (longest facts to retrieve)
+# virtual - gather min and virtual facts
+# facter - import facts from facter
+# ohai - import facts from ohai
+# You can combine them using comma (ex: network,virtual)
+# You can negate them using ! (ex: !hardware,!facter,!ohai)
+# A minimal set of facts is always gathered.
+gather_subset = network
+
+# additional paths to search for roles in, colon separated
+# N/B: This depends on how ansible is called
+#roles_path    = $WORKSPACE/kommandir_workspace/roles
+
+# uncomment this to disable SSH key host checking
+#host_key_checking = False
+host_key_checking = False
+
+# change the default callback
+#stdout_callback = skippy
+# enable additional callbacks
+#callback_whitelist = timer, mail
+
+# Determine whether includes in tasks and handlers are "static" by
+# default. As of 2.0, includes are dynamic by default. Setting these
+# values to True will make includes behave more like they did in the
+# 1.x versions.
+task_includes_static = True
+handler_includes_static = True
+
+# change this for alternative sudo implementations
+#sudo_exe = sudo
+
+# What flags to pass to sudo
+# WARNING: leaving out the defaults might create unexpected behaviours
+#sudo_flags = -H -S -n
+
+# SSH timeout
+#timeout = 10
+
+# default user to use for playbooks if user is not specified
+# (/usr/bin/ansible will use current user as default)
+#remote_user = root
+remote_user = root
+
+# logging is off by default unless this path is defined
+# if so defined, consider logrotate
+log_path = $ARTIFACTS/main.log
+
+# default module name for /usr/bin/ansible
+#module_name = command
+
+# use this shell for commands executed under sudo
+# you may need to change this to bin/bash in rare instances
+# if sudo is constrained
+# executable = /bin/sh
+
+# if inventory variables overlap, does the higher precedence one win
+# or are hash values merged together?  The default is 'replace' but
+# this can also be set to 'merge'.
+hash_behaviour = replace
+
+# by default, variables from roles will be visible in the global variable
+# scope. To prevent this, the following option can be enabled, and only
+# tasks and handlers within the role will see the variables there
+private_role_vars = False
+
+# list any Jinja2 extensions to enable here:
+#jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n
+
+# if set, always use this private key file for authentication, same as
+# if passing --private-key to ansible or ansible-playbook
+#private_key_file = /path/to/file
+
+# If set, configures the path to the Vault password file as an alternative to
+# specifying --vault-password-file on the command line.
+#vault_password_file = /path/to/vault_password_file
+
+# format of string {{ ansible_managed }} available within Jinja2
+# templates indicates to users editing templates files will be replaced.
+# replacing {file}, {host} and {uid} and strftime codes with proper values.
+#ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}
+# This short version is better used in templates as it won't flag the file as changed every run.
+#ansible_managed = Ansible managed: {file} on {host}
+
+# by default, ansible-playbook will display "Skipping [host]" if it determines a task
+# should not be run on a host.  Set this to "False" if you don't want to see these "Skipping"
+# messages. NOTE: the task header will still be shown regardless of whether or not the
+# task is skipped.
+#display_skipped_hosts = True
+display_skipped_hosts = False
+
+# by default, if a task in a playbook does not include a name: field then
+# ansible-playbook will construct a header that includes the task's action but
+# not the task's args.  This is a security feature because ansible cannot know
+# if the *module* considers an argument to be no_log at the time that the
+# header is printed.  If your environment doesn't have a problem securing
+# stdout from ansible-playbook (or you have manually specified no_log in your
+# playbook on all of the tasks where you have secret information) then you can
+# safely set this to True to get more informative messages.
+display_args_to_stdout = False
+
+# by default (as of 1.3), Ansible will raise errors when attempting to dereference
+# Jinja2 variables that are not set in templates or action lines. Uncomment this line
+# to revert the behavior to pre-1.3.
+#error_on_undefined_vars = False
+
+# by default (as of 1.6), Ansible may display warnings based on the configuration of the
+# system running ansible itself. This may include warnings about 3rd party packages or
+# other conditions that should be resolved if possible.
+# to disable these warnings, set the following value to False:
+system_warnings = False
+
+# by default (as of 1.4), Ansible may display deprecation warnings for language
+# features that should no longer be used and will be removed in future versions.
+# to disable these warnings, set the following value to False:
+deprecation_warnings = False
+
+# (as of 1.8), Ansible can optionally warn when usage of the shell and
+# command module appear to be simplified by using a default Ansible module
+# instead.  These warnings can be silenced by adjusting the following
+# setting or adding warn=yes or warn=no to the end of the command line
+# parameter string.  This will for example suggest using the git module
+# instead of shelling out to the git command.
+command_warnings = False
+
+
+# set plugin path directories here, separate with colons
+#action_plugins     = /usr/share/ansible/plugins/action
+#callback_plugins   = /usr/share/ansible/plugins/callback
+#connection_plugins = /usr/share/ansible/plugins/connection
+#lookup_plugins     = /usr/share/ansible/plugins/lookup
+#vars_plugins       = /usr/share/ansible/plugins/vars
+#filter_plugins     = /usr/share/ansible/plugins/filter
+#test_plugins       = /usr/share/ansible/plugins/test
+#strategy_plugins   = /usr/share/ansible/plugins/strategy
+
+# Most callbacks shipped with Ansible are disabled by default
+# and need to be whitelisted in your ansible.cfg file in order to function.
+callback_whitelist = default
+
+# by default callbacks are not loaded for /bin/ansible, enable this if you
+# want, for example, a notification or logging callback to also apply to
+# /bin/ansible runs
+#bin_ansible_callbacks = False
+
+
+# don't like cows?  that's unfortunate.
+# set to 1 if you don't want cowsay support or export ANSIBLE_NOCOWS=1
+#nocows = 1
+
+# set which cowsay stencil you'd like to use by default. When set to 'random',
+# a random stencil will be selected for each task. The selection will be filtered
+# against the `cow_whitelist` option below.
+#cow_selection = default
+#cow_selection = random
+
+# when using the 'random' option for cowsay, stencils will be restricted to this list.
+# it should be formatted as a comma-separated list with no spaces between names.
+# NOTE: line continuations here are for formatting purposes only, as the INI parser
+#       in python does not support them.
+#cow_whitelist=bud-frogs,bunny,cheese,daemon,default,dragon,elephant-in-snake,elephant,eyes,\
+#              hellokitty,kitty,luke-koala,meow,milk,moofasa,moose,ren,sheep,small,stegosaurus,\
+#              stimpy,supermilker,three-eyes,turkey,turtle,tux,udder,vader-koala,vader,www
+
+# don't like colors either?
+# set to 1 if you don't want colors, or export ANSIBLE_NOCOLOR=1
+nocolor = 0
+
+# if set to a persistent type (not 'memory', for example 'redis') fact values
+# from previous runs in Ansible will be stored.  This may be useful when
+# wanting to use, for example, IP information from one group of servers
+# without having to talk to them in the same playbook run to get their
+# current IP information.
+#fact_caching = memory
+
+# retry files
+# When a playbook fails by default a .retry file will be created in ~/
+# You can disable this feature by setting retry_files_enabled to False
+# and you can change the location of the files by setting retry_files_save_path
+
+#retry_files_enabled = False
+retry_files_enabled = False
+
+# squash actions
+# Ansible can optimise actions that call modules with list parameters
+# when looping. Instead of calling the module once per with_ item, the
+# module is called once with all items at once. Currently this only works
+# under limited circumstances, and only with parameters named 'name'.
+squash_actions = apk,apt,dnf,package,pacman,pkgng,yum,zypper
+
+# prevents logging of task data, off by default
+#no_log = False
+
+# prevents logging of tasks, but only on the targets, data is still logged on the master/controller
+no_target_syslog = True
+
+# controls whether Ansible will raise an error or warning if a task has no
+# choice but to create world readable temporary files to execute a module on
+# the remote machine.  This option is False by default for security.  Users may
+# turn this on to have behaviour more like Ansible prior to 2.1.x.  See
+# https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user
+# for more secure ways to fix this than enabling this option.
+#allow_world_readable_tmpfiles = False
+
+# controls the compression level of variables sent to
+# worker processes. At the default of 0, no compression
+# is used. This value must be an integer from 0 to 9.
+#var_compression_level = 9
+
+# controls what compression method is used for new-style ansible modules when
+# they are sent to the remote system.  The compression types depend on having
+# support compiled into both the controller's python and the client's python.
+# The names should match with the python Zipfile compression types:
+# * ZIP_STORED (no compression. available everywhere)
+# * ZIP_DEFLATED (uses zlib, the default)
+# These values may be set per host via the ansible_module_compression inventory
+# variable
+#module_compression = 'ZIP_DEFLATED'
+
+# This controls the cutoff point (in bytes) on --diff for files
+# set to 0 for unlimited (RAM may suffer!).
+#max_diff_size = 1048576
+
+[privilege_escalation]
+#become=True
+#become_method=sudo
+#become_user=root
+become_user=root
+#become_ask_pass=False
+
+[paramiko_connection]
+
+# uncomment this line to cause the paramiko connection plugin to not record new host
+# keys encountered.  Increases performance on new host additions.  Setting works independently of the
+# host key checking setting above.
+#record_host_keys=False
+
+# by default, Ansible requests a pseudo-terminal for commands executed under sudo. Uncomment this
+# line to disable this behaviour.
+#pty=False
+
+[ssh_connection]
+
+# ssh arguments to use
+# Leaving off ControlPersist will result in poor performance, so use
+# paramiko on older platforms rather than removing it
+ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o PreferredAuthentications=publickey -o ConnectTimeout=13
+
+# The path to use for the ControlPath sockets. This defaults to
+# "%(directory)s/ansible-ssh-%%h-%%p-%%r", however on some systems with
+# very long hostnames or very long path names (caused by long user names or
+# deeply nested home directories) this can exceed the character limit on
+# file socket names (108 characters for most platforms). In that case, you
+# may wish to shorten the string below.
+#
+# Example:
+# control_path = %(directory)s/%%h-%%r
+#control_path = %(directory)s/ansible-ssh-%%h-%%p-%%r
+
+# Enabling pipelining reduces the number of SSH operations required to
+# execute a module on the remote server. This can result in a significant
+# performance improvement when enabled, however when using "sudo:" you must
+# first disable 'requiretty' in /etc/sudoers
+#
+# By default, this option is disabled to preserve compatibility with
+# sudoers configurations that have requiretty (the default on many distros).
+#
+#pipelining = False
+pipelining=True
+
+# if True, make ansible use scp if the connection type is ssh
+# (default is sftp)
+#scp_if_ssh = True
+
+# if False, sftp will not use batch mode to transfer files. This may cause some
+# types of file transfer failures impossible to catch however, and should
+# only be disabled if your sftp version has problems with batch mode
+#sftp_batch_mode = False
+
+[accelerate]
+#accelerate_port = 5099
+#accelerate_timeout = 30
+#accelerate_connect_timeout = 5.0
+
+# The daemon timeout is measured in minutes. This time is measured
+# from the last activity to the accelerate daemon.
+#accelerate_daemon_timeout = 30
+
+# If set to yes, accelerate_multi_key will allow multiple
+# private keys to be uploaded to it, though each user must
+# have access to the system via SSH to add a new key. The default
+# is "no".
+#accelerate_multi_key = yes
+
+[selinux]
+# file systems that require special treatment when dealing with security context
+# the default behaviour that copies the existing context or uses the user default
+# needs to be changed to use the file system dependent context.
+#special_context_filesystems=nfs,vboxsf,fuse,ramfs
+
+# Set this to yes to allow libvirt_lxc connections to work without SELinux.
+#libvirt_lxc_noseclabel = yes
+
+[colors]
+#highlight = white
+#verbose = blue
+#warn = bright purple
+#error = red
+#debug = dark gray
+#deprecate = purple
+#skip = cyan
+#unreachable = red
+#ok = green
+#changed = yellow
+#diff_add = green
+#diff_remove = red
+#diff_lines = cyan

contrib/test/integration/build/bats.yml

@@ -0,0 +1,17 @@
+---
+
+- name: clone bats source repo
+  git:
+    repo: "https://github.com/sstephenson/bats.git"
+    dest: "{{ ansible_env.GOPATH }}/src/github.com/sstephenson/bats"
+
+- name: install bats
+  command: "./install.sh /usr/local"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/sstephenson/bats"
+
+- name: link bats
+  file:
+    src: /usr/local/bin/bats
+    dest: /usr/bin/bats
+    state: link

contrib/test/integration/build/cri-o.yml

@@ -0,0 +1,79 @@
+---
+
+- name: stat the expected cri-o directory
+  stat:
+    path: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+  register: dir_stat
+
+- name: expect cri-o to be cloned already
+  fail:
+    msg: "Expected cri-o to be cloned at {{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o but it wasn't!"
+  when: not dir_stat.stat.exists
+
+- name: install cri-o tools
+  make:
+    target: install.tools
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+
+- name: build cri-o
+  make:
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+
+- name: install cri-o
+  make:
+    target: install
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+
+- name: install cri-o systemd files
+  make:
+    target: install.systemd
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+
+- name: install cri-o config
+  make:
+    target: install.config
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+
+- name: install configs
+  copy:
+    src: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o/{{ item.src }}"
+    dest: "{{ item.dest }}"
+    remote_src: yes
+  with_items:
+    - src: contrib/cni/10-crio-bridge.conf
+      dest: /etc/cni/net.d/10-crio-bridge.conf
+    - src: contrib/cni/99-loopback.conf
+      dest: /etc/cni/net.d/99-loopback.conf
+    - src: test/redhat_sigstore.yaml
+      dest: /etc/containers/registries.d/registry.access.redhat.com.yaml
+
+- name: run with overlay
+  replace:
+    regexp: 'storage_driver = ""'
+    replace: 'storage_driver = "overlay"'
+    name: /etc/crio/crio.conf
+    backup: yes
+
+- name: run with systemd cgroup manager
+  replace:
+    regexp: 'cgroup_manager = "cgroupfs"'
+    replace: 'cgroup_manager = "systemd"'
+    name: /etc/crio/crio.conf
+    backup: yes
+
+- name: add docker.io default registry
+  lineinfile:
+    dest: /etc/crio/crio.conf
+    line: '"docker.io"'
+    insertafter: 'registries = \['
+    regexp: 'docker\.io'
+    state: present
+
+- name: add overlay storage opts on RHEL/CentOS
+  lineinfile:
+    dest: /etc/crio/crio.conf
+    line: '"overlay.override_kernel_check=1"'
+    insertafter: 'storage_option = \['
+    regexp: 'overlay\.override_kernel_check=1'
+    state: present
+  when: ansible_distribution == 'RedHat' or ansible_distribution == 'CentOS'

contrib/test/integration/build/cri-tools.yml

@@ -0,0 +1,16 @@
+---
+
+- name: clone cri-tools source repo
+  git:
+    repo: "https://github.com/kubernetes-incubator/cri-tools.git"
+    dest: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-tools"
+    version: "16e6fe4d7199c5689db4630a9330e6a8a12cecd1"
+
+- name: install crictl
+  command: "/usr/bin/go install github.com/kubernetes-incubator/cri-tools/cmd/crictl"
+
+- name: link crictl
+  file:
+    src: "{{ ansible_env.GOPATH }}/bin/crictl"
+    dest: /usr/bin/crictl
+    state: link

contrib/test/integration/build/kubernetes.yml

@@ -0,0 +1,63 @@
+---
+
+- name: clone kubernetes source repo
+  git:
+    repo: "https://github.com/runcom/kubernetes.git"
+    dest: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes"
+    version: "cri-o-node-e2e-patched"
+
+- name: install etcd
+  command: "hack/install-etcd.sh"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes"
+
+- name: build kubernetes
+  make:
+    chdir: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes"
+
+- name: Add custom cluster service file for the e2e testing
+  copy:
+    dest: /etc/systemd/system/customcluster.service
+    content: |
+      [Unit]
+      After=network-online.target
+      Wants=network-online.target
+      [Service]
+      WorkingDirectory={{ ansible_env.GOPATH }}/src/k8s.io/kubernetes
+      ExecStart=/usr/local/bin/createcluster.sh
+      User=root
+      [Install]
+      WantedBy=multi-user.target
+
+- name: Add create cluster background script for e2e testing
+  copy:
+    dest: /usr/local/bin/createcluster.sh
+    content: |
+      #!/bin/bash
+
+      export PATH=/usr/local/go/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/root/bin:{{ ansible_env.GOPATH }}/bin:{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes/third_party/etcd:{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes/_output/local/bin/linux/amd64/
+      export CONTAINER_RUNTIME=remote
+      export CGROUP_DRIVER=systemd
+      export CONTAINER_RUNTIME_ENDPOINT='/var/run/crio.sock --runtime-request-timeout=5m'
+      export ALLOW_SECURITY_CONTEXT=","
+      export ALLOW_PRIVILEGED=1
+      export DNS_SERVER_IP={{ ansible_eth0.ipv4.address }}
+      export API_HOST={{ ansible_eth0.ipv4.address }}
+      export API_HOST_IP={{ ansible_eth0.ipv4.address }}
+      export KUBE_ENABLE_CLUSTER_DNS=true
+      ./hack/local-up-cluster.sh
+    mode: "u=rwx,g=rwx,o=x"
+
+- name: Set kubernetes_provider to be local
+  lineinfile:
+    dest: /etc/environment
+    line: 'KUBERNETES_PROVIDER=local'
+    regexp: 'KUBERNETES_PROVIDER='
+    state: present
+
+- name: Set KUBECONFIG
+  lineinfile:
+    dest: /etc/environment
+    line: 'KUBECONFIG=/var/run/kubernetes/admin.kubeconfig'
+    regexp: 'KUBECONFIG='
+    state: present

contrib/test/integration/build/plugins.yml

@@ -0,0 +1,50 @@
+---
+
+- name: clone plugins source repo
+  git:
+    repo: "https://github.com/containernetworking/plugins.git"
+    dest: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins"
+    version: "dcf7368eeab15e2affc6256f0bb1e84dd46a34de"
+
+- name: build plugins
+  command: "./build.sh"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins"
+
+- name: install plugins
+  copy:
+    src: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins/bin/{{ item }}"
+    dest: "/opt/cni/bin"
+    mode: "o=rwx,g=rx,o=rx"
+    remote_src: yes
+  with_items:
+    - bridge
+    - dhcp
+    - flannel
+    - host-local
+    - ipvlan
+    - loopback
+    - macvlan
+    - ptp
+    - sample
+    - tuning
+    - vlan
+
+- name: clone runcom plugins source repo
+  git:
+    repo: "https://github.com/runcom/plugins.git"
+    dest: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins"
+    version: "custom-bridge"
+    force: yes
+
+- name: build plugins
+  command: "./build.sh"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins"
+
+- name: install custom bridge
+  copy:
+    src: "{{ ansible_env.GOPATH }}/src/github.com/containernetworking/plugins/bin/bridge"
+    dest: "/opt/cni/bin/bridge-custom"
+    mode: "o=rwx,g=rx,o=rx"
+    remote_src: yes

contrib/test/integration/build/runc.yml

@@ -0,0 +1,23 @@
+---
+
+- name: clone runc source repo
+  git:
+    repo: "https://github.com/opencontainers/runc.git"
+    dest: "{{ ansible_env.GOPATH }}/src/github.com/opencontainers/runc"
+    version: "84a082bfef6f932de921437815355186db37aeb1"
+
+- name: build runc
+  make:
+    params: BUILDTAGS="seccomp selinux"
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/opencontainers/runc"
+
+- name: install runc
+  make:
+    target: "install"
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/opencontainers/runc"
+
+- name: link runc
+  file:
+    src: /usr/local/sbin/runc
+    dest: /usr/bin/runc
+    state: link

contrib/test/integration/callback_plugins/default.py

@@ -0,0 +1,156 @@
+'''Plugin to override the default output logic.'''
+
+# upstream: https://gist.github.com/cliffano/9868180
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+# For some reason this has to be done
+import imp
+import os
+
+ANSIBLE_PATH = imp.find_module('ansible')[1]
+DEFAULT_PATH = os.path.join(ANSIBLE_PATH, 'plugins/callback/default.py')
+DEFAULT_MODULE = imp.load_source(
+    'ansible.plugins.callback.default',
+    DEFAULT_PATH
+)
+
+try:
+    from ansible.plugins.callback import CallbackBase
+    BASECLASS = CallbackBase
+except ImportError: # < ansible 2.1
+    BASECLASS = DEFAULT_MODULE.CallbackModule
+
+
+class CallbackModule(DEFAULT_MODULE.CallbackModule):  # pylint: disable=too-few-public-methods,no-init
+    '''
+    Override for the default callback module.
+
+    Render std err/out outside of the rest of the result which it prints with
+    indentation.
+    '''
+    CALLBACK_VERSION = 2.0
+    CALLBACK_TYPE = 'stdout'
+    CALLBACK_NAME = 'default'
+
+    def __init__(self, *args, **kwargs):
+        # pylint: disable=non-parent-init-called
+        BASECLASS.__init__(self, *args, **kwargs)
+        self.failed_task = []
+        self.result_file = os.environ.get('AHT_RESULT_FILE')
+
+    def _dump_results(self, result):
+        '''Return the text to output for a result.'''
+        result['_ansible_verbose_always'] = True
+
+        save = {}
+        for key in ['stdout', 'stdout_lines', 'stderr', 'stderr_lines', 'msg']:
+            if key in result:
+                save[key] = result.pop(key)
+
+        output = BASECLASS._dump_results(self, result)  # pylint: disable=protected-access
+
+        for key in ['stdout', 'stderr', 'msg']:
+            if key in save and save[key]:
+                output += '\n\n%s:\n---\n%s\n---' % (key.upper(), save[key])
+
+        for key, value in save.items():
+            result[key] = value
+
+        return output
+
+    def v2_runner_on_unreachable(self, result):
+        self.failed_task = result
+
+        if self._play.strategy == 'free' and self._last_task_banner != result._task._uuid:
+            self._print_task_banner(result._task)
+
+        delegated_vars = result._result.get('_ansible_delegated_vars', None)
+        if delegated_vars:
+            self._display.display("fatal: [%s -> %s]: UNREACHABLE! => %s" % (result._host.get_name(), delegated_vars['ansible_host'], self._dump_results(result._result)), color=C.COLOR_UNREACHABLE)
+        else:
+            self._display.display("fatal: [%s]: UNREACHABLE! => %s" % (result._host.get_name(), self._dump_results(result._result)), color=C.COLOR_UNREACHABLE)
+
+    def v2_runner_on_failed(self,result, ignore_errors=False):
+        if ignore_errors is not True:
+            # Sets environment variable for test failures for use in playboks.
+            # Handlers tasks can conditionalize themselves using this variable
+            # to run only on failure.
+            os.environ["AHT_FAILURE"] = "1"
+
+            # Save last failure
+            self.failed_task = result
+
+        if self._play.strategy == 'free' and self._last_task_banner != result._task._uuid:
+            self._print_task_banner(result._task)
+
+        delegated_vars = result._result.get('_ansible_delegated_vars', None)
+        if 'exception' in result._result:
+            if self._display.verbosity < 3:
+                # extract just the actual error message from the exception text
+                error = result._result['exception'].strip().split('\n')[-1]
+                msg = "An exception occurred during task execution. To see the full traceback, use -vvv. The error was: %s" % error
+            else:
+                msg = "An exception occurred during task execution. The full traceback is:\n" + result._result['exception']
+
+            self._display.display(msg, color=C.COLOR_ERROR)
+
+        if result._task.loop and 'results' in result._result:
+            self._process_items(result)
+
+        else:
+            if delegated_vars:
+                self._display.display("fatal: [%s -> %s]: FAILED! => %s" % (result._host.get_name(), delegated_vars['ansible_host'], self._dump_results(result._result)), color=C.COLOR_ERROR)
+            else:
+                self._display.display("fatal: [%s]: FAILED! => %s" % (result._host.get_name(), self._dump_results(result._result)), color=C.COLOR_ERROR)
+
+        if ignore_errors:
+            self._display.display("...ignoring", color=C.COLOR_SKIP)
+
+    def v2_playbook_on_stats(self, stats):
+        self._display.banner("PLAY RECAP")
+
+        hosts = sorted(stats.processed.keys())
+        for h in hosts:
+            t = stats.summarize(h)
+
+            self._display.display(u"%s : %s %s %s %s" % (
+                hostcolor(h, t),
+                colorize(u'ok', t['ok'], C.COLOR_OK),
+                colorize(u'changed', t['changed'], C.COLOR_CHANGED),
+                colorize(u'unreachable', t['unreachable'], C.COLOR_UNREACHABLE),
+                colorize(u'failed', t['failures'], C.COLOR_ERROR)),
+                screen_only=True
+            )
+
+            self._display.display(u"%s : %s %s %s %s" % (
+                hostcolor(h, t, False),
+                colorize(u'ok', t['ok'], None),
+                colorize(u'changed', t['changed'], None),
+                colorize(u'unreachable', t['unreachable'], None),
+                colorize(u'failed', t['failures'], None)),
+                log_only=True
+            )
+
+        self._display.display("", screen_only=True)
+        # Save result to file if environment variable exists
+        if self.result_file is not None:
+            if self.failed_task:
+                with open(self.result_file, 'w') as f:
+                    f.write("PLAY: %s\n%s\n%s" % (self._play, \
+                                                  self.failed_task._task, \
+                                                  self._dump_results(self.failed_task._result)))
+            else:
+                open(self.result_file, 'w').close()

contrib/test/integration/e2e.yml

@@ -0,0 +1,57 @@
+---
+
+- name: enable and start CRI-O
+  systemd:
+    name: crio
+    state: started
+    enabled: yes
+    daemon_reload: yes
+
+- name: update the server address for the custom cluster
+  lineinfile:
+    dest: /usr/local/bin/createcluster.sh
+    line: "export {{ item }}={{ ansible_eth0.ipv4.address }}"
+    regexp: "^export {{ item }}="
+    state: present
+  with_items:
+    - DNS_SERVER_IP
+    - API_HOST
+    - API_HOST_IP
+
+- name: enable and start the custom cluster
+  systemd:
+    name: customcluster.service
+    state: started
+    enabled: yes
+    daemon_reload: yes
+
+- name: wait for the cluster to be running
+  command: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes/_output/bin/kubectl get service kubernetes --namespace default"
+  register: kube_poll
+  until: kube_poll | succeeded
+  retries: 100
+  delay: 30
+
+- name: ensure directory exists for e2e reports
+  file:
+    path: "{{ artifacts }}"
+    state: directory
+
+- name: Buffer the e2e testing command to workaround Ansible YAML folding "feature"
+  set_fact:
+    e2e_shell_cmd: >
+        /usr/bin/go run hack/e2e.go
+            --test
+            --test_args="-host=https://{{ ansible_default_ipv4.address }}:6443
+                        --ginkgo.focus=\[Conformance\]
+                        --report-dir={{ artifacts }}"
+            &> {{ artifacts }}/e2e.log
+  # Fix vim syntax hilighting: "
+
+- name: disable SELinux
+  command: setenforce 0
+
+- name: run e2e tests
+  shell: "{{ e2e_shell_cmd | regex_replace('\\s+', ' ') }}"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes"

contrib/test/integration/golang.yml

@@ -0,0 +1,51 @@
+---
+
+- name: fetch Golang
+  unarchive:
+    remote_src: yes
+    src: https://storage.googleapis.com/golang/go1.8.4.linux-amd64.tar.gz
+    dest: /usr/local
+
+- name: link go toolchain
+  file:
+    src: "/usr/local/go/bin/{{ item }}"
+    dest: "/usr/bin/{{ item }}"
+    state: link
+  with_items:
+    - go
+    - gofmt
+    - godoc
+
+- name: ensure user profile exists
+  file:
+    path: "{{ ansible_user_dir }}/.profile"
+    state: touch
+
+- name: set up PATH for Go toolchain and built binaries
+  lineinfile:
+    dest: "{{ ansible_user_dir }}/.profile"
+    line: 'PATH={{ ansible_env.PATH }}:{{ ansible_env.GOPATH }}/bin:/usr/local/go/bin'
+    regexp: '^PATH='
+    state: present
+
+- name: set up directories
+  file:
+    path: "{{ item }}"
+    state: directory
+  with_items:
+    - "{{ ansible_env.GOPATH }}/src/github.com/containernetworking"
+    - "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator"
+    - "{{ ansible_env.GOPATH }}/src/github.com/k8s.io"
+    - "{{ ansible_env.GOPATH }}/src/github.com/sstephenson"
+    - "{{ ansible_env.GOPATH }}/src/github.com/opencontainers"
+
+- name: install Go tools and dependencies
+  shell: /usr/bin/go get -u "github.com/{{ item }}"
+  with_items:
+    - tools/godep
+    - onsi/ginkgo/ginkgo
+    - onsi/gomega
+    - cloudflare/cfssl/cmd/...
+    - jteeuwen/go-bindata/go-bindata
+    - vbatts/git-validation
+    - cpuguy83/go-md2man

contrib/test/integration/main.yml

@@ -0,0 +1,58 @@
+- hosts: all
+  remote_user: root
+  vars_files:
+    - "{{ playbook_dir }}/vars.yml"
+  tags:
+    - setup
+  tasks:
+    - name: set up the system
+      include: system.yml
+
+    - name: install Golang tools
+      include: golang.yml
+
+    - name: clone build and install bats
+      include: "build/bats.yml"
+
+    - name: clone build and install cri-tools
+      include: "build/cri-tools.yml"
+
+    - name: clone build and install kubernetes
+      include: "build/kubernetes.yml"
+
+    - name: clone build and install runc
+      include: "build/runc.yml"
+
+    - name: clone build and install networking plugins
+      include: "build/plugins.yml"
+
+- hosts: all
+  remote_user: root
+  vars_files:
+    - "{{ playbook_dir }}/vars.yml"
+  tags:
+    - integration
+    - e2e
+  tasks:
+    - name: clone build and install cri-o
+      include: "build/cri-o.yml"
+
+- hosts: all
+  remote_user: root
+  vars_files:
+    - "{{ playbook_dir }}/vars.yml"
+  tags:
+    - integration
+  tasks:
+    - name: run cri-o integration tests
+      include: test.yml
+
+- hosts: all
+  remote_user: root
+  vars_files:
+    - "{{ playbook_dir }}/vars.yml"
+  tags:
+    - e2e
+  tasks:
+    - name: run k8s e2e tests
+      include: e2e.yml

contrib/test/integration/results.yml

@@ -0,0 +1,62 @@
+---
+# vim-syntax: ansible
+
+- hosts: '{{ hosts | default("all") }}'
+  vars_files:
+    - "{{ playbook_dir }}/vars.yml"
+  vars:
+    _result_filepaths: []  # do not use
+    _dstfnbuff: []  # do not use
+  tasks:
+    - name: The crio_integration_filepath is required
+      tags:
+          - integration
+      set_fact:
+       _result_filepaths: "{{ _result_filepaths + [crio_integration_filepath] }}"
+
+    - name: The crio_node_e2e_filepath is required
+      tags:
+          - e2e
+      set_fact:
+       _result_filepaths: "{{ _result_filepaths + [crio_node_e2e_filepath] }}"
+
+    - name: Verify expectations
+      assert:
+        that:
+          - 'result_dest_basedir | default(False, True)'
+          - '_result_filepaths | default(False, True)'
+          - '_dstfnbuff == []'
+          - 'results_fetched is undefined'
+
+    - name: Results directory exists
+      file:
+          path: "{{ result_dest_basedir }}"
+          state: directory
+      delegate_to: localhost
+
+    - name: destination file paths are buffered for overwrite-checking and jUnit conversion
+      set_fact:
+          _dstfnbuff: >
+              {{ _dstfnbuff |
+                 union( [result_dest_basedir ~ "/" ~ inventory_hostname ~ "/" ~ item | basename] ) }}
+      with_items: '{{ _result_filepaths }}'
+
+    - name: Overwriting existing results assumed very very bad
+      fail:
+          msg: "Cowardly refusing to overwrite {{ item }}"
+      when: item | exists
+      delegate_to: localhost
+      with_items: '{{ _dstfnbuff }}'
+
+    # fetch module doesn't support directories
+    - name: Retrieve results from all hosts
+      synchronize:
+          checksum: True  # Don't rely on date/time being in sync
+          archive: False  # Don't bother with permissions or times
+          copy_links: True  # We want files, not links to files
+          recursive: True
+          mode: pull
+          dest: '{{ result_dest_basedir }}/{{ inventory_hostname }}/'  # must end in /
+          src: '{{ item }}'
+      register: results_fetched
+      with_items: '{{ _result_filepaths }}'

contrib/test/integration/system.yml

@@ -0,0 +1,117 @@
+---
+
+- name: Make sure we have all required packages
+  package:
+    name: "{{ item }}"
+    state: present
+  with_items:
+    - container-selinux
+    - curl
+    - device-mapper-devel
+    - expect
+    - findutils
+    - gcc
+    - git
+    - glib2-devel
+    - glibc-devel
+    - glibc-static
+    - gpgme-devel
+    - hostname
+    - iproute
+    - iptables
+    - krb5-workstation
+    - libassuan-devel
+    - libffi-devel
+    - libgpg-error-devel
+    - libguestfs-tools
+    - libseccomp-devel
+    - libvirt-client
+    - libvirt-python
+    - libxml2-devel
+    - libxslt-devel
+    - make
+    - mlocate
+    - nfs-utils
+    - nmap-ncat
+    - oci-register-machine
+    - oci-systemd-hook
+    - oci-umount
+    - openssl
+    - openssl-devel
+    - ostree-devel
+    - pkgconfig
+    - python
+    - python2-boto
+    - python2-crypto
+    - python-devel
+    - python-virtualenv
+    - PyYAML
+    - redhat-rpm-config
+    - rpcbind
+    - rsync
+    - sed
+    - skopeo-containers
+    - socat
+    - tar
+    - wget
+  async: 600
+  poll: 10
+
+- name: Add Btrfs for Fedora
+  package:
+    name: "{{ item }}"
+    state: present
+  with_items:
+   - btrfs-progs-devel
+  when: ansible_distribution in ['Fedora']
+
+- name: Update all packages
+  package:
+    name: '*'
+    state: latest
+  async: 600
+  poll: 10
+
+- name: Setup swap to prevent kernel firing off the OOM killer
+  shell: |
+    truncate -s 8G /root/swap && \
+    export SWAPDEV=$(losetup --show -f /root/swap | head -1) && \
+    mkswap $SWAPDEV && \
+    swapon $SWAPDEV && \
+    swapon --show
+
+- name: ensure directories exist as needed
+  file:
+    path: "{{ item }}"
+    state: directory
+  with_items:
+    - /opt/cni/bin
+    - /etc/cni/net.d
+
+- name: set sysctl vm.overcommit_memory=1 for CentOS
+  sysctl:
+    name: vm.overcommit_memory
+    state: present
+    value: 1
+  when: ansible_distribution == 'CentOS'
+
+- name: inject hostname into /etc/hosts
+  lineinfile:
+    dest: /etc/hosts
+    line: '{{ ansible_default_ipv4.address }} {{ ansible_nodename }}'
+    insertafter: 'EOF'
+    regexp: '{{ ansible_default_ipv4.address }}\s+{{ ansible_nodename }}'
+    state: present
+
+- name: Flush the iptables
+  command: iptables -F
+
+- name: Enable localnet routing
+  command: sysctl -w net.ipv4.conf.all.route_localnet=1
+
+- name: Add masquerade for localhost
+  command: iptables -t nat -I POSTROUTING -s 127.0.0.1 ! -d 127.0.0.1 -j MASQUERADE
+
+- name: Update the kernel cmdline to include quota support
+  command: grubby --update-kernel=ALL --args="rootflags=pquota"
+  when: ansible_distribution in ['RedHat', 'CentOS']

contrib/test/integration/test.yml

@@ -0,0 +1,25 @@
+---
+
+- name: Make testing output verbose so it can be converted to xunit
+  lineinfile:
+    dest: "{{ ansible_env.GOPATH }}/src/k8s.io/kubernetes/hack/make-rules/test.sh"
+    line: ' go test -v "${goflags[@]:+${goflags[@]}}" \'
+    regexp: ' go test \"\$'
+    state: present
+
+- name: set extra storage options
+  set_fact:
+    extra_storage_opts: " --storage-opt overlay.override_kernel_check=1"
+  when: ansible_distribution == 'RedHat' or ansible_distribution == 'CentOS'
+
+- name: ensure directory exists for e2e reports
+  file:
+    path: "{{ artifacts }}"
+    state: directory
+
+- name: run integration tests
+  shell: "CGROUP_MANAGER=cgroupfs STORAGE_OPTIONS='--storage-driver=overlay{{ extra_storage_opts | default('') }}' make localintegration >& {{ artifacts }}/testout.txt"
+  args:
+    chdir: "{{ ansible_env.GOPATH }}/src/github.com/kubernetes-incubator/cri-o"
+  async: 5400
+  poll: 30

contrib/test/integration/vars.yml

@@ -0,0 +1,8 @@
+---
+
+# For results.yml Paths use rsync 'source' conventions
+artifacts: "/tmp/artifacts"  # Base-directory for collection
+crio_integration_filepath: "{{ artifacts }}/testout.txt"
+crio_node_e2e_filepath: "{{ artifacts }}/junit_01.xml"
+result_dest_basedir: '{{ lookup("env","WORKSPACE") |
+                         default(playbook_dir, True) }}/artifacts'

contrib/test/requirements.txt

@@ -0,0 +1,54 @@
+# Pip requirements file for Ansible-based integration-testing environment.
+# Intended to be utilized by venv-ansible-playbook.sh script
+#
+# N/B: Hashes are required here | versions frozen for stability
+
+ansible==2.3.1.0 --hash=sha256:cd4b8f53720fcd0c351156b840fdd15ecfbec22c951b5406ec503de49d40b9f5
+
+asn1crypto==0.22.0 --hash=sha256:d232509fefcfcdb9a331f37e9c9dc20441019ad927c7d2176cf18ed5da0ba097 \
+                   --hash=sha256:cbbadd640d3165ab24b06ef25d1dca09a3441611ac15f6a6b452474fdf0aed1a
+
+bcrypt==3.1.3 --hash=sha256:05b35b9842b009b44496fa5433ce462f69966291e50fbd471dbb427f399f748f \
+              --hash=sha256:6645c8d0ad845308de3eb9be98b6fd22a46ec5412bfc664a423e411cdd8f5488
+
+cffi==1.10.0 --hash=sha256:c49187260043bd4c1d6a52186f9774f17d9b1da0a406798ebf4bfc12da166ade \
+             --hash=sha256:b3b02911eb1f6ada203b0763ba924234629b51586f72a21faacc638269f4ced5
+
+cryptography==1.9 --hash=sha256:5518337022718029e367d982642f3e3523541e098ad671672a90b82474c84882
+
+enum34==1.1.6 --hash=sha256:6bd0f6ad48ec2aa117d3d141940d484deccda84d4fcd884f5c3d93c23ecd8c79 \
+              --hash=sha256:8ad8c4783bf61ded74527bffb48ed9b54166685e4230386a9ed9b1279e2df5b1
+
+idna==2.5 --hash=sha256:cc19709fd6d0cbfed39ea875d29ba6d4e22c0cebc510a76d6302a28385e8bb70 \
+          --hash=sha256:3cb5ce08046c4e3a560fc02f138d0ac63e00f8ce5901a56b32ec8b7994082aab
+
+ipaddress==1.0.18 --hash=sha256:d34cf15d95ce9a734560f7400a8bd2ac2606f378e2a1d0eadbf1c98707e7c74a \
+                  --hash=sha256:5d8534c8e185f2d8a1fda1ef73f2c8f4b23264e8e30063feeb9511d492a413e1
+
+Jinja2==2.9.6 --hash=sha256:2231bace0dfd8d2bf1e5d7e41239c06c9e0ded46e70cc1094a0aa64b0afeb054 \
+              --hash=sha256:ddaa01a212cd6d641401cb01b605f4a4d9f37bfc93043d7f760ec70fb99ff9ff
+
+MarkupSafe==1.0 --hash=sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665
+
+paramiko==2.2.1 --hash=sha256:9c9402377ba8594889aab1e44a13b78eda685eb2145dc00b2353b4fbb25088cf \
+                --hash=sha256:ff94ae65379914ec3c960de731381f49092057b6dd1d24d18842ead5a2eb2277
+
+pyasn1==0.2.3 --hash=sha256:0439b9bd518418260c2641a571f0e07fce4370cab13b68f19b5e023306c03cad \
+              --hash=sha256:738c4ebd88a718e700ee35c8d129acce2286542daa80a82823a7073644f706ad
+
+pycparser==2.17 --hash=sha256:0aac31e917c24cb3357f5a4d5566f2cc91a19ca41862f6c3c22dc60a629673b6
+
+pycrypto==2.6.1 --hash=sha256:f2ce1e989b272cfcb677616763e0a2e7ec659effa67a88aa92b3a65528f60a3c
+
+PyNaCl==1.1.2 --hash=sha256:57314a7bad4bd39501dc622942f9921923673e52e126b0fc4f0214b5d25d619a \
+              --hash=sha256:32f52b754abf07c319c04ce16905109cab44b0e7f7c79497431d3b2000f8af8c
+
+PyYAML==3.12 --hash=sha256:592766c6303207a20efc445587778322d7f73b161bd994f227adaa341ba212ab
+
+six==1.10.0 --hash=sha256:0ff78c403d9bccf5a425a6d31a12aa6b47f1c21ca4dc2573a7e2f32a97335eb1 \
+            --hash=sha256:105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a
+
+virtualenv==15.1.0 --hash=sha256:39d88b533b422825d644087a21e78c45cf5af0ef7a99a1fc9fbb7b481e5c85b0 \
+                   --hash=sha256:02f8102c2436bb03b3ee6dede1919d1dac8a427541652e5ec95171ec8adbc93a
+
+pip==9.0.1 --hash=sha256:690b762c0a8460c303c089d5d0be034fb15a5ea2b75bdf565f40421f542fefb0

contrib/test/venv-ansible-playbook.sh

@@ -0,0 +1,106 @@
+#!/bin/bash
+
+# example usage
+#   $ ./venv-ansible-playbook.sh \
+#               -i 192.168.169.170 \
+#               --private-key=/path/to/key \
+#               --extra-vars "pullrequest=42" \
+#               --extra-vars "commit=abcd1234" \
+#               --user root \
+#               --verbose \
+#               $PWD/crio-integration-playbook.yaml
+
+# All errors are fatal
+set -e
+
+SCRIPT_PATH=`realpath $(dirname $0)`
+REQUIREMENTS="$SCRIPT_PATH/requirements.txt"
+
+echo
+
+if ! type -P virtualenv &> /dev/null
+then
+    echo "Could not find required 'virtualenv' binary installed on system."
+    exit 1
+fi
+
+if [ "$#" -lt "1" ]
+then
+    echo "No ansible-playbook command-line options specified."
+    echo "usage: $0 -i whatever --private-key=something --extra-vars foo=bar playbook.yml"
+    exit 2
+fi
+
+# Avoid dirtying up repository, keep execution bits confined to a known location
+if [ -z "$WORKSPACE" ] || [ ! -d "$WORKSPACE" ]
+then
+    export WORKSPACE="$(mktemp -d)"
+    echo "Using temporary \$WORKSPACE=\"$WORKSPACE\" for execution environment."
+    echo "Directory will be removed upon exit.  Export this variable with path"
+    echo "to an existing directory to preserve contents."
+    trap 'rm -rf "$WORKSPACE"' EXIT
+else
+    echo "Using existing \$WORKSPACE=\"$WORKSPACE\" for execution environment."
+    echo "Directory will be left as-is upon exit."
+    # Don't recycle cache, next job may have different requirements
+    trap 'rm -rf "$PIPCACHE"' EXIT
+fi
+
+# Create a directory to contain logs and test artifacts
+export ARTIFACTS=$(mkdir -pv $WORKSPACE/artifacts | tail -1 | cut -d \' -f 2)
+[ -d "$ARTIFACTS" ] || exit 3
+
+# All command failures from now on are fatal
+set -e
+echo
+echo "Bootstrapping trusted virtual environment, this may take a few minutes, depending on networking."
+echo "(logs: \"$ARTIFACTS/crio_venv_setup_log.txt\")"
+echo
+
+
+(
+    set -x
+    cd "$WORKSPACE"
+    # When running more than once, make it fast by skipping the bootstrap
+    if [ ! -d "./.cri-o_venv" ]; then
+        # N/B: local system's virtualenv binary - uncontrolled version fixed below
+        virtualenv --no-site-packages --python=python2.7 ./.venvbootstrap
+        # Set up paths to install/operate out of $WORKSPACE/.venvbootstrap
+        source ./.venvbootstrap/bin/activate
+        # N/B: local system's pip binary - uncontrolled version fixed below
+        # pip may not support --cache-dir, force it's location into $WORKSPACE the ugly-way
+        OLD_HOME="$HOME"
+        export HOME="$WORKSPACE"
+        export PIPCACHE="$WORKSPACE/.cache/pip"
+        pip install --force-reinstall --upgrade pip==9.0.1
+        # Undo --cache-dir workaround
+        export HOME="$OLD_HOME"
+        # Install fixed, trusted, hashed versions of all requirements (including pip and virtualenv)
+        pip --cache-dir="$PIPCACHE" install --require-hashes \
+            --requirement "$SCRIPT_PATH/requirements.txt"
+
+        # Setup trusted virtualenv using hashed binary from requirements.txt
+        ./.venvbootstrap/bin/virtualenv --no-site-packages --python=python2.7 ./.cri-o_venv
+        # Exit untrusted virtualenv
+        deactivate
+    fi
+    # Enter trusted virtualenv
+    source ./.cri-o_venv/bin/activate
+    # Upgrade stock-pip to support hashes
+    pip install --force-reinstall --cache-dir="$PIPCACHE" --upgrade pip==9.0.1
+    # Re-install from cache but validate all hashes (including on pip itself)
+    pip --cache-dir="$PIPCACHE" install --require-hashes \
+        --requirement "$SCRIPT_PATH/requirements.txt"
+    # Remove temporary bootstrap virtualenv
+    rm -rf ./.venvbootstrap
+    # Exit trusted virtualenv
+
+) &> $ARTIFACTS/crio_venv_setup_log.txt;
+
+echo
+echo "Executing \"$WORKSPACE/.cri-o_venv/bin/ansible-playbook $@\""
+echo
+
+# Execute command-line arguments under virtualenv
+source ${WORKSPACE}/.cri-o_venv/bin/activate
+${WORKSPACE}/.cri-o_venv/bin/ansible-playbook $@

crio-umount.conf

@@ -0,0 +1,8 @@
+# This contains a list of paths on host which will be unmounted inside
+# container. (If they are mounted inside container).
+
+# If there is a "/*" at the end, that means only mounts underneath that
+# mounts (submounts) will be unmounted but top level mount will remain
+# in place.
+/var/run/containers/*
+/var/lib/containers/storage/*

docs/crio.8.md

@@ -0,0 +1,177 @@
+% crio(8) Open Container Initiative Daemon
+% Dan Walsh
+% SEPTEMBER 2016
+# NAME
+crio - OCI Kubernetes Container Runtime daemon
+
+# SYNOPSIS
+**crio**
+[**--apparmor-profile**=[*value*]]
+[**--cgroup-manager**=[*value*]]
+[**--cni-config-dir**=[*value*]]
+[**--cni-plugin-dir**=[*value*]]
+[**--config**=[*value*]]
+[**--conmon**=[*value*]]
+[**--cpu-profile**=[*value*]]
+[**--default-transport**=[*value*]]
+[**--help**|**-h**]
+[**--insecure-registry**=[*value*]]
+[**--listen**=[*value*]]
+[**--log**=[*value*]]
+[**--log-format value**]
+[**--log-level value**]
+[**--pause-command**=[*value*]]
+[**--pause-image**=[*value*]]
+[**--registry**=[*value*]]
+[**--root**=[*value*]]
+[**--runroot**=[*value*]]
+[**--runtime**=[*value*]]
+[**--seccomp-profile**=[*value*]]
+[**--selinux**]
+[**--signature-policy**=[*value*]]
+[**--storage-driver**=[*value*]]
+[**--storage-opt**=[*value*]]
+[**--version**|**-v**]
+
+# DESCRIPTION
+OCI-based implementation of Kubernetes Container Runtime Interface Daemon
+
+crio is meant to provide an integration path between OCI conformant runtimes and the kubelet. Specifically, it implements the Kubelet Container Runtime Interface (CRI) using OCI conformant runtimes. The scope of crio is tied to the scope of the CRI.
+
+	* Support multiple image formats including the existing Docker image format
+	* Support for multiple means to download images including trust & image verification
+	* Container image management (managing image layers, overlay filesystems, etc)
+	* Container process lifecycle management
+	* Monitoring and logging required to satisfy the CRI
+	* Resource isolation as required by the CRI
+
+**crio [GLOBAL OPTIONS]**
+
+**crio [GLOBAL OPTIONS] config [OPTIONS]**
+
+# GLOBAL OPTIONS
+
+**--apparmor_profile**=""
+  Name of the apparmor profile to be used as the runtime's default (default: "crio-default")
+
+**--cgroup-manager**=""
+  cgroup manager (cgroupfs or systemd)
+
+**--config**=""
+  path to configuration file
+
+**--conmon**=""
+  path to the conmon executable (default: "/usr/local/libexec/crio/conmon")
+
+**--cpu-profile**=""
+set the CPU profile file path
+
+**--default-transport**
+  A prefix to prepend to image names that can't be pulled as-is.
+
+**--help, -h**
+  Print usage statement
+
+**--insecure-registry=**
+  Enable insecure registry  communication,  i.e.,  enable  un-encrypted
+  and/or untrusted communication.
+
+  List  of  insecure registries can contain an element with CIDR notation
+  to specify a whole  subnet.  Insecure  registries  accept  HTTP  and/or
+  accept HTTPS with certificates from unknown CAs.
+
+  Enabling  --insecure-registry  is useful when running a local registry.
+  However, because its use creates  security  vulnerabilities  it  should
+  ONLY  be  enabled  for testing purposes.  For increased security, users
+  should add their CA to their system's list of trusted  CAs  instead  of
+  using --insecure-registry.
+
+**--image-volumes**=""
+  Image volume handling ('mkdir', 'bind' or 'ignore') (default: "mkdir")
+  mkdir: A directory is created inside the container root filesystem for the volumes.
+  bind: A directory is created inside container state directory and bind mounted into
+  the container for the volumes.
+  ignore: All volumes are just ignored and no action is taken.
+
+**--listen**=""
+  Path to CRI-O socket (default: "/var/run/crio.sock")
+
+**--log**=""
+  Set the log file path where internal debug information is written
+
+**--log-format**=""
+  Set the format used by logs ('text' (default), or 'json') (default: "text")
+
+**--log-level**=""
+  log crio messages above specified level: debug, info (default), warn, error, fatal or panic
+
+**--log-size-max**=""
+  Maximum log size in bytes for a container (default: -1 (no limit)).
+  If it is positive, it must be >= 8192 (to match/exceed conmon read buffer).
+
+**--pause-command**=""
+  Path to the pause executable in the pause image (default: "/pause")
+
+**--pause-image**=""
+  Image which contains the pause executable (default: "kubernetes/pause")
+
+**--pids-limit**=""
+  Maximum number of processes allowed in a container (default: 1024)
+
+**--root**=""
+  The crio root dir (default: "/var/lib/containers/storage")
+
+**--registry**=""
+  Registry host which will be prepended to unqualified images, can be specified multiple times
+
+**--runroot**=""
+  The crio state dir (default: "/var/run/containers/storage")
+
+**--runtime**=""
+  OCI runtime path (default: "/usr/bin/runc")
+
+**--selinux**=*true*|*false*
+  Enable selinux support (default: false)
+
+**--seccomp-profile**=""
+  Path to the seccomp json profile to be used as the runtime's default (default: "/etc/crio/seccomp.json")
+
+**--signature-policy**=""
+  Path to the signature policy json file (default: "", to use the system-wide default)
+
+**--storage-driver**
+  OCI storage driver (default: "devicemapper")
+
+**--storage-opt**
+  OCI storage driver option (no default)
+
+**--cni-config-dir**=""
+  CNI configuration files directory (default: "/etc/cni/net.d/")
+
+**--cni-plugin-dir**=""
+  CNI plugin binaries directory (default: "/opt/cni/bin/")
+
+**--cpu-profile**
+  Set the CPU profile file path
+
+**--version, -v**
+  Print the version
+
+# COMMANDS
+CRI-O's default command is to start the daemon. However, it currently offers a
+single additional subcommand.
+
+## config
+
+Outputs a commented version of the configuration file that would've been used
+by CRI-O. This allows you to save you current configuration setup and then load
+it later with **--config**. Global options will modify the output.
+
+**--default**
+  Output the default configuration (without taking into account any configuration options).
+
+# SEE ALSO
+crio.conf(5)
+
+# HISTORY
+Sept 2016, Originally compiled by Dan Walsh <dwalsh@redhat.com> and Aleksa Sarai <asarai@suse.de>

docs/crio.conf.5.md

@@ -0,0 +1,159 @@
+% crio.conf(5) Open Container Initiative Daemon
+% Aleksa Sarai
+% OCTOBER 2016
+
+# NAME
+crio.conf - CRI-O configuration file
+
+# DESCRIPTION
+The CRI-O configuration file specifies all of the available command-line options
+for the crio(8) program, but in a TOML format that can be more easily modified
+and versioned.
+
+# FORMAT
+The [TOML format][toml] is used as the encoding of the configuration file.
+Every option and subtable listed here is nested under a global "crio" table.
+No bare options are used. The format of TOML can be simplified to:
+
+    [table]
+    option = value
+
+    [table.subtable1]
+    option = value
+
+    [table.subtable2]
+    option = value
+
+## CRIO TABLE
+
+The `crio` table supports the following options:
+
+
+**root**=""
+  CRIO root dir (default: "/var/lib/containers/storage")
+
+**runroot**=""
+  CRIO state dir (default: "/var/run/containers/storage")
+
+**storage_driver**=""
+  CRIO storage driver (default is "overlay")
+
+Note:
+  **overlay** and **overlay2** are the same driver
+
+
+**storage_option**=[]
+  CRIO storage driver option list (no default)
+
+  Values:
+
+	"STORAGE_DRIVER.imagestore=/PATH",
+
+	Paths to additional container image stores. These are read/only and are usually stored on remote network shares, based on overlay storage format.
+	storage_option=[ "overlay.imagestore=/mnt/overlay", ]
+
+	"STORAGE_DRIVER.size=SIZE"
+
+	Maximum size of a container image.  Default is 10GB. The size flag sets quota on the size of container images.
+	storage_option=[ "overlay.size=1G", ]
+
+Note: Not all drivers support all options.
+
+Note:  In order to use the **size** option for quota on *overlay* storage you must use the *xfs* file system.  The mount point that the *overlay* file system must be setup with the *pquota* flag at mount time. If you are setting up / to be used with quota, you have to modify the linux boot line in /etc/grubq2.conf and add the rootflags=pquota flag.
+
+Example:
+	linux16 /vmlinuz-4.12.13-300.fc26.x86_64 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap rhgb quiet LANG=en_US.UTF-8 rootflags=pquota
+
+
+## CRIO.API TABLE
+
+**listen**=""
+  Path to crio socket (default: "/var/run/crio.sock")
+
+## CRIO.RUNTIME TABLE
+
+**conmon**=""
+  Path to the conmon executable (default: "/usr/local/libexec/crio/conmon")
+
+**conmon_env**=[]
+  Environment variable list for conmon process (default: ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",])
+
+**log_size_max**=""
+  Maximum sized allowed for the container log file (default: -1)
+  Negative numbers indicate that no size limit is imposed.
+  If it is positive, it must be >= 8192 (to match/exceed conmon read buffer).
+  The file is truncated and re-opened so the limit is never exceeded.
+
+**pids_limit**=""
+  Maximum number of processes allowed in a container (default: 1024)
+
+**runtime**=""
+  OCI runtime path (default: "/usr/bin/runc")
+
+**selinux**=*true*|*false*
+  Enable selinux support (default: false)
+
+**signature_policy**=""
+  Path to the signature policy json file (default: "", to use the system-wide default)
+
+**seccomp_profile**=""
+  Path to the seccomp json profile to be used as the runtime's default (default: "/etc/crio/seccomp.json")
+
+**apparmor_profile**=""
+  Name of the apparmor profile to be used as the runtime's default (default: "crio-default")
+
+**no_pivot**=*true*|*false*
+  Instructs the runtime to not use pivot_root, but instead use MS_MOVE
+
+**default_mounts**=[]
+  List of mount points, in the form host:container, to be mounted in every container
+
+## CRIO.IMAGE TABLE
+
+**default_transport**
+  A prefix to prepend to image names that can't be pulled as-is (default: "docker://")
+
+**image_volumes**=""
+  Image volume handling ('mkdir', 'bind' or 'ignore') (default: "mkdir")
+  mkdir: A directory is created inside the container root filesystem for the volumes.
+  bind: A directory is created inside container state directory and bind mounted into
+  the container for the volumes.
+  ignore: All volumes are just ignored and no action is taken.
+
+**insecure_registries**=""
+  Enable insecure registry  communication,  i.e.,  enable  un-encrypted
+  and/or untrusted communication.
+
+  List  of  insecure registries can contain an element with CIDR notation
+  to specify a whole  subnet.  Insecure  registries  accept  HTTP  and/or
+  accept HTTPS with certificates from unknown CAs.
+
+  Enabling  --insecure-registry  is useful when running a local registry.
+  However, because its use creates  security  vulnerabilities  it  should
+  ONLY  be  enabled  for testing purposes.  For increased security, users
+  should add their CA to their system's list of trusted  CAs  instead  of
+  using --insecure-registry.
+
+**pause_command**=""
+  Path to the pause executable in the pause image (default: "/pause")
+
+**pause_image**=""
+  Image which contains the pause executable (default: "kubernetes/pause")
+
+**registries**=""
+  Comma separated list of registries that will be prepended when pulling
+  unqualified images
+
+## CRIO.NETWORK TABLE
+
+**network_dir**=""
+  Path to CNI configuration files (default: "/etc/cni/net.d/")
+
+**plugin_dir**=""
+  Path to CNI plugin binaries (default: "/opt/cni/bin/")
+
+# SEE ALSO
+crio(8)
+
+# HISTORY
+Oct 2016, Originally compiled by Aleksa Sarai <asarai@suse.de>

docs/kpod-attach.1.md

@@ -0,0 +1,31 @@
+% kpod(1) kpod-attach - See the output of pid 1 of a container or enter the container
+% Dan Walsh
+# kpod-attach "1" "September 2017" "kpod"
+
+## NAME
+kpod-attach - Attach to a running container
+
+## Description
+
+We chose not to implement the `attach` feature in `kpod` even though the upstream Docker
+project has it. The upstream project has had lots of issues with attaching to running
+processes that we did not want to replicate. The `kpod exec` and `kpod log` commands
+offer you the same functionality far more dependably.
+
+**Reasons to attach to the primary PID of a container:**
+
+
+1) Executing commands inside of the container
+
+  We recommend that you use `kpod exec` to execute a command within a container
+
+  `kpod exec CONTAINERID /bin/sh`
+
+2) Viewing the output stream of the primary process in the container
+
+  We recommend that you use `kpod logs` to see the output from the container
+
+  `kpod logs CONTAINERID`
+
+## SEE ALSO
+kpod(1), kpod-exec(1), kpod-logs(1)

docs/kpod-cp.1.md

@@ -0,0 +1,46 @@
+% kpod(1) kpod-cp - Copy content between container's file system and the host
+% Dan Walsh
+# kpod-cp "1" "August 2017" "kpod"
+
+## NAME
+kpod-cp - Copy files/folders between a container and the local filesystem.
+
+## Description
+We chose not to implement the `cp` feature in `kpod` even though the upstream Docker
+project has it. We have a much stronger capability.  Using standard kpod-mount
+and kpod-umount, we can take advantage of the entire linux tool chain, rather
+then just cp.
+
+If a user wants to copy contents out of a container or into a container, they
+can execute a few simple commands.
+
+You can copy from the container's file system to the local machine or the
+reverse, from the local filesystem to the container. 
+
+If you want to copy the /etc/foobar directory out of a container and onto /tmp
+on the host, you could execute the following commands:
+
+	mnt=$(kpod mount CONTAINERID)
+	cp -R ${mnt}/etc/foobar /tmp
+	kpod umount CONTAINERID
+
+If you want to untar a tar ball into a container, you can execute these commands:
+
+	mnt=$(kpod mount CONTAINERID)
+	tar xf content.tgz -C ${mnt}
+	kpod umount CONTAINERID
+
+One last example, if you want to install a package into a container that
+does not have dnf installed, you could execute something like:
+
+	mnt=$(kpod mount CONTAINERID)
+	dnf install --installroot=${mnt} httpd
+	chroot ${mnt} rm -rf /var/log/dnf /var/cache/dnf
+	kpod umount CONTAINERID
+
+This shows that using `kpod mount` and `kpod umount` you can use all of the
+standard linux tools for moving files into and out of containers, not just
+the cp command.
+
+## SEE ALSO
+kpod(1), kpod-mount(1), kpod-umount(1)

docs/kpod-diff.1.md

@@ -0,0 +1,45 @@
+% kpod(1) kpod-diff - Inspect changes on a container or image's filesystem
+% Dan Walsh
+# kpod-diff "1" "August 2017" "kpod"
+
+## NAME
+kpod diff - Inspect changes on a container or image's filesystem
+
+## SYNOPSIS
+**kpod** **diff** [*options* [...]] NAME
+
+## DESCRIPTION
+Displays changes on a container or image's filesystem.  The container or image will be compared to its parent layer
+
+## OPTIONS
+
+**--format**
+
+Alter the output into a different format.  The only valid format for diff is `json`.
+
+
+## EXAMPLE
+
+kpod diff redis:alpine
+C /usr
+C /usr/local
+C /usr/local/bin
+A /usr/local/bin/docker-entrypoint.sh
+
+kpod diff --format json redis:alpine
+{
+  "changed": [
+    "/usr",
+    "/usr/local",
+    "/usr/local/bin"
+  ],
+  "added": [
+    "/usr/local/bin/docker-entrypoint.sh"
+  ]
+}
+
+## SEE ALSO
+kpod(1)
+
+## HISTORY
+August 2017, Originally compiled by Ryan Cole <rycole@redhat.com>

docs/kpod-export.1.md

@@ -0,0 +1,44 @@
+% kpod(1) kpod-export - Simple tool to export a container's filesystem as a tarball
+% Urvashi Mohnani
+# kpod-export "1" "July 2017" "kpod"
+
+## NAME
+kpod-export - Export container's filesystem contents as a tar archive
+
+## SYNOPSIS
+**kpod export**
+**CONTAINER**
+[**--output**|**-o**]
+[**--help**|**-h**]
+
+## DESCRIPTION
+**kpod export** exports the filesystem of a container and saves it as a tarball
+on the local machine. **kpod export** writes to STDOUT by default and can be
+redirected to a file using the **output flag**.
+
+**kpod [GLOBAL OPTIONS]**
+
+**kpod export [GLOBAL OPTIONS]**
+
+**kpod export [OPTIONS] CONTAINER**
+
+## OPTIONS
+
+**--output, -o**
+Write to a file, default is STDOUT
+
+## EXAMPLES
+
+```
+# kpod export -o redis-container.tar 883504668ec465463bc0fe7e63d53154ac3b696ea8d7b233748918664ea90e57
+```
+
+```
+# kpod export > redis-container.tar 883504668ec465463bc0fe7e63d53154ac3b696ea8d7b233748918664ea90e57
+```
+
+## SEE ALSO
+kpod(1), kpod-import(1), crio(8), crio.conf(5)
+
+## HISTORY
+August 2017, Originally compiled by Urvashi Mohnani <umohnani@redhat.com>

docs/kpod-history.1.md

@@ -0,0 +1,106 @@
+% kpod(1) kpod-history - Simple tool to view the history of an image
+% Urvashi Mohnani
+% kpod-history "1" "JULY 2017" "kpod"
+
+## NAME
+kpod-history - Shows the history of an image
+
+## SYNOPSIS
+**kpod history**
+**IMAGE[:TAG|DIGEST]**
+[**--human**|**-H**]
+[**--no-trunc**]
+[**--quiet**|**-q**]
+[**--format**]
+[**--help**|**-h**]
+
+## DESCRIPTION
+**kpod history** displays the history of an image by printing out information
+about each layer used in the image. The information printed out for each layer
+include Created (time and date), Created By, Size, and Comment. The output can
+be truncated or not using the **--no-trunc** flag. If the **--human** flag is
+set, the time of creation and size are printed out in a human readable format.
+The **--quiet** flag displays the ID of the image only when set and the **--format**
+flag is used to print the information using the Go template provided by the user.
+
+Valid placeholders for the Go template are listed below:
+
+| **Placeholder** | **Description**                                                               |
+| --------------- | ----------------------------------------------------------------------------- |
+| .ID             | Image ID                                                                      |
+| .Created        | if **--human**, time elapsed since creation, otherwise time stamp of creation |
+| .CreatedBy      | Command used to create the layer                                              |
+| .Size           | Size of layer on disk                                                         |
+| .Comment        | Comment for the layer                                                         |
+
+**kpod [GLOBAL OPTIONS]**
+
+**kpod history [GLOBAL OPTIONS]**
+
+**kpod history [OPTIONS] IMAGE[:TAG|DIGEST]**
+
+## OPTIONS
+
+**--human, -H**
+    Display sizes and dates in human readable format
+
+**--no-trunc**
+    Do not truncate the output
+
+**--quiet, -q**
+    Print the numeric IDs only
+
+**--format**
+    Alter the output for a format like 'json' or a Go template.
+
+
+## EXAMPLES
+
+```
+# kpod history debian
+ID              CREATED       CREATED BY                                      SIZE       COMMENT
+b676ca55e4f2c   9 weeks ago   /bin/sh -c #(nop) CMD ["bash"]                  0 B
+<missing>       9 weeks ago   /bin/sh -c #(nop) ADD file:ebba725fb97cea4...   45.14 MB
+```
+
+```
+# kpod history --no-trunc=true --human=false debian
+ID              CREATED                CREATED BY                                      SIZE       COMMENT
+b676ca55e4f2c   2017-07-24T16:52:55Z   /bin/sh -c #(nop) CMD ["bash"]                  0
+<missing>       2017-07-24T16:52:54Z   /bin/sh -c #(nop) ADD file:ebba725fb97cea4...   45142935
+```
+
+```
+# kpod history --format "{{.ID}} {{.Created}}" debian
+b676ca55e4f2c   9 weeks ago
+<missing>       9 weeks ago
+```
+
+```
+# kpod history --format json debian
+[
+    {
+        "id": "b676ca55e4f2c0ce53d0636438c5372d3efeb5ae99b676fa5a5d1581bad46060",
+        "created": "2017-07-24T16:52:55.195062314Z",
+        "createdBy": "/bin/sh -c #(nop)  CMD [\"bash\"]",
+        "size": 0,
+        "comment": ""
+    },
+    {
+        "id": "b676ca55e4f2c0ce53d0636438c5372d3efeb5ae99b676fa5a5d1581bad46060",
+        "created": "2017-07-24T16:52:54.898893387Z",
+        "createdBy": "/bin/sh -c #(nop) ADD file:ebba725fb97cea45d0b1b35ccc8144e766fcfc9a78530465c23b0c4674b14042 in / ",
+        "size": 45142935,
+        "comment": ""
+    }
+]
+```
+
+## history
+Show the history of an image
+
+## SEE ALSO
+kpod(1), crio(8), crio.conf(5)
+
+## HISTORY
+July 2017, Originally compiled by Urvashi Mohnani <umohnani@redhat.com>

docs/kpod-images.1.md

@@ -0,0 +1,60 @@
+% kpod(1) kpod-images - List images in local storage
+% Dan Walsh
+# kpod-images "1" "March 2017" "kpod"
+
+## NAME
+kpod images - List images in local storage
+
+## SYNOPSIS
+**kpod** **images** [*options* [...]]
+
+## DESCRIPTION
+Displays locally stored images, their names, and their IDs.
+
+## OPTIONS
+
+**--digests**
+
+Show image digests
+
+**--filter, -f=[]**
+
+Filter output based on conditions provided (default [])
+
+**--format**
+
+Change the default output format.  This can be of a supported type like 'json'
+or a Go template.
+
+**--noheading, -n**
+
+Omit the table headings from the listing of images.
+
+**--no-trunc, --notruncate**
+
+Do not truncate output.
+
+**--quiet, -q**
+
+Lists only the image IDs.
+
+
+## EXAMPLE
+
+kpod images
+
+kpod images --quiet
+
+kpod images -q --noheading --notruncate
+
+kpod images --format json
+
+kpod images --format "{{.ID}}"
+
+kpod images --filter dangling=true
+
+## SEE ALSO
+kpod(1)
+
+## HISTORY
+March 2017, Originally compiled by Dan Walsh <dwalsh@redhat.com>

docs/kpod-info.1.md

@@ -0,0 +1,36 @@
+% kpod(1) kpod-version - Simple tool to view version information
+% Vincent Batts
+% kpod-version "1" "JULY 2017" "kpod"
+
+## NAME
+kpod-info - Display system information
+
+
+## SYNOPSIS
+**kpod** **info** [*options* [...]]
+
+
+## DESCRIPTION
+
+Information display here pertain to the host, current storage stats, and build of kpod. Useful for the user and when reporting issues.
+
+
+## OPTIONS
+
+**--debug, -D**
+
+Show additional information
+
+**--format**
+
+Change output format to "json" or a Go template.
+
+
+## EXAMPLE
+
+`kpod info`
+
+`kpod info --debug --format json| jq .host.kernel`
+
+## SEE ALSO
+crio(8), crio.conf(5)

docs/kpod-inspect.1.md

@@ -0,0 +1,171 @@
+% kpod(1) kpod-inspect - Display a container or image's configuration
+% Dan Walsh
+# kpod-inspect "1" "July 2017" "kpod"
+
+## NAME
+kpod inspect - Display a container or image's configuration
+
+## SYNOPSIS
+**kpod** **inspect** [*options* [...]] name
+
+## DESCRIPTION
+This displays the low-level information on containers and images identified by name or ID. By default, this will render all results in a JSON array. If the container and image have the same name, this will return container JSON for unspecified type. If a format is specified, the given template will be executed for each result.
+
+## OPTIONS
+
+**--type, t="TYPE"**
+
+Return data on items of the specified type.  Type can be 'container', 'image' or 'all' (default: all)
+
+**--format, -f="FORMAT"**
+
+Format the output using the given Go template
+
+**--size**
+
+Display the total file size if the type is a container
+
+
+## EXAMPLE
+
+kpod inspect redis:alpine
+
+{
+    "ArgsEscaped": true,
+    "AttachStderr": false,
+    "AttachStdin": false,
+    "AttachStdout": false,
+    "Cmd": [
+        "/bin/sh",
+        "-c",
+        "#(nop) ",
+        "CMD [\"redis-server\"]"
+    ],
+    "Domainname": "",
+    "Entrypoint": [
+        "entrypoint.sh"
+    ],
+    "Env": [
+        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+        "REDIS_VERSION=3.2.9",
+        "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.2.9.tar.gz",
+        "REDIS_DOWNLOAD_SHA=6eaacfa983b287e440d0839ead20c2231749d5d6b78bbe0e0ffa3a890c59ff26"
+    ],
+    "ExposedPorts": {
+        "6379/tcp": {}
+    },
+    "Hostname": "e1ede117fb1e",
+    "Image": "sha256:75e877aa15b534396de82d385386cc4dda7819d5cbb018b9f97b77aeb8f4b55a",
+    "Labels": {},
+    "OnBuild": [],
+    "OpenStdin": false,
+    "StdinOnce": false,
+    "Tty": false,
+    "User": "",
+    "Volumes": {
+        "/data": {}
+    },
+    "WorkingDir": "/data"
+}
+{
+    "ID": "b3f2436bdb978c1d33b1387afb5d7ba7e3243ed2ce908db431ac0069da86cb45",
+    "Names": [
+        "docker.io/library/redis:alpine"
+    ],
+    "Digests": [
+        "sha256:88286f41530e93dffd4b964e1db22ce4939fffa4a4c665dab8591fbab03d4926",
+        "sha256:07b1ac6c7a5068201d8b63a09bb15358ec1616b813ef3942eb8cc12ae191227f",
+        "sha256:91e2e140ea27b3e89f359cd9fab4ec45647dda2a8e5fb0c78633217d9dca87b5",
+        "sha256:08957ceaa2b3be874cde8d7fa15c274300f47185acd62bca812a2ffb6228482d",
+        "sha256:acd3d12a6a79f772961a771f678c1a39e1f370e7baeb9e606ad8f1b92572f4ab",
+        "sha256:4ad88df090801e8faa8cf0be1f403b77613d13e11dad73f561461d482f79256c",
+        "sha256:159ac12c79e1a8d85dfe61afff8c64b96881719139730012a9697f432d6b739a"
+    ],
+    "Parent": "",
+    "Comment": "",
+    "Created": "2017-06-28T22:14:36.35280993Z",
+    "Container": "ba8d6c6b0d7fdd201fce404236136b44f3bfdda883466531a3d1a1f87906770b",
+    "ContainerConfig": {
+        "Hostname": "e1ede117fb1e",
+        "Domainname": "",
+        "User": "",
+        "AttachStdin": false,
+        "AttachStdout": false,
+        "AttachStderr": false,
+        "Tty": false,
+        "OpenStdin": false,
+        "StdinOnce": false,
+        "Env": [
+            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+            "REDIS_VERSION=3.2.9",
+            "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.2.9.tar.gz",
+            "REDIS_DOWNLOAD_SHA=6eaacfa983b287e440d0839ead20c2231749d5d6b78bbe0e0ffa3a890c59ff26"
+        ],
+        "Cmd": [
+            "/bin/sh",
+            "-c",
+            "#(nop) ",
+            "CMD [\"redis-server\"]"
+        ],
+        "ArgsEscaped": true,
+        "Image": "sha256:75e877aa15b534396de82d385386cc4dda7819d5cbb018b9f97b77aeb8f4b55a",
+        "Volumes": {
+            "/data": {}
+        },
+        "WorkingDir": "/data",
+        "Entrypoint": [
+            "entrypoint.sh"
+        ],
+        "Labels": {},
+        "OnBuild": []
+    },
+    "Author": "",
+    "Config": {
+        "ExposedPorts": {
+            "6379/tcp": {}
+        },
+        "Env": [
+            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+            "REDIS_VERSION=3.2.9",
+            "REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.2.9.tar.gz",
+            "REDIS_DOWNLOAD_SHA=6eaacfa983b287e440d0839ead20c2231749d5d6b78bbe0e0ffa3a890c59ff26"
+        ],
+        "Entrypoint": [
+            "entrypoint.sh"
+        ],
+        "Cmd": [
+            "redis-server"
+        ],
+        "Volumes": {
+            "/data": {}
+        },
+        "WorkingDir": "/data"
+    },
+    "Architecture": "amd64",
+    "OS": "linux",
+    "Size": 3965955,
+    "VirtualSize": 19808086,
+    "GraphDriver": {
+        "Name": "overlay",
+        "Data": {
+            "MergedDir": "/var/lib/containers/storage/overlay/2059d805c90e034cb773d9722232ef018a72143dd31113b470fb876baeccd700/merged",
+            "UpperDir": "/var/lib/containers/storage/overlay/2059d805c90e034cb773d9722232ef018a72143dd31113b470fb876baeccd700/diff",
+            "WorkDir": "/var/lib/containers/storage/overlay/2059d805c90e034cb773d9722232ef018a72143dd31113b470fb876baeccd700/work"
+        }
+    },
+    "RootFS": {
+        "type": "layers",
+        "diff_ids": [
+            "sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0",
+            "sha256:c92a8fc997217611d0bfc9ff14d7ec00350ca564aef0ecbf726624561d7872d7",
+            "sha256:d4c406dea37a107b0cccb845611266a146725598be3e82ba31c55c08d1583b5a",
+            "sha256:8b4fa064e2b6c03a6c37089b0203f167375a8b49259c0ad7cb47c8c1e58b3fa0",
+            "sha256:c393e3d0b00ddf6b4166f1e2ad68245e08e9e3be0a0567a36d0a43854f03bfd6",
+            "sha256:38047b4117cb8bb3bba82991daf9a4e14ba01f9f66c1434d4895a7e96f67d8ba"
+        ]
+    }
+}
+
+
+## SEE ALSO
+kpod(1)

docs/kpod-kill.1.md

@@ -0,0 +1,33 @@
+% kpod(1) kpod-kill- Kill one or more containers with a signal
+% Brent Baude
+# kpod-kill"1" "September 2017" "kpod"
+
+## NAME
+kpod kill - Kills one or more containers with a signal
+
+## SYNOPSIS
+**kpod kill [OPTIONS] CONTAINER [...]**
+
+## DESCRIPTION
+The main process inside each container specified will be sent SIGKILL, or any signal specified with option --signal.
+
+## OPTIONS
+
+**--signal, s**
+
+Signal to send to the container. For more information on Linux signals, refer to *man signal(7)*.
+
+
+## EXAMPLE
+
+kpod kill mywebserver
+
+kpod kill 860a4b23
+
+kpod kill --signal TERM 860a4b23
+
+## SEE ALSO
+kpod(1), kpod-stop(1)
+
+## HISTORY
+September 2017, Originally compiled by Brent Baude <bbaude@redhat.com>

docs/kpod-load.1.md

@@ -0,0 +1,76 @@
+% kpod(1) kpod-load - Simple tool to load an image from an archive to containers-storage
+% Urvashi Mohnani
+# kpod-load "1" "July 2017" "kpod"
+
+## NAME
+kpod-load - Load an image from docker archive
+
+## SYNOPSIS
+**kpod load**
+**NAME[:TAG|@DIGEST]**
+[**--input**|**-i**]
+[**--quiet**|**-q**]
+[**--help**|**-h**]
+
+## DESCRIPTION
+**kpod load** copies an image from either **docker-archive** or **oci-archive** stored
+on the local machine. **kpod load** reads from stdin by default or a file if the **input** flag is set.
+The **quiet** flag suppresses the output when set.
+
+**kpod [GLOBAL OPTIONS]**
+
+**kpod load [GLOBAL OPTIONS]**
+
+**kpod load [OPTIONS] NAME[:TAG|@DIGEST]**
+
+## OPTIONS
+
+**--input, -i**
+Read from archive file, default is STDIN
+
+**--quiet, -q**
+Suppress the output
+
+**--signature-policy="PATHNAME"**
+
+Pathname of a signature policy file to use.  It is not recommended that this
+option be used, as the default behavior of using the system-wide default policy
+(frequently */etc/containers/policy.json*) is most often preferred
+
+## EXAMPLES
+
+```
+# kpod load --quiet -i fedora.tar
+```
+
+```
+# kpod load -q --signature-policy /etc/containers/policy.json -i fedora.tar
+```
+
+```
+# kpod load < fedora.tar
+Getting image source signatures
+Copying blob sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0
+ 0 B / 4.03 MB [---------------------------------------------------------------]
+Copying config sha256:7328f6f8b41890597575cbaadc884e7386ae0acc53b747401ebce5cf0d624560
+ 0 B / 1.48 KB [---------------------------------------------------------------]
+Writing manifest to image destination
+Storing signatures
+```
+
+```
+# cat fedora.tar | kpod load
+Getting image source signatures
+Copying blob sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0
+ 0 B / 4.03 MB [---------------------------------------------------------------]
+Copying config sha256:7328f6f8b41890597575cbaadc884e7386ae0acc53b747401ebce5cf0d624560
+ 0 B / 1.48 KB [---------------------------------------------------------------]
+Writing manifest to image destination
+Storing signatures
+```
+
+## SEE ALSO
+kpod(1), kpod-save(1), crio(8), crio.conf(5)
+
+## HISTORY
+July 2017, Originally compiled by Urvashi Mohnani <umohnani@redhat.com>

docs/kpod-login.1.md

@@ -0,0 +1,65 @@
+% kpod(1) kpod-login - Simple tool to login to a registry server
+% Urvashi Mohnani
+# kpod-login "1" "August 2017" "kpod"
+
+## NAME
+kpod-login - Login to a container registry
+
+## SYNOPSIS
+**kpod login**
+[**--help**|**-h**]
+[**--authfile**]
+[**--user**|**-u**]
+[**--password**|**-p**]
+**REGISTRY**
+
+## DESCRIPTION
+**kpod login** logs into a specified registry server with the correct username
+and password. **kpod login** reads in the username and password from STDIN.
+The username and password can also be set using the **username** and **password** flags.
+The path of the authentication file can be specified by the user by setting the **authfile**
+flag. The default path used is **${XDG\_RUNTIME_DIR}/containers/auth.json**.
+
+**kpod [GLOBAL OPTIONS]**
+
+**kpod login [GLOBAL OPTIONS]**
+
+**kpod login [OPTIONS] REGISTRY [GLOBAL OPTIONS]**
+
+## OPTIONS
+
+**--password, -p**
+Password for registry
+
+**--username, -u**
+Username for registry
+
+**--authfile**
+Path of the authentication file. Default is ${XDG_\RUNTIME\_DIR}/containers/auth.json
+
+## EXAMPLES
+
+```
+# kpod login docker.io
+Username: umohnani
+Password:
+Login Succeeded!
+```
+
+```
+# kpod login -u testuser -p testpassword localhost:5000
+Login Succeeded!
+```
+
+```
+# kpod login --authfile authdir/myauths.json docker.io
+Username: umohnani
+Password:
+Login Succeeded!
+```
+
+## SEE ALSO
+kpod(1), kpod-logout(1), crio(8), crio.conf(5)
+
+## HISTORY
+August 2017, Originally compiled by Urvashi Mohnani <umohnani@redhat.com>

docs/kpod-logout.1.md

@@ -0,0 +1,56 @@
+% kpod(1) kpod-logout - Simple tool to logout of a registry server
+% Urvashi Mohnani
+# kpod-logout "1" "August 2017" "kpod"
+
+## NAME
+kpod-logout - Logout of a container registry
+
+## SYNOPSIS
+**kpod logout**
+[**--help**|**-h**]
+[**--authfile**]
+[**--all**|**-a**]
+**REGISTRY**
+
+## DESCRIPTION
+**kpod logout** logs out of a specified registry server by deleting the cached credentials
+stored in the **auth.json** file. The path of the authentication file can be overrriden by the user by setting the **authfile** flag.
+The default path used is **${XDG\_RUNTIME_DIR}/containers/auth.json**.
+All the cached credentials can be removed by setting the **all** flag.
+
+**kpod [GLOBAL OPTIONS]**
+
+**kpod logout [GLOBAL OPTIONS]**
+
+**kpod logout [OPTIONS] REGISTRY [GLOBAL OPTIONS]**
+
+## OPTIONS
+
+**--authfile**
+Path of the authentication file. Default is ${XDG_\RUNTIME\_DIR}/containers/auth.json
+
+**--all, -a**
+Remove the cached credentials for all registries in the auth file
+
+## EXAMPLES
+
+```
+# kpod logout docker.io
+Remove login credentials for https://registry-1.docker.io/v2/
+```
+
+```
+# kpod logout --authfile authdir/myauths.json docker.io
+Remove login credentials for https://registry-1.docker.io/v2/
+```
+
+```
+# kpod logout --all
+Remove login credentials for all registries
+```
+
+## SEE ALSO
+kpod(1), kpod-login(1), crio(8), crio.conf(5)
+
+## HISTORY
+August 2017, Originally compiled by Urvashi Mohnani <umohnani@redhat.com>

docs/kpod-logs.1.md

@@ -0,0 +1,61 @@
+% kpod(1) kpod-logs - Fetch the logs of a container
+% Ryan Cole
+# kpod-logs "1" "March 2017" "kpod"
+
+## NAME
+kpod logs - Fetch the logs of a container
+
+## SYNOPSIS
+**kpod** **logs** [*options* [...]] container
+
+## DESCRIPTION
+The kpod logs command batch-retrieves whatever logs are present for a container at the time of execution.  This does not guarantee execution order when combined with kpod run (i.e. your run may not have generated any logs at the time you execute kpod logs
+
+## OPTIONS
+
+**--follow, -f**
+
+Follow log output.  Default is false
+
+**--since=TIMESTAMP**
+
+Show logs since TIMESTAMP
+
+**--tail=LINES**
+
+Ouput the specified number of LINES at the end of the logs.  LINES must be a positive integer.  Defaults to 0, which prints all lines
+
+## EXAMPLE
+
+kpod logs b3f2436bdb978c1d33b1387afb5d7ba7e3243ed2ce908db431ac0069da86cb45
+
+2017/08/07 10:16:21 Seeked /var/log/crio/pods/eb296bd56fab164d4d3cc46e5776b54414af3bf543d138746b25832c816b933b/c49f49788da14f776b7aa93fb97a2a71f9912f4e5a3e30397fca7dfe0ee0367b.log - &{Offset:0 Whence:0}
+1:C 07 Aug 14:10:09.055 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
+1:C 07 Aug 14:10:09.055 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=1, just started
+1:C 07 Aug 14:10:09.055 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
+1:M 07 Aug 14:10:09.055 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
+1:M 07 Aug 14:10:09.055 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
+1:M 07 Aug 14:10:09.055 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
+1:M 07 Aug 14:10:09.056 * Running mode=standalone, port=6379.
+1:M 07 Aug 14:10:09.056 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
+1:M 07 Aug 14:10:09.056 # Server initialized
+
+
+kpod logs --tail 2 b3f2436bdb97
+
+1:M 07 Aug 14:10:09.056 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
+1:M 07 Aug 14:10:09.056 # Server initialized
+
+kpod logs 224c375f27cd --since 2017-08-07T10:10:09.055837383-04:00 myserver
+
+1:M 07 Aug 14:10:09.055 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
+1:M 07 Aug 14:10:09.055 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
+1:M 07 Aug 14:10:09.056 * Running mode=standalone, port=6379.
+1:M 07 Aug 14:10:09.056 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
+1:M 07 Aug 14:10:09.056 # Server initialized
+
+## SEE ALSO
+kpod(1)
+
+## HISTORY
+August 2017, Originally compiled by Ryan Cole <rycole@redhat.com>

docs/kpod-mount.1.md

@@ -0,0 +1,50 @@
+% kpod(1) kpod-mount - Mount a working container's root filesystem.
+% Dan Walsh
+# kpod-mount "1" "July 2017" "kpod"
+
+## NAME
+kpod mount - Mount a working container's root filesystem
+
+## SYNOPSIS
+**kpod** **mount**
+
+**kpod** **mount** **containerID**
+
+## DESCRIPTION
+Mounts the specified container's root file system in a location which can be
+accessed from the host, and returns its location.
+
+If you execute the command without any arguments, the tool will list all of the
+currently mounted containers.
+
+## RETURN VALUE
+The location of the mounted file system.  On error an empty string and errno is
+returned.
+
+## OPTIONS
+
+**--format**
+    Print the mounted containers in specified format (json)
+
+**--notruncate**
+
+Do not truncate IDs in output.
+
+**--label**
+
+SELinux label for the mount point
+
+## EXAMPLE
+
+kpod mount c831414b10a3
+
+/var/lib/containers/storage/overlay/f3ac502d97b5681989dff84dfedc8354239bcecbdc2692f9a639f4e080a02364/merged
+
+kpod mount
+
+c831414b10a3 /var/lib/containers/storage/overlay/f3ac502d97b5681989dff84dfedc8354239bcecbdc2692f9a639f4e080a02364/merged
+
+a7060253093b /var/lib/containers/storage/overlay/0ff7d7ca68bed1ace424f9df154d2dd7b5a125c19d887f17653cbcd5b6e30ba1/merged
+
+## SEE ALSO
+kpod(1), kpod-umount(1), mount(8)