Compare commits

..

8 Commits

Author SHA1 Message Date
Aidan Delaney 96daeb5dfa
Merge branch 'main' into feature/ratchet-vale 2025-08-12 05:47:38 +01:00
Richard Schneeman 44292df114
Fix `launch.toml` in use-inline-buildpacks.md (#843)
The `launch.toml` command takes a slice and not a string

```
remote: ERROR: failed to build: toml: line 3 (last key "processes.command"): incompatible types: TOML value has type string; destination has type slice
```

Spec: d29ba81518/buildpack.md (launchtoml-toml)

Signed-off-by: Richard Schneeman <richard.schneeman+no-recruiters@gmail.com>
2025-08-12 05:47:14 +01:00
Richard Schneeman b927876fd3
Clear up `clean-env` docs (#841)
* Clear up `clean-env` docs

## Context

I was asked to validate that several buildpacks were correctly passing user-supplied environment variables to sub-processes for the purpose of configuring access to dependencies that require credentials.

The docs clearly state that a `buildpack.toml` with `clear-env` should have specific behavior regarding environment variable visibility, but it's not clear that it means that the `CNB_PLATFORM_DIR/env/` is ALWAYS provided and that `clear-env = false` means that information is merged into the current environment.

If you read it literally and you know that `$CNB_PLATFORM_DIR/env/` is a mechanism to provide the environment variables to a buildpack, you might interpret it as the ability to hide these platform envs from the buildpack. When really, it's saying "lifecycle will make things easy for you if you just want a firehose of all the envs, it will copy the user-provided ones into the current environment so you don't need to." It does this by default with the ability to opt out by setting `clear-env = true`.

## Change

I'm suggesting we tighten up the language here to make it clear that `$CNB_PLATFORM_DIR/env/` is always available, and that setting `clear-env = true` will tell the lifecycle to copy the contents of `CNB_PLATFORM_DIR/env/` into the current environment. Specific language suggestions pending. This logic is in several places:

- https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/clear-env/
- https://buildpacks.io/docs/reference/config/buildpack-config/
- https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/

Signed-off-by: Schneems <richard.schneeman+foo@gmail.com>

* Fix vale failures

Signed-off-by: Schneems <richard.schneeman+foo@gmail.com>

---------

Signed-off-by: Schneems <richard.schneeman+foo@gmail.com>
2025-08-12 05:46:43 +01:00
Charles Moulliard 85ca62d2ef
Updated the tekton documentation to use the new buildpack task (#837)
* Updated the tekton documentation to use the new buildpack task supporting the extension. #814

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Add additional words to vale dictionary

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Extend the documentation to have a chapter covering extension. Add new pipeline parameters

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Review the scenario and tested it using ubi8 and jammy builder images

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Fix some vale vocabularies

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Remove the cache workspace as not used and documented part of task's documentation

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

* Switch the builder image to use: cnbs/sample-builder

Signed-off-by: cmoulliard <cmoulliard@redhat.com>

---------

Signed-off-by: cmoulliard <cmoulliard@redhat.com>
2025-06-30 09:52:40 +01:00
Aidan Delaney cffa864ed4
Extract podman ssh key (#828)
Pull in the root podman ssh key by invoking podman system
connection

Signed-off-by: Aidan Delaney <aidan.delaney@gmail.com>
2025-06-26 12:30:21 -05:00
dependabot[bot] c5243ea67f
Bump buildpacks/github-actions from 5.8.8 to 5.9.1 (#836)
Bumps [buildpacks/github-actions](https://github.com/buildpacks/github-actions) from 5.8.8 to 5.9.1.
- [Release notes](https://github.com/buildpacks/github-actions/releases)
- [Commits](https://github.com/buildpacks/github-actions/compare/v5.8.8...v5.9.1)

---
updated-dependencies:
- dependency-name: buildpacks/github-actions
  dependency-version: 5.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-26 07:58:22 +00:00
DaxServer d596d3ddeb
Update 06_caching.md (#832)
Signed-off-by: DaxServer <7479937+DaxServer@users.noreply.github.com>
2025-06-26 08:58:05 +01:00
dependabot[bot] 522552f1d0
Bump buildpacks/github-actions from 5.8.3 to 5.8.8 (#800)
Bumps [buildpacks/github-actions](https://github.com/buildpacks/github-actions) from 5.8.3 to 5.8.8.
- [Release notes](https://github.com/buildpacks/github-actions/releases)
- [Commits](https://github.com/buildpacks/github-actions/compare/v5.8.3...v5.8.8)

---
updated-dependencies:
- dependency-name: buildpacks/github-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Aidan Delaney <adelaney21@bloomberg.net>
2025-04-10 08:52:23 +01:00
9 changed files with 150 additions and 67 deletions

View File

@ -10,4 +10,10 @@ rebase
Rebasing
SBOM(?s)
semver
Syft
Syft
[Tt]ekton
[Kk]ubernetes
uid
Minikube
kubectl
Quarkus

View File

@ -24,7 +24,7 @@ jobs:
- name: Install Dependencies
run: sudo apt-get install make curl jq
- name: Install pack
uses: buildpacks/github-actions/setup-pack@v5.8.3
uses: buildpacks/github-actions/setup-pack@v5.9.1
with:
pack-version: '0.36.0'
- name: Test

View File

@ -11,7 +11,12 @@ Environment variables are a common way to configure various buildpacks at build-
<!--more-->
Below are a few ways you can do so. All of them will use our [samples] repo for simplicity.
Below are a few ways you can do so. All of them use the [samples] repository for
simplicity. The following documentation assumes that all participating buildpack
either use `clear-env = false` as the default in their
[buildpack.toml](https://buildpacks.io/docs/reference/config/buildpack-config/),
or if they use `clear-env = true`, that they merge in filtered user supplied
environment variables.
### Using CLI arguments (`--env`)

View File

@ -50,7 +50,7 @@ find . -type f -name $(my_data_files) -delete
cat <<EOF > ${1}/launch.toml
[[processes]]
type = 'bash'
command = 'bin/bash'
command = ['bin/bash']
EOF
"""
```

View File

@ -3,20 +3,30 @@ title="Clear the buildpack environment"
weight=99
+++
"Clearing" the buildpack environment with `clear-env` is the process of preventing end-users from customizing a buildpack's behavior through environment variables.
"Clearing" the buildpack environment with `clear-env` is the process of preventing the lifecycle from automatically merging user-provided environment variables into the buildpack's executing process's environment variables.
<!--more-->
Buildpack authors may elect to clear user-provided environment variables when `bin/detect` and `bin/build` are executed. This is achieved by setting `clear-env` to `true` in [buildpack.toml](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml); by default `clear-env` is set to `false`.
User-provided environment variables are always written to disk at `$CNB_PLATFORM_DIR/env/` as "platform" environment variables and are available to a buildpack regardless of the `clear-env` setting. For example, if someone runs `pack build --env HELLO=world`, there is always a `$CNB_PLATFORM_DIR/env/hello` file with the contents of `world`.
* When `clear-env` is set to `true` for a given buildpack, the `lifecycle` will not set user-provided environment variables when running `/bin/detect` or `/bin/build`.
* If a buildpack does allow customization by the end-user through the environment (`clear-env` is `false`), there is a special convention for naming the environment variables recognized by the buildpack, shown in the following table:
By default with `clear-env = false`, the lifecycle automatically copies these user-provided environment variables into the current process environment when executing `/bin/detect` and `/bin/build`. This provides a comprehensive stream of all user environment variables for a buildpack that wants easy access to user customization.
| Env Variable | Description | Detect | Build | Launch |
Setting `clear-env = true` in the [buildpack.toml](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpacktoml-toml) prevents this automatic merging, giving a buildpack complete control over which user-provided environment variables to use. This provides maximum isolation and predictability for a buildpack that wants to be more selective about environment variable usage.
For example, if a user has specified the `PATH` environment variable, then a buildpack written in bash might unexpectedly find that tools it relies on such as `cp` aren't the ones it expects. However, setting `clear-env = true` won't change runtime behavior.
* When you set `clear-env` to `true` for a given buildpack, the `lifecycle` writes user-provided environment variables to disk at `$CNB_PLATFORM_DIR/env/` but it won't copy those variables into the buildpack process when running `/bin/detect` or `/bin/build`.
* If a buildpack uses `clear-env = false` which allows customization by the end-user through the environment, there is a special convention for naming the environment variables recognized by the buildpack, shown in the following table:
| Environment Variable | Description | Detect | Build | Launch |
|------------------------|---------------------------------------------------|--------|-------|--------|
| `BP_*` | User-provided variable for buildpack | [x] | [x] | |
| `BPL_*` | User-provided variable for exec.d | | | [x] |
### Further Reading
Buildpack that use `clear-env = true` should filter and export environment variables from `$CNB_PLATFORM_DIR/env/` while warning on the filtered variables. Emitting a warning helps users understand if runtime behavior differs from build time behavior. Sourcing the user environment variables is critical to allowing sub-processes access to credentials.
For more about how environment variables are specified by end-users, see the page for how to [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/).
For example, if a private gem server hosted on `gems.example.com` needs access in a sub-process such as `bundle install`, the user must provide `BUNDLE_GEMS__EXAMPLE__COM=<username>:<password>`. If `clear-env = true` and the buildpack doesn't use platform environment variables, then trying to access that resource would fail.
### Further reading
For more about how end-users specify environment variables, see the page for how to [customize buildpack behavior with build-time environment variables](https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/configure-build-time-environment/).

View File

@ -84,7 +84,7 @@ Reusing layer 'examples/node-js:node-js'
Now, let's implement the caching logic. We need to record the version of the NodeJS runtime that is used in a build. On subsequent builds, the caching logic will detect if the NodeJS version is the same as the version in the cached layer. We restore the previous layer from the cache if the current requested NodeJS version matches the previous NodeJS runtime version.
<!-- test:file=node-js-buildpack/bin/build -->
```
```bash
#!/usr/bin/env bash
set -eo pipefail

View File

@ -9,16 +9,14 @@ weight=6
[Tekton][tekton] is an open-source CI/CD system running on k8s.
The CNB project has created two reference "tasks" for performing buildpacks builds,
both of which use the [lifecycle][lifecycle] directly (i.e. they do not use `pack`).
The CNB project has created a reference "task" for performing buildpacks builds with or without extensions (aka Dockerfile to be applied) top
of the [lifecycle][lifecycle] tool (i.e. they do not use `pack`).
<!--more-->
The [Buildpacks Phases Task][buildpacks-phases] calls the individual [lifecycle][lifecycle] binaries (prepare, analyze, detect, restore, build or extender, export), to run each phase in a separate container.
They are:
The uid and gid as defined part of the builder image will be used to build the image.
1. [buildpacks][buildpacks-task] `task` &rarr; This task, which we recommend using, calls the `creator` binary of the
[lifecycle][lifecycle] to construct, and optionally publish, a runnable image.
2. [buildpacks-phases][buildpacks-phases] `task` &rarr; This task calls the individual [lifecycle][lifecycle] binaries, to run each phase in a separate container.
The different parameters to customize the task are defined part of the task's documentation under the section `parameters`.
## Set Up
@ -32,28 +30,22 @@ Before we get started, make sure you've got the following installed:
### 1. Install Tekton and Tekton Dashboard
To start, set up `Tekton`, using the Tekton [documentation][tekton-setup].
To start, set up a `Tekton` version `>= 1.0`, using the Tekton [documentation][tekton-setup].
We also recommend using the `Tekton dashboard`. To install it, follow the steps in the [dashboard docs][tekton-dashboard-setup], and
start the dashboard server.
> NOTE: If you run Tekton on a Kind or Minikube Kubernetes cluster, be sure to set the `coschedule` flag to `disabled` within the `feature-flags` ConfigMap.
### 2. Install the Buildpacks Task
Install the latest version of the buildpacks task (currently `0.6`), by running:
Install the latest version of the buildpacks task (currently `0.3`), by running:
```shell
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/buildpacks/0.6/buildpacks.yaml
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/refs/heads/main/task/buildpacks-phases/0.3/buildpacks-phases.yaml
```
### 3. Install git-clone Task
For our `pipeline`, we will use the `git-clone` task to clone a repository. Install the latest version (currently `0.4`), by running:
```shell
kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/task/git-clone/0.4/git-clone.yaml
```
### 4. Define and Apply Tekton Pipeline Resources
### 3. Define and Apply Tekton Pipeline Resources
In order to set up our pipeline, we will need to define a few things:
@ -63,9 +55,9 @@ In order to set up our pipeline, we will need to define a few things:
- PersistentVolumeClaim &rarr; A `PersistentVolumeClaim` (a general Kubernetes concept, generally shortened to PVC) is
a request for storage by a user.
#### 4.1 PVCs
#### 4.1 Persistent Volume
Create a file `resources.yml` that defines a `PersistentVolumeClaim`:
Create a file `resources.yml` that defines a `PersistentVolumeClaim` able to store the git cloned project and buildpacks files:
```yaml
apiVersion: v1
@ -83,12 +75,12 @@ spec:
#### 4.2 Authorization
> NOTE: You don't need to use authorization if you are pushing to a local registry. However, if you are pushing to a
> remote registry (e.g. `DockerHub`, `GCR`), you need to add authorization
> remote registry (e.g. `DockerHub`, `GCR`, `quay.io), you need to add authorization
Create a `Secret` containing username and password that the build should use to authenticate to the container registry.
```shell
kubectl create secret docker-registry docker-user-pass \
kubectl create secret docker-registry registry-user-pass \
--docker-username=<USERNAME> \
--docker-password=<PASSWORD> \
--docker-server=<LINK TO REGISTRY, e.g. https://index.docker.io/v1/ > \
@ -103,57 +95,70 @@ kind: ServiceAccount
metadata:
name: buildpacks-service-account
secrets:
- name: docker-user-pass
- name: registry-user-pass
```
> NOTE: This service account will be used by Tekton in order to mount the credentials as docker config file part of the pod running buildpacks
#### 4.3 Pipeline
Create a file `pipeline.yml` that defines the `Pipeline`, and relevant resources:
```yaml
apiVersion: tekton.dev/v1beta1
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: buildpacks-test-pipeline
spec:
params:
- name: git-url
type: string
description: URL of the project to git clone
- name: source-subpath
type: string
description: The subpath within the git project
- name: image
type: string
description: image URL to push
- name: builder
type: string
description: builder image URL
- name: env-vars
type: array
description: env vars to pass to the lifecycle binaries
workspaces:
- name: source-workspace # Directory where application source is located. (REQUIRED)
- name: cache-workspace # Directory where cache is stored (OPTIONAL)
tasks:
- name: fetch-repository # This task fetches a repository from github, using the `git-clone` task you installed
taskRef:
name: git-clone
resolver: http
params:
- name: url
value: https://raw.githubusercontent.com/tektoncd/catalog/refs/heads/main/task/git-clone/0.9/git-clone.yaml
workspaces:
- name: output
workspace: source-workspace
params:
- name: url
value: https://github.com/buildpacks/samples
- name: subdirectory
value: ""
value: "$(params.git-url)"
- name: deleteExisting
value: "true"
- name: buildpacks # This task uses the `buildpacks` task to build the application
- name: buildpacks # This task uses the `buildpacks phases` task to build the application
taskRef:
name: buildpacks
name: buildpacks-phases
runAfter:
- fetch-repository
workspaces:
- name: source
workspace: source-workspace
- name: cache
workspace: cache-workspace
params:
- name: APP_IMAGE
value: "$(params.image)"
- name: SOURCE_SUBPATH
value: "apps/java-maven" # This is the path within the samples repo you want to build (OPTIONAL, default: "")
- name: BUILDER_IMAGE
value: paketobuildpacks/builder:base # This is the builder we want the task to use (REQUIRED)
value: "$(params.source-subpath)"
- name: CNB_BUILDER_IMAGE
value: "$(params.builder)"
- name: CNB_ENV_VARS
value: "$(params.env-vars[*])"
- name: display-results
runAfter:
- buildpacks
@ -185,12 +190,13 @@ kubectl apply -f resources.yml -f sa.yml -f pipeline.yml
Create a file `run.yml`, which defines the `PipelineRun`:
```yaml
apiVersion: tekton.dev/v1beta1
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: buildpacks-test-pipeline-run
spec:
serviceAccountName: buildpacks-service-account # Only needed if you set up authorization
taskRunTemplate:
serviceAccountName: buildpacks-service-account # Only needed if you set up authorization
pipelineRef:
name: buildpacks-test-pipeline
workspaces:
@ -198,11 +204,16 @@ spec:
subPath: source
persistentVolumeClaim:
claimName: buildpacks-source-pvc
- name: cache-workspace
subPath: cache
persistentVolumeClaim:
claimName: buildpacks-source-pvc
params:
- # The url of the git project to clone (REQURED).
name: git-url
value: https://github.com/buildpacks/samples
- # This is the path within the git project you want to build (OPTIONAL, default: "")
name: source-subpath
value: "apps/java-maven"
- # This is the builder image we want the task to use (REQUIRED).
name: builder
value: cnbs/sample-builder:noble
- name: image
value: <REGISTRY/IMAGE NAME, eg gcr.io/test/image > # This defines the name of output image
```
@ -225,27 +236,73 @@ kubectl describe pipelinerun buildpacks-test-pipeline-run
or by using the Tekton Dashboard.
Once the application is successfully built, you can pull it and run it by running:
Once the application is successfully built, you can pull and run it by running:
```shell
docker pull some-output-image
docker | podman pull <REGISTRY/IMAGE NAME>
docker | podman run -it <REGISTRY/IMAGE NAME>
```
### 7. Cleanup (Optional)
### 7. Using extension
If your builder image supports the [extension][extension] mechanism able to customize the [build][extension-build] or the [run (aka execution)][extension-run], then you can replay this scenario by simply changing within the `PipelineRun` resource file the builder parameter
```yaml
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: buildpacks-test-pipeline-run
spec:
taskRunTemplate:
serviceAccountName: buildpacks-service-account
pipelineRef:
name: buildpacks-test-pipeline
workspaces:
- name: source-workspace
subPath: source
persistentVolumeClaim:
claimName: buildpacks-source-pvc
params:
- name: image
value: <REGISTRY/IMAGE NAME, eg gcr.io/test/image>
- name: git-url
value: https://github.com/quarkusio/quarkus-quickstarts
- name: source-subpath
value: "getting-started"
- name: builder
value: paketobuildpacks/builder-ubi8-base:0.1.30
- name: env-vars
value:
- BP_JVM_VERSION=21
```
When the build process starts, then you should see, part of the extender step, if you build a Java runtime (Quarkus, Spring boot, etc) such log messages if the extension installs by example a different JDK
```txt
2025-06-27T11:32:25.067007701Z time="2025-06-27T11:32:25Z" level=info msg="Performing slow lookup of group ids for root"
2025-06-27T11:32:25.067243910Z time="2025-06-27T11:32:25Z" level=info msg="Running: [/bin/sh -c echo ${build_id}]"
2025-06-27T11:32:25.095150183Z 9e447871-e415-4018-a860-d5a66d925a57
2025-06-27T11:32:25.096877516Z time="2025-06-27T11:32:25Z" level=info msg="Taking snapshot of full filesystem..."
2025-06-27T11:32:25.280396774Z time="2025-06-27T11:32:25Z" level=info msg="Pushing layer oci:/kaniko/cache/layers/cached:a035cdb3949daa8f4e7b2c523ea0d73741c7c2d5b09981c261ebae99fd2f3233 to cache now"
2025-06-27T11:32:25.280572023Z time="2025-06-27T11:32:25Z" level=info msg="RUN microdnf --setopt=install_weak_deps=0 --setopt=tsflags=nodocs install -y openssl-devel java-21-openjdk-devel nss_wrapper which && microdnf clean all"
2025-06-27T11:32:25.280577315Z time="2025-06-27T11:32:25Z" level=info msg="Cmd: /bin/sh"
2025-06-27T11:32:25.280578398Z time="2025-06-27T11:32:25Z" level=info msg="Args: [-c microdnf --setopt=install_weak_deps=0 --setopt=tsflags=nodocs install -y openssl-devel java-21-openjdk-devel nss_wrapper which && microdnf clean all]"
...
```
### 8. Cleanup (Optional)
To clean up, run:
```shell
kubectl delete taskrun --all
kubectl delete pvc --all
kubectl delete pv --all
kubectl delete -n default pipelinerun buildpacks-test-pipeline-run
kubectl delete -n default pipeline buildpacks-test-pipeline
kubectl delete -n default buildpacks-phases
kubectl delete -n default pvc buildpacks-source-pvc
```
## References
The Buildpacks tasks can be accessed at:
The Buildpacks task can be accessed at:
- [Buildpacks Task Source][buildpacks-task]
- [Buildpacks Phases Task Source][buildpacks-phases]
Some general resources for Tekton are:
@ -262,6 +319,8 @@ Some general resources for Tekton are:
[tekton-setup]: https://tekton.dev/docs/getting-started/
[tekton-dashboard-setup]: https://tekton.dev/docs/dashboard/
[tekton-concepts]: https://tekton.dev/docs/concepts/
[git-clone-task]: https://github.com/tektoncd/catalog/tree/master/task/git-clone
[kubectl-install]: https://kubernetes.io/docs/tasks/tools/install-kubectl/
[tekton-auth]: https://tekton.dev/docs/pipelines/auth/
[extension]: https://buildpacks.io/docs/for-buildpack-authors/tutorials/basic-extension/02_why-dockerfiles/
[extension-build]: https://buildpacks.io/docs/for-buildpack-authors/tutorials/basic-extension/04_build-dockerfile/
[extension-run]: https://buildpacks.io/docs/for-buildpack-authors/tutorials/basic-extension/06_run-dockerfile-extend/

View File

@ -29,7 +29,10 @@ The schema is as follows:
Human readable name.
- **`clear-env`** _(boolean, optional, default: `false`)_\
Clears user-defined environment variables when `true` on executions of `bin/detect` and `bin/build`.
When `true`, prevents the lifecycle from automatically merging user-provided
environment variables into the buildpack process environment during
`bin/detect` and `bin/build`. User-provided environment variables remain
available at `$CNB_PLATFORM_DIR/env/` regardless of this setting.
- **`homepage`** _(string, optional)_\
Buildpack homepage.

View File

@ -75,7 +75,7 @@
1. Add the SSH key for the podman VM to your keychain:
```shell=zsh
ssh-add -k "$HOME/.ssh/podman-machine-default"
ssh-add -k $(podman system connection list --format="{{.URI}} {{.Identity}}" | grep root | cut -d' ' -f2)
```
2. Change default connection to use `root`:
```shell=zsh