I noticed that building the "vendor" target was copying the source twice;
once in the "build-base" stage, and once in the "update-modules" stage:
docker buildx bake --set vendor.args.MODULE=github.com/docker/cli vendor
...
=> [build-base 1/3] COPY --from=hugo /go/bin/hugo /bin/hugo 0.2s
=> [build-base 2/3] COPY --from=node /src/node_modules /src/node_modules 1.6s
=> [build-base 3/3] COPY . . 3.1s
=> [update-modules 1/4] WORKDIR /src 0.0s
=> [update-modules 2/4] COPY . . 5.0s
=> [update-modules 3/4] RUN hugo mod get -u 14.1s
=> [update-modules 4/4] RUN hugo mod vendor 17.3s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Linting was only running on a subset of our Markdown files in `content`.
This change fixes the glob path, and adds appropriate exclude paths.
Because the glob pattern was incorrect before, we had lots of files that
didn't comply with the linting rules. This commit also fixes those
issues.
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
The updated version uses new anchors for flags, so updates are needed
elsewhere to update those anchors.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This page will be updated once the associated PR in docker/cli is merged.
Excluding it until then.
Running ["LinkCheck", "ImageCheck", "ScriptCheck"] on ["./_site"] on *.html...
Ran on 2173 files!
- ./_site/engine/reference/commandline/cli/index.html
* linking to internal hash #format-the-output that does not exist (line 314)
<a href="/engine/reference/commandline/config_ls/#format-the-output"><strong>format the output</strong> section in the documentation</a>
* linking to internal hash #format-the-output that does not exist (line 318)
<a href="/engine/reference/commandline/images/#format-the-output"><strong>format the output</strong> section in the documentation</a>
* linking to internal hash #format-the-output that does not exist (line 334)
<a href="/engine/reference/commandline/secret_ls/#format-the-output"><strong>format the output</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 326)
<a href="/engine/reference/commandline/plugin_ls/#formatting"><strong>formatting</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 322)
<a href="/engine/reference/commandline/node_ls/#formatting"><strong>formatting</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 330)
<a href="/engine/reference/commandline/ps/#formatting"><strong>formatting</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 338)
<a href="/engine/reference/commandline/service_inspect/#formatting"><strong>formatting</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 342)
<a href="/engine/reference/commandline/service_ls/#formatting"><strong>formatting</strong> section in the documentation</a>
* linking to internal hash #formatting that does not exist (line 346)
<a href="/engine/reference/commandline/stats/#formatting"><strong>formatting</strong> section in the documentation</a>
HTML-Proofer found 9 failures!
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The buildx build docs refers to anchors that are being updated,
but are maintained in a separate repository; exclude that page
from the validator until the anchor changes are live and the
buildx docs updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Dockerfile: add option to enforce git log history for local dev
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* jekyll(last_modified_at): do not override if already set with frontmatter
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* jekyll(last_modified_at): use data files for commands reference instead of stub file
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
* fix broken links
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
This adds information on how the docs use linting, and more detailed
instructions on how to enable it in an IDE, using VSCode as a concrete
and widespread example.
Additionally, to do this, we add a duplicate style config for
markdownlint in JSON format for the Node.js implementation of
markdownlint that is used by various editor plugins. These both
implement the same core set of warnings, and appear to function
identically.
Add GitHub Actions annotations that will appear in the CI checks - this
should make it easier to see the reasons for the linting failures,
without needing to deep dive into the logs.
* First inclusion of extensions-sdk docs from upstream repo
Signed-off-by: Guillaume Tardif <guillaume.tardif@gmail.com>
* Mark extension as Beta, not Preview
Signed-off-by: Guillaume Tardif <guillaume.tardif@gmail.com>
The 20.10 release branch now exists, and "master" will be used for
Docker Engine 21.xx development.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
As we don't generate a metadata.json, auto-complete is non-functional,
so disabling the script for previews.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch updates the default configuration to match a "development"
situation, and introduces build-options to produce a "production"
build.
By default (dev environment):
- Google Analytics / GTM and PollDaddy are disabled
- SASS builds non-minified stylesheets (for easier readabililty)
- Excludes "enterprise" stubs
Building a "production" build locally is still possible by overriding
the `JEKYLL_ENV` build-arg;
JEKYLL_ENV=production docker-compose up --build
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Canonical links are expected to be full URLs, not relative.
For this to work, the Dockerfile had to be updated, because we're stripping
the domain-name from links ("<a href..."), but the script currently also included
"<link rel='canonical' .." tags.
With the change, canonical links are left alone;
These hrefs will be replaced
echo '<a class=foo href="https://docs.docker.com/foo">hello</a>' | sed -e 's#\(<a[^>]* href="\)https://docs.docker.com/#\1/#g'
# <a class=foo href="/foo">hello</a>
echo '<a href="https://docs.docker.com/foo">hello</a>' | sed -e 's#\(<a[^>]* href="\)https://docs.docker.com/#\1/#g'
# <a href="/foo">hello</a>
But, for example, this one is left alone
echo '<link rel="canonical" href="https://docs.docker.com/foo/bar" />' | sed -e 's#\(<a[^>]* href="?\)https://docs.docker.com/#\1/#g'
# <link rel="canonical" href="https://docs.docker.com/foo/bar" />
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We generate the website with relative URLs to allow hosting the content
on preview domains.
However, the sitemap.xml should always contain absolute URLs (including
the domain name).
I couldn't find a way to configure the sitemap URL's separately, so using
the same hack we use for stripping URL's in the HTML.
Before this, the sitemap.xml would contain;
<url>
<loc>/engine/reference/commandline/volume_prune/</loc>
</url>
After this, the full URL is included
<url>
<loc>https://docs.docker.com/engine/reference/commandline/volume_prune/</loc>
</url>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
For intermediate stages, it's generally ok to perform steps
separately, however in this case, fixing the links would
happen every time the HTML was generated, so we might as
well do it in the same step, to reduce the size of the local
build cache, as it was adding 54MB for each rebuild:
IMAGE CREATED CREATED BY SIZE COMMENT
7d97d86ae290 3 minutes ago RUN /bin/sh -c find ${TARGET} -type f -name … 53.7MB buildkit.dockerfile.v0
<missing> 3 minutes ago RUN /bin/sh -c jekyll build -d ${TARGET} # b… 375MB buildkit.dockerfile.v0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "fetch-upstream-resources" script also updated the toc.yaml,
based on values in the _config.yml.
As a result, any change in either the "_config.yml" or "toc.yaml"
would also result in the upstream resources to be fetched again.
This patch separates the step to update the toc, so that the
upstream resources can be cached. This _does_ mean that remote
sources can get outdated (only will be rebuilt when changing
ENGINE_BRANCH or DISTRIBUTION_BRANCH). That should not be a
problem for actual deployments, which don't use caching.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "deploybase" stage was only used for the final stage.
This patch inlines the steps from that stage to the final stage
itself. In the process, changing the order of copying the NGINX
configuration file to _after_ the docs were copied. This prevents
having to copy the archived docs when making a change in the NGINX
configuration. The NGINX configuration file _will_ now be copied
on each rebuild, but the file is small, and as such is still faster
than copying all archives.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "deploy-source" stage was added in a previous refactor. While
this stage is useful to "export" all docs (including archives if
needed), and _convenient_ to collect all those files to the final
stage, it caused a regression in performance.
This patch changes the final ("deploy") to copy the "archives" and
"current" docs separately, so that the archived versions can be
cached in the final stage, and don't have to be copied again on
each rebuild.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This part of the script was setting variables for any "_version"
property in the configuration file.
We (currently) only need the engine API version, so simplifying
the code to just do that, and fix some spaces -> tabs indentations.
With this change the script only uses standard (POSIX) scripting,
so switching to use the standard (/bin/sh) shell.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The docs/docker.github.io:nginx-onbuild has not been
updated for a while, and having a local file present
allows for easier updating it (for example, to add
wildcard redirects).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Docker Enterprise actively supports the 17.06, 18.03, 18.09, and
19.03 versions, with 17.06 and 18.03 reaching EOL soon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't require these to be set upfront, as we can create them
locally in the fetch-upstream-resources.sh script
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch adds a stage that only contains the generated files. It can
be used to export the generated HTML for hosting the documentation on a
non-containerised service (e.g. to deploy to an s3 bucket).
When using BuildKit, use the `--output` option to build the files and to
copy them to your local filesystem.
For example, to build current docs, including archives:
DOCKER_BUILDKIT=1 docker build --target=deploy-source --output=./_site .
And to build without archives:
DOCKER_BUILDKIT=1 docker build --target=deploy-source --build-arg ENABLE_ARCHIVES=false --output=./_site .
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds a `ENABLE_ARCHIVES` build-arg, which allows building
the documentation without archives.
Note that currently, the archives drop-down is still added
unconditionally (so also included if archives are disabled).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Although the docs still rendered ok, some pieces were missing
when generating the "samples", which resulted in warnings during
build:
```
Adding front-matter to ./_samples/library/rethinkdb/README.md ...
cat: can't open './_samples/boilerplate.txt': No such file or directory
Adding front-matter to ./_samples/library/tomcat/README.md ...
cat: can't open './_samples/boilerplate.txt': No such file or directory
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The implementation of the Dockerfile first copied the current
version of the documentation, to be followed by fetching upstream
resources, and adding the archived documentation.
As a result, the build-cache was "busted" on every change that was
made locally, causing a full rebuild of both "fetching upstream resources",
building the reference documentation, and generating the archived
versions of the documentation.
With the changes made to the "fetch-upstream-resources.sh" script,
it's now possible to build those docs in advance (in a separate build-
stage), and append the pre-built docs _after_ the current version
of the documentation was built.
This patch updates the Dockerfile to build reference-docs and
archived docs in separate build-stages, optimizing the build for
caching (wich the assumption that no changes are made in upstream
resources between builds). Not only caused this builds to take longer
than needed, but each rebuild also caused new "dangling" images to
be created (in case of the "classic" builder), or new build-caches
to be created (when using BuildKit).
Note that when _deploying_ the documentation, no caching should be
used (to enforce a full rebuild of the documentation, and guarantee
that reference-docs are "fresh").
Before this change, rebuilding the documentation (after a previous
build, so with cache present) took ~185 seconds;
```
docker build -t docs .
[+] Building 184.7s (26/26) FINISHED
=> [builder 2/5] COPY . md_source 2.8s
=> [builder 3/5] RUN bash ./md_source/_scripts/fetch-upstream-resources.sh md_source 52.7s
=> [builder 4/5] RUN jekyll build -s md_source -d /usr/share/nginx/html --config md_source/_config.yml 113.0s
=> [builder 5/5] RUN find /usr/share/nginx/html -type f -name '*.html' | grep -vE "v[0-9]+\." | while read i; do sed -i 's#href="https://docs.docker.com/#href="/#g' "$i"; done 4.4s
=> CACHED [stage-1 2/8] COPY --from=docs/docker.github.io:nginx-onbuild /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf 0.0s
=> CACHED [stage-1 3/8] COPY --from=docs/docker.github.io:v17.03 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [stage-1 4/8] COPY --from=docs/docker.github.io:v17.06 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [stage-1 5/8] COPY --from=docs/docker.github.io:v17.09 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [stage-1 6/8] COPY --from=docs/docker.github.io:v17.12 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [stage-1 7/8] COPY --from=docs/docker.github.io:v18.03 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> [stage-1 8/8] COPY --from=builder /usr/share/nginx/html /usr/share/nginx/html 3.7s
=> exporting to image 5.1s
=> => exporting layers 5.1s
=> => writing image sha256:70f77631001bd0271455f893b87dd75a1fd6f5db84a00feb8afe5ddf5a697d7d 0.0s
=> => naming to docker.io/library/docs 0.0s
```
After this change, archives and reference-docs are using the build-cache,
and building was reduced to ~126 seconds (most of that taken by Jekyll to
generate the static HTML);
```
docker build -t docs .
+] Building 126.4s (30/30) FINISHED
...
=> CACHED [upstream-resources 1/4] COPY ./_scripts/fetch-upstream-resources.sh ./_scripts/ 0.0s
=> CACHED [upstream-resources 2/4] COPY ./_config.yml . 0.0s
=> CACHED [upstream-resources 3/4] COPY ./_data/toc.yaml ./_data/ 0.0s
=> CACHED [upstream-resources 4/4] RUN bash ./_scripts/fetch-upstream-resources.sh . 0.0s
=> [current 2/4] COPY --from=upstream-resources /usr/src/app/md_source/. ./ 1.5s
=> [current 3/4] RUN jekyll build -d /usr/share/nginx/html 108.9s
=> [current 4/4] RUN find /usr/share/nginx/html -type f -name '*.html' | grep -vE "v[0-9]+\." | while read i; do sed -i 's#href="https://docs.docker.com/#href="/#g' "$i"; done 4.8s
=> CACHED [deploybase 2/2] COPY --from=docs/docker.github.io:nginx-onbuild /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf 0.0s
=> CACHED [archives 1/5] COPY --from=docs/docker.github.io:v17.03 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [archives 2/5] COPY --from=docs/docker.github.io:v17.06 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [archives 3/5] COPY --from=docs/docker.github.io:v17.09 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [archives 4/5] COPY --from=docs/docker.github.io:v17.12 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> CACHED [archives 5/5] COPY --from=docs/docker.github.io:v18.03 /usr/share/nginx/html /usr/share/nginx/html 0.0s
=> [deploy 1/1] COPY --from=current /usr/share/nginx/html /usr/share/nginx/html 4.1s
=> exporting to image 1.9s
=> => exporting layers 1.9s
=> => writing image sha256:e76359b937fb4d956fb17e889a7333687929d4c9ae093909fe5630a8f3e4de64 0.0s
=> => naming to docker.io/library/docs 0.0s
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This variable is no longer needed, as the scripts are
always executed relative to the current directory (`.`).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Go back to small /edge
* Update site front page
Add info about Editions
Reorganize components
Update headings
* Move logic of fetching content to a script
* Add v1.28 API
* Add info about versioned API
The master branch is called trunk in svn parlance
and the directory structure is a little bit different.
Also trust the certificates for the https svn connections,
otherwise we may be prompted.
Fix Dockerfile in vnext-engine
The master branch is called trunk in svn parlance
and the directory structure is a little bit different.
Also trust the certificates for the https svn connections,
otherwise we may be prompted.
The master branch is called trunk in svn parlance
and the directory structure is a little bit different.
Also trust the certificates for the https svn connections,
otherwise we may be prompted.
Sebastiaan van Stijn
David Karlsson
David Karlsson
Guillaume Lours
CrazyMax
CrazyMax
Craig Osterhout
Justin Chadwell
Usha Mandya
Guillaume Tardif
Ally Smith
paigehargrave
Maria Bermudez
Joao Fernandes
Misty Stanley-Jones
Misty Stanley-Jones
John Mulhausen
Victor Vieux