Commit Graph

1589 Commits

Author SHA1 Message Date
Alan Guo Xiang Tan b47bd562cc
Release `discourse/base` images for `discourse/discourse` stable branch (#864)
This commit updates our `build` workflow to push the following manifests
which can be used to run Discourse against the stable branch.

1. discourse/base:2.0.$TIMESTAMP-stable
2. discourse/base:release-stable

Co-authored-by: David Taylor <david@taylorhq.com>
2024-09-09 19:49:14 +08:00
David Taylor 3384f2c71c
Improve node_modules cleanup logic for stable branch (#869)
- makes decision based on current state of directory, instead of `$version`
- cleans up the correct directories
- only cleans up the contents. This is important if node_modules directores are mounted volumes (e.g. in devcontainer)
2024-09-09 11:13:18 +01:00
Jeff Wong 34f016a7a6
DEV: export default base image (#868)
allows default base image to be more portable.
2024-09-08 23:11:55 -07:00
cheungtitus d00c2e75d5
Added support of Linux on Mac with ARM processors (#631)
Co-authored-by: TItus Cheung <cheungtitus@kenrui-group.com>
2024-09-09 10:09:20 +08:00
Jeremy 6c37c5dacd
Update Discourse installation support in China (pnpm) (#867)
This commit updates 2 things.
1. Updates the yarn hook to replace the npm mirror before `pnpm install`.
2. Removes the `yarn.lock` patch as pnpm is now used.

After applying these modifications, I successfully installed Discourse on the Tencent Cloud China server. No more network problems.
2024-09-06 17:10:52 +01:00
David Taylor 928adfd09f
Suppress pnpm upgrade notices in logs (#866) 2024-09-06 10:28:15 +01:00
Alan Guo Xiang Tan 189cfe2ac9
Clean up nginx folder after installation. (#865) 2024-09-06 15:09:59 +08:00
Jeff Wong e84629a754
DEV: update golang versions (#863)
update golang versions and bump dependencies
2024-09-05 08:08:17 +08:00
Jeff Wong 9e25686a99
FEATURE: add an option for custom namespaces (#862)
Add an option to select a targeted namespace, env var DISCOURSE_NAMESPACE

Configure:
Add `source-tag` to select which tag to configure from.
Rename old `tag` option to `target-tag` to differentiate from `source-tag` option

Migrate:
Add `tag` to select which tag to migrate with
2024-09-04 06:42:02 -07:00
Alan Guo Xiang Tan 3186478f51
Reduce min docker version to 20.10.0 (#861)
Follow up to 31931deae8

Docker 20.10.0 works for now so be less aggressive in forcing people to
upgrade.
2024-09-04 11:35:26 +08:00
Alan Guo Xiang Tan 31931deae8
DEV: Bump minimal Docker version to 24.0.7 (#860)
When running the newer Debian bookworm based images, we are seeing
`(ThreadError) can't create Thread: Operation not permitted` errors when
trying to spawn a thread in Ruby.

A similar issue was reported in https://github.com/docker-library/ruby/issues/429#issuecomment-1708908819
and the fix here is to upgrade Docker. Either way, we should probably
update because Docker 17 has been EOF for many many years.
2024-09-04 10:47:58 +08:00
Jeff Wong fbfff14a83
add tmp directory to gitignore (#859) 2024-09-03 12:12:34 +08:00
Jeff Wong 3e93a3e9b2
FEATURE: add autocomplete (#857) 2024-09-02 18:54:14 -07:00
Jeff Wong d87acd4ccd
FEATURE: add runtime features (#849)
adds commands for: start, run, stop, cleanup, destroy, logs, enter, restart, rebuild -- carrying over existing run commands from launcher1.

Rebuild will also do its best to minimize downtime with the following steps:

* Detect if Discourse is running as a single container or external DB
* Detect if db:migrate is configured to run on container boot
* Build initial container (keeping existing one online)
* Exit running containers if it's a single container (otherwise keeps existing online)
* Run migrations
  * Defer migrations if db:migrate is configured to run on container boot
  * Run migrations with SKIP_POST_DEPLOYMENT_MIGRATIONS=1 if it's a 2 container setup
  * Otherwise, run all migrations
* Destroy the old container (finally stopping the current, if it's still up here)
* Start the new container
* Run post-deploy migrations
  * Run migrations with SKIP_POST_DEPLOYMENT_MIGRATIONS=0 if it's a 2 container setup
2024-09-03 06:09:38 +08:00
Jeff Wong de17bef722
DEV: install redis data folder if none exists (#804) 2024-09-02 13:19:43 -07:00
David Taylor 87871bc430
Make compatible with pnpm (#854)
Switches behavior based on the presence of `yarn.lock`

See https://github.com/discourse/discourse/pull/28671
2024-09-02 20:55:57 +01:00
Alan Guo Xiang Tan fe48fa2c5c
Switch to `discourse/ruby:3.3.4-bookworm-slim` (#853) 2024-09-02 11:49:43 +08:00
Jeff Wong 44134a21bb
DEV: remove bundle cache (#852)
On the current image this removes ~78MB from the layer

Uncompressed image sizes:
before: 3.3GB
after: 3.23GB
2024-08-30 18:05:34 -07:00
Alan Guo Xiang Tan cfe8aa0df0
Remove use of `discourse/base:aarch64` in launcher2 (#851)
Since fc61b8a850, we started shipping
manifests for the `discourse/base` images so we don't have to
specifically pull the tag for arm64.
2024-08-29 19:03:17 -07:00
Alan Guo Xiang Tan 9cd8186278
Drop `mac-os` tests for launcher_go (#850)
The runners cost 10 times more and there are no macOS specific tests
which we are running.
2024-08-29 17:39:27 -07:00
Jeff Wong 85a6e38644
FEATURE: Add workflow for `launcher_go` (#848) 2024-08-29 15:59:12 +08:00
Jeff Wong b7e54edb04
FEATURE: add migrate and bootstrap commands (#842)
Adds command for migrate only. Adds batch commands for bootstrap (build+migrate+configure)
2024-08-29 11:01:28 +08:00
Jeff Wong 7c18fce1b4
DEV: remove oxipng archive in layer (#846)
* DEV: remove oxipng archive in layer

Saves ~52MB for the layer

* DEV: Remove nginx archives

Remove another 1.2MB from image
2024-08-28 19:34:24 -07:00
Alan Guo Xiang Tan 1740b529a6
DEV: Improve assertions of `cli_build_test.go` (#844)
This commit improves the assertions by testing against the entire
command string and env so that we can be sure of the full command we are
running.
2024-08-28 10:02:49 -07:00
David Taylor 83c7e4e587
Increase timeout for scheduled build job (#845)
Now that base & dev images are built in the same job, we need a little more time
2024-08-28 17:19:59 +08:00
Alan Guo Xiang Tan 595dd6af1c
DEV: Minor readability improvements to launcher2 (#843) 2024-08-28 14:20:17 +08:00
Jeff Wong d3ae996af6
FEATURE: add configure command (#841)
Add 'configure' command - If run after the "build" command, this is equivalent to today's 'bootstrap' command.

Note that unlike build command, a docker run+commit pattern needs to be used here as this requires a running database + mounted volumes.
2024-08-28 13:13:39 +08:00
Jeff Wong 0808c17a04
[WIP] Launcher2 (#791)
Partial merge of Launcher2's CLI

* FEATURE: merge launcher2 - build command

merge launcher2, only build commands
2024-08-26 12:36:41 -07:00
Alan Guo Xiang Tan ceb92647b4
Fix timeout for arm64 builds not correctly set (#839) 2024-08-26 15:05:20 +08:00
Alan Guo Xiang Tan def0f9ec3f
Base image in launcher (#838)
This pulls in the Nginx 1.26.1 update, Ruby 3.3.4 update and Debian
bookworm update.
2024-08-26 13:22:33 +08:00
David Taylor fc61b8a850
DEV: Introduce arm64 dev image and multi-arch manifests (#829)
- Combines dev image build into the `base` job
- Calculates a single timestamp for all builds
- Pushes timestamped per-arch images to Dockerhub for base, base-slim and dev images
- If both arch build jobs are successful, multiarch manifests are generated & pushed
2024-08-26 13:10:28 +08:00
David Taylor b110891382
Update GitHub actions runners to ubuntu-22.04 (#837) 2024-08-22 11:09:50 +01:00
Alan Guo Xiang Tan 84644fcbea
Bump Ruby to 3.3.4 (#836)
Ruby 3.3.4 has fixes for some regressions in 3.3.3. I don't think it
affects us but still good to upgrade.
2024-08-22 12:25:15 +08:00
Alan Guo Xiang Tan 4c58e2b75b
Switch to debian:bookworm-slim image (#831)
Debian Bullseye is EOL.
2024-08-22 10:57:56 +08:00
Alan Guo Xiang Tan 1d7323204b
Ensure uid, gid for postgres, redis and discourse stays consistent (#835)
When updating to `debian/base:bookworm` from `debian/base:bullseye`, we
noticed that the uid of the `postgres` and `redis` user changed leading
to permission issues when trying to access directories of mounted volumn
which was previously created with the old uid. The change is because the
`_apt` user is assigned a uid of `42` in Debian bookworm instead of `100`
in Debian bullseye. As a result, the `postgres` user created by the `postgres`
package is automatically assigned a uid of `100` in Debian bookworm instead of `101` in Debian
bullseye.

This commit updates the `slim.Dockefile` to manually add the `postgres`
user and group assigning it a gid of `104` and uid of `101`. The `redis`
user and group is assigned a uid of `103` and a gid of `106`. The
`discourse` user and group is assigned a uid of `1000` and gid of
`1000`.
2024-08-22 10:57:29 +08:00
David Taylor 98c5337c12
Warmup yarn and bundle caches in dev image (#832)
Uses a multi-stage build with bind mount to avoid adding the discourse repo itself to the layers
2024-08-20 14:51:32 +01:00
David Taylor 7b0e4b45b5
dev: add user-installed gems to PATH (#833)
This is important for usability of things like discourse_theme, and editor tooling (e.g. ruby-lsp)
2024-08-20 14:51:14 +01:00
David Taylor 8aff5b6588
Use flaky-test-retry system (#834)
We use this on the main core repository, so it makes sense to use it here as well. It should reduce the overall flakiness of the discourse_docker build.
2024-08-20 13:44:25 +01:00
David Taylor f1c9d8afb2
DEV: Simplify discourse_dev postgres setup (#830)
- Remove manual database creation, and instead promote discourse user to postgres SUPERUSER. This means that `db:drop` and `db:create` commands can be run in the dev image, just like in other local development environments. As well as simplifying things, it fixes turbo_rspec, which was previously impossible in the docker dev environment (because `discourse` didn't have permissions to create the parallel databases)

- Stop pre-migrating test database in dev image. It adds additional build time & image size, and doesn't actually help because core's `bin/docker/boot_dev` script overwrites the container's postgres directory with a volume mount
2024-08-19 16:44:23 +01:00
Alan Guo Xiang Tan 6f8c17dbf4
Upgrade nginx to 1.26.1 since it is the new stable version (#827)
Since 1.26.x has been marked stable, 1.25.x is automatically EOL as
nginx only maintains one mainline version and one stable version.
2024-08-19 09:19:54 +08:00
Pablo Ganuza Vidal f953eb623a
Fix verification of ports (#818)
nc needs -p to publish verification code correctly, at least in Debian 12
2024-08-19 08:39:29 +10:00
Derek J. Lambert fc9fbedfac
FIX: Broken config and variable handling in setup script (#693)
* FIX: Variable isn't being set

* DEV: Goodbye tabs, they're probably feeling lonely anyway

* FIX: Variable is never used. No need for command substitution

* FIX: The likelihood of knowing the PID for the script before execution is exceptionally low
2024-08-19 08:35:56 +10:00
Jay Pfaffman b7a6888a2b
bash tab completion for launcher + discourse-setup (#676)
add tab completion for launcher and discourse-setup.

For launcher, offers command (e.g., rebuild, start) and then offers yml files from containers directory. After that switches (e.g., --run-image) are offered. (Will not offer switches except in final position, sorry.)

discourse-setup offers switches (e.g., --two-container).

discourse-docter has no command line arguments.
2024-08-19 08:31:55 +10:00
Jeff Wong 9f80d83b32
DEV: allow multiple configure runs (#828)
when we have already run an initial setup, fall back to just checking for
socket, rather than outright failing if the init script has already been run.

This allows 'configure' steps to be re-run in standalone cases.

eg: `launcher2 configure app && launcher2 configure app`

current version: fails as it's missing the install_postgres file
with PR: checks for psql socket, and builds.

doing something like `launcher2 start app && launcher2 configure app` would also
print out a more correct error message, "postgres already running stop container"
2024-08-12 07:43:26 -07:00
Rafael dos Santos Silva 618ab51850
Update pgvector to 0.7 (#825)
* Update pgvector to 0.7

* escape

* drop version
2024-08-09 11:16:38 -03:00
Alan Guo Xiang Tan 494e353c2d
Revert "Switch to Chrome for Testing and drop support for Chromium (#824)" (#826)
This reverts commit bdfcc8ad23.

Broke the test build
2024-08-09 11:05:25 +08:00
Alan Guo Xiang Tan bdfcc8ad23
Switch to Chrome for Testing and drop support for Chromium (#824)
We started installing Chromium because there is no linux ARM support
for Chrome yet. However, trying to run tests on Chromium seems to be
extra challenging. For example, upgrading to Debian 12 causes our
Javascript tests to fail on Chromium but not on Chrome.

Chrome for Testing was built specifically for web app testing so let's
follow Google's recommendation.
2024-08-09 09:55:50 +08:00
Alan Guo Xiang Tan 21a0351e30
Build `discourse/base` and `discourse/discourse_test` image for bookworm (#823)
We need to upgrade to bookworm because bullseye is EOL. This commit when merged into branch will push the following images to Docker hub:

1. `discourse/base:slim-bookworm`
2. `discourse/base:release-bookworm`
3. `discourse/discourse_test:slim-bookworm`
4. `discourse/discourse_test:slim-browsers-bookworm`
5. `discourse/discourse_test:release-bookworm`
2024-08-08 16:29:53 +08:00
Alan Guo Xiang Tan 9e253b704c
FIX: Remove `bundle config jobs` (#821)
This fixes a regression introduced in
bbefa1e5f3. Basically, we cannot configure
the default bundle jobs when building the image because the number of
cores used to build the image can be different from the number of cores
on the machine running the image.
2024-08-08 11:23:54 +08:00
David Taylor 6f7f62ceca
Increase `yarn` network-timeout from 30s to 60s (#820)
A number of people have reported hitting yarn timeouts on low-spec DO droplets, which causes the build to fail. This should provide a little more leeway
2024-08-07 14:13:50 +01:00