Commit Graph

1614 Commits

Author SHA1 Message Date
Jeff Wong 136aefe49d
Move DB initialization scripts for postgres and redis into service files.
This resolves a race condition with unconfigured images attempting to bring up
DBs for the first time. This does not affect fully bootstrapped images.

Currently, all jobs start at boot - this includes postgres.

Issue with the current is - postgres starts and adds the corresponding .s/.pid
files to /var/run/postgres.

Simultaneously, the unicorn job gets started, checks to see if postgres is
running (it is already at this point from boot), and runs install_postgres.

Inside the install_postgres script, we mount the shared postgres folder and
remove .s/.pid files -- after postgres has already been started. In this case,
we remove the (in-use) .s and .pid files.

Subsequent unicorn tasks fail, erroring out the service and forcing it into
a restart loop. Since postgres never restarts, it never regenerates the .s/.pid
files, and unicorn can never run successfully.

This proposal moves install_postgres into the postgres job file, eliminating the
race condition. Since they are part of the same service, install_postgres will
always run before starting postgres - it will no longer be able to remove valid
.s and .pid files.

Redis has a similar race condition with the creation of its data folder. This
isn't as disastrous as the redis service restarts until the folder exists from
unicorn run, but it provides better reasoning about the running services.

Add early exit from unicorn boot scripts to properly retry migrate as well.

Use pg_isready to check if pg is ready directly in create_db.
Merge the ready check into create_db.

Run create_db in a subshell on postgres job start, rather than in unicorn script.

remove postgres-config call
2025-05-19 10:18:07 -07:00
Alan Guo Xiang Tan 64f31dbbe3
DEV: Update labels for our self hosted runners (#965)
We have 8 and 16 cores runner now but 16 core runners are not quite
ready for prime time yet.
2025-05-16 15:46:19 +08:00
Jeff Wong 280b40613d
DEV: add base_image specifications to data templates (#964)
Add base_image to other potentially standalone templates in addition to the web template
2025-05-13 13:52:33 -07:00
Jeff Wong aa547bc082
DEV: include base image in web template (#963)
This will move base image bumps from launcher into web.template.yml
2025-05-12 19:22:03 -07:00
David Taylor 3fba2aeee8
DEV: Move `RETRY_AND_LOG_FLAKY_TESTS` to correct location (#962)
We need to set the env inside the docker container, not in the github runner
2025-05-12 10:57:18 +01:00
Joffrey JAFFEUX c622ee5b24
Separate dependencies and browser installation (#956)
- deps as root
- browser as discourse user
2025-05-12 10:11:35 +01:00
Krzysztof Kotlarek 5bb42d2865
Revert "DEV: Set fixed playwright browsers path for test Docker image (#960)" (#961)
This reverts commit 9c2bedb521.
2025-05-12 14:40:20 +08:00
Alan Guo Xiang Tan 9c2bedb521
DEV: Set fixed playwright browsers path for test Docker image (#960) 2025-05-10 06:50:19 +08:00
Alan Guo Xiang Tan 3c2234830a
Revert "Use Nginx config with outlets (#913)" (#958)
This reverts commit 7b042d6157.

`web.socketed.template.yml` is broken
2025-05-09 11:09:35 +08:00
Bianca Nenciu 7b042d6157
Use Nginx config with outlets (#913)
This config has several 'include' statements that serve as extension
outlets. Most of the "find & replace" instructions from templates
files have been replaced to use outlets.

This commit also deprecates IPv6 template in favor of an automatic
implementation that relies on container's connectivity.
2025-05-07 18:56:22 +03:00
Joffrey JAFFEUX 4cb4a5912d
Install playwright dependencies in test image (#955)
This step avoids having to install playwright in each CI run.

Note I also removed the yarn check as it shouldn't be necessary anymore.
2025-05-07 15:52:25 +02:00
Philo Gray 09d5eb60bb
Clarified SMTP-related comments in samples/standalone.yml (#948)
This is just a small change and goes along with changes to docs at
https://meta.discourse.org/t/troubleshoot-email-on-a-new-discourse-install/16326.
2025-05-06 09:23:26 +08:00
Alan Guo Xiang Tan 9ec9c0cb6a
Bump Ruby to 3.3.7 (#954) 2025-05-06 09:22:32 +08:00
Rafael dos Santos Silva 721facba64
FIX: Setup swap when RAM <= 4GB (#953) 2025-04-17 15:04:27 -03:00
Alan Guo Xiang Tan e42fa9711e
Drop `pry` as default runtime console. (#947)
Using `pry` as the default runtime console in the production environment
is forcing us to account for it in Discourse core's codebase. In order
to avoid the additional complexity in Discourse core's codebase, we have
decided to drop `pry` as the default runtime console and rely on IRB
which is just as good these days. We will however be keeping the
dependency on `pry` around so those that would like to use it can start
it manually.
2025-03-04 12:28:25 +08:00
Ella E. 3f9f4fa4aa
Update README.md (#942)
Include blurb about hosting.

Internal: /t/122265/
2025-03-02 23:18:35 -07:00
Jay Pfaffman a275b59627
Removing unicorn service from phpbb template broke the build (#934)
* Removing unicorn service from phpbb template broke the build

https://meta.discourse.org/t/migrate-a-phpbb3-forum-to-discourse/30810/786?u=pfaffman

* Prevent Sidekiq from running

Usually we don't want Sidekiq to run during an import because it might mess things up while there's incomplete or inconsistent data.

---------

Co-authored-by: Gerhard Schlager <gerhard.schlager@discourse.org>
2025-02-27 11:24:26 -05:00
Jarek Radosz 526b43200c
Bump default base image to 2.0.20250226-0128 (#946) 2025-02-26 12:02:42 +01:00
Jarek Radosz 45c0a02cc7
DEV: Re-add ghostscript to the base image (#945) 2025-02-26 02:28:38 +01:00
Jarek Radosz 78e8b2a4cc
DEV: Use more recent version of libheif (#943) 2025-02-26 01:17:03 +01:00
SouthpawKB 375bcf1675
DEV: Launcher Cleanup - Remove redundant word in error message (#944) 2025-02-25 16:26:10 -05:00
David Taylor 3715498fc1
Update firefox tar format (#939)
Firefox started distributing `.tar.xz` instead of `.tar.bz2`
2025-02-11 12:11:29 +00:00
David Taylor efd6bc8ccb
FIX: web.ssl.template compat with latest core (#938)
649505d869 added some new content between `listen 80;` and `gzip on;`, which broke these replacements.

This commit reorders & updates the replacements so that they should work reliably with versions of core before and after that change.
2025-02-04 20:01:48 +00:00
Gary Pendergast e570a8c5c1
DEV: Force PNPM v9 to be installed. (#937)
Before upgrading PNPM to v10, we need to get Docker builds running again, so this change temporarily forces PNPM v9 to be installed.
2025-02-04 18:17:32 +11:00
mwaniki-wairungu 98f470fc25
FIX: attempt the postgresql update if a stale socket file exists (#935) 2025-02-03 14:45:51 +03:00
mwaniki-wairungu ad89eb3b10
FIX: install pgvector package for old PG versions during update (#932) 2025-01-29 18:32:39 +03:00
mwaniki-wairungu 7b4839c5ea
Bump default base image to discourse/base:2.0.20250129-0720 (#931)
Follow-up to d9c837c783
2025-01-29 11:53:40 +03:00
mwaniki-wairungu d9c837c783
FEATURE: Update base image and set default to postgresql 15 (#909)
This updates the default PostgreSQL version to 15.

  * image/base: update default postgres version to 15
  * launcher: bump base image to include postgres 15
  * postgres.template.yml: update default version to 15
  * postgres.15.template.yml: current version template

Legacy version templates:

  * postgres.13.template.yml
  * postgres.12.template.yml
  * postgres.10.template.yml
  * postgres.9.5.template.yml
2025-01-29 10:20:49 +03:00
Jarek Radosz 92ac6cbc9b
DEV: Update ImageMagick to 7.1.1-43 (#929)
We could use a bump, after two years since the last one.
2025-01-28 12:07:56 +01:00
Jarek Radosz c41284d0d1
DEV: Update nginx to 1.26.1 (#930) 2025-01-28 12:07:44 +01:00
nta~ 79724e9e28
Fix TLS in socketed template (#925)
After 7d548ad4ae, the replacement pattern in web.socketed.template.yml no longer matches, making nginx.https.sock not exist.

As such, remove the `http2` field from the `listen` directive to match aforementioned commit.
2025-01-28 08:59:53 +08:00
Jarek Radosz 79ccbcc532
DEV: Update the workflow to Ubuntu 24.04 (#928)
The current LTS
2025-01-28 08:58:35 +08:00
Mwaniki Wairungu 90a1b46e98
FIX: use appropriate manifest for pg-15 image from 8a956d4bb
Follow-up to ab957b2f64
2025-01-27 12:40:00 +03:00
Alan Guo Xiang Tan 8a956d4bb1
Push docker manifest for `discourse/base:2.0.${{ env.TIMESTAMP }}-15` (#926)
Follow-up to ab957b2f64
2025-01-27 16:41:30 +08:00
Michael Fitz-Payne 8057b695b7
DEV: bump timeout for non-scheduled builds (#924)
The most recent build timed out at 30 minutes. Bump timeout temporarily
to get a gauge on how long self-hosted builds take (push to Dockerhub
looks to be slower than Github-hosted runners).
2025-01-24 09:21:00 +08:00
Michael Fitz-Payne e72554ed7f
DEV: switch amd64 builds to debian-12 runners
We can use our self-hosted action/runner machines to build the amd64
Docker images, falling back to the Github action/runner machines for the
arm64 builds.

Additionally, simplify the expression for how arm runners are selected.
The cost savings from switching to self-hosted runners dwarfs the
increase in 8core vs 2core arm runners.
2025-01-24 08:32:50 +10:00
Bianca Nenciu e44bc08778
Revert "Add new Nginx config to base image (#918)" (#923)
We have decided to go another route and keep the Nginx config in the
Discourse repository, but add the outlets there.

This reverts commit 32dc866c85.
2025-01-23 15:46:29 +08:00
Bianca Nenciu 32dc866c85
Add new Nginx config to base image (#918)
This commit moves the Nginx config from discourse/discourse repository
and updates it with several 'include' statements that serve as extension
outlets.

This change was made to place the config closer to where it is used and
implements an alternative to the "find & replace" hacks.

It is unused at this moment, but all templates and samples will be
updated in a future commit.
2025-01-22 20:13:17 +02:00
Alan Guo Xiang Tan 0c7d085b38
DEV: Correct manifest name for images with PG 15 installed (#919)
Follow-up to ab957b2f64
2025-01-21 16:01:07 +08:00
Alan Guo Xiang Tan ab957b2f64
Build images with PG 15 installed (#917)
This commit updates the build workflow to build and push an extra base Docker
image with PG 15 installed. The tag is not meant to be permanent and is
just meant to help us update our postgres templates to support PG 15
within the same commit.

The tag will be removed once our `discourse/base:release` ships with PG
15 by default.
2025-01-21 09:59:32 +08:00
Alan Guo Xiang Tan 136c638906
DEV: Remove invalid comment (#916)
This script is used by the build workflow which invalidates the comment
2025-01-16 08:48:16 +08:00
Jarek Radosz 6c5e2d1412
DEV: Remove unused docker images (#915)
1. `monitor` uses a 9 year old base image
2. `discourse_fast_switch` uses a 6 year old base image and references ruby 2.4 and 2.5
3. `discourse_bench` uses an 8 year old base image and references postgres 9.5
2025-01-16 07:52:39 +08:00
Jarek Radosz aeb28ff600
DEV: Minor cleanup of the workflow (#914) 2025-01-15 21:46:02 +01:00
Jeff Wong fa85268554
Separate nginx build step (#908)
* Revert "Revert "Move nginx installation into another build step (#897)" (#899)"

This reverts commit 0e767585d6.

* add nginx runtime dependencies
2025-01-14 13:29:19 -08:00
David Taylor 107ffb40fe
Revert "Add nginx config file and support for outlets (#871)" (#912)
This reverts commit 01cbf07622.
2025-01-14 16:44:52 +00:00
Bianca Nenciu 01cbf07622
Add nginx config file and support for outlets (#871)
The nginx config file used to be copied from the discourse/discourse
repository, but it has been now moved in this project, closer to the
place where it is used.

The config has several 'include' statements that implement support for
outlets that templates can then use to extend the default configuration
for various features. This is an alternative to the "find & replace"
hacks.
2025-01-14 17:56:45 +02:00
Jarek Radosz 4f56ee9f72 Bump default base image to discourse/base:2.0.20250114-0014 2025-01-14 15:46:23 +01:00
Jarek Radosz b345430a82
FEATURE: Update node to version 22.x (#910) 2025-01-13 14:01:08 +01:00
Alan Guo Xiang Tan d4ebd314de
DEV: Periodic bump of launcher (#906) 2025-01-07 14:18:56 +08:00
Alan Guo Xiang Tan ae194861f8
Bump Redis to 7.0.15 (#907)
Pulls in bug and security fixes since 7.0.7
2025-01-07 13:46:53 +08:00