Commit Graph

297 Commits

Author SHA1 Message Date
cjihrig 878047be0b test_runner: simplify test start time tracking
This commit simplifies the logic for tracking test start time.
The start time is now set only when a test/suite begins running.
If the test/suite never runs, a fallback is provided in postRun().

PR-URL: https://github.com/nodejs/node/pull/52182
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-25 13:23:34 +00:00
cjihrig 4648c83dbc test_runner: don't await the same promise for each test
Prior to this commit, each top level test awaited the same
global promise for setting up test reporters. This commit
updates the logic to only await the promise the first time
it is encountered.

PR-URL: https://github.com/nodejs/node/pull/52185
Refs: https://github.com/nodejs/node/pull/47164
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-25 01:01:41 +00:00
Moshe Atlow f9755f6f79
test_runner: emit diagnostics when watch mode drains
PR-URL: https://github.com/nodejs/node/pull/52130
Fixes: https://github.com/nodejs/node/issues/51253
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2024-03-19 21:55:25 +00:00
Colin Ihrig 5276c0d5d4
test_runner: add suite()
This commit adds a suite() function to the test runner and makes
describe() an alias for it. This matches the it() alias for
test().

Fixes: https://github.com/nodejs/node/issues/51430
PR-URL: https://github.com/nodejs/node/pull/52127
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-19 15:38:17 +00:00
Colin Ihrig 4ba9f45d99
test_runner: ignore todo flag when running suites
This commit removes a check for the todo flag when determining
if a suite should run. In general, the todo flag should have
no impact on whether or not a test/suite runs. Instead, it should
only impact how the result of the test/suite is handled.

PR-URL: https://github.com/nodejs/node/pull/52117
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-18 18:11:33 +00:00
Colin Ihrig 6f4d6011ea
test_runner: skip each hooks for skipped tests
When a test is skipped, the corresponding beforeEach and afterEach
hooks should also be skipped.

Fixes: https://github.com/nodejs/node/issues/52112
PR-URL: https://github.com/nodejs/node/pull/52115
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-18 16:29:36 +00:00
Colin Ihrig 05db979c01
test_runner: run top level tests in a microtask
This commit updates the test harness to prevent top level
tests from executing immediately. This allows certain config
data, such as filtering options, to be discovered before running
the tests.

PR-URL: https://github.com/nodejs/node/pull/52092
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-17 04:17:36 +00:00
Colin Ihrig 97b2c5344d
test_runner: remove redundant report call
This commit removes a redundant call to `reportStarted()`. It is
redundant because a few lines down, `subtest.finalize()` is
called. `finalize()` will find the first test that is ready to
report its data, and then call `report()`, which also calls
`reportStarted()`. This will trigger the `'test:start'` as high
up the test tree as necessary.

PR-URL: https://github.com/nodejs/node/pull/52089
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2024-03-16 21:36:58 +00:00
atlowChemi 780d030bdf test_runner: use internal addAbortListener
PR-URL: https://github.com/nodejs/node/pull/52081
Refs: https://github.com/nodejs/node/pull/48596
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2024-03-16 09:08:59 +00:00
Moshe Atlow 814fa1ae74
test_runner: use source maps when reporting coverage
PR-URL: https://github.com/nodejs/node/pull/52060
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-15 06:33:42 +00:00
Colin Ihrig 84de97a61e
test_runner: support forced exit
This commit updates the test runner to allow a forced exit once
all known tests have finished running.

Fixes: https://github.com/nodejs/node/issues/49925
PR-URL: https://github.com/nodejs/node/pull/52038
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-13 10:49:15 +00:00
Colin Ihrig 3c5764a0e2
test_runner: handle undefined test locations
This commit updates the built in reporters to check for the
documented case of a test's location being undefined.

As a drive by fix, the C++ code for computing the test location
now returns undefined if the script location is empty. This lets
tests run inside of eval().

PR-URL: https://github.com/nodejs/node/pull/52036
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-03-12 18:16:28 +00:00
cjihrig 328642bbb9 test_runner: use paths for test locations
This commit transforms test locations to paths when V8 provides
file URLs (which seems to be for ESM files).

Fixes: https://github.com/nodejs/node/issues/51610
PR-URL: https://github.com/nodejs/node/pull/52010
Fixes: https://github.com/nodejs/node/issues/51392
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-11 08:56:16 +00:00
cjihrig 6d625fe616 test_runner: support source mapped test locations
This commit adds support for source mapping test locations
when the --enable-source-maps flag is present.

Fixes: https://github.com/nodejs/node/issues/51392
PR-URL: https://github.com/nodejs/node/pull/52010
Fixes: https://github.com/nodejs/node/issues/51610
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-11 08:56:16 +00:00
Colin Ihrig 592c6907bf
test_runner: avoid overwriting root start time
This commit ensures the root test start time is not overwritten
when top level before()/after() hooks are run.

PR-URL: https://github.com/nodejs/node/pull/52020
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-03-11 03:18:07 +00:00
Joyee Cheung 075c95f61f
module: refactor ESM loader initialization and entry point handling
Split the `internal/process/esm_loader` file which contains the
singleton cascaded loader:

- The the singleton cascaded loader now directly resides in
  `internal/modules/esm/loader`, where the constructor also lives.
  This file is the root of most circular dependency of ESM code,
  (because components of the loader need the singleton itself),
  so this makes the dependency more obvious. Added comments about
  loading it lazily to avoid circular dependency.
- The getter to the cascaded loader is also turned into a method
  to make the side effect explicit.
- The sequence of `loadESM()` and `handleMainPromise` is now merged
  together into `runEntryPointWithESMLoader()` in
  `internal/modules/run_main` because this is intended to run entry
  points with the ESM loader and not just any module.
- Documents how top-level await is handled.

PR-URL: https://github.com/nodejs/node/pull/51999
Fixes: https://github.com/nodejs/node/issues/42868
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2024-03-10 08:21:18 +08:00
Colin Ihrig 29b231763e
test_runner: abort unfinished tests on async error
This commit updates the test runner's uncaughtException handler
to abort tests instead of assuming they finished running.

Fixes: https://github.com/nodejs/node/issues/51381
PR-URL: https://github.com/nodejs/node/pull/51996
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-09 21:30:02 +00:00
Moshe Atlow 5d13419dbd
test_runner: run before hook immediately if test started
PR-URL: https://github.com/nodejs/node/pull/52003
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-09 13:19:08 +00:00
cjihrig 0e8070d4f2 Revert "test_runner: do not invoke after hook when test is empty"
This reverts commit a53fd95d36.

This caused a regression because the original issue this commit
was attempting to fix is not a bug. The after() hook should
always run.

Fixes: https://github.com/nodejs/node/issues/51997
PR-URL: https://github.com/nodejs/node/pull/51998
Reviewed-By: Matthew Aitken <maitken033380023@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2024-03-07 09:16:36 +00:00
Malthe Borch 8451990668
test_runner: add support for null and date value output
PR-URL: https://github.com/nodejs/node/pull/51920
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2024-03-02 08:48:26 +00:00
Michał Drobniak 00dc6d9d97
test_runner: improve `--test-name-pattern` to allow matching single test
Try to match a test by name prefixed with all its ancestors
to ensure uniqueness of the name

Fixes: https://github.com/nodejs/node/issues/46728
PR-URL: https://github.com/nodejs/node/pull/51577
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-01 12:58:11 +00:00
Moshe Atlow aac5ad901d
test_runner: add `test:complete` event to reflect execution order
PR-URL: https://github.com/nodejs/node/pull/51909
Fixes: https://github.com/nodejs/node/issues/51907
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2024-03-01 10:08:09 +00:00
Malthe Borch 3ef7117877
test_runner: serialize 'expected' and 'actual' in isolation
Previously, a value seen in 'actual' would be serialized as a circular
reference if it had also appeared in 'expected'.

PR-URL: https://github.com/nodejs/node/pull/51851
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-02-29 08:05:59 +00:00
Marco Ippolito 6cb8a60bcc
test_runner: add ref methods to mocked timers
Fixes: https://github.com/nodejs/node/issues/51701
PR-URL: https://github.com/nodejs/node/pull/51809
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Tierney Cyren <hello@bnb.im>
2024-02-26 15:57:05 +00:00
Ben Richeson 17187dd2ed
test_runner: check if timeout was cleared by own callback
PR-URL: https://github.com/nodejs/node/pull/51673
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-02-18 11:08:44 +00:00
Tim Stableford d755f1ae88
lib: allow checking the test result from afterEach
PR-URL: https://github.com/nodejs/node/pull/51485
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2024-01-19 14:56:06 +00:00
Marco Ippolito a53fd95d36
test_runner: do not invoke after hook when test is empty
PR-URL: https://github.com/nodejs/node/pull/51389
Fixes: https://github.com/nodejs/node/issues/51371
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2024-01-08 10:03:45 +00:00
Pulkit Gupta dad666ad06
test_runner: fixed test object is incorrectly passed to setup()
PR-URL: https://github.com/nodejs/node/pull/50982
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-12-24 08:23:54 +00:00
Pulkit Gupta 71e19e8f08
test_runner: fixed to run after hook if before throws an error
PR-URL: https://github.com/nodejs/node/pull/51062
Refs: https://github.com/nodejs/node/issues/50842
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2023-12-17 21:15:02 +00:00
Pulkit Gupta 452d29c62b
test_runner: format coverage report for tap reporter
PR-URL: https://github.com/nodejs/node/pull/51119
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
2023-12-15 06:53:08 +00:00
Pulkit Gupta f801b58e77
test_runner: fix infinite loop when files are undefined in test runner
PR-URL: https://github.com/nodejs/node/pull/51047
Fixes: https://github.com/nodejs/node/issues/48823
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-12-10 09:35:22 +00:00
Moshe Atlow cb08703c80
test: fix test runner colored output test
PR-URL: https://github.com/nodejs/node/pull/51064
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-12-10 07:44:43 +00:00
Mika Fischer 314c8f95c5
test_runner: add tests for various mock timer issues
PR-URL: https://github.com/nodejs/node/pull/50384
Fixes: https://github.com/nodejs/node/issues/50365
Fixes: https://github.com/nodejs/node/issues/50381
Fixes: https://github.com/nodejs/node/issues/50382
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-11-12 08:26:28 +00:00
Moshe Atlow 3cce03a03f
test_runner: pass abortSignal to test files
PR-URL: https://github.com/nodejs/node/pull/50630
Fixes: https://github.com/nodejs/node/issues/50583
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-11-10 22:09:15 +00:00
Moshe Atlow 5fab505c76
test_runner: output errors of suites
PR-URL: https://github.com/nodejs/node/pull/50361
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-10-26 09:10:24 +00:00
Phil Nash c60c11aae1
test_runner: adds built in lcov reporter
Fixes https://github.com/nodejs/node/issues/49626

PR-URL: https://github.com/nodejs/node/pull/50018
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-10-25 12:11:32 +00:00
Mika Fischer 4f5db8b26d
test_runner: test return value of mocked promisified timers
PR-URL: https://github.com/nodejs/node/pull/50331
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-10-25 07:15:55 +00:00
Lucas Santos 45a0b153b3
test_runner: add Date to the supported mock APIs
signed-off-by: Lucas Santos <lhs.santoss@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/48638
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-10-23 11:23:12 +00:00
Raz Luvaton 527589b755
test_runner: disallow array in `run` options
PR-URL: https://github.com/nodejs/node/pull/49935
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-10-06 20:21:15 +00:00
Colin Ihrig 4b35a9cfda
test_runner: add test location for FileTests
This commit adds the previously missing test location for
FileTest tests.

Fixes: https://github.com/nodejs/node/issues/49926
Fixes: https://github.com/nodejs/node/issues/49927
PR-URL: https://github.com/nodejs/node/pull/49999
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-10-03 18:19:51 +00:00
Colin Ihrig c935d4c8fa
test_runner: replace spurious if with else
There is an `if` statement that likely should have been
an `else` in the original PR.

Refs: https://github.com/nodejs/node/pull/48915
PR-URL: https://github.com/nodejs/node/pull/49943
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
2023-09-29 11:04:58 +00:00
Moshe Atlow c9e72e34ab
test_runner: accept `testOnly` in `run`
PR-URL: https://github.com/nodejs/node/pull/49753
Fixes: https://github.com/nodejs/node/issues/49733
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
2023-09-24 10:46:13 +00:00
Moshe Atlow a4c7f81241
test_runner: catch reporter errors
PR-URL: https://github.com/nodejs/node/pull/49646
Fixes: https://github.com/nodejs/node/issues/48937
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-09-21 23:22:21 +03:00
Moshe Atlow 22907ce366
test_runner: add junit reporter
PR-URL: https://github.com/nodejs/node/pull/49614
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-09-14 08:43:53 +00:00
Caio Borghi 4efa3744a0
test_runner: add jsdocs to mock.js
PR-URL: https://github.com/nodejs/node/pull/49555
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-09-14 04:24:39 +00:00
Deokjin Kim b3fc9173ed
lib: use internal `fileURLToPath`
For internal usage, `internal/url` should be used.

Refs: https://github.com/nodejs/node/pull/49553
PR-URL: https://github.com/nodejs/node/pull/49558
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2023-09-10 14:57:37 +00:00
Erick Wendel a61bc9d933
test_runner: fix invalid timer call
Signed-off-by: Erick Wendel <erick.workspace@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/49477
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-09-05 21:17:28 +00:00
Erick Wendel f139cc3ee8
test_runner: add jsdocs to MockTimers
Signed-off-by: Erick Wendel <erick.workspace@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/49476
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-09-05 16:53:07 +00:00
Moshe Atlow 47c5152545
test_runner: fix typescript coverage
PR-URL: https://github.com/nodejs/node/pull/49406
Fixes: https://github.com/nodejs/node/issues/49398
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-09-04 21:45:07 +00:00
Erick Wendel 8e82cfcbb3
test_runner: preserve original property descriptor
PR-URL: https://github.com/nodejs/node/pull/49433
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Raz Luvaton <rluvaton@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-09-03 19:08:26 +00:00
Erick Wendel c750049318
test_runner: add support for setImmediate
PR-URL: https://github.com/nodejs/node/pull/49397
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
2023-08-31 06:42:03 +00:00
Phil Nash 3a6a80a4e1
test_runner: report covered lines, functions and branches to reporters
This is a breaking change for the format of test:coverage events. But
the test coverage is still experimental, so I don't believe it requires
a semver-major bump.

Fixes https://github.com/nodejs/node/issues/49303

PR-URL: https://github.com/nodejs/node/pull/49320
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-08-30 20:22:45 +00:00
Colin Ihrig b5740756ca
test_runner: reland run global after() hook earlier
This commit reverts the revert in
bb52656fc6. It also includes the
fix for the issue that required the revert
(https://github.com/nodejs/node/pull/49059#issuecomment-1675171959)
and an additional common.mustCall() in the added test.

Refs: https://github.com/nodejs/node/pull/49059
Refs: https://github.com/nodejs/node/pull/49110
PR-URL: https://github.com/nodejs/node/pull/49116
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-08-14 14:29:29 +00:00
Joyee Cheung bb52656fc6
Revert "test_runner: run global after() hook earlier"
This reverts commit 6346bdc526.

Reason for revert: breaking CI

PR-URL: https://github.com/nodejs/node/pull/49110
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-08-12 04:19:05 +00:00
cjihrig 6346bdc526
test_runner: run global after() hook earlier
This commit moves the global after() hook execution from the
'beforeExit' event to the point where all tests have finished
running. This gives the global after() a chance to clean up
handles that would otherwise prevent the 'beforeExit' event
from being emitted.

PR-URL: https://github.com/nodejs/node/pull/49059
Fixes: https://github.com/nodejs/node/issues/49056
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-08-11 09:16:51 -04:00
cjihrig 4e61c22699
test_runner: expose location of tests
This commit adds each test's line and column number to the reporter
output. This will aid in debugging test suite failures when error
stacks are not helpful, test suites are large, or tests have the
same name. This data is also exposed on the spec reporter.

This commit also replaces the filename that was previously being
reported, with the filename where the test actually exists. These
are normally correct, but could be wrong if tests were run from
a file other than the user's entrypoint.

PR-URL: https://github.com/nodejs/node/pull/48975
Fixes: https://github.com/nodejs/node/issues/48457
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-08-11 09:12:49 -04:00
Raz Luvaton c58e8fc31f
test_runner: fix global after not failing the tests
PR-URL: https://github.com/nodejs/node/pull/48913
Fixes: https://github.com/nodejs/node/issues/48867
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-08-02 20:06:25 +00:00
Raz Luvaton 88094bba3c
test_runner: fix timeout in *Each hook failing further tests
PR-URL: https://github.com/nodejs/node/pull/48925
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-31 22:13:02 +00:00
Raz Luvaton 7a33f2b88d
test_runner: cleanup test timeout abort listener
fix #48475

PR-URL: https://github.com/nodejs/node/pull/48915
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-31 08:10:36 +00:00
Moshe Atlow a955c534a8 test_runner: dont set exit code on todo tests
PR-URL: https://github.com/nodejs/node/pull/48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-29 20:45:16 +00:00
Moshe Atlow 040865c648 test_runner: fix todo and only in spec reporter
PR-URL: https://github.com/nodejs/node/pull/48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-29 20:45:16 +00:00
Colin Ihrig bef3942820
test_runner: unwrap error message in TAP reporter
The TAP reporter unwraps errors that come from user code. It
was not properly unwrapping the error message when the failure
originated from a test hook. This lead to difficult to debug
errors when TAP output was used. This commit updates the TAP
reporter to properly unwrap the error message.

Fixes: https://github.com/nodejs/node/issues/48941
PR-URL: https://github.com/nodejs/node/pull/48942
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-29 18:36:55 +00:00
Raz Luvaton ee391f3781
test_runner: add `__proto__` null
PR-URL: https://github.com/nodejs/node/pull/48663
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-29 13:22:16 +00:00
Moshe Atlow 48345d0f62 watch: decrease debounce rate
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:14 +00:00
Moshe Atlow b814de6d77 watch: use debounce instead of throttle
PR-URL: https://github.com/nodejs/node/pull/48926
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-07-27 20:39:13 +00:00
Raz Luvaton a0f3ed8ac4
test_runner: fix global before not called when no global test exists
PR-URL: https://github.com/nodejs/node/pull/48877
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-07-24 11:38:23 +00:00
Raz Luvaton 44b8a061df
test_runner: fix async callback in describe not awaited
PR-URL: https://github.com/nodejs/node/pull/48856
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-23 10:22:35 +00:00
Raz Luvaton 24c3d8a1b3
test_runner: call abort on test finish
PR-URL: https://github.com/nodejs/node/pull/48827
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
2023-07-21 12:53:58 +00:00
Raz Luvaton 99881304c4
test_runner: add shards support
PR-URL: https://github.com/nodejs/node/pull/48639
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-07-05 03:57:28 +00:00
Shocker d312bd96d3
test_runner: fixed `test` shorthands return type
`test.todo`, `test.only` and `test.skip` are expected to return the
same as `test`. This commit corrects the inconsistent behavior of
these shorthands.

Fixes: https://github.com/nodejs/node/issues/48557
PR-URL: https://github.com/nodejs/node/pull/48555
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-07-03 09:04:11 +00:00
Benjamin Gruenbaum 1683e6030e
test: add Symbol.dispose support to mock timers
PR-URL: https://github.com/nodejs/node/pull/48549
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Erick Wendel <erick.workspace@gmail.com>
2023-06-29 07:45:44 +00:00
Moshe Atlow 7cd4e70948 test_runner: support passing globs
PR-URL: https://github.com/nodejs/node/pull/47653
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-06-25 14:40:34 +00:00
Chemi Atlow 78f6952751
fs: use kResistStopPropagation
PR-URL: https://github.com/nodejs/node/pull/48521
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-06-24 15:52:38 +00:00
Erick Wendel da80964a3d
test_runner: add initial draft for fakeTimers
Signed-off-by: Erick Wendel <erick.workspace@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/47775
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-06-22 04:01:52 +00:00
Moshe Atlow 8bc6e193a0
test_runner: add enqueue and dequeue events
PR-URL: https://github.com/nodejs/node/pull/48428
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-13 18:33:43 +00:00
Moshe Atlow c21fe3ad87
test_runner: make `--test-name-pattern` recursive
PR-URL: https://github.com/nodejs/node/pull/48382
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-06-11 15:02:18 +00:00
Damien Seguin 23c7f65ead
test_runner: refactor coverage report output for readability
Add a "table" parameter to getCoverageReport.
Keep the tap coverage output intact.
Change the output by adding padding and truncating the tables' cells.
Add separation lines for table head/body/foot.
Group uncovered lines as ranges.
Add yellow color for coverage between 50 and 90.

Refs: https://github.com/nodejs/node/pull/46674
PR-URL: https://github.com/nodejs/node/pull/47791
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-06-07 09:08:27 +00:00
Moshe Atlow 5c27cc2afb
test_runner: apply `runOnly` on suites
PR-URL: https://github.com/nodejs/node/pull/48279
Fixes: https://github.com/nodejs/node/issues/47937
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-05 00:05:34 +03:00
Moshe Atlow 5d685e4198 test_runner: emit `testdrained` event
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:09 +00:00
Moshe Atlow ca71ae6549 test_runner: stop watch mode when abortSignal aborted
PR-URL: https://github.com/nodejs/node/pull/48259
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-06-02 00:28:08 +00:00
Moshe Atlow 5e98a74327
test_runner: fix global after hook
PR-URL: https://github.com/nodejs/node/pull/48231
Fixes: https://github.com/nodejs/node/issues/48230
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-31 15:33:30 +00:00
Colin Ihrig b47fce0762
test_runner: remove redundant check from coverage
The code coverage reporting logic already filters out URLs that
don't start with 'file:', so there is no need to also filter
out URLs that start with 'node:'.

PR-URL: https://github.com/nodejs/node/pull/48070
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
2023-05-29 04:47:21 +00:00
Moshe Atlow e74465d678 test_runner: pass FORCE_COLOR to child process
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:39 +00:00
Moshe Atlow 40a7a055c6 test_runner: dont split lines on `test:stdout`
PR-URL: https://github.com/nodejs/node/pull/48057
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-25 10:49:38 +00:00
Moshe Atlow f0e70eede2
test_runner: fix test deserialize edge cases
PR-URL: https://github.com/nodejs/node/pull/48106
Fixes: https://github.com/nodejs/node/issues/48103
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-23 21:03:19 +00:00
Shiba 1229a22e51
test_runner: delegate stderr and stdout formatting to reporter
Introduce new `TestsStream` events `test:stderr` and `test:stdout`
to delegate `stderr` and `stdout` (e.g. `console.log()`) formatting
to the reporter. And patch existing reporters to:
- Spec: output the message as it is
- TAP: stay the same with existing `test:diagnostic`

PR-URL: https://github.com/nodejs/node/pull/48045
Fixes: https://github.com/nodejs/node/issues/48011
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-23 04:44:48 +00:00
Raz Luvaton 7e0c12fd56
test_runner: display dot report as wide as the terminal width
PR-URL: https://github.com/nodejs/node/pull/48038
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-23 04:44:38 +00:00
Moshe Atlow 05693acb15
lib: support FORCE_COLOR for non TTY streams
PR-URL: https://github.com/nodejs/node/pull/48034
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-18 12:36:38 +00:00
Benjamin Gruenbaum 3cf05be50e
test: fix suite signal
PR-URL: https://github.com/nodejs/node/pull/47800
Fixes: https://github.com/nodejs/node/issues/47882
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-05-17 08:57:21 +00:00
Moshe Atlow 7116bc08d7
test_runner: use v8.serialize instead of TAP
PR-URL: https://github.com/nodejs/node/pull/47867
Fixes: https://github.com/nodejs/node/issues/44656
Fixes: https://github.com/nodejs/node/issues/47955
Fixes: https://github.com/nodejs/node/issues/47481
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-05-15 14:27:45 +00:00
Chemi Atlow 8b3777d0c8
test_runner: add shorthands to `test`
PR-URL: https://github.com/nodejs/node/pull/47909
Fixes: https://github.com/nodejs/node/issues/47897
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-05-15 05:23:22 +00:00
Phil Nash 24615bd409
test_runner: fix ordering of test hooks
For tests with subtests the before hook was being run after the
beforeEach hook, which is the opposite to test suites and expectations.

Also, a function was being used to close over the after hooks, but at
the point it was being run the after hooks were not yet set up.

Fixes #47915

PR-URL: https://github.com/nodejs/node/pull/47931
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-05-11 10:10:56 +00:00
cjihrig 4360389d67
test_runner: omit inaccessible files from coverage
If V8 generates code coverage for a file that is later
inaccessible to the test runner, then omit that file from the
coverage report.

PR-URL: https://github.com/nodejs/node/pull/47850
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Beth Griggs <bethanyngriggs@gmail.com>
2023-05-10 14:08:00 +00:00
cjihrig b31d587dc8
test_runner: support combining coverage reports
This commit adds support for combining code coverage reports
in the test runner. This allows coverage to be collected for
child processes, and by extension, the test runner CLI.

PR-URL: https://github.com/nodejs/node/pull/47686
Fixes: https://github.com/nodejs/node/issues/47669
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2023-04-28 13:13:53 +00:00
cjihrig ab3a54f2ee
test_runner: remove no-op validation
The code already checks if testNamePatterns is an
array, and converts it to an array if it is not.

PR-URL: https://github.com/nodejs/node/pull/47687
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Debadree Chatterjee <debadree333@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Deokjin Kim <deokjin81.kim@gmail.com>
2023-04-25 16:03:51 +00:00
Moshe Atlow dccd25e5bb test_runner: fix test runner concurrency
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:01 +00:00
Moshe Atlow dac02632a6 test_runner: fix test counting
PR-URL: https://github.com/nodejs/node/pull/47675
Fixes: https://github.com/nodejs/node/issues/47365
Fixes: https://github.com/nodejs/node/issues/47696
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-24 22:32:00 +00:00
Moshe Atlow 95e40b528d
test_runner: fix nested hooks
PR-URL: https://github.com/nodejs/node/pull/47648
Fixes: https://github.com/nodejs/node/issues/47643
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-23 17:55:01 +00:00
atlowChemi c6279690be
test_runner: add testNamePatterns to run api
Accept a `testNamePatterns` value in the `run` fn, and drill those
patterns to the spawned processes.

PR-URL: https://github.com/nodejs/node/pull/47648
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-23 15:30:23 +03:00
cjihrig 53e3c0ebf8
test_runner: support coverage of unnamed functions
This commit updates the code coverage logic to skip the first
function in a file (which does not correspond to an actual
function) instead of skipping all functions without a name.

PR-URL: https://github.com/nodejs/node/pull/47652
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-23 07:17:49 +00:00
cjihrig ecb023be8a
test_runner: move coverage collection to root.postRun()
This commit moves code coverage collection from the test
harness exit handler to the postRun() function of the root
test.

This is necessary preparatory work for supporting
code coverage with --test. The reason is that --test is
implemented on top of run(), and that function calls the root
test's postRun() function, which outputs the test summary. This
happens before the harness exit handler.

PR-URL: https://github.com/nodejs/node/pull/47651
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-23 06:57:10 +00:00
Chemi Atlow 2ec4ef7c24
test_runner: execute before hook on test
Execute the before hook for Test as well, and execute it on root before
executing any tests.

Fixes: https://github.com/nodejs/node/issues/47518
PR-URL: https://github.com/nodejs/node/pull/47586
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-04-18 20:59:49 +00:00
Moshe Atlow d3b0a2a68b
test_runner: avoid reporting parents of failing tests in summary
PR-URL: https://github.com/nodejs/node/pull/47579
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-04-18 07:36:56 +00:00
Moshe Atlow 46a3cff698
test_runner: fix spec skip detection
PR-URL: https://github.com/nodejs/node/pull/47537
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-04-16 07:26:37 +00:00
Moshe Atlow 7ec93fb296
test_runner: color errors only when colors are available
PR-URL: https://github.com/nodejs/node/pull/47394
Fixes: https://github.com/nodejs/node/issues/47393
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2023-04-04 19:00:25 +03:00
Moshe Atlow af8ed02067
test_runner: hide failing tests title when all tests pass
PR-URL: https://github.com/nodejs/node/pull/47370
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-04-04 10:39:43 +03:00
Moshe Atlow 102540aef9
test_runner: stringify AssertError expected and actual
PR-URL: https://github.com/nodejs/node/pull/47088
Fixes: https://github.com/nodejs/node/issues/47075
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-04-04 04:14:23 +00:00
Pulkit Gupta 6d065863c2
test_runner: add code coverage support to spec reporter
PR-URL: https://github.com/nodejs/node/pull/46674
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-04-02 17:18:48 +00:00
HinataKah0 2e8f8eb4e3
test_runner: report failing tests after summary
Re-output failing tests after summary has been printed.
This behavior follows other popular test runners
(e.g. jest, mocha, etc...).

Updated SpecReporter:
1. When there is a 'test:fail' event, the test will be
stored.
2. After no more input, all the failed tests will be
flushed.
3. Extract the logic for formatting a test report
into a re-usable function.

Fixes: https://github.com/nodejs/node/issues/47110
PR-URL: https://github.com/nodejs/node/pull/47164
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-23 18:45:53 +00:00
Moshe Atlow d1eaded0d1
test_runner: count nested tests
PR-URL: https://github.com/nodejs/node/pull/47094
Fixes: https://github.com/nodejs/node/issues/46762
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-21 08:16:38 +00:00
Debadree Chatterjee d51541b720
test_runner: accept \x1b as a escape symbol
Fixes: https://github.com/nodejs/node/issues/46959
PR-URL: https://github.com/nodejs/node/pull/47050
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-03-16 20:39:36 +00:00
Steve Herzog 334bb179df
test_runner: support defining test reporter in NODE_OPTIONS
Adds --test-reporter and --test-reporter-destination as
allowable options in NODE_OPTIONS. Also adds the CLI flag
--test-child-process to allow forcing the default
test-reporter for inter-process communication.

Fixes: https://github.com/nodejs/node/issues/46484
PR-URL: https://github.com/nodejs/node/pull/46688
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-03-14 16:59:35 +00:00
Moshe Atlow 9960c36f3a
test_runner: default to spec reporter when on TTY environment
PR-URL: https://github.com/nodejs/node/pull/46969
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-09 00:22:19 +02:00
cjihrig d4c3e3b496
test_runner: remove root tracking set
The wasRootSetup Set in the test harness appears to be
redundant, since the startTime field can be used interchangeably.
This commit removes wasRootSetup.

PR-URL: https://github.com/nodejs/node/pull/46961
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 14:03:02 -05:00
cjihrig 3d63d5333d
test_runner: handle errors not bound to tests
This commit addresses a previously untested branch of the code.
It is possible when using the test runner that an error occurs
outside of a test. In this case, the test runner would simply
rethrow the error. This commit updates the logic to handle the
error in the same fashion as other uncaughtExceptions.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:47 -05:00
cjihrig e079aa80f0
test_runner: throw if harness is not bootstrapped
This commit updates the test harness to re-throw uncaught errors
if bootstrapping has not completed. This updates the existing
logic which tried to detect a specific error code.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:41 -05:00
cjihrig 4e3bc06e65
test_runner: track bootstrapping process
This commit updates the test harness and root test to track
when bootstrapping has completed.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:37 -05:00
cjihrig b191f09e6e
test_runner: give the root test a harness reference
This commit replaces the 'coverage' reference inside of the Test
class with a more generic harness reference which includes
coverage. This will let the root test more easily track process
level state such as code coverage, uncaughtException handlers,
and the state of bootstrapping.

PR-URL: https://github.com/nodejs/node/pull/46962
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
2023-03-07 13:59:28 -05:00
Moshe Atlow ca033c16fe
test_runner: align behavior of it and test
PR-URL: https://github.com/nodejs/node/pull/46889
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-05 09:42:29 +02:00
Moshe Atlow a37b72da87
test_runner: avoid running twice tests in describe
PR-URL: https://github.com/nodejs/node/pull/46888
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-03-03 07:27:16 +02:00
Moshe Atlow c7e6a0c1b1
test_runner: fix reconstruction of errors extracted from YAML
PR-URL: https://github.com/nodejs/node/pull/46872
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Nitzan Uziely <linkgoron@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-03-01 22:16:39 +00:00
Moshe Atlow 9c61d414c4
test_runner: reset count on watch mode
PR-URL: https://github.com/nodejs/node/pull/46577
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-26 16:28:30 +02:00
Colin Ihrig 0c90be9072
test_runner: better handle async bootstrap errors
The test runner is bootstrapped synchronously, with the exception
of importing custom reporters. To better handle asynchronously
throw errors, this commit introduces an internal error type that
can be checked for from the test runner's uncaughtException
handler.

After https://github.com/nodejs/node/pull/46707 and this change
land, the other throw statement in the uncaughtException handler
can be removed. This will allow the test runner to handle errors
thrown from outside of tests (which currently prevents the test
runner from reporting results).

PR-URL: https://github.com/nodejs/node/pull/46720
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-24 14:13:24 +00:00
Richie McColl 0093fd3ca8
test_runner: add `describe.only` and `it.only` shorthands
PR-URL: https://github.com/nodejs/node/pull/46604
Fixes: https://github.com/nodejs/node/issues/46562
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-21 22:28:03 +00:00
Moshe Atlow ce49f79ed5
test_runner: bootstrap reporters before running tests
PR-URL: https://github.com/nodejs/node/pull/46737
Fixes: https://github.com/nodejs/node/issues/46747
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-02-21 23:06:13 +02:00
Richie McColl c90ea9301b
test_runner: emit test-only diagnostic warning
PR-URL: https://github.com/nodejs/node/pull/46540
Fixes: https://github.com/nodejs/node/issues/46448
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-21 08:39:17 +00:00
Colin Ihrig 5e954c3ce3
test_runner: centralize CLI option handling
The test runner relies on a few CLI options. That code was spread
across a few locations. This commit centralizes that logic.

PR-URL: https://github.com/nodejs/node/pull/46707
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-20 01:26:04 +00:00
Richie McColl 261bf9652b
test_runner: display skipped tests in spec reporter output
PR-URL: https://github.com/nodejs/node/pull/46651
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-18 22:36:08 +00:00
Mert Can Altın 3c6547fcc8
test_runner: parse non-ascii character correctly
PR-URL: https://github.com/nodejs/node/pull/45736
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-18 18:10:16 +00:00
Moshe Atlow 3354f89dcf
test_runner: flatten TAP output when running using `--test`
PR-URL: https://github.com/nodejs/node/pull/46440
Fixes: https://github.com/nodejs/node/issues/45833
Refs: https://github.com/nodejs/node/issues/45833
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-02-18 16:19:24 +00:00
Moshe Atlow 2787e2dfc2
test_runner: allow nesting test within describe
PR-URL: https://github.com/nodejs/node/pull/46544
Fixes: https://github.com/nodejs/node/issues/46478
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-17 13:42:01 +00:00
Antoine du Hamel cbcefdf253
lib: enforce use of trailing commas in more files
Some part of the codebase already use trailing commas, this commit is
adding a lint rule to ensure it stays this way.
This commit also adds the rule for a few files that were missing only
one or two trailing commas.

PR-URL: https://github.com/nodejs/node/pull/46655
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-16 18:47:24 +01:00
Antoine du Hamel fe514bf960
lib: enforce use of trailing commas for functions
PR-URL: https://github.com/nodejs/node/pull/46629
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2023-02-14 18:45:16 +01:00
Pulkit Gupta 4c08c20e57
test: fix tap parser fails if a test logs a number
PR-URL: https://github.com/nodejs/node/pull/46056
Fixes: https://github.com/nodejs/node/issues/46048
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-02-06 16:58:51 +00:00
Moshe Atlow 1118db718c
test_runner: fix missing test diagnostics
PR-URL: https://github.com/nodejs/node/pull/46450
Fixes: https://github.com/nodejs/node/issues/45911
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 20:00:06 +00:00
Pulkit Gupta 61c65b066b
test_runner: top-level diagnostics not ommited when running with --test
PR-URL: https://github.com/nodejs/node/pull/46441
Fixes: https://github.com/nodejs/node/issues/45910
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-02-02 10:42:43 +00:00
Pulkit Gupta 2f38c74e26
test: fix tap escaping with and without --test
PR-URL: https://github.com/nodejs/node/pull/46311
Fixes: https://github.com/nodejs/node/issues/45836
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-01-28 21:07:47 +00:00
Antoine du Hamel 931f035bac test_runner: update comment to comply with eslint no-fallthrough rule
PR-URL: https://github.com/nodejs/node/pull/46258
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2023-01-19 16:47:39 +00:00
Jordan Harband 757c104147
tools: add `prefer-proto` rule
fixup: add support for `Object.create(null)`

fixup: extend to any 1-argument Object.create call

fixup: add tests
PR-URL: https://github.com/nodejs/node/pull/46083
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2023-01-10 05:38:36 +00:00
cjihrig 1ecc03eccd
doc,lib,src,test: rename --test-coverage
Add experimental to the name as requested during review.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:10:00 -05:00
cjihrig 8c95cb012b
test_runner: add initial code coverage support
This commit adds code coverage functionality to the node:test
module. When node:test is used in conjunction with the new
--test-coverage CLI flag, a coverage report is created when
the test runner finishes. The coverage summary is forwarded to
any test runner reporters so that the display can be customized
as desired. This new functionality is compatible with the
existing NODE_V8_COVERAGE environment variable as well.

There are still several limitations, which will be addressed in
subsequent pull requests:

- Coverage is only reported for a single process. It is possible
  to merge coverage reports together. Once this is done, the
  --test flag will be supported as well.
- Source maps are not currently supported.
- Excluding specific files or directories from the coverage
  report is not currently supported. Node core modules and
  node_modules/ are excluded though.

PR-URL: https://github.com/nodejs/node/pull/46017
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
2023-01-08 16:09:41 -05:00
Colin Ihrig 4788c0d2a0
test_runner: make built in reporters internal
This commit updates the test runner to make the built in test
reporters internal modules.

PR-URL: https://github.com/nodejs/node/pull/46092
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-01-06 19:34:12 +00:00
Moshe Atlow 9eb363a3e0
test_runner: report `file` in test runner events
PR-URL: https://github.com/nodejs/node/pull/46030
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2023-01-02 21:22:54 +00:00
Colin Ihrig fe5710effc
test_runner: use os.availableParallelism()
This commit addresses an existing TODO in the code by moving
to the new os.availableParallelism() instead of os.cpus().length.

PR-URL: https://github.com/nodejs/node/pull/45969
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-27 03:30:19 +00:00
Geoffrey Booth 12c0571c8f
module: move test reporter loading
Move the logic for handling --test-reporter out of the
general module loader and into the test_runner subsystem.

PR-URL: https://github.com/nodejs/node/pull/45923
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-22 21:40:32 +00:00
Moshe Atlow a1b27b25bb
test_runner: add reporters
PR-URL: https://github.com/nodejs/node/pull/45712
Fixes: https://github.com/nodejs/node/issues/45648
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2022-12-19 17:35:57 +00:00
Erick Wendel 929aada39d
test: fix mock.method to support class instances
It fixes a problem when trying to spy a method
from a class instance or static functions
on a class instance

PR-URL: https://github.com/nodejs/node/pull/45608
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-17 14:35:09 +00:00
Colin Ihrig 385d595a4f
test_runner: run t.after() if test body throws
This commit fixes a bug where t.after() was not called if the
test body threw an exception.

PR-URL: https://github.com/nodejs/node/pull/45870
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
2022-12-17 06:36:15 +00:00
Moshe Atlow 232efb06fe
test_runner: parse yaml
PR-URL: https://github.com/nodejs/node/pull/45815
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2022-12-13 14:51:35 +00:00
Antoine du Hamel 2483da743c
test_runner: refactor `tap_lexer` to use more primordials
PR-URL: https://github.com/nodejs/node/pull/45744
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2022-12-12 07:30:47 +01:00