Having an experimental feature behind a flag makes change
if we are expecting significant breaking changes to its API.
Since the Worker API has been essentially stable since
its initial introduction, and no noticeable doubt about
possibly not keeping the feature around has been voiced,
removing the flag and thereby reducing the barrier to experimentation,
and consequently receiving feedback on the implementation,
seems like a good idea.
PR-URL: https://github.com/nodejs/node/pull/25361
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Yuta Hiroto <hello@hiroppy.me>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Masashi Hirano <shisama07@gmail.com>
Reviewed-By: Weijia Wang <starkwang@126.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Some tests which create files and check file permissions assume the
umask is compatible with 022, and break when set to something like 007.
Explicitly set umask to 022
PR-URL: https://github.com/nodejs/node/pull/25213
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Partition test/addons-napi into test/js-native-api and test/node-api to
isolate the Node.js-agnostic portion of the N-API tests from the
Node.js-specific portion.
PR-URL: https://github.com/nodejs/node/pull/24557
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
While running the test suite the progress bar shows former line
endings if the new line is shorter than the former line. The length
was calculated without the line ending. It is now an empty string
to prevent the off by one error instead of using extra whitespace.
PR-URL: https://github.com/nodejs/node/pull/24748
Refs: https://github.com/nodejs/node/pull/24486
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Currently, configuring --without-ssl will cause the following test to
fail:
=== release test-https-agent-additional-options ===
Path: parallel/test-https-agent-additional-options
out/Release/node: bad option: --tls-v1.1
Command: out/Release/node --tls-v1.1
/node/test/parallel/test-https-agent-additional-options.js
=== release test-https-agent-session-eviction ===
Path: parallel/test-https-agent-session-eviction
out/Release/node: bad option: --tls-v1.0
Command: out/Release/node --tls-v1.0
/node/test/parallel/test-https-agent-session-eviction.js
This commit adds a check for the --tls-v.x flags and skips them if node
was built without crypto support.
PR-URL: https://github.com/nodejs/node/pull/24376
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Move benchmark tests (which are slow) out of the main test suite. We can
hopefully add them to node-daily-master so that they are still run daily
on CI.
PR-URL: https://github.com/nodejs/node/pull/24265
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Add the code cache tests to the default test suite, and test
the bookkeeping when the binary is not built with the code cache.
- Test the code cache generator to make sure we do not accidentally
break it - until we enable code cache in the CI.
Refs: https://github.com/nodejs/node/issues/21563
PR-URL: https://github.com/nodejs/node/pull/23855
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Ref: https://github.com/nodejs/node/pull/23585#issuecomment-430585490
Python's `list.remove` will throw if the element is not found and also
it removes only the first occurrence.
This patch replaces the use of `list.remove` with a `filter` which
solves both of the above mentioned problems.
PR-URL: https://github.com/nodejs/node/pull/23727
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Matheus Marchini <mat@mmarchini.me>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
* Hoist common code to base class
(`GetTestStatus`, and the `section` property to `TestConfiguration`)
* Replace ListSet with the built in set
* Remove ClassifiedTest
* Inline PrintReport
* How cases_to_run are filtered
PR-URL: https://github.com/nodejs/node/pull/23251
Reviewed-By: Rich Trott <rtrott@gmail.com>
Since faking TTY input is not otherwise fake-able, we need
support in the test runner for it.
PR-URL: https://github.com/nodejs/node/pull/23053
Reviewed-By: James M Snell <jasnell@gmail.com>
There’s no reason to have a separate addon just for
testing GC anymore.
PR-URL: https://github.com/nodejs/node/pull/22001
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jon Moss <me@jonathanmoss.me>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Removes a couple of unused/empty functions inside of `tools/test.py`
PR-URL: https://github.com/nodejs/node/pull/22010
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
- Updates `test/README.md` with new suites
- Fixes some outdated `IGNORED_SUITES` listings
- Allows for `test/known_issues` suite to be run by default
PR-URL: https://github.com/nodejs/node/pull/21910
Reviewed-By: Vse Mozhet Byt <vsemozhetbyt@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Lance Ball <lball@redhat.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
This patch speeds up the startup time and reduce the startup memory
footprint by using V8 code cache when comiling builtin modules.
The current approach is demonstrated in the `with-code-cache`
Makefile target (no corresponding Windows target at the moment).
1. Build the binary normally (`src/node_code_cache_stub.cc` is used),
by now `internalBinding('code_cache')` is an empty object
2. Run `tools/generate_code_cache.js` with the binary, which generates
the code caches by reading source code of builtin modules off source
code exposed by `require('internal/bootstrap/cache').builtinSource`
and then generate a C++ file containing static char arrays of the
code cache, using a format similar to `node_javascript.cc`
3. Run `configure` with the `--code-cache-path` option so that
the newly generated C++ file will be used when compiling the
new binary. The generated C++ file will put the cache into
the `internalBinding('code_cache')` object with the module
ids as keys
4. The new binary tries to read the code cache from
`internalBinding('code_cache')` and use it to compile
builtin modules. If the cache is used, it will put the id
into `require('internal/bootstrap/cache').compiledWithCache`
for bookkeeping, otherwise the id will be pushed into
`require('internal/bootstrap/cache').compiledWithoutCache`
This patch also added tests that verify the code cache is
generated and used when compiling builtin modules.
The binary with code cache:
- Is ~1MB bigger than the binary without code cahe
- Consumes ~1MB less memory during start up
- Starts up about 60% faster
PR-URL: https://github.com/nodejs/node/pull/21405
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Gus Caplan <me@gus.host>
`v8-updates` holds all tests related to V8 updates, for example, testing
for postmortem metadata chages.
PR-URL: https://github.com/nodejs/node/pull/20783
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Enable running tests inside workers by passing `--worker`
to `tools/test.py`. A number of tests are marked as skipped,
or have been slightly altered to fit the different environment.
PR-URL: https://github.com/nodejs/node/pull/20876
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Shingo Inoue <leko.noor@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: John-David Dalton <john.david.dalton@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Make test runner capable of skipping tests, which makes it possible
to skip the failing test/message/core_line_numbers.js test.
Make nyc no longer generate compact instrumentation (this causes
significantly different code output, which leads to failing test
assertions).
PR-URL: https://github.com/nodejs/node/pull/20794
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
When a test times out, the contents of stdout and stderr can often be
highly valuable in debugging. Provide that information.
Refs: https://github.com/nodejs/node/issues/19906#issuecomment-382073888
PR-URL: https://github.com/nodejs/node/pull/20260
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Include the exit code in test failures. This will give us more
information during the currently-puzzling failures that provide no
information in CI such as:
```
03:10:10 not ok 563 parallel/test-fs-truncate
03:10:10 ---
03:10:10 duration_ms: 1.119
03:10:10 severity: fail
03:10:10 stack: |-
03:10:10 ...
```
PR-URL: https://github.com/nodejs/node/pull/19855
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
common.PORT should not be used in parallelized tests. (There can be a
port collision if another tests requests an arbitrary open port from the
operating system and ends up getting common.PORT before a test that uses
common.PORT uses the port.) In such a situation, throw an error.
PR-URL: https://github.com/nodejs/node/pull/17559
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Adds a way to mark a specified test as 'flaky' on fips compliant
systems.
Earlier, the ``tools/test.py`` script supported only 'mode',
'system' and 'arch' for test environment specification. This limits the
ability to specify the behavior of tests and setting pre-determined
behavior of the same on other types of systems. As an example, the
feature request below indicates the need to specify certain tests as
'flaky' on fips compliant systems. It hints at future possibility of a
shared library, which in turn may need a specifier for running tests.
This commit introduces a new item in the ``env`` dict, called ``type``
which defaults to ``simple`` type. It also adds an optional command
line argument ``--type``, which inputs strings. Current functionality
extends to setting ``simple`` or ``fips`` for this ``type`` variable.
However, extending it to further uses is rather simple by adding "if"
conditions at appropriate places in the ``tools/test.py`` script.
PR-URL: https://github.com/nodejs/node/pull/16329
Fixes: https://github.com/nodejs/node/issues/14746
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
`test-make-doc` fails in CI on Raspberry Pi devices where `test-ci-js`
runs the test but does not build the docs. We do not want to build the
docs in these cases. Move the test to the `doctool` suite and add that
suite to `IGNORED_SUITES` in `test.py`. Specify `doctool` as a test
suite to run with `make test` or `make test-ci` but not with the
`make test-ci-js` job run on cross-compiled fanned CI tests like we do
with Raspberry Pi devices in CI.
PR-URL: https://github.com/nodejs/node/pull/16301
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Anatoli Papirovski <apapirovski@mac.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
The 'Main' function in tools/test.py file was using a variable named
``all_outcomes`` to store a value not being used. It is a best practice
to name unused variables, often return values of functions/methods (as
in this case) as ``_`` [1]. This just helps keep the code a bit cleaner
and avoid any silly mistakes.
PR-URL: https://github.com/nodejs/node/pull/16171
Refs: [1] https://stackoverflow.com/a/5477153
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Claudio Rodriguez <cjrodr@yahoo.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
You can now run suites for subsystem using shorthand, e.g., http.
Switch to black-list of default test folders from white-list.
Tests run by 'make test', 'make coverage', etc., now configurable.
Stop running known_issues suite when collecting test coverage.
PR-URL: https://github.com/nodejs/node/pull/15450
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
When ESM support was added it created a regression in the test runner
that broke the ability to run individual tests. This commit
re-introduces the use of `NormalizePath` which fixes the regression
in the test runner
Refs: https://github.com/nodejs/node/pull/15300
PR-URL: https://github.com/nodejs/node/pull/15329
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This follows the EPS an allows the node CLI to have ESM as an entry point.
`node ./example.mjs`. A newer V8 is needed for `import()` so that is not
included. `import.meta` is still in specification stage so that also is not
included.
PR-URL: https://github.com/nodejs/node/pull/14369
Author: Bradley Farias <bradley.meck@gmail.com>
Author: Guy Bedford <guybedford@gmail.com>
Author: Jan Krems <jan.krems@groupon.com>
Author: Timothy Gu <timothygu99@gmail.com>
Author: Michaël Zasso <targos@protonmail.com>
Author: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Currently, tests in test/abort do not run in CI.
This change configures the test runner to not write core files for abort
tests and to run them.
PR-URL: https://github.com/nodejs/node/pull/14013
Fixes: https://github.com/nodejs/node/issues/14012
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
At least temporarily until snapshots are restored, increase timeout for
tests. Tests that spawn many processes are timing out across many
platforms on CI.
PR-URL: https://github.com/nodejs/node/pull/14197
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
This fixes a regression from 53c88fa411 so that special arguments
can once again be passed to the node executable when running tests.
PR-URL: https://github.com/nodejs/node/pull/13384
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Async wrap providers tested:
- crypto.randomBytes
- crypto.pbkdf2
- fs event wrap
- fsreqwrap access
- fsreqwrap readFile
- getaddrinforeq wrap
- getnameinforeq wrap
- pipe connect wrap
- query wrap
- pipewrap
- processwrap
- shutdown wrap
- tcpwrap
- udpwrap
- send wrap
- detailed signal wrap
- statwatcher
- timerwrap via setTimeout
- timerwrap via setInterval
- for Immediate
- http parser request
- http parser response
- connection via ssl server
- tls wrap
- write wrap
- ttywrap via readstream
- ttywrap via wriream
- zctx via zlib binding deflate
Embedder API:
- async-event tests
- one test looks at the happy paths
- another ensures that in cases of events emitted in an order that
doesn't make sense, the order is enforced by async hooks throwing a
meaningful error
- embedder enforcement tests are split up since async hook stack
corruption now the process
- therefore we launch a child and check for error output of the offending code
Additional tests:
- tests that show that we can enable/disable hooks inside their lifetime
events
- tests that verify the graph of resources triggering the creation of
other resources
Test Helpers:
- init-hooks:
- returns one collector instance
- when created an async hook is created and the lifetime events are
registered to call the appropriate collector functions
- the collector also exposes `enable` and `disable` functions which call
through to the async hook
- hook checks:
- checks invocations of life time hooks against the actual invocations
that were collected
- in some cases like `destroy` a min/max range of invocations can be
supplied since in these cases the exact number is non-deterministic
- verify graph:
- verifies the triggerIds of specific async resources are as expected,
i.e. the creation of resources was triggered by the resource we expect
- includes a printGraph function to generate easily readable test
input for verify graph
- both functions prune TickObjects to create less brittle and easier
to understand tests
PR-URL: https://github.com/nodejs/node/pull/12892
Ref: https://github.com/nodejs/node/pull/11883
Ref: https://github.com/nodejs/node/pull/8531
Reviewed-By: Andreas Madsen <amwebdk@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
PR-URL: https://github.com/nodejs/node/pull/12806
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
look for the actual produced `exe` not just the directory
PR-URL: https://github.com/nodejs/node/pull/12120
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>
Add support for abi stable module API (N-API) as "Experimental feature".
The goal of this API is to provide a stable Node API for native
module developers. N-API aims to provide ABI compatibility guarantees
across different Node versions and also across different
Node VMs - allowing N-API enabled native modules to just work
across different versions and flavors of Node.js without recompilation.
A more detailed introduction is provided in:
https://github.com/nodejs/node-eps/blob/master/005-ABI-Stable-Module-API.md
and https://github.com/nodejs/abi-stable-node/blob/doc/VM%20Summit.pdf.
The feature, during its experimental state, will be guarded by a runtime
flag "--napi-modules". Only when this flag is added to the command line
will N-API modules along with regular non N-API modules be supported.
The API is defined by the methods in "src/node_api.h" and
"src/node_api_types.h". This is the best
starting point to review the API surface. More documentation will follow.
In addition to the implementation of the API using V8, which is included
in this PR, the API has also been validated against chakracore and that
port is available in
https://github.com/nodejs/abi-stable-node/tree/api-prototype-chakracore-8.x.
The current plan is to provide N-API support in versions 8.X and 6.X
directly. For older versions, such as 4.X or pre N-API versions of 6.X,
we plan to create an external npm module to provide a migration path
that will allow modules targeting older Node.js versions to use the API,
albeit without getting the advantage of not having to recompile.
In addition, we also plan an external npm package with C++ sugar to
simplify the use of the API. The sugar will be in-line only and will
only use the exported N-API methods but is not part of the N-API
itself. The current version is in:
https://github.com/nodejs/node-api.
This PR is a result of work in the abi-stable-node repo:
https://github.com/nodejs/abi-stable-node/tree/doc,
with this PR being the cumulative work on the api-prototype-8.x
branch with the following contributors in alphabetical order:
Author: Arunesh Chandra <arunesh.chandra@microsoft.com>
Author: Gabriel Schulhof <gabriel.schulhof@intel.com>
Author: Hitesh Kanwathirtha <hiteshk@microsoft.com>
Author: Ian Halliday <ianhall@microsoft.com>
Author: Jason Ginchereau <jasongin@microsoft.com>
Author: Michael Dawson <michael_dawson@ca.ibm.com>
Author: Sampson Gao <sampsong@ca.ibm.com>
Author: Taylor Woll <taylor.woll@microsoft.com>
PR-URL: https://github.com/nodejs/node/pull/11975
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Currently when node is build --without-ssl and the test are run,
there are a number of failing test due to tests expecting crypto
support to be available. This commit fixes fixes the failure and
instead skips the tests that expect crypto to be available.
PR-URL: https://github.com/nodejs/node/pull/11631
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Unset NODE_PATH environment variable when running tests.
On Ubuntu 16.04, some users experience test failures due to internal
libraries being installed in /usr/lib/nodejs/internal and NODE_PATH
including /usr/lib/nodejs. Tests that expect internal libraries to be
off limits without the --expose-internals flag will fail in this
situation. Currently, those tests are test/parallel/test-repl.js and
test/parallel/test-internal-modules.js.
This situation seems to (probably) be caused by some
not-entirely-uncommon package that gets installed.
Regardless, tests should ignore the user's NODE_PATH. (NODE_PATH is
tested in test/parallel/test-module-globalpaths-nodepath.js and
test/parallel/test-require-dot.js.)
PR-URL: https://github.com/nodejs/node/pull/11612
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Bryan English <bryan@bryanenglish.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Refs: https://twitter.com/trott/status/835729396900061184
https://github.com/nodejs/node/pull/11086 had introduced a regression
that broke command line options processing for tools/test.py.
Basically, it made tools/test.py discard the command line argument that
would be passed after `--abort-on-timeout`. For instance, when running:
```
$ python tools/test.py --abort-on-timeout path/to/some-test
```
all tests would be run because the last command line argument
(`/path/to/some-test`) would be discarded.
This change fixes this regression.
Refs: https://github.com/nodejs/node/pull/11086
PR-URL: https://github.com/nodejs/node/pull/11153
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Currently, when a process times out, it is terminated by sending it the
SIGTERM signal. Sending SIGBART instead allows the operating system to
generate a core file that can be investigated later using post-mortem
debuggers such as llnode or mdb_v8.
This can be very useful when investigating flaky tests that time out,
since in that case the failure is difficult to reproduce, and being able
to look at a core file makes a big difference.
With these changes, passing the --abort-on-timeout command line option
to tools/test.py now sends SIGABRT to processes timing out on all
platforms but Windows.
PR-URL: https://github.com/nodejs/node/pull/11086
Ref: https://github.com/nodejs/node/issues/11026
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
If a temp-dir is specified and already exists, the NODE_TEST_DIR
environment variable will never be set. This fixes that.
PR-URL: https://github.com/nodejs/node/pull/10723
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
This pr adds the ability to run `tools/test.py --path`. This means that
instead of defaulting to out/Release we can use the node version from
the path.
PR-URL: https://github.com/nodejs/node/pull/9674
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Allow test.py to run tests with a 'tests/' prefix or a '.js' postfix
PR-URL: https://github.com/nodejs/node/pull/9694
Fixes: https://github.com/nodejs/node/issues/9684
Reviewed-By: Prince John Wesley <princejohnwesley@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Produce a tap13-compatible output which makes it
simpler to parse. Output is still readable by
the jenkins tap plugin.
PR-URL: https://github.com/nodejs/node/pull/9262
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
The repeat option in test.py did not work as expected if `-j` was set to
more than one. Repeated tests running at the same time could share temp
directories and cause test failures. This was observed with:
tools/test.py -J --repeat=10 parallel/test-fs-watch-recursive
By using copy.deepCopy(), the repeated tests are separate objects and
not references to the same objects. Setting `thread_id` on one of them
will now not change the `thread_id` on all of them. And `thread_id` is
how the temp directory (and common.PORT as well) are determined.
Refs: https://github.com/nodejs/node/pull/9228
PR-URL: https://github.com/nodejs/node/pull/9249
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
This test executes a simple debug session over the inspector protocol.
PR-URL: https://github.com/nodejs/node/pull/8429
Reviewed-By: ofrobots - Ali Ijaz Sheikh <ofrobots@google.com>
On every platform but `Windows`. Also, print the crash information when
using the tap reporter.
PR-URL: https://github.com/nodejs/node/pull/7859
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
As the `no-build` and `build-only` options are not used anymore, they
can be safely removed.
PR-URL: https://github.com/nodejs/node/pull/7620
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: https://github.com/nodejs/node/pull/7483
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
PR-URL: https://github.com/nodejs/node/pull/7216
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
This reverts commit d413378e51.
PR-URL: https://github.com/nodejs/node/pull/7216
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
The test script (tools/test.py) logs duration as "duration_ms: x.y".
This is confusing (as the duration is measured in seconds).
New example output: duration: 0.212s
PR-URL: https://github.com/nodejs/node/pull/7133
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Claudio Rodriguez <cjrodr@yahoo.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Commit 084b2ec ("test: include component in tap output") introduced
an in hindsight glaringly obvious but fortunately not very critical
Windows-specific bug by failing to take the path separator into account.
This commit rectifies that, the prefix is now correctly stripped.
PR-URL: https://github.com/nodejs/node/pull/6915
Refs: https://github.com/nodejs/node/pull/6653
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This makes it so you can see why the check fails if it does.
Typically that sort of thing can happen if you are modifying
bootstrapping or `process`.
PR-URL: https://github.com/nodejs/node/pull/6786
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Print test name as (for example) "parallel/test-assert". Tests that are
scraped from the addons documentation are all named test.js, making it
hard to decipher what test is running when only the filename is printed.
Fixes: https://github.com/nodejs/node/issues/6651
PR-URL: https://github.com/nodejs/node/pull/6653
Reviewed-By: James M Snell <jasnell@gmail.com>
I often want to run a test many times to see if a failure
can be recreated and I believe this is a common
use case. We even have this job in the CI
https://ci.nodejs.org/job/node-stress-single-test/configure
but often you want to run it on a specific machine.
This patch adds the --repeat option so that
you can repeat the selected set of tests a
number of times. Given existing options
in test.py this will allow you to run
one or more tests for the number of
repeats specified. For example:
tools/test.py -j8 --repeat 1000 parallel/test-process-exec-argv
runs the test-process-exec-argv test 1000 times,
running 8 copies in parallel
tools/test.py --repeat 2
would run the entire test suite twice.
PR-URL: https://github.com/nodejs/node/pull/6700
Reviewed-By: Ben Noorhduis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Reviewed-By: thefourtheye - Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
* Test the toHTML function in html.js. Check that given valid markdown
it produces the expected html. One test case will prevent regressions
of #5873.
* Check that when given valid markdown toJSON produces valid JSON with
the expected schema.
* Add doctool to the list of built in tests so it runs in CI.
PR-URL: https://github.com/nodejs/node/pull/6031
Fixes: https://github.com/nodejs/node/issues/5955
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Rich Trott <rtrott@gmail.com>
The value is retrieved from `process.arch` in node itself.
PR-URL: https://github.com/nodejs/node/pull/5997
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Changes to Node core in order to allow compilation for linuxOne.
The ../archs/linux32-s390x/opensslconf.h and
../archs/linux64-s390x/opensslconf.h were automatically
generated by running make linux-ppc linux-ppc64 in the
deps/openssl/config directory as per our standard
practice
After these changes we still need a version of v8
which supports linuxOne but that will be coming soon
in the 5.1 version of v8. Until then with these changes
we'll be able to create a hybrid build which pulls in
v8 from the http://github/andrewlow repo.
PR-URL: https://github.com/nodejs/node/pull/5941
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit adds a known_issues directory to the test directory
for scripts that reproduce known bugs. Since these scripts are
expected to fail, it also adds a --expect-fail flag to test.py
which reports tests as successful when they fail.
Refs: https://github.com/nodejs/testing/issues/18
PR-URL: https://github.com/nodejs/node/pull/5528
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Wyatt Preul <wpreul@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
In some virtualized environments the amount of available resources
are misleading; for instance `multiprocessing.cpu_count()` on our current
4-core smartos vm's returns `48`. This is not a bug, merely how the vm
host provides information about available hardware.
Avoid running into issues by overriding `cpu_count()` with `JOBS`.
PR-URL: https://github.com/nodejs/node/pull/4495
Reviewed-By: Brian White <mscdex@mscdex.net>
In CI we previously passed `NODE_COMMON_PIPE` to the test runner to
avoid long filenames. Add an option to the test runner that allows the
user to change the temporary directory instead. This also allows us to
run test suites in parallel since `NODE_COMMON_PIPE` otherwise would
have been used from multiple tests at the same time.
PR-URL: https://github.com/nodejs/node/pull/3325
Reviewed-By: Joao Reis <reis@janeasystems.com>
Tests on SmartOS are sometimes retried due to a SmartOS issue on CI.
When this happens, a TAP diagnostic message is written.
PR-URL: https://github.com/nodejs/node/pull/3960
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Be slightly more verbose in cases where tests time out.
PR-URL: https://github.com/nodejs/node/pull/2647
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
By default the logfile is opened in append mode. This commit makes sure
that the file is opened in write-binary mode, so that the file will be
created if it doesn't exist or overwrite if it exists.
Fixes: #2834
PR-URL: #2837
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <orangemocha@nodejs.org>
As it is, the TAP result shows an extra hyphen in front of test names.
Sample:
ci.nodejs.org/job/node-test-commit-osx/nodes=osx1010/454/tapResults/
This patch removes the extra hyphen.
PR-URL: https://github.com/nodejs/node/pull/2718
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Make the test runner return a 0 exit code when only
flaky tests fail and --flaky-tests=dontcare is specified.
Ported from a9b642cf5b
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Ported from df3a2b2cf2
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
This commit replaces instances of io.js with Node.js, based on the
recent convergence. There are some remaining instances of io.js,
related to build and the installer.
Fixes: https://github.com/nodejs/node/issues/2361
PR-URL: https://github.com/nodejs/node/pull/2367
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
With this option, tests can be divided in groups and only one is run
from each group. This enables us to parallelize the tests across
multiple machines.
PR-URL: https://github.com/nodejs/io.js/pull/2260
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <alexis@janeasystems.com>
These are the core changes that allow pLinux BE/LE compile. They
don't include all of the changes needed for AIX which will follow
once we have pLinux up and running in the builds
PR-URL: https://github.com/nodejs/io.js/pull/2124
Reviewed-By: Ben Noordhuis <ben@strongloop.com>
Reviewed-By: Johan Bergstrom <bugs@bergstroem.nu>
In the TAP protocol, skips are flagged as ok. Expose more
information so we can understand if the test was skipped or not.
PR-URL: https://github.com/nodejs/io.js/pull/2130
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit introduces platform-specific test timeouts for the ARM
architectures. ARMv6 is notoriously slow so gets very large timeouts on
both the timeout value for each test, as well as certain problematic
individual tests. ARMv7 and ARMv8 also get slightly increased headroom.
PR-URL: https://github.com/iojs/io.js/pull/1366
Fixes: https://github.com/iojs/io.js/issues/1343
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The ARM buildbots are notoriously slow. Update the test runner to
double the per-test time limit when it's running on one of them.
PR-URL: https://github.com/iojs/io.js/pull/1357
Reviewed-By: Roman Reiss <me@silverwind.io>
This is a minimal effort to support test output written both to
stdout and file in order to get our buildbots understanding
test output.
PR-URL: https://github.com/iojs/io.js/pull/934
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
test: add test/addons to default test list
`make test-all` and `python tools/test.py` will now also run the addon
tests in test/addons.
test: remove test-npm from test-all make target
The test-npm target builds documentation, changes versioned files,
clutters the current working directory with artifacts, etc. In short,
it doesn't seem quite ready for inclusion in `make test-all`.
test: add test-ci target, reduced test-all
Add a test-ci target that is like test-all minus the (slow) pummel and
gc test suites.
This is primarily intended for the continuous integration, where we want
decent coverage but don't want to wait for ages for tests to complete.
cpplint: add -license/copyright to default filters
Commit 3e1b1dd ("Remove excessive copyright/license boilerplate") trips
up the copyright boilerplate style check. Disable it.
PR-URL: https://github.com/iojs/io.js/pull/314
Reviewed-By: Rod Vagg <rod@vagg.org>
Now that parallel tests are enabled, the test runner spits out a ton of
'access denied' errors while running the tests. These happen because a
virus scanner or the indexing service temporarily open the file after it
has been updated, and the test runner tries to unlink() them at the same
time.
This patch resolves this issue by attempting to unlink the file until it
succeeds.
PR-URL: https://github.com/iojs/io.js/pull/284
Reviewed-by: Ben Noordhuis <info@bnoordhuis.nl>
* rename the build targets
* update the test runner to use `out/{Debug,Release}/iojs`
* update the installer to install the iojs binary
* update one test that explicitly checks for the binary name
PR-URL: https://github.com/iojs/io.js/pull/262
Reviewed-By: Bert Belder <bertbelder@gmail.com>
Anna Henningsen
Thomas Chung
Rich Trott
cclauss
Gabriel Schulhof
Ruben Bridgewater
Daniel Bevenius
Joyee Cheung
Sakthipriyan Vairamani (thefourtheye)
Refael Ackermann
Jon Moss
Matheus Marchini
Benjamin Coe
Richard Lau
Ashok
Nikhil Komawar
Gibson Fahnestock
Timothy Gu
Bradley Farias
Brian White
Thorsten Lorenz
Jason Ginchereau
Julien Gilli
Gibson Fahnestock
George Adams
Francis Gulotta
Johan Bergström
Eugene Ostroukhov
Santiago Gimeno
Rod Vagg
Jeremiah Senkpiel
Ben Noordhuis
Michael Dawson
Ian Kronquist
cjihrig
Stefan Budeanu
Karl Skomski
Alexis Campailla
João Reis
Roman Reiss
Bert Belder
Chris Dickinson