For detecting whether `require(esm)` is supported without triggering
the experimental warning.
PR-URL: https://github.com/nodejs/node/pull/55241
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/54563
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Before the fix, the pseudocode was describing an infinite loop.
PR-URL: https://github.com/nodejs/node/pull/55117
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.
There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.
PR-URL: https://github.com/nodejs/node/pull/55085
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
This patch implements a "module-sync" exports condition
for packages to supply a sycnrhonous ES module to the
Node.js module loader, no matter it's being required
or imported. This is similar to the "module" condition
that bundlers have been using to support `require(esm)`
in Node.js, and allows dual-package authors to opt into
ESM-first only newer versions of Node.js that supports
require(esm) while avoiding the dual-package hazard.
```json
{
"type": "module",
"exports": {
"node": {
// On new version of Node.js, both require() and import get
// the ESM version
"module-sync": "./index.js",
// On older version of Node.js, where "module" and
// require(esm) are not supported, use the transpiled CJS version
// to avoid dual-package hazard. Library authors can decide
// to drop support for older versions of Node.js when they think
// it's time.
"default": "./dist/index.cjs"
},
// On any other environment, use the ESM version.
"default": "./index.js"
}
}
```
We end up implementing a condition with a different name
instead of reusing "module", because existing code in the
ecosystem using the "module" condition sometimes also expect
the module resolution for these ESM files to work in CJS
style, which is supported by bundlers, but the native
Node.js loader has intentionally made ESM resolution
different from CJS resolution (e.g. forbidding `import
'./noext'` or `import './directory'`), so it would be
semver-major to implement a `"module"` condition
without implementing the forbidden ESM resolution rules.
For now, this just implments a new condition as semver-minor
so it can be backported to older LTS.
Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages
PR-URL: https://github.com/nodejs/node/pull/54648
Fixes: https://github.com/nodejs/node/issues/52173
Refs: https://github.com/joyeecheung/test-module-condition
Refs: https://github.com/nodejs/node/issues/52697
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/54488
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Ulises Gascón <ulisesgascongonzalez@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Jake Yuesong Li <jake.yuesong@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/53619
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Tooling in the ecosystem have been using the __esModule property to
recognize transpiled ESM in consuming code. For example, a 'log'
package written in ESM:
export function log(val) { console.log(val); }
Can be transpiled as:
exports.__esModule = true;
exports.default = function log(val) { console.log(val); }
The consuming code may be written like this in ESM:
import log from 'log'
Which gets transpiled to:
const _mod = require('log');
const log = _mod.__esModule ? _mod.default : _mod;
So to allow transpiled consuming code to recognize require()'d real ESM
as ESM and pick up the default exports, we add a __esModule property by
building a source text module facade for any module that has a default
export and add .__esModule = true to the exports. We don't do this to
modules that don't have default exports to avoid the unnecessary
overhead. This maintains the enumerability of the re-exported names
and the live binding of the exports.
The source of the facade is defined as a constant per-isolate property
required_module_facade_source_string, which looks like this
export * from 'original';
export { default } from 'original';
export const __esModule = true;
And the 'original' module request is always resolved by
createRequiredModuleFacade() to wrap which is a ModuleWrap wrapping
over the original module.
PR-URL: https://github.com/nodejs/node/pull/52166
Refs: https://github.com/nodejs/node/issues/52134
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
This patch:
1. Adds ESM syntax detection to compileFunctionForCJSLoader()
for --experimental-detect-module and allow it to emit the
warning for how to load ESM when it's used to parse ESM as
CJS but detection is not enabled.
2. Moves the ESM detection of --experimental-detect-module for
the entrypoint from executeUserEntryPoint() into
Module.prototype._compile() and handle it directly in the
CJS loader so that the errors thrown during compilation *and
execution* during the loading of the entrypoint does not
need to be bubbled all the way up. If the entrypoint doesn't
parse as CJS, and detection is enabled, the CJS loader will
re-load the entrypoint as ESM on the spot asynchronously using
runEntryPointWithESMLoader() and cascadedLoader.import(). This
is fine for the entrypoint because unlike require(ESM) we don't
the namespace of the entrypoint synchronously, and can just
ignore the returned value. In this case process.mainModule is
reset to undefined as they are not available for ESM entrypoints.
3. Supports --experimental-detect-module for require(esm).
PR-URL: https://github.com/nodejs/node/pull/52047
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
This patch adds `require()` support for synchronous ESM graphs under
the flag `--experimental-require-module`
This is based on the the following design aspect of ESM:
- The resolution can be synchronous (up to the host)
- The evaluation of a synchronous graph (without top-level await) is
also synchronous, and, by the time the module graph is instantiated
(before evaluation starts), this is is already known.
If `--experimental-require-module` is enabled, and the ECMAScript
module being loaded by `require()` meets the following requirements:
- Explicitly marked as an ES module with a `"type": "module"` field in
the closest package.json or a `.mjs` extension.
- Fully synchronous (contains no top-level `await`).
`require()` will load the requested module as an ES Module, and return
the module name space object. In this case it is similar to dynamic
`import()` but is run synchronously and returns the name space object
directly.
```mjs
// point.mjs
export function distance(a, b) {
return (b.x - a.x) ** 2 + (b.y - a.y) ** 2;
}
class Point {
constructor(x, y) { this.x = x; this.y = y; }
}
export default Point;
```
```cjs
const required = require('./point.mjs');
// [Module: null prototype] {
// default: [class Point],
// distance: [Function: distance]
// }
console.log(required);
(async () => {
const imported = await import('./point.mjs');
console.log(imported === required); // true
})();
```
If the module being `require()`'d contains top-level `await`, or the
module graph it `import`s contains top-level `await`,
[`ERR_REQUIRE_ASYNC_MODULE`][] will be thrown. In this case, users
should load the asynchronous module using `import()`.
If `--experimental-print-required-tla` is enabled, instead of throwing
`ERR_REQUIRE_ASYNC_MODULE` before evaluation, Node.js will evaluate the
module, try to locate the top-level awaits, and print their location to
help users fix them.
PR-URL: https://github.com/nodejs/node/pull/51977
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
https://github.com/nodejs/node/pull/44004#discussion_r1067599238
PR-URL: https://github.com/nodejs/node/pull/46178
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Harshitha K P <harshitha014@gmail.com>
Return type of PACKAGE_EXPORTS_RESOLVE was changed when subpath folder
mappings reached EOL. But RESOLVE_ESM_MATCH was not updated.
Refs: https://github.com/nodejs/node/pull/40121
PR-URL: https://github.com/nodejs/node/pull/45280
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Some core modules can be loaded with or without the `node:` prefix.
Using the prefix disambiguates which specifiers refer to core modules.
This commit updates the docs to use the prefix everywhere a core module
is referenced.
PR-URL: https://github.com/nodejs/node/pull/42752
Fixes: https://github.com/nodejs/node/issues/38343
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Mestery <mestery@protonmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Akhil Marsonya <akhil.marsonya27@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
The ellipsis in `All together` is not needed. I've had it on my list of
"small things to take care of when you get a chance" for a while. The
Microsoft Style Guide says to not use ellipses except in certain cases,
and that reminded me to do it. The day has arrived!
PR-URL: https://github.com/nodejs/node/pull/41888
Reviewed-By: Derek Lewis <DerekNonGeneric@inf.is>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Mestery <mestery@protonmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/41728
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Mestery <mestery@protonmail.com>
Update mdast-util-to-markdown to 1.2.4 which reduces unnecessary
escaping of `_` and some other characters. Re-run markdown formatter.
PR-URL: https://github.com/nodejs/node/pull/40645
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
PR-URL: https://github.com/nodejs/node/pull/40403
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Zijian Liu <lxxyxzj@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Upcoming remark-based formatting/linting will add a blank line between
comments in markdown. This is in preparation for that change.
PR-URL: https://github.com/nodejs/node/pull/40160
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Main changes:
- Replace current HTML anchor generation to match
header anchor generation in Github markdown.
- Remove unnecessary double namespacing on generated anchors/links (E.G.
`esm.md#loaders` instead of `esm.md#esm_loaders`).
- Anchors/links are automatically prefixed with their respective modules
when concatenated for usage in `all.html`.
Benefits:
- All anchor links within and between markdown API docs actually work.
- Adding new anchor links no longer requires contributors to generate
the HTML docs first to look up the correct anchors.
- Anchors are much shorter.
- All previous anchor links are preserved by generating hidden legacy
anchors.
PR-URL: https://github.com/nodejs/node/pull/39304
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Notable Changes:
Diagnostics channel (experimental module):
`diagnostics_channel` is a new experimental module that provides an API
to create named channels to report arbitrary message data for
diagnostics purposes.
The module was initially introduced in Node.js v15.1.0 and is
backported to v14.17.0 to enable testing it at a larger scale.
With `diagnostics_channel`, Node.js core and module authors can publish
contextual data about what they are doing at a given time. This could
be the hostname and query string of a mysql query, for example. Just
create a named channel with `dc.channel(name)` and call
`channel.publish(data)` to send the data to any listeners to that
channel.
```js
const dc = require('diagnostics_channel');
const channel = dc.channel('mysql.query');
MySQL.prototype.query = function query(queryString, values, callback) {
// Broadcast query information whenever a query is made
channel.publish({
query: queryString,
host: this.hostname,
});
this.doQuery(queryString, values, callback);
};
```
Channels are like one big global event emitter but are split into
separate objects to ensure they get the best performance. If nothing is
listening to the channel, the publishing overhead should be as close to
zero as possible. Consuming channel data is as easy as using
`channel.subscribe(listener)` to run a function whenever a message is
published to that channel.
```js
const dc = require('diagnostics_channel');
const channel = dc.channel('mysql.query');
channel.subscribe(({ query, host }) => {
console.log(`mysql query to ${host}: ${query}`);
});
```
The data captured can be used to provide context for what an app is
doing at a given time. This can be used for things like augmenting
tracing data, tracking network and filesystem activity, logging
queries, and many other things. It's also a very useful data source
for diagnostics tools to provide a clearer picture of exactly what the
application is doing at a given point in the data they are presenting.
Contributed by Stephen Belanger (https://github.com/nodejs/node/pull/34895).
UUID support in the crypto module:
The new `crypto.randomUUID()` method now allows to generate random
[RFC 4122](https://www.rfc-editor.org/rfc/rfc4122.txt) Version 4
UUID strings:
```js
const { randomUUID } = require('crypto');
console.log(randomUUID());
// 'aa7c91a1-f8fc-4339-b9db-f93fc7233429'
```
Contributed by James M Snell (https://github.com/nodejs/node/pull/36729).
Experimental support for `AbortController` and `AbortSignal`:
Node.js 14.17.0 adds experimental partial support for `AbortController`
and `AbortSignal`.
Both constructors can be enabled globally using the
`--experimental-abortcontroller` flag.
Additionally, several Node.js APIs have been updated to support
`AbortSignal` for cancellation.
It is not mandatory to use the built-in constructors with them. Any
spec-compliant third-party alternatives should be compatible.
`AbortSignal` support was added to the following methods:
* `child_process.exec`
* `child_process.execFile`
* `child_process.fork`
* `child_process.spawn`
* `dgram.createSocket`
* `events.on`
* `events.once`
* `fs.readFile`
* `fs.watch`
* `fs.writeFile`
* `http.request`
* `https.request`
* `http2Session.request`
* The promisified variants of `setImmediate` and `setTimeout`
Other notable changes:
* doc:
* revoke deprecation of legacy url, change status to legacy (James M Snell) (https://github.com/nodejs/node/pull/37784)
* add legacy status to stability index (James M Snell) (https://github.com/nodejs/node/pull/37784)
* upgrade stability status of report API (Gireesh Punathil) (https://github.com/nodejs/node/pull/35654)
* deps:
* V8: Backport various patches for Apple Silicon support (BoHong Li) (https://github.com/nodejs/node/pull/38051)
* update ICU to 68.1 (Michaël Zasso) (https://github.com/nodejs/node/pull/36187)
* upgrade to libuv 1.41.0 (Colin Ihrig) (https://github.com/nodejs/node/pull/37360)
* http:
* add http.ClientRequest.getRawHeaderNames() (simov) (https://github.com/nodejs/node/pull/37660)
* report request start and end with diagnostics\_channel (Stephen Belanger) (https://github.com/nodejs/node/pull/34895)
* util:
* add getSystemErrorMap() impl (eladkeyshawn) (https://github.com/nodejs/node/pull/38101)
PR-URL: https://github.com/nodejs/node/pull/38507
Fixes: https://github.com/nodejs/node/issues/36098
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Guy Bedford <guybedford@gmail.com>
Co-authored-by: Darshan Sen <raisinten@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37246
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Linked PACKAGE_EXPORTS_RESOLVE to "Resolver Algorithm Specification" in
"ECMAScript modules" page, so that the `require` pseudocode is easier to
understand.
PR-URL: https://github.com/nodejs/node/pull/37135
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
remark-parse@9.0.0 combined with our html.js tool ends a list if a
multi-line item does not include indentation. Update our docs for this
formatting.
I looked around for a lint rule to enforce this but didn't find one
readily available. (Happy to be shown that I'm wrong about that!) We may
need to write one.
PR-URL: https://github.com/nodejs/node/pull/36049
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().
PR-URL: https://github.com/nodejs/node/pull/35915
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
RafaelGSS
Joyee Cheung
Antoine du Hamel
Guy Bedford
chirsz
shallow-beach
Geoffrey Booth
Claudio W
Rich Trott
翠 / green
Konv
Simone Busoli
Jannis R
Tobias Nießen
Tobias Hernstig
Qingyu Deng
Michaël Zasso
foxxyz
Danielle Adams
Beth Griggs
ExE Boss
Utku Gultopu
bcoe