The term "native module" dates back to some of the oldest code
in the code base. Within the context of Node.js core it usually
refers to modules that are native to Node.js (e.g. fs, http),
but it can cause confusion for people who don't work on this
part of the code base, as "native module" can also refer to
native addons - which is even the case in some of the API
docs and error messages.
This patch tries to make the usage of these terms more consistent.
Now within the context of Node.js core:
- JavaScript scripts that are built-in to Node.js are now referred
to as "built-in(s)". If they are available as modules,
they can also be referred to as "built-in module(s)".
- Dynamically-linked shared objects that are loaded into
the Node.js processes are referred to as "addons".
We will try to avoid using the term "native modules" because it could
be ambiguous.
Changes in this patch:
File names:
- node_native_module.h -> node_builtins.h,
- node_native_module.cc -> node_builtins.cc
C++ binding names:
- `native_module` -> `builtins`
`node::Environment`:
- `native_modules_without_cache` -> `builtins_without_cache`
- `native_modules_with_cache` -> `builtins_with_cache`
- `native_modules_in_snapshot` -> `builtins_in_cache`
- `native_module_require` -> `builtin_module_require`
`node::EnvSerializeInfo`:
- `native_modules` -> `builtins
`node::native_module::NativeModuleLoader`:
- `native_module` namespace -> `builtins` namespace
- `NativeModuleLoader` -> `BuiltinLoader`
- `NativeModuleRecordMap` -> `BuiltinSourceMap`
- `NativeModuleCacheMap` -> `BuiltinCodeCacheMap`
- `ModuleIds` -> `BuiltinIds`
- `ModuleCategories` -> `BuiltinCategories`
- `LoadBuiltinModuleSource` -> `LoadBuiltinSource`
`loader.js`:
- `NativeModule` -> `BuiltinModule` (the `NativeModule` name used in
`process.moduleLoadList` is kept for compatibility)
And other clarifications in the documentation and comments.
PR-URL: https://github.com/nodejs/node/pull/44135
Fixes: https://github.com/nodejs/node/issues/44036
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Now that we include the code cache into the embedded snapshot,
there is no point in splitting an Environment-independent
NativeModuleLoader out of NativeModuleEnv. Merge the two
classes for simplicity.
PR-URL: https://github.com/nodejs/node/pull/43824
Refs: https://github.com/nodejs/node/issues/31074
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
There are no clear indicators anyone is using the dtrace USDT probes.
ETW support is very intertwined with the dtrace infrastructure. It's not
clear if anyone uses ETW so to keep things simple it too is removed.
Fixes: https://github.com/nodejs/node/issues/43649
PR-URL: https://github.com/nodejs/node/pull/43652
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
This commit adds the setting of an appname (configuration section
name), 'nodejs_conf', to be used when reading OpenSSL configuration
files.
The motivation for this is that currently the default OpenSSL
configuration, 'openssl_conf', element will be used which may be
undesirable as it might configure OpenSSL in unwanted ways. With this
commit it is still possible to use a default openssl.cnf file but the
only section that Node.js will read from is a section named
'nodejs_conf'.
PR-URL: https://github.com/nodejs/node/pull/43124
Refs: https://github.com/nodejs/node/issues/40366
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Since V8 code cache encodes indices to the read-only space
it is safer to make sure that the code cache is generated in the
same heap used to generate the embdded snapshot. This patch
merges the code cache builder into the snapshot builder and
makes the code cache part of node::SnapshotData that is
deserialized into the native module loader during bootstrap.
PR-URL: https://github.com/nodejs/node/pull/43023
Fixes: https://github.com/nodejs/node/issues/31074
Refs: https://github.com/nodejs/node/issues/35711
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Since all its uses are now gone, it's time to say goodbye to
AllocatedBuffer.
Refs: https://github.com/nodejs/node/pull/39941
Signed-off-by: Darshan Sen <raisinten@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/43008
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.
Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:
* A number of functions used executables such as `mksnapshot` are
not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
claiming to export a number of functions that are actually in
`libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
generated causing native extensions to try to link against
`node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
native extensions don't know to look for `libnode.lib` rather
than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
`node` in the same folder. This works fine from the
`out/Release` folder but not from an installed prefix, where
`node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
setting correctly for `bin/node` to find `lib/libnode.so`.
For the `libnode.lib` vs `node.lib` issue there are two possible
options:
1. Ensure `node.lib` from `node.exe` does not get generated, and
instead copy `libnode.lib` to `node.lib`. This means addons
compiled when referencing the correct `node.lib` file will
correctly depend on `libnode.dll`. The down side is that
native addons compiled with stock Node.js will still try to
resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
and process this to generate a `node.def` file to be linked into
`node.exe` with the `/DEF:node.def` flag. The export entries
in `node.def` will all read
```
my_symbol=libnode.my_symbol
```
so that `node.exe` will redirect all exported symbols back to
`libnode.dll`. This has the benefit that addons compiled with
stock Node.js will load correctly into `node.exe` from a shared
library build, but means that every embedding executable also
needs to perform this same trick.
I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.
For example
```
>npx node-gyp configure --nodedir
C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\Release\binding.node
File Type: DLL
Image has the following dependencies:
KERNEL32.dll
libnode.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
...
```
PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
So that the embedded snapshot can be reused by the worker.
PR-URL: https://github.com/nodejs/node/pull/42702
Refs: https://github.com/nodejs/node/issues/35711
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
This adds a --build-snapshot runtime option which is currently only
supported by the node_mksnapshot binary, and a --node-snapshot-main
configure option that makes use it to run a custom script when
building the embedded snapshot. The idea is to have this experimental
feature in core as a configure-time feature for now, and investigate
the renaming V8 bugs before we make it available to more users via
making it a runtime option.
PR-URL: https://github.com/nodejs/node/pull/42466
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
Reviewed-By: Khaidi Chu <i@2333.moe>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
I noticed that we were taking `TimerCb` as a `const&` and then copying
that into the member. This is completely fine when the constructor is
called with an lvalue. However, when called with an rvalue, we can allow
the `std::function` to be moved into the member instead of falling back
to a copy, so I changed the constructors to take in universal
references. Also, `std::function` constructors can take in multiple
arguments, so I further modified the constructors to use variadic
templates.
Signed-off-by: Darshan Sen <darshan.sen@postman.com>
PR-URL: https://github.com/nodejs/node/pull/40665
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Currently using the --openssl-is-fips configuration option in
combination with --ninja is broken.
This commit fixes two issues, one being an issue with the linker/version
script path variable. The second is that the locations of built
artifacts that differ for ninja and make.
ninja:
$ ./configure --openssl-is-fips --ninja
$ ninja -C out/Release
$ ./node --enable-fips -p 'crypto.getFips()'
1
make:
$ ./configure --openssl-is-fips
$ make -j8
$ ./node --enable-fips -p 'crypto.getFips()'
1
PR-URL: https://github.com/nodejs/node/pull/40518
Refs: https://github.com/nodejs/node/issues/40509
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
This pull request updates the OpenSSL version that is statically
linked with Node.js from OpenSSl 1.1.1 to quictls OpenSSL 3.0.0+quic.
This pull request will replace the OpenSSL version that is currently
in the deps directory and when performing a normal build
OpenSSL 3.0+quic will be statically linked to the Node.js executable.
We will still be able to dynamically link to OpenSSL 1.1.1 and we have
a CI job which dynamically links to OpenSSL 1.1.1 which is run for
every pull request to make sure that we maintain backward compatibility.
PR-URL: https://github.com/nodejs/node/pull/38512
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
man ld -export_dynamic:
```
Preserves all global symbols in main executables during LTO.
Without this option, Link Time Optimization is allowed to inline
and remove global functions.
This option is used when a main executable may load a plug-in which
requires certain symbols from the main executable.
```
Bug: vercel/pkg#1155
Signed-off-by: Jesse Chan <jc@linux.com>
PR-URL: https://github.com/nodejs/node/pull/39839
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
GYP uses the system path when parsing node.gyp;
However, if system python is different from our
gyp runtime python, like '2.7', gyp would crash.
Co-authored-by: Michaël Zasso <targos@protonmail.com>
PR-URL: https://github.com/nodejs/node/pull/39293
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
We can remove the Acorn plugins as their features are now supported
by default.
PR-URL: https://github.com/nodejs/node/pull/39166
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Guy Bedford <guybedford@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jiawen Geng <technicalcute@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
On Windows there is a limit to the length of commands, so there
will be an error once the lengths of the JS file names combined
exceed that limit. This patch modifies js2c.py so that
it now takes a --directory argument to glob for .js and
.mjs files in addition to the list of files passed directly.
We still pass the additional files we include from deps/
directly through the command line, as we only includes some of
them so we cannot simply glob, but those are limited so listing
them out should be fine.
Refs: https://docs.microsoft.com/en-us/troubleshoot/windows-client/shell-experience/command-line-string-limitation
PR-URL: https://github.com/nodejs/node/pull/39069
Refs: https://github.com/nodejs/node/pull/38971
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
When I moved these files from node-inspect to Node.js core, I put them
in lib/internal/inspector. That was a mistake. They should be in
lib/internal/debugger.
PR-URL: https://github.com/nodejs/node/pull/39080
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
The purpose of separating is for readability and maintainability.
PR-URL: https://github.com/nodejs/node/pull/38988
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
This patch:
- Moves the snapshot building code to src/ so that we can reuse it
later when generating custom snapshots from an entry point accepted
by the node binary.
- Create a SnapshotData struct that incorporates all the data useful
for a snapshot blob, including both the V8 data and the Node.js
data.
PR-URL: https://github.com/nodejs/node/pull/38902
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/38758
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Adds a new `makeTransferable()` utility that can construct a
`JSTransferable` object that does not directly extend the
`JSTransferable` JavaScript class.
Because JavaScript does not support multiple inheritance, it is
not possible (without help) to implement a class that extends
both `JSTransferable` and, for instance, `EventTarget` without
incurring a significant additional complexity and performance
cost by making all `EventTarget` instances extend `JSTransferable`...
That is, we *don't* want:
```js
class EventTarget extends JSTransferable { ... }
```
The `makeTransferable()` allows us to create objects that are
backed internally by `JSTransferable` without having to actually
extend it by leveraging the magic of `Reflect.construct()`.
```js
const {
JSTransferable,
kClone,
kDeserialize,
kConstructor,
makeTransferable,
} = require('internal/worker/js_transferable');
class E {
constructor(b) {
this.b = b;
}
}
class F extends E {
[kClone]() { /** ... **/ }
[kDeserialize]() { /** ... **/ }
static [kConstructor]() { return makeTransferable(F); }
}
const f = makeTransferable(F, 1);
f instanceof F; // true
f instanceof E; // true
f instanceof JSTransferable; // false
const mc = new MessageChannel();
mc.port1.onmessage = ({ data }) => {
data instanceof F; // true
data instanceof E; // true
data instanceof JSTransferable; // false
};
mc.port2.postMessage(f); // works!
```
The additional `internal/test/transfer.js` file is required for the
test because successfully deserializing transferable classes requires
that they be located in `lib/internal` for now.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/38383
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Khaidi Chu <i@2333.moe>
node-inspect developers have agreed to move node-inspect into core
rather than vendor it as a dependency.
Refs: https://github.com/nodejs/node/discussions/36481
PR-URL: https://github.com/nodejs/node/pull/38161
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
PR-URL: https://github.com/nodejs/node/pull/37792
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Create a utils module for isIterable(), isReadable(), and isStream().
PR-URL: https://github.com/nodejs/node/pull/37508
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
* Update the user timing implementation to conform to
User Timing Level 3.
* Reimplement user timing and timerify with pure JavaScript
implementations
* Simplify the C++ implementation for gc and http2 perf
* Runtime deprecate additional perf entry properties
in favor of the standard detail argument
* Disable the `buffered` option on PerformanceObserver,
all entries are queued and dispatched on setImmediate.
Only entries with active observers are buffered.
* This does remove the user timing and timerify
trace events. Because the trace_events are still
considered experimental, those can be removed without
a deprecation cycle. They are removed to improve
performance and reduce complexity.
Old: `perf_hooks/usertiming.js n=100000: 92,378.01249733355`
New: perf_hooks/usertiming.js n=100000: 270,393.5280638482`
PR-URL: https://github.com/nodejs/node/pull/37136
Refs: https://github.com/nodejs/diagnostics/issues/464
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
1. Put the v8 binding data class into a header so we can reuse
the class definition during deserialization.
2. Put the v8 binding code into node::v8_utils namespace for
clarity.
3. Move the binding data property initialization into its
constructor so that we can reuse it during deserialization
4. Reorder the v8 binding initialization so that we don't
unnecessarily initialize the properties in a loop
PR-URL: https://github.com/nodejs/node/pull/37112
Refs: https://github.com/nodejs/node/pull/36943
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/37048
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
The `Blob` object is an immutable data buffer. This is a first step
towards alignment with the `Blob` Web API.
Signed-off-by: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/36811
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Using an iterable WeakMap (a data-structure that uses WeakRef and
WeakMap), we are able to: stop relying on Module._cache to
serialize source maps; stop requiring an error object when calling
findSourceMap().
PR-URL: https://github.com/nodejs/node/pull/35915
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Joyee Cheung
Ben Noordhuis
Brian White
Tobias Nießen
Daniel Bevenius
Darshan Sen
William Marlow
legendecas
Michael Dawson
Michaël Zasso
Darshan Sen
Jesse Chan
himself65
Rich Trott
XadillaX
Antoine du Hamel
James M Snell
Anna Henningsen
Jiawen Geng
Thiago Padilha
Benjamin Gruenbaum
bcoe