Commit Graph

34 Commits

Author SHA1 Message Date
Livia Medeiros 9ab976397d
fs: close dir before throwing if `options.bufferSize` is invalid
PR-URL: https://github.com/nodejs/node/pull/58856
Fixes: https://github.com/nodejs/node/issues/58854
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
2025-07-06 05:32:45 +00:00
Livia Medeiros d824914e61
fs: throw `ERR_INVALID_THIS` on illegal invocations
PR-URL: https://github.com/nodejs/node/pull/58848
Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2025-06-28 16:58:54 +00:00
René ce546e468a
fs: make `Dir` disposers idempotent
PR-URL: https://github.com/nodejs/node/pull/58692
Refs: https://github.com/nodejs/node/pull/58206
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2025-06-20 19:03:34 +00:00
Livia Medeiros 823ca6991f
fs: make `processReadResult()` and `readSyncRecursive()` private
PR-URL: https://github.com/nodejs/node/pull/58672
Fixes: https://github.com/nodejs/node/issues/58671
Refs: https://github.com/nodejs/node/pull/41439
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Filip Skokan <panva.ip@gmail.com>
Reviewed-By: Ulises Gascón <ulisesgascongonzalez@gmail.com>
2025-06-14 17:23:47 +00:00
Livia Medeiros 6c9a7cd61e
lib: make ERM functions into wrappers returning undefined
PR-URL: https://github.com/nodejs/node/pull/58400
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2025-05-25 13:44:43 +00:00
Antoine du Hamel 4a86be255c
fs: add to `Dir` support for explicit resource management
PR-URL: https://github.com/nodejs/node/pull/58206
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
2025-05-11 14:53:21 +00:00
Antoine du Hamel 942862f287
fs: ensure `dir.read()` does not throw synchronously
PR-URL: https://github.com/nodejs/node/pull/58228
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2025-05-10 10:37:32 +02:00
0hm☘️🏳️‍⚧️ a8a86b3adb
lib: resolve the issue of not adhering to the specified buffer size
We create a `queueHandler`, and in every iteration we execute
the handlers in the `queueHandler` until we get a non-null result.

PR-URL: https://github.com/nodejs/node/pull/55896
Refs: https://github.com/nodejs/node/issues/55764
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: James M Snell <jasnell@gmail.com>
2025-04-26 15:38:15 +00:00
Ethan Arrowood e0ef622c54
fs: fix bufferSize option for opendir recursive
The bufferSize option was not respected in recursive mode. This PR
implements a naive solution to fix this issue until a better
implementation can be designed.

Fixes: https://github.com/nodejs/node/issues/48820
PR-URL: https://github.com/nodejs/node/pull/55744
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
2024-11-08 16:03:55 +00:00
Yagiz Nizipli 3ad2e12073
fs: move `ToNamespacedPath` dir calls to c++
PR-URL: https://github.com/nodejs/node/pull/53630
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
2024-07-11 18:51:28 +00:00
Yagiz Nizipli 307430e490
fs: improve error performance of `fs.dir`
PR-URL: https://github.com/nodejs/node/pull/53667
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2024-07-09 14:38:01 +00:00
Jungku Lee c14133503a
fs: use private fields instead of symbols for `Dir`
PR-URL: https://github.com/nodejs/node/pull/51037
Refs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_properties
Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2023-12-28 06:16:13 +00:00
Antoine du Hamel 2e458d9736
fs: introduce `dirent.parentPath`
The goal is to replace `dirent.path` using a name that's less likely to
create confusion.
`dirent.path` value has not been stable, moving it to a different
property name should avoid breaking some upgrading user expectations.

PR-URL: https://github.com/nodejs/node/pull/50976
Reviewed-By: Ethan Arrowood <ethan@arrowood.dev>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
2023-12-02 22:49:21 +00:00
Yagiz Nizipli 571ecbf992
fs: improve error performance of `opendirSync`
PR-URL: https://github.com/nodejs/node/pull/49705
Refs: https://github.com/nodejs/performance/issues/106
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
2023-09-21 17:04:13 +00:00
William Marlow d8b378cb3f
fs: fix readdir and opendir recursive with unknown file types
If the libuv operations invoked by `readdir`/`opendir` return
`uv_dirent_t` values where the `type` is `UV_DIRENT_UNKNOWN` then a
further `lstat` is issued to fully construct the `Dirent` values. In the
recursive versions of these functions, the `path` parameter was
incorrectly assumed to be the path to the entry when it should be the
path to the directory containing the entry.

Fixes #49499.

PR-URL: https://github.com/nodejs/node/pull/49603
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-09-11 19:08:08 +00:00
Ethan Arrowood 12a93ce630
fs: make readdir recursive algorithm iterative
PR-URL: https://github.com/nodejs/node/pull/47650
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-05-10 08:00:40 +00:00
Ethan Arrowood 7b39e8099a
fs: add recursive option to readdir and opendir
Adds a naive, linear recursive algorithm for the following methods:
readdir, readdirSync, opendir, opendirSync, and the promise based
equivalents.

Fixes: https://github.com/nodejs/node/issues/34992
PR-URL: https://github.com/nodejs/node/pull/41439
Refs: https://github.com/nodejs/tooling/issues/130
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
2023-04-20 17:50:27 +00:00
Antoine du Hamel 9f192530f2
fs: add trailing commas in source files
PR-URL: https://github.com/nodejs/node/pull/46696
Reviewed-By: Moshe Atlow <moshe@atlow.co.il>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: LiviaMedeiros <livia@cirno.name>
Reviewed-By: James M Snell <jasnell@gmail.com>
2023-02-20 01:58:32 +01:00
Antoine du Hamel fe514bf960
lib: enforce use of trailing commas for functions
PR-URL: https://github.com/nodejs/node/pull/46629
Reviewed-By: Jacob Smith <jacob@frende.me>
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Mohammed Keyvanzadeh <mohammadkeyvanzade94@gmail.com>
2023-02-14 18:45:16 +01:00
Antoine du Hamel 06d8606960
lib: use null-prototype objects for property descriptors
Refs: https://github.com/nodejs/node/pull/42921

PR-URL: https://github.com/nodejs/node/pull/43270
Reviewed-By: Paolo Insogna <paolo@cowtech.it>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Chengzhong Wu <legendecas@gmail.com>
Reviewed-By: Minwoo Jung <nodecorelab@gmail.com>
2022-06-03 09:23:58 +01:00
Mohammed Keyvanzadeh 8c4b8b201a
lib: replace validator and error
Refs: https://github.com/nodejs/node/pull/41660

PR-URL: https://github.com/nodejs/node/pull/41678
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Darshan Sen <raisinten@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
2022-02-05 08:36:48 -08:00
Antoine du Hamel 9e1e89000d fs: refactor to avoid unsafe array iteration
PR-URL: https://github.com/nodejs/node/pull/36699
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
2021-01-06 11:31:53 +01:00
ZiJian Liu b00bb01db9 lib: refactor to use validateCallback
PR-URL: https://github.com/nodejs/node/pull/36609
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
2020-12-29 15:48:57 +00:00
Zijian Liu b938f88204 fs: pass ERR_DIR_CLOSED asynchronously to dir.close
Pass the error to the callback or returns a rejected Promise instead of
throwing a synchonous error.

Fixes: https://github.com/nodejs/node/issues/36237

PR-URL: https://github.com/nodejs/node/pull/36243
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Yongsheng Zhang <zyszys98@gmail.com>
2020-11-27 18:43:41 +01:00
Antoine du Hamel 8d6c2f2ada fs: refactor to use more primordials
PR-URL: https://github.com/nodejs/node/pull/36196
Reviewed-By: James M Snell <jasnell@gmail.com>
2020-11-27 18:40:51 +01:00
Anna Henningsen d3a8a23089
fs: forbid concurrent operations on Dir handle
libuv does not expect concurrent operations on `uv_dir_t` instances,
and will gladly create memory leaks, corrupt data, or crash the
process.

This patch forbids that, and:

- Makes sure that concurrent async operations are run sequentially
- Throws an exception if sync operations are attempted during an
  async operation

The assumption here is that a thrown exception is preferable to
a potential hard crash.

This fully fixes flakiness from `parallel/test-fs-opendir` when
run under ASAN.

PR-URL: https://github.com/nodejs/node/pull/33274
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2020-05-15 19:36:50 +02:00
Jesus Hernandez 4076e043b8 lib: removes unnecessary params
PR-URL: https://github.com/nodejs/node/pull/32694
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gerhard Stöbich <deb2001-github@yahoo.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>
2020-04-10 16:47:21 +08:00
Sebastien Ahkrin 593240d1a1
lib: replace Symbol.asyncIterator by SymbolAsyncIterator
PR-URL: https://github.com/nodejs/node/pull/30947
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: James M Snell <jasnell@gmail.com>
2019-12-16 09:23:50 +01:00
Sebastien Ahkrin eac3f0adc4
lib: replace Symbol global by the primordials Symbol
PR-URL: https://github.com/nodejs/node/pull/30737
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
2019-12-08 13:38:58 +01:00
Michaël Zasso 0646eda4fc
lib: flatten access to primordials
Store all primordials as properties of the primordials object.
Static functions are prefixed by the constructor's name and prototype
methods are prefixed by the constructor's name followed by "Prototype".
For example: primordials.Object.keys becomes primordials.ObjectKeys.

PR-URL: https://github.com/nodejs/node/pull/30610
Refs: https://github.com/nodejs/node/issues/29766
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>
2019-11-25 10:28:15 +01:00
Anna Henningsen b35181f877
fs: add `bufferSize` option to `fs.opendir()`
Add an option that controls the size of the internal
buffer.

Fixes: https://github.com/nodejs/node/issues/29941

PR-URL: https://github.com/nodejs/node/pull/30114
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
2019-10-28 14:34:25 +01:00
Anna Henningsen 5c93aab278
fs: buffer dir entries in opendir()
Read up to 32 directory entries in one batch when `dir.readSync()`
or `dir.read()` are called.

This increases performance significantly, although it introduces
quite a bit of edge case complexity.

                                                                 confidence improvement accuracy (*)    (**)    (***)
     fs/bench-opendir.js mode='async' dir='lib' n=100                  ***    155.93 %      ±30.05% ±40.34%  ±53.21%
     fs/bench-opendir.js mode='async' dir='test/parallel' n=100        ***    479.65 %      ±56.81% ±76.47% ±101.32%
     fs/bench-opendir.js mode='sync' dir='lib' n=100                           10.38 %      ±14.39% ±19.16%  ±24.96%
     fs/bench-opendir.js mode='sync' dir='test/parallel' n=100         ***     63.13 %      ±12.84% ±17.18%  ±22.58%

PR-URL: https://github.com/nodejs/node/pull/29893
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
2019-10-11 23:09:39 +02:00
Jeremiah Senkpiel 7f22aaf62b fs: remove options.encoding from Dir.read*()
This is unlikely to be necessary in any case, and causes much
unwarrented complexity when implementing further
optimizations.

Refs: https://github.com/nodejs/node/pull/29893#discussion_r333179482

PR-URL: https://github.com/nodejs/node/pull/29908
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
2019-10-09 15:05:40 -07:00
Jeremiah Senkpiel cbd8d715b2 fs: introduce `opendir()` and `fs.Dir`
This adds long-requested methods for asynchronously interacting and
iterating through directory entries by using `uv_fs_opendir`,
`uv_fs_readdir`, and `uv_fs_closedir`.

`fs.opendir()` and friends return an `fs.Dir`, which contains methods
for doing reads and cleanup. `fs.Dir` also has the async iterator
symbol exposed.

The `read()` method and friends only return `fs.Dirent`s for this API.
Having a entry type or doing a `stat` call is deemed to be necessary in
the majority of cases, so just returning dirents seems like the logical
choice for a new api.

Reading when there are no more entries returns `null` instead of a
dirent. However the async iterator hides that (and does automatic
cleanup).

The code lives in separate files from the rest of fs, this is done
partially to prevent over-pollution of those (already very large)
files, but also in the case of js allows loading into `fsPromises`.

Due to async_hooks, this introduces a new handle type of `DIRHANDLE`.

This PR does not attempt to make complete optimization of
this feature. Notable future improvements include:
- Moving promise work into C++ land like FileHandle.
- Possibly adding `readv()` to do multi-entry directory reads.
- Aliasing `fs.readdir` to `fs.scandir` and doing a deprecation.

Refs: https://github.com/nodejs/node-v0.x-archive/issues/388
Refs: https://github.com/nodejs/node/issues/583
Refs: https://github.com/libuv/libuv/pull/2057

PR-URL: https://github.com/nodejs/node/pull/29349
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: David Carlier <devnexen@gmail.com>
2019-10-08 10:34:48 -07:00