Also factor out common parts in querystring and url.
PR-URL: https://github.com/nodejs/node/pull/11161
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Removes the non-standard options on WHATWG URL toString
and extends the existing url.format() API to support
customizable serialization of the WHATWG URL object.
This does not yet include the documentation updates
because the documentation for the new WHATWG URL object
has not yet landed.
Example:
```js
const url = require('url');
const URL = url.URL;
const myURL = new URL('http://example.org/?a=b#c');
const str = url.format(myURL, {fragment: false, search: false});
console.log(str);
// Prints: http://example.org/
```
PR-URL: https://github.com/nodejs/node/pull/10857
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Timothy Gu <timothygu99@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Move non-standard methods to `url` module instead of exposing as
static methods on the `URL` object.
PR-URL: https://github.com/nodejs/node/pull/10512
Reviewed-By: Italo A. Casas <me@italoacasas.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Currently, around line 417 lib/url.js is truncating hostname and put the
rest of hostname to the path if hostname length after `.` is equal or
more than 63. This behavior is different from browser behavior. I
changed the code so that it doesn’t truncate.
I also added the test example which has more than 63 length in after
`.` in hostname in test url.
PR-URL: https://github.com/nodejs/node/pull/9292
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
This is to be consistent with the other operators and helps
understanding the context when the code is grepped.
PR-URL: https://github.com/nodejs/node/pull/10213
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Teddy Katz <teddy.katz@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add a hostPattern variable for readable purposes
PR-URL: https://github.com/nodejs/node/pull/9653
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
- rename variables in autoEscapeStr so they are easier to understand
- comment the escaping algorithm
- increase coverage for autoEscapeStr
PR-URL: https://github.com/nodejs/node/pull/10083
Reviewed-By: Anna Henningsen <anna@addaleax.net>
The `/` character does not need to be escaped when occurring inside a
character class in a regular expression. Remove such instances of
escaping in the code base.
PR-URL: https://github.com/nodejs/node/pull/9591
Reviewed-By: Teddy Katz <teddy.katz@gmail.com>
The `/` character does not need to be escaped when occurring inside a
character class in a regular expression. Remove such instances of
escaping in the code base.
PR-URL: https://github.com/nodejs/node/pull/9485
Reviewed-By: Prince John Wesley <princejohnwesley@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Teddy Katz <teddy.katz@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Implements WHATWG URL support. Example:
```
var u = new url.URL('http://example.org');
```
Currently passing all WHATWG url parsing tests and all but two of the
setter tests. The two setter tests are intentionally skipped for now
but will be revisited.
PR-URL: https://github.com/nodejs/node/pull/7448
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
Fixes an error where a loop, used to traverse an array of length `n`,
ran `n + 1` times instead of `n`.
PR-URL: https://github.com/nodejs/node/pull/8420
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
ICU has a punycode implementation built in. Use it instead of the
javascript implementation because it's much faster.
PR-URL: https://github.com/nodejs/node/pull/7355
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
`file:` URLs that do not start with `file://` are invalid. Browsers
convert `file:/etc/passwd` to `file:///etc/passwd`. This is also what
the docs indicate we are doing, but we're not.
PR-URL: https://github.com/nodejs/node/pull/7234
Fixes: https://github.com/nodejs/node/issues/3361
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Brian White <mscdex@mscdex.net>
This makes things consistent with the way that the querystring
module creates parsed results.
PR-URL: https://github.com/nodejs/node/pull/6289
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Minwoo Jung <jmwsoft@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reorder slashed protocols so they are grouped by protocol name. This is
done so it doesn't look like we're duplicating protocol names at the
bottom of the list.
PR-URL: https://github.com/nodejs/node/pull/5380
Reviewed-By: Evan Lucas <evanlucas@me.com>
The deopt issues arose from the use of const in specific situations
that v8 does not fully support yet.
Fixes: https://github.com/nodejs/node/issues/5299
PR-URL: https://github.com/nodejs/node/pull/5300
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
This commit improves url.parse() performance by 50-210%
with the existing url/url-parse benchmarks. Also, the
optimizations made in url.format() result in a 40%
increase in performance for url.resolve().
Some optimization strategies used in this commit include:
* Combining multiple searches on the same string into a
single loop
* Avoiding unnecessary string.split() and array.join()
* Minimizing creation of temporary strings
* Using a faster alternative to encodeURIComponent,
borrowed from the querystring module
PR-URL: https://github.com/nodejs/node/pull/4892
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ryan Graham <r.m.graham@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Also changes some `var`s to `const` as they never change.
PR-URL: https://github.com/nodejs/node/pull/4867
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
With an indentation style of two spaces, it is not possible to indent
multiline variable declarations by four spaces. Instead, the var keyword
is used on every new line.
Use const instead of var where applicable for changed lines.
PR-URL: https://github.com/nodejs/io.js/pull/2286
Reviewed-By: Roman Reiss <me@silverwind.io>
PR-URL: https://github.com/nodejs/node/pull/4390
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This commit fixes some error messages that are not consistent with
some general rules which most of the error messages follow.
PR-URL: https://github.com/nodejs/node/pull/3374
Reviewed-By: Roman Reiss <me@silverwind.io>
This reverts commit 3fd7fc429c.
It was agreed that this change contained too much potential ecosystem
breakage, particularly around the inability to `delete` properties off a
`Url` object. It may be re-introduced for a later release, along with
better work on ecosystem compatibility.
PR-URL: https://github.com/iojs/io.js/pull/1602
Reviewed-By: Mikeal Rogers <mikeal.rogers@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Forrest L Norvell <forrest@npmjs.com>
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Isaac Z. Schlueter <i@izs.me>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
This reverts commit dbdd81a91b.
It was agreed that this change contained too much potential ecosystem
breakage, particularly around the inability to `delete` properties off a
`Url` object. It may be re-introduced for a later release, along with
better work on ecosystem compatibility.
PR-URL: #1602
Reviewed-By: Mikeal Rogers <mikeal.rogers@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Forrest L Norvell <forrest@npmjs.com>
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Isaac Z. Schlueter <i@izs.me>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
This reverts commit 66877216bd.
It was agreed that this change contained too much potential ecosystem
breakage, particularly around the inability to `delete` properties off a
`Url` object. It may be re-introduced for a later release, along with
better work on ecosystem compatibility.
PR-URL: #1602
Reviewed-By: Mikeal Rogers <mikeal.rogers@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Forrest L Norvell <forrest@npmjs.com>
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Isaac Z. Schlueter <i@izs.me>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
In addition to null, undefined and the empty string
are treated as empty (removing the component from the url).
The string '#' is treated same as empty values when
setting .hash.
The string '?' is treated same as empty values when
setting .search.
PR-URL: https://github.com/iojs/io.js/pull/1589
Fixes: https://github.com/iojs/io.js/issues/1588
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
When resolving a reference URL with the 'file' scheme an no host
against a base URL without the 'file' scheme, the first path element
of the reference URL is used as the host for the target URL. This
results in an invalid target URL.
This change makes an exception for file URLs so that the host is not
mangled during URL resolution.
PR-URL: https://github.com/iojs/io.js/pull/1277
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Petka Antonov <petka_antonov@hotmail.com>
`'use strict'` changes the behavior for `Function.prototype.call` when
the context is `undefined`. In earlier versions of node the value
`undefined` would make `url.format` look for fields in the global scope.
The docs states that `url.format` takes a parsed URL object and returns
a formatted URL string. So with this change it will now throw for other
values.
The exception is if the input is a string. Then it will call `url.parse`
on the string and then format it. The reason for that is that you can
call `url.format` on strings to clean up potentially wonky urls.
Fixes: https://github.com/iojs/io.js/issues/1033
PR-URL: https://github.com/iojs/io.js/pull/1036
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Julian Duque <julianduquej@gmail.com>
'.' and '..' are directory specs and resolving urls with or
without the hostname with '.' and '..' should add a trailing
slash to the end of the url.
Fixes: https://github.com/joyent/node/issues/8992
PR-URL: https://github.com/iojs/io.js/pull/278
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Many of the util.is*() methods used to check data types
simply compare against a single value or the result of
typeof. This commit replaces calls to these methods with
equivalent checks. This commit does not touch calls to the
more complex methods (isRegExp(), isDate(), etc.).
Fixes: https://github.com/iojs/io.js/issues/607
PR-URL: https://github.com/iojs/io.js/pull/647
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit replaces a number of var statements throughout
the lib code with const statements.
PR-URL: https://github.com/iojs/io.js/pull/541
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The copyright and license notice is already in the LICENSE file. There
is no justifiable reason to also require that it be included in every
file, since the individual files are not individually distributed except
as part of the entire package.
Fix regression introduced in 6120472036
that broke parsing of some ssh: urls.
An example url is ssh://git@github.com:npm/npm.git
PR-URL: https://github.com/iojs/io.js/pull/299
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
I was originally going to do this after the v0.11.15 release, but as
that release is three weeks overdue now, I decided not to wait any
longer; we don't want the delta to get too big.
Conflicts:
lib/net.js
test/simple/simple.status
PR-URL: https://github.com/iojs/io.js/pull/236
Reviewed-By: Bert Belder <bertbelder@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
The url.parse() function now checks whether an escapable character is in
the URL before trying to escape it.
PR-URL: https://github.com/joyent/node/pull/8638
[trev.norris@gmail.com: Switch to use continue instead of if]
Signed-off-by: Trevor Norris <trev.norris@gmail.com>
Replace the call to Array#splice() with a faster open-coded version
that creates less garbage.
Add a new benchmark to prove it. With the change applied, it scores
about 5% higher and that is nothing to sneeze at.
PR-URL: https://github.com/iojs/io.js/pull/184
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Turn on strict mode for the files in the lib/ directory. It helps
catch bugs and can have a positive effect on performance.
PR-URL: https://github.com/node-forward/node/pull/64
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Match the behavior of the slow path by setting url.query to an empty
object when the url contains no query, but query parsing is requested.
Also add a test for this case, and update the documents to clearly
reflect this behavior.
Fixes: https://github.com/joyent/node/issues/8332
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
There is no need to split the host by hand in `url.js` – Punycode.js
takes care of it anyway. This not only simplifies the code, but also
adds support for RFC 3490 separators (i.e. not just U+002E, but U+3002,
U+FF0E, and U+FF61 as well).
Closes#6055.
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Fixes an issue that caused the first querystring to be parsed prepending
a "?" in the first variable name on relative urls with no #fragment
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
This patch adds a fast path for parsing of simple path-only URLs, as commonly
found in HTTP requests received by a server.
Benchmark results [ms], before / after patch:
/foo/bar 0.008956 0.000418 (fast path used)
http://example.com/ 0.011426 0.011437 (normal slow path, no change)
In a simple 'ab' benchmark of a single-threaded web server, this patch
increases the request rate from around 6400 to 7400 req/s.
Reviewed-By: Fedor Indutny <fedor@indutny.com>
See https://code.google.com/p/chromium/issues/detail?id=25916
Parse URLs with backslashes the same as web browsers, by replacing all
backslashes with forward slashes, except those that occur after the
first # character.
Manual rebase of 9520ade
Signed-off-by: Trevor Norris <trev.norris@gmail.com>
When using url.parse(), path and pathname usually return '/' when there
is no path available. However when you have a protocol that contains
non-lowercase letters and the input string does not have a trailing
slash, both path and pathname will be undefined.
In cases where there are multiple @-chars in a url, Node currently
parses the hostname and auth sections differently than web browsers.
This part of the bug is serious, and should be landed in v0.10, and
also ported to v0.8, and releases made as soon as possible.
The less serious issue is that there are many other sorts of malformed
urls which Node either accepts when it should reject, or interprets
differently than web browsers. For example, `http://a.com*foo` is
interpreted by Node like `http://a.com/*foo` when web browsers treat
this as `http://a.com%3Bfoo/`.
In general, *only* the `hostEndingChars` should be the characters that
delimit the host portion of the URL. Most of the current `nonHostChars`
that appear in the hostname should be escaped, but some of them (such as
`;` and `%` when it does not introduce a hex pair) should raise an
error.
We need to have a broader discussion about whether it's best to throw in
these cases, and potentially break extant programs, or return an object
that has every field set to `null` so that any attempt to read the
hostname/auth/etc. will appear to be empty.
`url.format` should escape ? and # chars in pathname, and # chars in
search, because they change the semantics of the operation otherwise.
Don't escape % chars, or anything else. (see: #4082)
added a .path property = .pathname + .search for use with http.request
And tests to verify everything.
With the tests, I changed over to deepEqual, and I would note the comment on the test
['.//g', 'f:/a', 'f://g'], which I think is a fundamental problem
This supersedes pull 1596
The file:// protocol *always* has a hostname; it's frequently
abbreviated as an empty string, which represents 'localhost'
implicitly.
According to RFC 1738 (http://tools.ietf.org/html/rfc1738):
A file URL takes the form:
file://<host>/<path>
where <host> is the fully qualified domain name of the system on
which the <path> is accessible...
As a special case, <host> can be the string "localhost" or the empty
string; this is interpreted as 'the machine from which the URL is
being interpreted'.
The change for #954 introduced a regression that would cause
the url parser to fail on special chars found in the auth
segment. Fix that, and also don't create invalid urls when
format() is called on an object containing an auth member
containing '@' characters or delimiters.
1. Allow single-quotes in urls, but escape them.
2. Add comments about which RFCs we're following for guidance.
3. Handle any invalid character in the hostname portion.
4. lcase protocol and hostname portions, since they are
case-insensitive.
This does 3 things:
1. Delimiters and "unwise" characters are never included in the
hostname or path.
2. url.format will sanitize string URLs that are passed to it.
3. The parsed url's 'href' member will be the sanitized url, which may
not match the argument to url.parse.
Timothy Gu
DavidCai
James M Snell
Junshu Okamoto
Michaël Zasso
Axel Monroy
joyeecheung
Prince J Wesley
Rich Trott
Luigi Pinca
Ilkka Myller
Alex Kocharin
Brian White
nettofarah
Kári Tristan Helgason
Michaël Zasso
Nik Nyby
micnic
Yosuke Furukawa
Rod Vagg
Petka Antonov
Jeffrey Jagoda
Christian Tellnes
Amir Saboury
cjihrig
isaacs
Ben Noordhuis
Evan Lucas
CGavrila
Jonathan Johnson
Yazhong Liu
Trevor Norris
Evan Rutledge Borden
Gabriel Wicke
Timothy J Fontaine
Majid Arif Siddiqui
Mathias Bynens
Ezequiel Rabinovich
Yuki KAN
Shuan Wang
J. Lee Coltrane
Łukasz Walukiewicz
Jordan Sissel
seebees
Maciej Małecki
Robert Mustacchi
Jeremy Selier
Ryan Petrello
Ryan Dahl