Commit Graph

315 Commits

Author SHA1 Message Date
Matthew Soulanille 51577688f1
Move nightly publishing tests and release branch tests to github actions (#8555)
[Example nightly release and publish test](https://github.com/mattsoulanille/tfjs/actions/runs/14672995746/job/41183624273)

[Example release PR test](https://github.com/mattsoulanille/tfjs/actions/runs/14674243426/job/41187438555?pr=1)
2025-04-28 09:07:00 -07:00
Ping Yu c465681739
nightly test fix (#8214)
* testing nightly fix

* disable mobilenet test

* fix converter and e2e tests
2024-04-06 00:56:02 -07:00
Linchenn 00831deee1
Update iOS and OSX versions for testing (#8023)
* update versions

* upd

* upd

* upd

* Update karma.conf.js
2023-10-23 14:56:32 -07:00
Ping Yu 906cbe30b2
support string dtype input generation (#8003)
FEATURE
2023-10-13 15:13:45 -07:00
Matthew Soulanille ac9519e537
Show meaningful error message in benchmark tool (#7990) 2023-10-03 13:27:29 -07:00
Ping Yu 89a0b8f103
[converter] fixed py import for auto tracking (#7982)
* update py import for auto tracking

* [e2e] Use ios12 and report karma seed (#7984)

* [e2e] Use ios12 and report karma seed

* Update karma.conf.js

---------

Co-authored-by: Matthew Soulanille <msoulanille@google.com>
2023-10-03 11:01:24 -07:00
Matthew Soulanille 0bd03f7e58
Re-enable nightly firefox tests (#7954)
* Re-enable nightly firefox tests

* Use ios 15 webgl2 for layers tests

---------

Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2023-09-12 17:50:57 -07:00
Matthew Soulanille d500a0d039
Disable firefox browserstack tests temporarily (#7935)
* Disable firefox browserstack tests temporarily

* Remove unused WebGL scripts

* Update test-ci.sh

* Always use port 9876 for karma

* Always use port 9200 for karma

Browserstack supports ports 9200-9400 for safari
testing (https://www.browserstack.com/question/39572), and when karma launches
and its port is in use, it automatically increments the port. Starting at 9200
ensures the port is always supported by BrowserStack.

* Fix bash npm-run-all script

* Fix tfjs-vis test script

* Update karma.conf.js

* Update automl macos test version

---------

Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2023-08-25 17:05:49 -07:00
Matthew Soulanille 983dc16a4e
Update TypeScript to 5.0.4 (#7681) 2023-07-21 14:55:05 -07:00
dependabot[bot] 4a113009f2
build(deps): bump semver in /e2e/custom_module/blazeface (#7821)
Bumps [semver](https://github.com/npm/node-semver) from 6.3.0 to 6.3.1.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/v6.3.1/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v6.3.0...v6.3.1)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-14 13:16:01 -07:00
Linchenn 22196bb7ab
[Local benchmark tool] Add parallel compile option (#7755)
FEATURE

* add parallel compile benchmark

* Update index.html
2023-06-27 00:32:45 +00:00
wrighkv1 2cbc737417
Add npmVersion command line argument to benchmark NPM versions (#7674)
* npmVersion CLIarg

* add readme description

---------

Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
2023-05-30 20:56:51 +00:00
Matthew Soulanille de94eec199
Fix tfjs-release not updating all tfjs versions of subpackages (#7550)
Some TFJS packages, like wasm, have examples or demos in them. These usually depend on the parent package, but the parent package is not marked as to be updated when updating the subpackage dependency versions. For an example of this, see #7547.

Update the TFJS dependencies of these subpackages to the release version if they are `link:` dependencies.
2023-05-17 18:50:25 +00:00
Matthew Soulanille 089efca029
Fix Verdaccio nightly e2e tests (#7644)
Replace e2e's custom local publishing logic with the publish-npm script. This more accurately represents how we release TFJS.

Fix publish-npm not waiting for Verdaccio to start before pushing packages.

Avoid testing WebGPU on platforms other than Chrome MacOS (for now).

---------

Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
2023-05-15 14:03:03 -07:00
Matthew Soulanille 825b04ba06
Remove duplicate e2e test (#7682)
The removed test is already present on line 29.
2023-05-11 15:48:31 -07:00
Matthew Soulanille 03b436933e
Fix golden integration tests not awaiting results (#7676)
Golden e2e model tests were not waiting for the asynchronous `tensor.data()` function to finish when checking results, making it hard to tell which test is failing. This PR fixes that.

After this fix, golden model tests are consistently timing out on BrowserStack Mac
Chrome. They work locally when tested on an M1 mac. Disable them on BrowserStack
for now.
2023-05-11 12:50:48 -07:00
Matthew Soulanille f62dd9d999
Disable e2e WebGPU golden model tests (#7677)
WebGPU golden model tests are freezing on BrowserStack for unknown reasons (they work locally). Disable them for now to unblock presubmits.
2023-05-10 16:54:57 -07:00
Xu Xing cb9a98b90f
[e2e] Enable webgpu intergration test (#7543) 2023-05-06 14:18:27 +08:00
Linchenn 3f01eb5e17
Fix local benchmark (#7608)
FIX
2023-04-24 09:56:59 -07:00
Xu Xing 8d40c4ffc5
[e2e] Support set env flag from url (#7582)
FEATURE

For most tfjs app, URL parameter like tfjsflags=WEBGPU_PRINT_SHADER:binary
will be recognized as setting tfjs environment flag value (If it has been
registered). For e2e, this logic has been broken due to setEnvFlags will
override values for all existing flags.

This change supports setting environment flag values from URL parameters
in e2e, example: tfjsflags=WEBGPU_PRINT_SHADER:binary. When a flag is
set in both GUI(tunable flag) and URL parameter, URL parameter will
override flag from GUI.

Co-authored-by: Yang Gu <yang.gu@intel.com>
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
2023-04-21 20:14:06 -07:00
Xu Xing 3832a961a3
[e2e] Remove pose-detection (#7518)
BUG

* [e2e] Remove pose-detection

* Nit

---------

Co-authored-by: Yang Gu <yang.gu@intel.com>
Co-authored-by: Jiajia Qin <jiajia.qin@intel.com>
2023-04-18 19:46:06 -07:00
Linchenn 237a99104e
[Local benchmark] Fix run button (#7575)
BUG

* Fix run button

* Update index.html
2023-04-11 17:36:03 -07:00
Linchenn 8e4723ef95
Update dat.gui to lil.gui for local benchmark tool (#7538) 2023-04-10 18:48:31 -07:00
Tirumalesh 77e8e53fc4
Fixed the broken link in README.md (#7551)
Fixed the broken link for `tfjs/e2e/benchmarks/model_config.js`  in line 152
2023-04-06 22:37:50 -07:00
Matthew Soulanille a17c0b7312
Fix release script not updating all package.jsons (#7542)
release-tfjs updated only the root package.json of each package, but e2e tests
have some other package.json files that must be updated as well. Fix this by
updating all package.json files in each package so that they use the npm release
versions instead of `link:` dependencies.
2023-04-04 16:52:58 -07:00
Matthew Soulanille 430bc13045
Further parallelize e2e ci tests (#7533)
Apply the following optimizations:
* Parallelize python model conversions.
* Parallelize custom module bundling.
* Parallelize test-ci.
2023-03-31 11:24:13 -07:00
chunnienc ca7ce8f1e9
[e2e] Ignore golden tests in some nightly tests (#7532)
* Add golden tag

* Remove a test

* Fix tags

* fix
2023-03-30 13:30:12 -07:00
Matthew Soulanille 10f44523bc
Parallelize e2e browserstack tests (#7531)
Use GNU parallel to run browserstack tests in parallel.
2023-03-30 11:51:22 -07:00
chunnienc ed5dd8001a
Add golden data fetch command (#7526) 2023-03-29 10:14:07 -07:00
Matthew Soulanille 95a2873035
Fix golden file test indexing nonexistent tensor (#7527)
Use `.toEqual(jasmine.anything())` instead of `.not.toEqual(null)` to catch both
`null` and `undefined`.

Use `model.outputs.map(output => output.name)` instead of `model.outputNodes` to
get the output node name without the numerical index.
2023-03-28 15:52:04 -07:00
chunnienc 9923395b76
Add e2e golden model tests (#7520)
* Add script to build golden model data

* Add graph model predict test

* Add intermediate tensor tests

* Fix

* Add script

* Add comments

* Remove unused deps

* Update e2e/integration_tests/graph_model_golden_tests.ts

Co-authored-by: Matthew Soulanille <matthew@soulanille.net>

* Update e2e/integration_tests/graph_model_golden_tests.ts

Co-authored-by: Matthew Soulanille <matthew@soulanille.net>

* Add check

* Fix

* Add script to fetch data

* Add cloudbuild step

* Fix

* Fix cloudbuild tests

* Remove parallel

* Update names

* Fix bucket name

* Add null checks

* Fix null checks in test

* Update fetch script

---------

Co-authored-by: Matthew Soulanille <matthew@soulanille.net>
2023-03-27 19:20:42 -07:00
Matthew Soulanille 8e0b81af6b
Use verdaccio on 127.0.0.1 instead of localhost to avoid ipv6 (#7491)
Fix an intermittent bug in the npm publishing script that causes the script to depend on the network configuration of the host. I think what's happening is that when ipv6 is available, sometimes Verdaccio binds [::1]:5783 instead of 127.0.0.1:5783, but npm tries to publish to 127.0.0.1:5783. Explicitly specifying the loopback address 127.0.0.1 fixes this.
2023-03-20 12:27:43 -07:00
dependabot[bot] 231c23dfaa
build(deps-dev): bump webpack in /e2e/custom_module/blazeface (#7481)
SECURITY
Bumps [webpack](https://github.com/webpack/webpack) from 5.9.0 to 5.76.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.9.0...v5.76.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-15 09:50:19 -07:00
dependabot[bot] 90a44706d9
build(deps-dev): bump webpack (#7478)
SECURITY
Bumps [webpack](https://github.com/webpack/webpack) from 5.70.0 to 5.76.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.70.0...v5.76.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-15 09:50:02 -07:00
dependabot[bot] 7011910a90
build(deps-dev): bump webpack in /e2e/custom_module/dense_model (#7477)
SECURITY
Bumps [webpack](https://github.com/webpack/webpack) from 5.70.0 to 5.76.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.70.0...v5.76.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-15 09:49:44 -07:00
dependabot[bot] a40cbc8e1f
build(deps-dev): bump webpack from 5.46.0 to 5.76.0 in /e2e/webpack_test (#7475)
SECURITY
Bumps [webpack](https://github.com/webpack/webpack) from 5.46.0 to 5.76.0.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Commits](https://github.com/webpack/webpack/compare/v5.46.0...v5.76.0)

---
updated-dependencies:
- dependency-name: webpack
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
2023-03-15 09:49:18 -07:00
Wanming Lin 252bf326ac
[e2e] Move tflite backend to worker thread (#7453)
This PR moves all operations of tflite backend to worker thread
in order to make preparation for WebNN external delegate integration
because the WebNN spec restricts its Sync API in the worker thread.
Besides, this PR also fixes some minor issues.
2023-03-14 10:45:06 -07:00
dependabot[bot] ddb202aa4c
build(deps): bump ua-parser-js in /e2e/benchmarks/browserstack-benchmark (#7317)
Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.31 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](https://github.com/faisalman/ua-parser-js/compare/0.7.31...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2023-01-27 13:05:36 -08:00
dependabot[bot] a38b5a10c9
build(deps): bump ua-parser-js from 0.7.31 to 0.7.33 in /e2e (#7314)
Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.31 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](https://github.com/faisalman/ua-parser-js/compare/0.7.31...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-27 12:46:07 -08:00
Linchenn 4efeb24702
rm (#7305)
FEATURE
2023-01-26 10:35:51 -08:00
Matthew Soulanille 0a32b672ef
Improve npm publishing infra (#7287)
The current npm publishing infra is more manual than necessary. This PR makes the following improvements:

1. Publish all packages in a release unit with one command. Publishing no longer involves running yarn publish-npm multiple times.

2. Stage all packages to a local verdaccio repository before publishing. This catches any compatibility issues that may appear when publishing to npm and allows us to build all packages before publishing any to npm.

3. Publish with a single OTP. The previous infra required an OTP per package to publish to npm. This new infra publishes all packages sequentially after they're built, so it can use a single OTP for all of them. If another OTP is required due to the first one timing out, it asks for another.

A future PR will move our nightly verdaccio e2e tests to use this infra, which will allow us to avoid setting the npm registry globally. This will let us run verdaccio tests in parallel with the rest of nightly tests, which should reduce the time taken.
2023-01-23 16:27:41 -08:00
dependabot[bot] a9f687249b
build(deps): bump json5 from 2.2.1 to 2.2.3 in /e2e (#7264)
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-11 13:15:38 -08:00
Linchenn 09535adfe5
Add averageTimeExclFirst to benchmark_util.timeInference (#7231)
FEATURE
2023-01-04 13:33:52 -08:00
dependabot[bot] eb04b2c04c
build(deps): bump json5 in /e2e/custom_module/universal_sentence_encoder (#7224)
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 10:34:18 -08:00
dependabot[bot] 8a2258f0ef
build(deps): bump json5 in /e2e/custom_module/blazeface (#7236)
SECURITY
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 10:33:54 -08:00
dependabot[bot] 0653851a69
build(deps): bump json5 in /e2e/custom_module/dense_model (#7235)
Bumps [json5](https://github.com/json5/json5) from 2.2.1 to 2.2.3.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v2.2.1...v2.2.3)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 10:33:36 -08:00
Linchenn 4ac5658806
Support batch-benchmark on local device (#7228)
FEATURE
2023-01-03 16:41:26 -08:00
dependabot[bot] 5b0ddc19b1
build(deps): bump fast-json-patch (#7210)
Bumps [fast-json-patch](https://github.com/Starcounter-Jack/JSON-Patch) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/Starcounter-Jack/JSON-Patch/releases)
- [Commits](https://github.com/Starcounter-Jack/JSON-Patch/compare/v3.1.0...3.1.1)

---
updated-dependencies:
- dependency-name: fast-json-patch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-03 09:19:36 -08:00
Linchenn f9d46b6343
Update date for nightly benchmark (#7184)
BUG
2022-12-19 23:15:32 -08:00
Linchenn b40c641664
Increase cycle from week to half-month for nightly benchmark (#7173)
FEATURE

* upd

* enable default date

* fix
2022-12-15 14:25:33 -08:00