Commit Graph

79 Commits

Author SHA1 Message Date
Matthew Soulanille d500a0d039
Disable firefox browserstack tests temporarily (#7935)
* Disable firefox browserstack tests temporarily

* Remove unused WebGL scripts

* Update test-ci.sh

* Always use port 9876 for karma

* Always use port 9200 for karma

Browserstack supports ports 9200-9400 for safari
testing (https://www.browserstack.com/question/39572), and when karma launches
and its port is in use, it automatically increments the port. Starting at 9200
ensures the port is always supported by BrowserStack.

* Fix bash npm-run-all script

* Fix tfjs-vis test script

* Update karma.conf.js

* Update automl macos test version

---------

Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2023-08-25 17:05:49 -07:00
wrighkv1 2cbc737417
Add npmVersion command line argument to benchmark NPM versions (#7674)
* npmVersion CLIarg

* add readme description

---------

Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
2023-05-30 20:56:51 +00:00
Matthew Soulanille de94eec199
Fix tfjs-release not updating all tfjs versions of subpackages (#7550)
Some TFJS packages, like wasm, have examples or demos in them. These usually depend on the parent package, but the parent package is not marked as to be updated when updating the subpackage dependency versions. For an example of this, see #7547.

Update the TFJS dependencies of these subpackages to the release version if they are `link:` dependencies.
2023-05-17 18:50:25 +00:00
Xu Xing 3832a961a3
[e2e] Remove pose-detection (#7518)
BUG

* [e2e] Remove pose-detection

* Nit

---------

Co-authored-by: Yang Gu <yang.gu@intel.com>
Co-authored-by: Jiajia Qin <jiajia.qin@intel.com>
2023-04-18 19:46:06 -07:00
Tirumalesh 77e8e53fc4
Fixed the broken link in README.md (#7551)
Fixed the broken link for `tfjs/e2e/benchmarks/model_config.js`  in line 152
2023-04-06 22:37:50 -07:00
dependabot[bot] ddb202aa4c
build(deps): bump ua-parser-js in /e2e/benchmarks/browserstack-benchmark (#7317)
Bumps [ua-parser-js](https://github.com/faisalman/ua-parser-js) from 0.7.31 to 0.7.33.
- [Release notes](https://github.com/faisalman/ua-parser-js/releases)
- [Changelog](https://github.com/faisalman/ua-parser-js/blob/master/changelog.md)
- [Commits](https://github.com/faisalman/ua-parser-js/compare/0.7.31...0.7.33)

---
updated-dependencies:
- dependency-name: ua-parser-js
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2023-01-27 13:05:36 -08:00
Linchenn 4efeb24702
rm (#7305)
FEATURE
2023-01-26 10:35:51 -08:00
Linchenn 09535adfe5
Add averageTimeExclFirst to benchmark_util.timeInference (#7231)
FEATURE
2023-01-04 13:33:52 -08:00
Linchenn 4ac5658806
Support batch-benchmark on local device (#7228)
FEATURE
2023-01-03 16:41:26 -08:00
dependabot[bot] 5b0ddc19b1
build(deps): bump fast-json-patch (#7210)
Bumps [fast-json-patch](https://github.com/Starcounter-Jack/JSON-Patch) from 3.1.0 to 3.1.1.
- [Release notes](https://github.com/Starcounter-Jack/JSON-Patch/releases)
- [Commits](https://github.com/Starcounter-Jack/JSON-Patch/compare/v3.1.0...3.1.1)

---
updated-dependencies:
- dependency-name: fast-json-patch
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-03 09:19:36 -08:00
Linchenn f9d46b6343
Update date for nightly benchmark (#7184)
BUG
2022-12-19 23:15:32 -08:00
Linchenn b40c641664
Increase cycle from week to half-month for nightly benchmark (#7173)
FEATURE

* upd

* enable default date

* fix
2022-12-15 14:25:33 -08:00
Linchenn 11835343f9
Increase timeout and add Gpu info to nightly benchmark (#7145)
FEATURE

* fix firestore

* add test

* fix
2022-12-07 17:14:48 -08:00
dependabot[bot] b426bb4f79
Bump socket.io-parser in /e2e/benchmarks/browserstack-benchmark (#7032)
Bumps [socket.io-parser](https://github.com/socketio/socket.io-parser) from 4.0.4 to 4.0.5.
- [Release notes](https://github.com/socketio/socket.io-parser/releases)
- [Changelog](https://github.com/socketio/socket.io-parser/blob/main/CHANGELOG.md)
- [Commits](https://github.com/socketio/socket.io-parser/compare/4.0.4...4.0.5)

---
updated-dependencies:
- dependency-name: socket.io-parser
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-14 10:11:09 -08:00
Linchenn 6d238b3142
[Benchmark] Add weekly scheduling for nightly benchmark (#6959)
FEATURE

* add weeklyRun

* Update app.js

* add tests

* add weeklyCycleRun

* add doc

* Update app.js
2022-10-20 13:26:42 -07:00
Linchenn 1bcdd05bb4
[Benchmark] Add GPU hardware info to the benchmark tools (#6907)
FEATURE

* add benchmark

* use webglBackend
2022-10-06 12:58:23 -07:00
Matthew Soulanille c5eb1d39bf
[e2e] Fix incorrect reference to link-package (#6895)
Some package.json files in e2e linked against link_package instead of link-package. This was not discovered earlier because these tests are usually run after the package.jsons are prepared for release and dependencies are rewritten to the release versions.
2022-10-05 09:52:39 -07:00
Linchenn ebc655ac4e
Fix nightly benchmark (#6771)
BUG

* fix

* Update cloudbuild.yml

* update bs

Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2022-08-24 14:37:49 -07:00
Linchenn d86a6dba74
add flg (#6772)
FEATURE
2022-08-19 14:43:27 -07:00
Linchenn 59af8c2342
[browserstack benchmark tool] Use stream to write to outfile (#6750)
FEATURE

* write

* Update app.js

* Update app.js
2022-08-18 11:09:49 -07:00
Linchenn 89f59aa6a7
[browserstack benchmark tool] batch benchmark code snippet (#6742)
FEATURE
* move code snippet

* Update app.js

* Update index.js
2022-08-12 13:14:39 -07:00
Linchenn 996bdf4ef4
[browserstack benchmark tool] Move CodeSnippet from backend to frontend (#6738)
* move code snippet

* merge codeSnippet

* Delete benchmark_results.html
2022-08-11 15:41:20 -07:00
Linchenn 8715f5816c
[browserstack benchmark tool] Add a html to present benchmark results (#6740)
FEATURE

* Update benchmark_results.html

* add html

* Update app.js

* upd names

* add args

Co-authored-by: Na Li <linazhao@google.com>
2022-08-11 14:48:53 -07:00
Linchenn 6ee743f2d0
Setup optimization target deices/browsers (#6722)
PERF

* update target browsers/devices

* Update browser_list.json

* Update index.js

* up
2022-08-05 16:39:42 -07:00
Linchenn f3e6c7fbd9
Fix app.js (#6727)
BUG
2022-08-05 15:01:59 -07:00
Matthew Soulanille 7ae8860dbc
Run the next BrowserStack benchmark immediately instead of in batches (#6717)
Before this change, the benchmark script would start x benchmarks, wait for them all to complete, and then start the next x. This change makes it start new benchmarks immediately when prior benchmarks complete so that there are always x benchmarks running.

This doesn't work across tests that require different karma configs (e.g. changing model / parameters), and all tests from one karma config must complete before tests with a different karma config can start.

Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
2022-08-03 16:28:55 -07:00
Linchenn 630913684f
Fix build script (#6716)
BUG
2022-08-03 14:45:19 -07:00
Linchenn 2bc408aba1
Update browser_list.json (#6713)
Adds/updates browsers/devices for the BrowserStack benchmark tool:

1. Windows: updates OS from 10 to 11 and updates browser version to the latest for all browser.
2. Mac: updates OS from Catalina to Monterey and updates browser version to the latest for all browser.
3. ios iphone: adds os_version as 15
4. ios ipad: adds os_version as 14 and 15
5. android: adds os_version as 12.0
2022-08-02 19:41:22 -07:00
Linchenn e5c138e927
[browserstack benchmark tool] Enable code snippet benchmark (#6704)
FEATURE
* benchmark code snippet

* Update benchmark_models.js

* upd

* show code snippet in tool

* typo
2022-08-02 16:09:47 -07:00
Matthew Soulanille 2dc7a7766c
Use the firebase-admin package instead of firebase to push benchmark results (#6711)
Replace the firebase package with firebase-admin. The firebase-admin package is more appropriate for our use case of running benchmarks in GCP and pushing them to Firestore. Per their docs, "The Firebase Admin Node.js SDK enables access to Firebase services from privileged environments (such as servers or cloud) in Node.js." It uses application default credentials, which are available in GCP.
2022-08-02 14:53:21 -07:00
Linchenn 5c5f446938
[browserstack benchmark tool] Enable customizing local builds (#6705)
FEATURE

* enable customizing local builds

* simplify build

* update yarn cmds
2022-08-02 11:48:22 -07:00
Linchenn bb6205704f
Fix browserstack-benchmark tool (#6703)
BUG
* fix bs

* update isTflite
2022-08-01 17:18:03 -07:00
Matthew Soulanille 2745ea9de9
[wasm] Build tfjs-backend-wasm entirely with Bazel (#6458)
Build tfjs-backend-wasm with Bazel. tfjs-backend-wasm already uses Bazel for Emscripten. This PR makes it use Bazel for TypeScript as well and replaces the build scripts with a single pkg_npm target.

Fixes #5282
2022-06-14 10:18:02 -07:00
dependabot[bot] b72b202c63
Bump protobufjs in /e2e/benchmarks/browserstack-benchmark (#6491)
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js) from 6.11.2 to 6.11.3.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases)
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/v6.11.3/CHANGELOG.md)
- [Commits](https://github.com/protobufjs/protobuf.js/compare/v6.11.2...v6.11.3)

---
updated-dependencies:
- dependency-name: protobufjs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-02 16:07:00 -07:00
dependabot[bot] cd8973e733
Bump minimist in /e2e/benchmarks/browserstack-benchmark (#6280)
SECURITY
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
- [Release notes](https://github.com/substack/minimist/releases)
- [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-31 10:41:09 -07:00
Ping Yu 33c62a005c
Security fixes (#6268)
* yarn.lock updates for security warning

* fix minimist security issue
2022-03-29 12:34:26 -07:00
Ping Yu fae0fe12fc
more security fixes (#6237)
SECURITY
2022-03-16 12:56:36 -07:00
Ping Yu e674862840
yarn.lock updates for security warning (#6229) 2022-03-14 14:44:34 -07:00
dependabot[bot] 40c41cd428
Bump karma in /e2e/benchmarks/browserstack-benchmark (#6193)
SECURITY
Bumps [karma](https://github.com/karma-runner/karma) from 6.3.14 to 6.3.16.
- [Release notes](https://github.com/karma-runner/karma/releases)
- [Changelog](https://github.com/karma-runner/karma/blob/master/CHANGELOG.md)
- [Commits](https://github.com/karma-runner/karma/compare/v6.3.14...v6.3.16)

---
updated-dependencies:
- dependency-name: karma
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-14 11:25:50 -07:00
dependabot[bot] a451c560c1
Bump follow-redirects in /e2e/benchmarks/browserstack-benchmark (#6134)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.12.1 to 1.14.8.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.12.1...v1.14.8)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
2022-02-16 08:58:32 -08:00
dependabot[bot] ef75020435
Bump karma in /e2e/benchmarks/browserstack-benchmark (#6139)
Bumps [karma](https://github.com/karma-runner/karma) from 4.4.1 to 6.3.14.
- [Release notes](https://github.com/karma-runner/karma/releases)
- [Changelog](https://github.com/karma-runner/karma/blob/master/CHANGELOG.md)
- [Commits](https://github.com/karma-runner/karma/compare/v4.4.1...v6.3.14)

---
updated-dependencies:
- dependency-name: karma
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-14 08:57:24 -08:00
Matthew Soulanille 2a7ce55363
[webgl] Build tfjs-backend-webgl with Bazel (#5562)
PROCESS
2021-09-22 10:03:32 -07:00
Claire 04633433fd
Fix GCP Timeout for BrowserStack Nightly Benchmarks (#5423)
* Ending Firestore hang in BrowserStack benchmarks

* Add catch to addResultToFirestore

* Fix NITs and return Promise.allSettled()

Co-authored-by: Na Li <linazhao@google.com>
2021-08-04 02:57:23 +00:00
Claire caa449ecf7
Add multiple device and model benchmarking capability and fix Firestore API key reference in cloudbuild (#5391)
* added benchmarking multiple backends and models

* updated documentation

* updated browser list

* fixed Firestore and GCP credentials

* NITs and db refactoring
2021-08-02 17:56:27 +00:00
Claire 6b78485fb2
Add retry benchmark capabilities and jasmine tests (#5367)
Add a command line argument to the BrowserStack benchmarker to retry benchmarks that fail.
2021-08-02 08:40:41 -07:00
Claire 9e37b37968
Add device and benchmark info to BrowserStack benchmarks (#5386)
* Spawn device and benchmark info

* Changing name of one info
2021-07-27 12:18:50 -07:00
Keith Williams 89b2fec26c
Push benchmark results to firestore (#5316)
Push BrowserStack benchmark results to firestore.
2021-07-27 09:31:23 -07:00
Claire 5c4d58cb4a
Runs preconfigured BrowserStack benchmark through GCP (#5323)
* Runs preconfiguration through GCP 3 benchmarks at a time

* Fix wording regarding cloud benchmarks

* remove redundant aspects

* karma config and maxBenchmark alterations

* remove disconnect tolerance

* Update command line args in README.md

Co-authored-by: Matthew Soulanille <msoulanille@google.com>
2021-07-20 10:12:37 -05:00
Claire 1951811016
Add limit to number of parallel BrowserStack benchmarks (#5305)
* Add limit to number of parallel benchmarks
2021-07-13 13:15:13 +00:00
Claire 2904f9faeb
BrowserStack: Add option to use CDNs instead of local dependencies (#5303)
Uses the CDN dependencies for BrowserStack benchmarks if the --webDeps argument is added as a command line argument.
2021-07-09 10:12:52 -07:00