Move GCP presubmits to Github actions. This allows us to run WebGL / WebGPU tests, since Github Actions have a MacOS runner with a GPU. This should unblock several PRs.
* Disable firefox browserstack tests temporarily
* Remove unused WebGL scripts
* Update test-ci.sh
* Always use port 9876 for karma
* Always use port 9200 for karma
Browserstack supports ports 9200-9400 for safari
testing (https://www.browserstack.com/question/39572), and when karma launches
and its port is in use, it automatically increments the port. Starting at 9200
ensures the port is always supported by BrowserStack.
* Fix bash npm-run-all script
* Fix tfjs-vis test script
* Update karma.conf.js
* Update automl macos test version
---------
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
Test the WebGPU backend on BrowserStack using MacOS. Windows machines appear to
be VMs, and WebGPU can not get context on them. MacOS devices seem to be real, so they work.
tfjs-tfdf and tfjs-tflite both depend on pre-compiled WASM files. These WASM files are downloaded by scripts (i.e. curl) during the build step. This violates Bazel's hermeticity requirements because build steps should not access remote resources. This PR fixes this by downloading these WASM files using http_archive in the WORKSPACE file.
Fixes#7103
Avoid opening a bunch of browser pages when running karma chrome tests where possible. This excludes WebGPU on Linux because headless webgpu is not supported on linux yet.
Add the flag `--//:headless=true|false` (default true) to control this and set it to false on Windows.
Enable testing the WebGPU backend on Linux using Chrome Dev (105) instead of Chrome Canary (even though base: 'ChromeCanary' is listed, it launches Chrome Dev on Linux). Fix incorrect passing of flags to Chrome. Flags enabled in chrome://flags need to appear between --flag-switches-begin and --flag-switches-end.
Note that not all tests are passing on Linux. This could be due to Chrome Dev being behind Chrome Canary, or due to a lack of testing on Linux before this PR.
Update iOS from 11 to 12.3.
Remove #/usr/bin/env node from test_async_backends.ts
Disable iOS worker tests, which need to load absolute URLs (unsupported in BrowserStack right now).
Fixes#6598
Update jasmine from 3.1.0 to 4.2.1 for Bazel packages. Update karma from 6.3.16 to 6.4.0.
Remove deprecated use of async function and done callback at the same time in Jasmine tests.
Add karma-jasmine-html-reporter to show test status in the browser window.
Add an experimental remote build execution config to run highly parallel builds on remote machines. Currently, only Linux is supported. Adding --config=rbe to any Bazel command (or any yarn command that uses Bazel) will make the build execute remotely. Note that this is only enabled for members of the TFJS team.
tslib was formerly included by chance since typescript depends on it. With this change, we explicitly depend on it and add it as a dependency to the tfjs_bundle Bazel rule. Rollup uses the es5 downleveling plugin, which uses tslib for bundle size optimization.
Build tfjs-backend-wasm with Bazel. tfjs-backend-wasm already uses Bazel for Emscripten. This PR makes it use Bazel for TypeScript as well and replaces the build scripts with a single pkg_npm target.
Fixes#5282
#6504 broke Karma tests that run with custom browsers other than BrowserStack. This PR fixes it.
Tested by running yarn test in tfjs-backend-webgl, which uses Chrome Canary (although I'm on Linux, which doesn't have a Chrome Canary build, so I saw "No binary for ChromeCanary browser on your platform.").
Create tools/make_rollup_config.ts, which exports a function to create a tfjs rollup configuration. The only thing tools/rollup_template.config.js does now is import and call that function.
Move the custom es5 downleveling plugin to its own file.
Co-authored-by: Na Li <linazhao@google.com>
Disable cpu forwarding for the new gather_nd tests that target webgl implementation.
Lock windows chrome version to 101.0 on browserstack, given the latest chrome (102.0) failed to provide webgl2 context.
Remove unused karma.config files.
Temporarily limit concurrent Bazel jobs to 25 (the number of browserstack tests we can run at the same time).
* Make ts_library tsconfig.json extend root tsconfig
* Remove tsetse settings from main tsconfig
* Remove baseUrl and paths from ts_library tsconfig
* Revert "Remove baseUrl and paths from ts_library tsconfig"
Reverted since this PR should not change the tsconfig behavior (although I don't really think this change does affect behavior).
This reverts commit 2a72331b1c.
* Lint bazel
Upgrade Bazel version to 4.2.2 (supported by m1), rules_nodejs to 4.6.1 (required for node 16.13.2), and node to 16.13.2 (same as our CI docker image and required for m1 compatibility).
tfjs-backend-wasm will be fixed in #6098.
Not yet working:
* tfjs-converter python. #6089 may fix this if you have a working python2.7 homebrew install.
BUG
* Allow bundles to keep 'require' calls as 'require' calls
@rollup/plugin-commonjs rewrites 'require' calls to 'import' statements, but this is not always desirable. This commit adds the 'leave_as_require' option to 'tfjs_bundle', which allows specifying a list of modules to leave as 'require' calls.
This commit also disables @rollup/plugin-commonjs when targeting commonjs, since require calls do not need to be transpiled for commonjs.
* Always use the commonjs plugin
It is required even for node cjs bundles because we still want to bundle the commonjs modules. Without the plugin, they're left as 'require()' calls, but with the plugin, their contents are included in the bundle.
DEV
INTERNAL
* Build tfjs-backend-webgpu with Bazel
* update
* bazel format
* use node:11 for yarn test-ci
* Revert "use node:11 for yarn test-ci"
This reverts commit 46b2e6ddd2f7afb08e402a4d534b4371bb007be5.
* add bazel test and test-dev
* update commands in package.json
* rebase
* bazel lint
* update setup_test structure
* save
* Use central `yarn lint` instead of mono webgpu backend
* Make WebGPU local testing run chrome with flag
* update yarn.lock
* bazel format
* skip browserstack testing for WebGPU
* update dependency
* Address comments
* lint
* lint
* update tsconfig.test.json
* Address Matt's comments
Pass the correct bash variables to Karma when running under Windows Subsystem for Linux (WSL). Add instructions for setting up WSL to build and test tfjs.
Run the correct browsers in nightly and pass the correct parameters to them to match the original test-ci.sh file.
Pin Firefox to 90.
Fix square test comparison precision on Safari webgl1.
Make the default ts_library macro behave like pre-4.0 versions of rules_nodejs by automatically setting package_name to module_name if its not set.
Temporarily mark webgl tests as not testonly to work around bazelbuild/rules_nodejs#2984 until the next release.
Add package_name to pkg_npm targets.
Make npm packages generated by Bazel match what we currently publish to npm.
Also, add a webpack regression test to e2e that makes sure webpack can consume the npm packages we publish.
Additionally, fix a bug in version tests that was caused by setting package_name of a js_library and module_name of a ts_library to the same value. See the following link for more details:
4e73b8820a/tools/defaults.bzl (L206-L218)Fixes#5382
Add an enumerate_tests Bazel rule that generates a typescript file containing import statements for a list of files. See tools/enumerate_tests.bzl for details.
Add a presubmit_browsers option to tfjs_web_test that controls which browsers run on PR presubmits. If not specified, tfjs_web_test runs only the first browser in the browsers list for PR presubmits. All browsers run in nightly.
Add the nightly Bazel tag to enable running tests in nightly without also making them run in PR presubmits. Tests tagged with ci will run as presubmit tests and in nightly. Tests tagged with nightly will run in nightly only (unless also tagged with ci).
Our node bundles are transpiled down to es5, so the ts_library commonjs outputs used by node should be es5 as well.
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
Matthew Soulanille