* Start verdaccio earlier and keep it alive longer
Start the verdaccio server immediately when running publish-npm and keep it
alive until the first OTP is entered. Also, add a tip for how to access the
server from your browser. This makes it easier to verify that all packages will
be published correctly to NPM.
* Fix CI timeout from asking to quit verdaccio
Killing the verdaccio process throws an error because the disconnect event emits when the process is killed. We throw an error on a disconnect to catch any unexpected verdaccio disconnections.
Fix this by deregistering the disconnect handler before killing the verdaccio process.
Some TFJS packages, like wasm, have examples or demos in them. These usually depend on the parent package, but the parent package is not marked as to be updated when updating the subpackage dependency versions. For an example of this, see #7547.
Update the TFJS dependencies of these subpackages to the release version if they are `link:` dependencies.
Replace e2e's custom local publishing logic with the publish-npm script. This more accurately represents how we release TFJS.
Fix publish-npm not waiting for Verdaccio to start before pushing packages.
Avoid testing WebGPU on platforms other than Chrome MacOS (for now).
---------
Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
release-tfjs updated only the root package.json of each package, but e2e tests
have some other package.json files that must be updated as well. Fix this by
updating all package.json files in each package so that they use the npm release
versions instead of `link:` dependencies.
Fix an intermittent bug in the npm publishing script that causes the script to depend on the network configuration of the host. I think what's happening is that when ipv6 is available, sometimes Verdaccio binds [::1]:5783 instead of 127.0.0.1:5783, but npm tries to publish to 127.0.0.1:5783. Explicitly specifying the loopback address 127.0.0.1 fixes this.
The current npm publishing infra is more manual than necessary. This PR makes the following improvements:
1. Publish all packages in a release unit with one command. Publishing no longer involves running yarn publish-npm multiple times.
2. Stage all packages to a local verdaccio repository before publishing. This catches any compatibility issues that may appear when publishing to npm and allows us to build all packages before publishing any to npm.
3. Publish with a single OTP. The previous infra required an OTP per package to publish to npm. This new infra publishes all packages sequentially after they're built, so it can use a single OTP for all of them. If another OTP is required due to the first one timing out, it asks for another.
A future PR will move our nightly verdaccio e2e tests to use this infra, which will allow us to avoid setting the npm registry globally. This will let us run verdaccio tests in parallel with the rest of nightly tests, which should reduce the time taken.
* Add script to launch tfjs debugger local server
* use jszip
* remove admzip
* address comments
* update version to the latest
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
Add tfjs-tfdf to the release script under alpha packages. Fix a bug where the release script throws an error if a package has never been published to npm.
Previously, nightly Verdaccio tests ran exclusively with Bazel tests because it was as convenient way to make the rest of the build steps wait for Verdaccio to finish. This PR properly sets up the waitFor field of the other steps to depend on Verdaccio tests when they're present.
To do this, the PR adds a new waitedForByPackages flag to the cloudbuild config, which makes all non-bazel packages waitFor any step it's added to.
Bazel test output is already logged in the results UI. Logging it in cloudbuild makes the log very long (> 5MB) which then truncates the rest of the logs.
Currently, we run Verdaccio tests only when we do a release. Since they're only tested after choosing a release commit from nightly CI, they are often a point of failure. This PR makes them run during nightly tests, so we can identify issues with them before creating a release PR.
This PR also enables Bazel sandboxing, which fixes the rules_nodejs linker issue.
* [webgpu] Remove 'yarn --cwd' from publish script
Some scripts call `yarn` multiple times. Each additional time `yarn` is nested, an additional `--` needs to be passed before arguments can be passed (since we're using yarn 1). Other packages' publishing scripts require two instances of `--`, and this is [what the publish-npm script assumes](https://github.com/tensorflow/tfjs/blob/master/scripts/publish-npm.ts#L132).
tfjs-backend-webgpu uses an additional call to yarn to change directory to the root of the monorepo, so it requires three instances of `--` before `--otp=######` can be passed during publishing.
This PR removes the extra call to yarn and instead uses the full name of the Bazel target, `//tfjs-backend-webgpu:tfjs-backend-webgpu_pkg.publish`. This should not affect the published artifacts since Bazel always creates a separate output tree for them anyway.
* Revert webgpu changes and add a special-case to publish-npm
* Fix column width of publish-npm.ts
Co-authored-by: Ping Yu <4018+pyu10055@users.noreply.github.com>
Build tfjs-backend-wasm with Bazel. tfjs-backend-wasm already uses Bazel for Emscripten. This PR makes it use Bazel for TypeScript as well and replaces the build scripts with a single pkg_npm target.
Fixes#5282
Disable cpu forwarding for the new gather_nd tests that target webgl implementation.
Lock windows chrome version to 101.0 on browserstack, given the latest chrome (102.0) failed to provide webgl2 context.
Remove unused karma.config files.
Temporarily limit concurrent Bazel jobs to 25 (the number of browserstack tests we can run at the same time).
* Use packages from package_dependencies.json for diff checking
* Clone to /tmp when checking for diffs
* Remove unused imports
* Return to original path correctly
Improve link_package speed by only building the dependencies that are required and by building them all in a single bazel build command.
Remove the link_package_core package, which is no longer needed now that link_package can build just the required dependencies.
.substr() is deprecated so we replace it with .slice() which works similarily but isn't deprecated
Signed-off-by: Tobias Speicher <rootcommander@gmail.com>
Co-authored-by: Na Li <linazhao@google.com>
Update e2e dependencies to the versions that are released when releasing the tfjs monorepo. This fixes a failure in release tests where e2e tries to install 'link:' dependencies.
Add a '--local' option to the release-tfjs script to prevent creating a release PR. This is just for development and makes it easier to see the output of the script.
* In release, replace only packages marked as deps
\#6209 made the release script overwrite every dependency in the set of packages being published. This causes tfjs-core's verdaccio tests to fail because tfjs-core has a devDependency on tfjs-backend-cpu, but core needs to be published before cpu. This PR reverts to the original behavior of overwriting dependency versions in the package.json only if they are listed in a given release phase's dependencies.
* Remove update-dependency.ts from e2e
* Add webgpu to the release script
* Check package.json for link and file deps before publishing
* Allow alpha monorepo packages to publish a different version
* Make alpha packages publish using the tfjs branch
* Update release-util.ts
Co-authored-by: Na Li <linazhao@google.com>
DEV
INTERNAL
* Build tfjs-backend-webgpu with Bazel
* update
* bazel format
* use node:11 for yarn test-ci
* Revert "use node:11 for yarn test-ci"
This reverts commit 46b2e6ddd2f7afb08e402a4d534b4371bb007be5.
* add bazel test and test-dev
* update commands in package.json
* rebase
* bazel lint
* update setup_test structure
* save
* Use central `yarn lint` instead of mono webgpu backend
* Make WebGPU local testing run chrome with flag
* update yarn.lock
* bazel format
* skip browserstack testing for WebGPU
* update dependency
* Address comments
* lint
* lint
* update tsconfig.test.json
* Address Matt's comments
Matthew Soulanille