* Fix for windows installation #7341
Fix a problem with the installation on windows.
My problem on windows was that `./lib/napi-v9` directory didn't exist and `./scripts/deps-stage.js` on line **60** needed the path to symlink or copy the dll.
My fix was to make sure that the directory `napi-v9` exists with:
`await fs.mkdir(path.dirname(destLibTensorFlowPath), {recursive: true})`;
I'm solving the copy issue with this code.
* Fix fs.mkdir usage in deps-stage.js
* update py import for auto tracking
* [e2e] Use ios12 and report karma seed (#7984)
* [e2e] Use ios12 and report karma seed
* Update karma.conf.js
---------
Co-authored-by: Matthew Soulanille <msoulanille@google.com>
* fixed range issue when creating int32 tensor with larger than 2 ^ 24 size
* reduce the size of the test tensor
* remove use of max op
* fixed test
* fixed the test
Chrome ArrayBuffers throw allocation errors above 2GB in size. This makes it impossible to load TFJS models above this size in Chrome (even with weight sharding) because model loading involves concatenating all the weights into a single ArrayBuffer.
This PR avoids this concatenation. Instead of slicing the weight tensors out of a single concatenated ArrayBuffer, it keeps the weight buffers in their original shards and slices them using the CompositeArrayBuffer class created in #7598.
* Test commit
* Support ensureShape in tfjs
* Applied changes
* Updates doc
* Updates doc
* Updates doc
* Add an example
* add dif length and null shape tests
* fix the lint error
* Update executor tests
* Update the header
* Implement bitwiseAnd() ops and add kernel in Webcpu backEnd
* Fix format
* Change to only support int32 type
A new function, `isTypedArray` was added to the `platform` interface by #7181
and first published in tfjs-core 4.2.0. This made 4.2.0 incompatible with
earlier versions of backends that implemented `platform`, such as node and
react-native. This change adds a fallback to the use of `isTypedArray` so
earlier versions of platforms that don't implement `isTypedArray` will not throw
an error.
Note that the fallback behavior may not be perfect, such as when running Jest
tests in node. See #7175 for more details and upgrade all @tensorflow scoped
packages to ^4.2.0 to avoid this.
BUG
* fixed summary writer memory leak due to the step Int64Scalar internal tensor not deleted after use.
* also fixed a similar issue related to saved model execution
The current npm publishing infra is more manual than necessary. This PR makes the following improvements:
1. Publish all packages in a release unit with one command. Publishing no longer involves running yarn publish-npm multiple times.
2. Stage all packages to a local verdaccio repository before publishing. This catches any compatibility issues that may appear when publishing to npm and allows us to build all packages before publishing any to npm.
3. Publish with a single OTP. The previous infra required an OTP per package to publish to npm. This new infra publishes all packages sequentially after they're built, so it can use a single OTP for all of them. If another OTP is required due to the first one timing out, it asks for another.
A future PR will move our nightly verdaccio e2e tests to use this infra, which will allow us to avoid setting the npm registry globally. This will let us run verdaccio tests in parallel with the rest of nightly tests, which should reduce the time taken.
Avoid casting through '{}' with 'foo as {} as Bar' and instead use 'unknown', i.e. 'foo as unknown as Bar'. This is the preferred method according to Google's style guide. Internal link: go/tsstyle#double-assertions.
This major release upgrades TypeScript from 3.5.3 to 4.8.4 (https://github.com/tensorflow/tfjs/pull/6346). It is intended to be as non-breaking as possible, but it technically has breaking changes for users of `typescript<4.4`. All other users should be unaffected.
When compiling projects that depend on TFJS with `typescript<4.4`, the following error will appear.
```
node_modules/@webgpu/types/dist/index.d.ts:587:16 - error TS2304: Cannot find name 'PredefinedColorSpace'.
587 colorSpace?: PredefinedColorSpace;
~~~~~~~~~~~~~~~~~~~~
...
```
This can be fixed by upgrading TypeScript to 4.4.2 or greater, or by adding the file `predefined_color_space.d.ts` (name and path can be changed) with the following contents to your project to define the missing type. If this file is added, it will need to be removed when TypeScript is upgraded past 4.3.
```typescript
type PredefinedColorSpace = "display-p3" | "srgb";
```
`typescript<3.7` has the following additional error.
```
node_modules/@tensorflow/tfjs-core/dist/engine.d.ts:127:9 - error TS1086: An accessor cannot be declared in an ambient context.
127 get backend(): KernelBackend;
~~~~~~~
...
```
Enabling [`skipLibCheck`](https://www.typescriptlang.org/tsconfig#skipLibCheck) suppresses this error, and upgrading to at least TypeScript 3.6.2 fixes it (although the above fix for `PredefinedColorSpace` will also need to be applied).
Build the tfjs-node and tfjs-node-gpu node addons with the release docker image when publishing them to npm. This ensures a consistent GLIBCXX version is used when publishing.
Some package.json files in e2e linked against link_package instead of link-package. This was not discovered earlier because these tests are usually run after the package.jsons are prepared for release and dependencies are rewritten to the release versions.
Currently, we run Verdaccio tests only when we do a release. Since they're only tested after choosing a release commit from nightly CI, they are often a point of failure. This PR makes them run during nightly tests, so we can identify issues with them before creating a release PR.
This PR also enables Bazel sandboxing, which fixes the rules_nodejs linker issue.
* Add kernel RaggedGather for CPU and WebGL backend
* Improve error message
* Mark function as async
* Mark function as async
* Fix expect call for async
* Fix datasync call
Make build-deps-for check for Bazel dependencies transitively instead of just on the packages passed to it. Additionally, change the behavior of `build-deps-for` so that when it is passed a Bazel package, it will directly build the package instead of building just its dependencies.
Co-authored-by: Linchenn <40653845+Linchenn@users.noreply.github.com>
Update jasmine from 3.1.0 to 4.2.1 for Bazel packages. Update karma from 6.3.16 to 6.4.0.
Remove deprecated use of async function and done callback at the same time in Jasmine tests.
Add karma-jasmine-html-reporter to show test status in the browser window.
Bump tfjs-node TensorFlow version from 2.7.0 to 2.7.3.
Add tfjs-node/scripts/build-libtensorflow-arm64.yml, a Google CloudBuild file to cross-compile TF 2.7.3 to arm64 and upload it to GCP.
Add a special case for arm64 like we do for arm32 and darwin arm64.
Fixes#5937
Tested in a Linux docker container running on Apple silicon.
Build tfjs-backend-wasm with Bazel. tfjs-backend-wasm already uses Bazel for Emscripten. This PR makes it use Bazel for TypeScript as well and replaces the build scripts with a single pkg_npm target.
Fixes#5282
Improve link_package speed by only building the dependencies that are required and by building them all in a single bazel build command.
Remove the link_package_core package, which is no longer needed now that link_package can build just the required dependencies.
gaikwadrahul8