Commit Graph

182 Commits

Author SHA1 Message Date
Terry Wilson 26be0c7665
buildscripts: add missing $GRADLE_FLAGS to enable AndroidX (#10504) 2023-08-17 16:32:23 -07:00
Terry Wilson beaeeefdd2
examples: Android examples to use AGP 7.4.0 (#10497)
- Upgrades AGP version to 7.4.0
- Upgrades appcomat to use AndroidX
- Updates the `android.sh` build script to set the `android.useAndroidX` flag
2023-08-17 11:24:42 -07:00
Zach Reyes 6d48271fa4
buildscripts: Add bootstrap generator to cron job that runs nightly (#10440) 2023-08-02 13:04:56 -07:00
Eric Anderson d654707838
buildscripts: Build Android with main build in linux_artifacts
Note that this changes the JDK used to compile releases to Java 11. That
should only impact the appearance of the Javadoc.

This adds the Android SDK to the build container, removing the
dependency on the Android SDK being available on the CI host. This
allows running on newer Kokoro images. 'Android' and 'Android interop'
CIs still depend on the Android SDK being available on the host, but
since they aren't used as part of the release process, they can more
easily migrate off Kokoro as part of future work.

This also causes Android components to now be built with -Werror, as we
use -PfailOnWarnings=true in unix.sh but were missing it from the
Android build invocations.

Gradle will auto-download the necessary version of build-tools. We don't
want to download it ourselves because the version we specify might not
even be used. Looking at logs, we were previously downloading a version
that was unused.

We now fork javac to avoid OOM. The build fails 2/3 times before the
forking, and 0/3 after.
2023-06-28 16:24:21 -07:00
Xuan Wang 1c9ce5d76b
[PSM interop] Don't fail url_map target if sub-target already failed (#10295) 2023-06-22 15:42:21 -07:00
Xuan Wang 02ac6f4393
buildscripts, psm interop: Don't fail target if sub-target already failed
We configured TestGrid to file bug separately for each 
failed sub-target, if we still fail the main target, 
TestGrid will fail duplicate bugs.

The same change in core: 
https://github.com/grpc/grpc/pull/33222.
2023-05-31 17:37:46 -07:00
Terry Wilson 0005029328
android,binder,cronet: .aar file when publishing (#10138) 2023-05-05 15:21:26 -07:00
Sergii Tkachenko c9fcd46e84
buildscripts: increase PSM Security test timeout to 4h (#10065)
The total time PSM Security takes to run was getting close to 3h lately.
Adding an extra hour to allow for a larger margin of error.

ref b/259690410
2023-04-19 10:35:12 -07:00
Eric Anderson 35852130d9 buildscripts: Cross-compile in Docker
Previously builds were done with Ubuntu 16.04, and now we are using
18.04. Thus the generated binaries will no longer work for
Ubuntu 16.04 and Debian 9 users, both of which are outside of their
support window and aren't supported by Abseil. RHEL users are
unaffected, as the binaries already didn't work on RHEL 7 and they will
remain working with RHEL 8. FWIW, Ubuntu 18.04 will leave its support
window in June.
2023-04-18 16:16:35 -07:00
Dirk Haubenreisser 99cbdd5d69
Add support for cross-compiling for s390x platform (#9455)
* Added s390x platform support
* Adapt to existing platform naming scheme
* Updated s390_64 library whitelist
* Use g++ compiler version 8.x for s390x
* Introduced dedicated Docker container for building s390x artifacts Minor fix

---------

Signed-off-by: Dirk Haubenreisser <haubenr@de.ibm.com>
Co-authored-by: Eric Anderson <ejona@google.com>
2023-03-23 13:21:31 -07:00
yifeizhuang c1ff4a856d
buildscript: iterate all example folder and build (#9961) 2023-03-17 15:09:58 -07:00
Terry Wilson 7ecb909776
buildscripts: Use Java 11 with android-interop (#9943) 2023-03-09 09:33:10 -08:00
Terry Wilson b5b7cacba9
Upgrade Android Gradle plugin to 7.4.0 (#9933)
This is the latest version of the plugin supported by the Gradle version
in use at the moment (7.6).

Note that this also upgrades the R8 optimizer to a version (4.0.48) that
now uses "full mode" optimization by default.

This also splits off Android projects to run under Java 11 (Gradle
plugin requirement) while the other projects continue to run under Java
8.
2023-03-08 11:18:27 -08:00
Sergii Tkachenko f2533f4fd8
xds interop: Fix buildscripts not continuing on a failed test suite (#9833)
Apparently there's a difference between bash 3 and bash 4.
OSX comes with bash 3 out-of-box, so for whoever wrote this logic
it "worked on my machine".

The `((` construct returns a 0 exit code if the value is non-zero.
Since the value starts at 0 and we do a post-increment,
it will always fail the first time.
Changing it to a pre-increment should fix it.
2023-01-17 13:15:40 -08:00
Sergii Tkachenko 42ba00ba65
Revert "xds interop: Fix buildscripts not continuing on a failed test suite (#9817)" (#9831)
This reverts commit d83a599c97.

Reverted in favor of better syntax suggested here: https://github.com/grpc/grpc-node/pull/2323#pullrequestreview-1248043432. This fix will be sent as another PR for the convenience of backporting.
2023-01-17 11:36:31 -08:00
Sergii Tkachenko d83a599c97
xds interop: Fix buildscripts not continuing on a failed test suite (#9817)
Apparently there's a difference between bash 3 and bash 4.
OSX comes with bash 3 out-of-box, so for whoever wrote this logic
it "worked on my machine".
2023-01-13 12:42:45 -08:00
Eric Anderson ec5bc6b1eb Drop Bazel 4 support
Bazel 6 has been released, and we track the two most recent major
versions.
2023-01-11 08:15:47 -08:00
Sergii Tkachenko f4cda008ee
buildscripts: drop xDS v2 interop test (#9783)
This disables xDS v2 support interop test.
xDS v2 support dropped in https://github.com/grpc/grpc-java/pull/9760.

Internal ref cl/499306755
2023-01-03 15:37:12 -08:00
Terry Wilson bf692c04e6
Add custom_lb to the test suites. (#9502)
Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2022-09-06 15:49:49 -07:00
Sergii Tkachenko c8c3d3d6b2
xDS interop: enable pod log collection in the buildscripts (#9495)
- Enables pod log collection in all PSM interop jobs implemented
  in https://github.com/grpc/grpc/pull/30594.
- Associate test suite runs with their own log file, so it's displayed
  on the "Target Log" tab
- Updates security job to not stop after a failed suite, so that
  authz_test run even if security_test failed
- Fix run_test not returning correct exit status, causing false
  positives in some cases. See https://github.com/grpc/grpc/pull/30768
2022-09-06 11:18:27 -07:00
Terry Wilson 1dd764a3a1
buildscripts: Add outlier_detection_test interop test. (#9461) 2022-08-22 14:27:20 -07:00
yifeizhuang 58cd6e1a7f
example: fix orca example to use new ORCA API (#9403) 2022-07-22 07:40:49 -07:00
Sergii Tkachenko 4aa9b92551
buildscripts: Fix kube contexts in the xds LB tests (#9389)
- The primary should've been `GKE_CLUSTER_PSM_LB`
- The secondary cluster was not activated for LB tests. This resulted
  in the failover test failing, as it relies on workloads running in
  different zones.
2022-07-18 17:40:46 -07:00
Sergii Tkachenko 10449d63bc Revert "buildscripts: Add missing secondary_kube_context to xds LB tests (#9380)"
This reverts commit e3e152a449.
2022-07-18 15:28:35 -07:00
Sergii Tkachenko e3e152a449
buildscripts: Add missing secondary_kube_context to xds LB tests (#9380)
Secondary cluster was not activated for LB tests. This resulted in the failover test failing, as it relies on workloads running in different zones.

ref b/238226704
2022-07-15 16:03:48 -07:00
Sergii Tkachenko d7a6c1ea31
xds: Allow Gradle to use more memory when building interop - GCE (#9354)
Same as #9347, but for GCE framework too (xds and xds_v3 jobs).

Should fix "Expiring Daemon because JVM heap space is exhausted".

PR #9269 probably pushed the build
over the edge, but there's been evidence via flakes for a good while
that we've been reaching the limit.

b/238334438
2022-07-08 19:28:23 -07:00
Eric Anderson 5f9ef98173 xds: Allow Gradle to use more memory when building interop
Should fix "Expiring Daemon because JVM heap space is exhausted".

https://github.com/grpc/grpc-java/pull/9269 probably pushed the build
over the edge, but there's been evidence via flakes for a good while
that we've been reaching the limit.

b/238334438
2022-07-08 12:59:27 -07:00
Eric Anderson 3de7e74c57
xds: Build third-party protos in separate build step
This dramatically shortens build time, even for full builds. A full
assemble of xds on my laptop goes from 1m 46s to 33s at least because
errorprone is disabled for the protos.
2022-07-07 07:26:38 -07:00
Sergii Tkachenko 91fcc33243
buildscripts: Fix Xmx JVM flag propagation in GRADLE_OPTS
* buildscripts: Fix Xmx JVM flag propagation in GRADLE_OPTS
* buildscripts: double Java memory allocation pool

To reduce periodic OOMs of the "GitHub Actions Linux Testing / tests (11) (pull_request)" job.
2022-06-30 14:39:03 -07:00
Eric Anderson c0790283ec
Bump protobuf to 3.21.1 (#9311)
Fixes #9264
2022-06-30 11:18:49 -07:00
sumitd2 04f9bdc1fd
compiler: Cross-compile for ppc64le platform (#9284)
Fixes #9274
2022-06-23 09:18:19 -07:00
Sergii Tkachenko 12984db6a7 kokoro: standard TESTING_VERSION in k8s framework build scripts
This Addresses the issue with skips not working due to the
missing/inconsistent `--testing_version` flag, ref b/235688697.

1. Uses the new `TESTING_VERSION` variable populated in the shared
grpc_xds_k8s_install_test_driver.sh - new approach for detecting
versions applicable to all languages.
2. Use `TESTING_VERSION` in all build files in `--testing_version` and
when tagging docker images. This will be backported to all active
test branches. Build Scripts in all other languages will be updated
as well.

Corresponding grpc core change: https://github.com/grpc/grpc/pull/30027
2022-06-21 09:08:24 -07:00
Lidi Zheng d6a23e4116 interop-test: Remove duplicated GCE tests 2022-05-25 15:56:29 -07:00
Lidi Zheng d196e588a2
Add more xds_lb test cases (#8998)
This test suite is only fully enabled for C++ and Python at this
moment. This commit enables the rest of the test suite for grpc-java.

* change_backend_service_test
* failover_test
* remove_neg_test
* round_robin_test
* affinity_test
2022-03-21 12:17:26 -07:00
Lidi Zheng d71bef5afe
Redirect xDS tests to use grpc/grpc's master branch (#8982)
We are solving the issue of grpc/grpc delaying the release, causing GCE
tests to fail. Updating the master branch prevents similar cases from
happening.

Tested: [prod:grpc/java/master/branch/xds_v3](http://sponge/010f5353-e65e-4b1a-b6d3-f5a84e31546b)
2022-03-17 09:55:41 -07:00
yifeizhuang 467985e958
add kokoro config for basic tests xds k8s (#8888) 2022-02-04 11:28:13 -08:00
Eric Anderson 39cc44e38c
kokoro: Pretty test results
Previously, only Windows had the plumbing to rename test results for
the Kokoro result viewers to pretty-print.

macos.cfg was the only CI that lacked a corresponding .sh, which maked
unix.sh harder to reason about. Created macos.sh so that unix.sh is now
just a helper script and will not be called directly by Kokoro.

We now avoid "gradle clean" to avoid wiping results. Still clean compiler
since we do re-run the build multiple times with varying platforms.
Shouldn't be necessary, but "just in case" since I want this commit to
be low risk. This improves Windows to produce detailed results even
if the CI was successful.
2022-01-12 15:44:01 -08:00
sanjaypujare d7f951a9d8
all: update netty to 4.1.72.Final and tcnative to 2.0.46.Final (#8780) 2022-01-07 15:34:59 -08:00
ZHANG Dapeng 5ae6f9ca25
all: Upgrade Android plugin to 4.2.0
- bump android plugin version to 4.2.0
- migrate deprecated android.support dependencies to androidx dependencies
- bump `targetSdkVersion` to 29
- temporarily ignore lint error for 'MissingClass' due to #8799
- run android CIs with `-Pandroid.useAndroidX=true -Pandroid.enableJetifier=true` flags
- android examples are still using android.support dependencies, will not be updated in this PR.
2022-01-07 09:52:26 -08:00
sanjaypujare e474e3dc7e
buildscripts: always make sure we are using jdk8 (#8800) 2022-01-04 13:01:02 -08:00
Lidi Zheng 80f1be03f0
Use xds-test-server-5 as the GCE interop server (#8777) 2021-12-21 16:39:31 -08:00
markb74 ee581bfdfa
buildscripts: add config for building grpc-binder artifact (#8722) 2021-11-23 18:11:54 +01:00
sanjaypujare b746bab97b
buildscripts: rename xds-k8s to psm-security as part of tech-debt cleanup and name clarity (#8695) 2021-11-12 17:40:07 -08:00
Kevin Wooten 6518d7bd6d
Copy macOS x86 artifacts to aarch during upload (#8680) 2021-11-09 10:36:04 -08:00
Eric Anderson fe9026ed8a kokoro: Increase xds-k8s timeout to 3 hours
The addition of the authz tests in 0d345721 is causing the tests to
exceed their timeout. By itself, the authz test takes about an hour in
this environment. Before the authz tests, xds-k8s was taking an hour
and a half.
2021-11-05 10:04:48 -07:00
Eric Anderson 0d34572149 kokoro: Enable xds authz_test
It is successfully passing against prod:
https://source.cloud.google.com/results/invocations/e2be0996-ed4d-4a4c-90ad-20bc706f9f70/targets
2021-11-04 16:31:16 -07:00
Ivo List bb51bb6dfa
java_grpc_library.bzl: Fix parameters of java_common.compile (#7598)
Parameter host_javabase is removed.

This is preparation for flipping incompatible_java_common_parameters in
Bazel 5. See https://github.com/bazelbuild/bazel/issues/12373

Bazel versions prior to 4 require host_javabase, so are no longer supported.
2021-10-08 13:48:22 -07:00
Terry Wilson 83d36104e1
Source k8s test driver install script from core repo (#8573)
The test driver install script is read directly from the core repo master branch and the copy in the Java repo is deleted.
2021-10-07 13:05:03 -07:00
Lidi Zheng 8ac9a4e7bd [xDS interop] add Docker tagging logic to the xds_url_map job 2021-10-07 09:08:23 -07:00
markb74 f57de6bd03
Make binder instrumentation tests run on kokoro. (#8563)
The tests run as part of the existing android-interop-testing job.

We needed to modify the manifest of the apk built under android-interop-testing to declare Android Services used by the binder tests.
2021-09-29 11:28:14 +02:00