Changes the xDS interop test client to support timeout test.
- Synced xDS test proto messages with grpc-proto.
- Changed RpcConfig to be the configuration for per test method type. Added timeoutSec for its deadline configuration.
- Changed accumulated stats to include RPC status instead of just succeeded/failed.
Resolves#7741
Some of the static methods in generated code have the same method name but different package name, such `ClientCalls.asyncClientStreamingCall` and `ServerCalls.asyncClientStreamingCall`. It's less readable using static import than using full-qualified method name in-place.
The proto field is named as num_failures but its comment is saying it is for number of RPCs that failed to record a remote peer. RPC failed == RPC failed to record a remote peer was true previously (so no existing tests should be affected by this changed) as server completed RPCs immediately. It is no longer true with server capability to keep the call open/delayed.
This change clarifies the proto definition for stats RPC. rpcs_by_peer is for recording RPCs succeeded and num_failures is for RPCs failed. RPCs in the flight when the stats call times out are not counted towards any of the stats.
Update xDS interop test proto to aggregate accumulated stats based on RPC methods (mirroring 643e5bcd1e8db931cf76a3be19cd9bba223ee987 in C-core's change). Updated the xDS interop test client to support querying accumulated stats aggregated to RPC methods.
The main() Thread will call tearDown() itself. It appears this
redundancy has existed since e813eaae2f, where the normal error handling
was enhanced at the same time as cleaning up resource management. The
cleanup should have made it obvious the hook was no longer needed, but
alas. Technically, it did originally provide a purpose if setup()
failed, but it would have been better to just move setup() into the
try-catch instead. Today it doesn't even provide that purpose.
Most of these are easy "replace X with Y."
The CreateStartScripts changes were because the scripts were being included in
the output zip/tar multiple times. The was because they were all using the same
output directory, and the entire output directory was being included for each.
The output directory tmp/ was particularly poor because other tasks were
dumping things into it, so our zip/tar was including those junk files as well.
This prevents grpc-xds and its transitive dependencies from being included
twice in distTar and distZip, which reduces the size from 60 MB to 40 MB. It
does mean that interop-testing as a whole depends on xds, but that should not
be an issue any longer. It was an issue before we started providing grpc-xds on
Maven Central.
If the server fails (e.g.: with an unimplemented status), the test still succeeds.
This change adds assertions to check the status code as well as the expected number of responses.
This is a java analogue of the C++ test added in grpc/grpc#19623.
Proto changes were synced from grpc/grpc-proto
This client is suitable for testing fallback with any "grpclb" load-balanced service, but is particularly meant to implement a set of test cases described in an internal doc titled "DirectPath Cloud-to-Prod End-to-End Test Cases", section "gRPC DirectPath-to-CFE fallback".
- Use gradle configuration `api` for dependencies that are part of grpc public api signatures.
- Replace deprecated gradle configurations `compile`, `testCompile`, `runtime` and `testRuntime`.
- With minimal change in dependencies: If we need dep X and Y to compile our code, and if X transitively depends on Y, then our build would still pass even if we only include X as `compile`/`implementation` dependency for our project. Ideally we should include both X and Y explicitly as `implementation` dependency for our project, but in this PR we don't add the missing Y if it is previously missing.
Currently the classpath for all startscripts in interop-testing includes alpnagent and grpc-xds, even if a lot of startscripts don't need them. Made a change to only include alpnagent or grpc-xds when a startscript really needs it.
This reverts commit c5f48b8e38. (#6780)
Revert because caused a regression in the ALTS tests. https://source.cloud.google.com/results/invocations/691d9965-fea1-487d-b606-352a5234039e/targets/grpc%2Fcore%2Fpull_request%2Flinux%2Fgrpc_interop_toprod/log
2020-03-01 20:02:12,491 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang3/SystemUtils
at io.grpc.alts.CheckGcpEnvironment.isRunningOnGcp(CheckGcpEnvironment.java:69)
at io.grpc.alts.CheckGcpEnvironment.isOnGcp(CheckGcpEnvironment.java:44)
at io.grpc.alts.ComputeEngineChannelBuilder.(ComputeEngineChannelBuilder.java:62)
at io.grpc.alts.ComputeEngineChannelBuilder.forTarget(ComputeEngineChannelBuilder.java:72)
at io.grpc.alts.ComputeEngineChannelBuilder.forAddress(ComputeEngineChannelBuilder.java:77)
at io.grpc.testing.integration.TestServiceClient$Tester.createChannel(TestServiceClient.java:399)
at io.grpc.testing.integration.AbstractInteropTest.setUp(AbstractInteropTest.java:309)
at io.grpc.testing.integration.TestServiceClient.setUp(TestServiceClient.java:198)
at io.grpc.testing.integration.TestServiceClient.main(TestServiceClient.java:56)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang3.SystemUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 9 more
In v1.27.0 release the grpc-interop-testing artifact in maven includes grpc-xds, but grpc-xds is not yet published. It should be removed from the dependency list in maven artifact.
Decouples grpc-core with census, while still preserve the default integration of census in grpc-core. Users wishing to enable census needs to add grpc-census to their runtime classpath.
- Created a grpc-census module:
- Moved CensusStatsModule.java and CensusTracingModule.java into grpc-census from grpc-core. CensusModuleTests.java is also moved. They now belong to io.grpc.census package.
Moved DeprecatedCensusConstants.java into io.grpc.census.internal (is this necessary?) in grpc-census.
- Created CensusStatsAccessor.java and CensusTracingAccessor.java, which are used to create census ClientInterceptor and ServerStreamTracer.Factory.
- Everything in grpc-census are package private, except the accessor classes. They only publicly expose ClientInterceptor and ServerStreamTracer.Factory, no Census specific types are exposed.
- Use runtime reflection to load and apply census stats/tracing to channel/server builders, if grpc-census is found in runtime classpath.
- Removed special APIs on AbstractManagedChannelImplBuilder and AbstractServerImplBuilder for overriding census module. They are only used for testing. Now we changed tests to apply Census ClientInterceptor and ServerStreamTracer.Factory just as normal interceptor/stream tracer factory. Test writer is responsible for taking care of the ordering concerns of interceptors and stream tracer factories.
This change adds two booleans to the ChannelBuilders to
allow transports to use get and put. These are currently defaulted to
on, but unset on the method descriptors. This change is 1/2 that will
allow the safe / idempotent bits to be set on generated proto code.
Part 2/2 will actually enable it.
The use case for this is for interceptors that implement caching logic.
They need to be able to access the safe/idempotent bits on the MD in
order to decide to how to handle the request, even if gRPC doesn't use
GET / PUT HTTP methods.
Examples and android projects were left unchanged. They can be changed
later.
No plugin versions were changed, to make this as non-functional of a
change as possible. Upgrading Gradle to 5.6 was necessary for
pluginManagement in settings.gradle.
Fixes#5593 and supersedes #5601
Now that https://github.com/census-instrumentation/opencensus-java/pull/1854 has been merged & released as 0.21.0. We can start using the method & status tags.
Background:
Opencensus introduced new tags for status and method (https://github.com/census-instrumentation/opencensus-java/pull/1115). The old views that used those tags were deprecated and new views were created that used the new tags. However grpc-java wasn't updated to use the new tags due to concern of breaking existing metrics. This resulted in the old views being deprecated while the new views were broken (goomics #50).
https://github.com/census-instrumentation/opencensus-java/pull/1854 added a compatibility layer to opencensus that would remap new tags to old tags for old views. This should unblock grpc to switching to the new tags while allowing old views to still be populated. That commit was released as part of opencensus 0.21, which grpc currently uses
* compiler: Use 'SERVICE_NAME' instead of duplicated '$Package$$service_name$'
* compiler: Align indentation
* Fix typo
* Add modified golden files and all re-generated code to meet Travis CI and Windows build requirements
See PR #5943
* Polishing
ServerCall.close() is meant to only be accessed from a single thread, but in the test it is accessed from the callbacks of several client calls. Synchronize access to sate TSAN.
Works for #4740
- Subclasses of `AbstractClientStream` include remote address in insight if available.
- `DelayedStream` adds buffered time, and the insight of real stream if it's set.
- `RetriableStream` insights outputs of Substreams.
Example error message:
```
deadline exceeded after 8112071ns. [buffered_nanos=24763, remote_addr=foo.test.google.fr/127.0.0.1:44749]
```
or
```
deadline exceeded after 8112071ns. [buffered_nanos=22344324763, waiting_for_connection]
```
This is related to #4776 but taking a more usage-specific approach.
This reverts commit e795f14bed. It breaks some of
our internal tests. At the very least the cast fails because we are reusing the
service for something without going through the normal stub. There may also
have been a test hang, but it'd take more effort to figure out.
The former is deprecated and replaced by the latter in Mockito 2.
However, there is a functional difference: ArgumentMatchers will reject
`null` and check the type if the matcher specified a type (e.g.
`any(Class)` or `anyInt()`). `any()` will remain to accept anything.
The `method` and `status` shouldn't be propagated in the first place,
but in previous OpenCensus implementation all tags are propagating by
default. Now with the TagMetadata it may make sense to change them to
local tags.
This will be a breaking change to users who depend on the behavior that
these tags propagate through process boundaries.
Also updated CensusModule to use the new helper methods ContextUtils.withValue() instead of directly manipulating the context keys. See census-instrumentation/opencensus-java#1864.
New fields are added to the test protos according to grpc/grpc-proto#51
This test needs to be run in an environment where "pick_first" or "grpclb" with child policy "pick_first" is configured.
This commit swaps to using a Sync task to place generated code in the
src/generated folder instead of the gradle-protobuf-plugin's
generatedFilesBaseDir. This provides much nicer results on failed
builds, and you will no longer see all the generated files deleted.
But at the same time the Sync task makes it easy to only copy the
grpc-generated code. This was not previously done because we were lazy
and using generatedFilesBaseDir, which made it difficult to treat the
services differently from the messages.
This will be a new override. The old override is now deprecated.
In order to pass new information without adding new overrides, I shoved most information
into an object called StreamInfo. The Metadata is left out to draw attention because
it's mutable.
Motivation: this is needed for correctly supporting pick_first in GRPCLB. GRPCLB needs to
add a token to the headers, and the token varies among servers. With round_robin, GRPCLB
create a Subchannel for each server, thus can attach the token when the Subchannel is picked.
To implement pick_first, all server addresses will be put in a single Subchannel, we will
need to add the header in newClientStreamTracer(), by looking up the server address from
the transport attributes and deciding which token to add.
For Bazel, we upgrade to protobuf 3.6.1.2 and javalite HEAD to fix
incompatibilities in newer Bazel releases.
compiler/Dockerfile is unused, so it was removed instead of being updated.
protoc no longer includes codegen for nano, so we remain on the older protoc
any time nano is used.
Protobuf now requires C++11 when compiling, so windows was swapped to
VC 14.
yifeizhuang
Chengyuan Zhang
Doug Fawley
ZHANG Dapeng
Sergii Tkachenko
sanjaypujare
Eric Anderson
Eric Gribkoff
Trustin Lee
Philippe Laflamme
Jihun Cho
apolcyn
ZhenLian
Tomo Suzuki
Carl Mastrangelo
Igor Bernstein
liym
Carl Mastrangelo
Kun Zhang
Tim van der Lippe
Yang Song
Tim van der Lippe
Jiangtao Li