diff --git a/docs/eventing/README.md b/docs/eventing/README.md index 2aa6c2da8..5576d7fc4 100644 --- a/docs/eventing/README.md +++ b/docs/eventing/README.md @@ -71,16 +71,17 @@ To learn how to use the registry, see the Knative Eventing also defines an event forwarding and persistence layer, called a -[**Channel**](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/channel_types.go#L36). -Messaging implementations may provide implementations of Channels via the -[ClusterChannelProvisioner](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/cluster_channel_provisioner_types.go#L35) -object. Events are delivered to Services or forwarded to other channels +[**Channel**](https://github.com/knative/eventing/blob/master/pkg/apis/messaging/v1alpha1/channel_types.go#L57). +Each channel is a separate Kubernetes [Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). +Events are delivered to Services or forwarded to other channels (possibly of a different type) using [Subscriptions](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/subscription_types.go#L35). This allows message delivery in a cluster to vary based on requirements, so that some events might be handled by an in-memory implementation while others would be persisted using Apache Kafka or NATS Streaming. +See the [List of Channel implementations](../eventing/channels/channels.yaml). + ### Future design goals The focus for the next Eventing release will be to enable easy implementation of @@ -111,7 +112,7 @@ The eventing infrastructure supports two forms of event delivery at the moment: Service is not available. 1. Fan-out delivery from a source or Service response to multiple endpoints using - [Channels](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/channel_types.go#L36) + [Channels](https://github.com/knative/eventing/blob/master/pkg/apis/messaging/v1alpha1/channel_types.go#L57) and [Subscriptions](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/subscription_types.go#L35). In this case, the Channel implementation ensures that messages are delivered diff --git a/docs/eventing/broker-trigger.md b/docs/eventing/broker-trigger.md index 007c6dc3f..840151871 100644 --- a/docs/eventing/broker-trigger.md +++ b/docs/eventing/broker-trigger.md @@ -23,11 +23,9 @@ kind: Broker metadata: name: default spec: - channelTemplate: - provisioner: - apiVersion: eventing.knative.dev/v1alpha1 - kind: ClusterChannelProvisioner - name: gcp-pubsub + channelTemplateSpec: + apiVersion: messaging.knative.dev/v1alpha1 + kind: InMemoryChannel ``` ## Trigger diff --git a/docs/eventing/channels/README.md b/docs/eventing/channels/README.md index b75cae915..5bcfc80a6 100644 --- a/docs/eventing/channels/README.md +++ b/docs/eventing/channels/README.md @@ -1,9 +1,4 @@ -Channels are Kubernetes Custom Resources that define a single event forwarding -and persistence layer in Knative. Messaging implementations provide -implementations of Channels via the -[ClusterChannelProvisioner](https://github.com/knative/eventing/blob/master/pkg/apis/eventing/v1alpha1/cluster_channel_provisioner_types.go#L35) -object, and support different technologies, such as Apache Kafka or NATS +Channels are Kubernetes [Custom Resources]((https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)) which define a single event forwarding +and persistence layer. Messaging implementations may provide implementations of +Channels via a Kubernetes Custom Resource, supporting different technologies, such as Apache Kafka or NATS Streaming. - -Note: Cluster Channel Provisioner (CCP) has been deprecated and will be -unsupported in v0.9. You should now use the Channels CRDs. diff --git a/docs/eventing/channels/channels-crds.md b/docs/eventing/channels/channels-crds.md index 29f14117f..4f384e8cb 100644 --- a/docs/eventing/channels/channels-crds.md +++ b/docs/eventing/channels/channels-crds.md @@ -21,18 +21,14 @@ This is a non-exhaustive list of the available Channels for Knative Eventing. Notes: -- Inclusion in this list is not an endorsement, nor does it imply any level of +* Inclusion in this list is not an endorsement, nor does it imply any level of support. -- Cluster Channel Provisioner (CCP) has been deprecated and will be unsupported - in v0.9. You should now use the Channels CRDs. +Name | Status | Support | Description +--- | --- | --- | --- +[GCP PubSub](https://github.com/google/knative-gcp) | Proof of Concept | None | Channels are backed by [GCP PubSub](https://cloud.google.com/pubsub/). +[InMemoryChannel](https://github.com/knative/eventing/tree/master/config/channels/in-memory-channel) | Proof of Concept | None | In-memory channels are a best effort Channel. They should NOT be used in Production. They are useful for development. +[KafkaChannel](https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config) | Proof of Concept | None | Channels are backed by [Apache Kafka](http://kafka.apache.org/) topics. +[NatssChannel](https://github.com/knative/eventing/tree/master/contrib/natss/config) | Proof of Concept | None | Channels are backed by [NATS Streaming](https://github.com/nats-io/nats-streaming-server#configuring). + -| Name | Status | Support | Description | -| -------------------------------------------------------------------------------------------------------------- | ---------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | -| [CCP - Apache Kafka](https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config/provisioner) | Proof of Concept | None | Deprecated: Channels are backed by [Apache Kafka](http://kafka.apache.org/) topics. | -| [CCP - GCP PubSub](https://github.com/knative/eventing/tree/master/contrib/gcppubsub/config) | Proof of Concept | None | Deprecated: Channels are backed by [GCP PubSub](https://cloud.google.com/pubsub/). | -| [CCP - In-Memory](https://github.com/knative/eventing/tree/v0.8.0/config/provisioners/in-memory-channel) | Proof of Concept | None | Deprecated: In-memory channels are a best effort Channel. They should NOT be used in Production. They are useful for development. | -| [CCP - Natss](https://github.com/knative/eventing/tree/master/contrib/natss/config/provisioner) | Proof of Concept | None | Deprecated: Channels are backed by [NATS Streaming](https://github.com/nats-io/nats-streaming-server#configuring). | -| [CRD - InMemoryChannel](https://github.com/knative/eventing/tree/master/config/channels/in-memory-channel) | Proof of Concept | None | In-memory channels are a best effort Channel. They should NOT be used in Production. They are useful for development. | -| [CRD - KafkaChannel](https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config) | Proof of Concept | None | Channels are backed by [Apache Kafka](http://kafka.apache.org/) topics. | -| [CRD - NatssChannel](https://github.com/knative/eventing/tree/master/contrib/natss/config) | Proof of Concept | None | Channels are backed by [NATS Streaming](https://github.com/nats-io/nats-streaming-server#configuring). | diff --git a/docs/eventing/channels/channels.yaml b/docs/eventing/channels/channels.yaml index 98d7ca1c0..4a2de1d30 100644 --- a/docs/eventing/channels/channels.yaml +++ b/docs/eventing/channels/channels.yaml @@ -1,44 +1,26 @@ -# List of available Channel implementation for persistence of the events associated with a given channel +# List of available Channel implementation for persistence of the events associated with a given channel channels: - - name: In-Memory - url: https://github.com/knative/eventing/tree/v0.8.0/config/provisioners/in-memory-channel - status: Proof of Concept - support: None - description: > - Deprecated: In-memory channels are a best effort Channel. They should NOT be used in Production. They are useful for development. - - name: CCP - Apache Kafka - url: https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config/provisioner - status: Proof of Concept - support: None - description: > - Deprecated: Channels are backed by [Apache Kafka](http://kafka.apache.org/) topics. - - name: CCP - GCP PubSub - url: https://github.com/knative/eventing/tree/master/contrib/gcppubsub/config - status: Proof of Concept - support: None - description: > - Deprecated: Channels are backed by [GCP PubSub](https://cloud.google.com/pubsub/). - - name: CCP - Natss - url: https://github.com/knative/eventing/tree/master/contrib/natss/config/provisioner - status: Proof of Concept - support: None - description: > - Deprecated: Channels are backed by [NATS Streaming](https://github.com/nats-io/nats-streaming-server#configuring). - - name: CRD - InMemoryChannel + - name: InMemoryChannel url: https://github.com/knative/eventing/tree/master/config/channels/in-memory-channel status: Proof of Concept support: None description: > In-memory channels are a best effort Channel. They should NOT be used in Production. They are useful for development. - - name: CRD - KafkaChannel + - name: KafkaChannel url: https://github.com/knative/eventing-contrib/tree/master/kafka/channel/config status: Proof of Concept support: None description: > Channels are backed by [Apache Kafka](http://kafka.apache.org/) topics. - - name: CRD - NatssChannel + - name: NatssChannel url: https://github.com/knative/eventing/tree/master/contrib/natss/config status: Proof of Concept support: None description: > Channels are backed by [NATS Streaming](https://github.com/nats-io/nats-streaming-server#configuring). + - name: GCP PubSub + url: https://github.com/google/knative-gcp + status: Proof of Concept + support: None + description: > + Channels are backed by [GCP PubSub](https://cloud.google.com/pubsub/). diff --git a/docs/eventing/channels/generator/ReadmeTemplate.gomd b/docs/eventing/channels/generator/ReadmeTemplate.gomd index 0240ce996..e63e0289f 100644 --- a/docs/eventing/channels/generator/ReadmeTemplate.gomd +++ b/docs/eventing/channels/generator/ReadmeTemplate.gomd @@ -29,9 +29,6 @@ Notes: * Inclusion in this list is not an endorsement, nor does it imply any level of support. -* Cluster Channel Provisioner (CCP) has been deprecated and will be - unsupported in v0.9. You should now use the Channels CRDs. - Name | Status | Support | Description --- | --- | --- | --- {{ range .Channels -}} diff --git a/docs/eventing/debugging/README.md b/docs/eventing/debugging/README.md index 7de973f7f..50f3e6060 100644 --- a/docs/eventing/debugging/README.md +++ b/docs/eventing/debugging/README.md @@ -16,7 +16,7 @@ know roughly how things fit together. ## Version -This Debugging content supports version v0.3.0 or later of +This Debugging content supports version v0.8.0 or later of [Knative Eventing](https://github.com/knative/eventing/releases/) and the [Eventing-contrib resources](https://github.com/knative/eventing-contrib/releases/). @@ -90,7 +90,7 @@ We will attempt to determine why from the most basic pieces out: 1. `fn` - The `Deployment` has no dependencies inside Knative. 1. `svc` - The `Service` has no dependencies inside Knative. -1. `chan` - The `Channel` depends on its backing `ClusterChannelProvisioner` and +1. `chan` - The `Channel` depends on its backing `channel implementation` and somewhat depends on `sub`. 1. `src` - The `Source` depends on `chan`. 1. `sub` - The `Subscription` depends on both `chan` and `svc`. @@ -144,18 +144,18 @@ This should return a single Pod, which if you inspect is the one generated by ##### `chan` `chan` uses the -[`in-memory-channel`](https://github.com/knative/eventing/tree/v0.8.0/config/provisioners/in-memory-channel) -as its `ClusterChannelProvisioner`. This is a very basic provisioner and has few +[`in-memory-channel`]( https://github.com/knative/eventing/tree/master/config/channels/in-memory-channel). +This is a very basic channel and has few failure modes that will be exhibited in `chan`'s `status`. ```shell -kubectl --namespace knative-debug get channel chan -o jsonpath='{.status.conditions[?(@.type == "Ready")].status}' +kubectl --namespace knative-debug get channel.messaging.knative.dev chan -o jsonpath='{.status.conditions[?(@.type == "Ready")].status}' ``` This should return `True`. If it doesn't, get the full resource: ```shell -kubectl --namespace knative-debug get channel chan --output yaml +kubectl --namespace knative-debug get channel.messaging.knative.dev chan --output yaml ``` If `status` is completely missing, it implies that something is wrong with the @@ -164,7 +164,7 @@ If `status` is completely missing, it implies that something is wrong with the Next verify that `chan` is addressable: ```shell -kubectl --namespace knative-debug get channel chan -o jsonpath='{.status.address.hostname}' +kubectl --namespace knative-debug get channel.messaging.knative.dev chan -o jsonpath='{.status.address.hostname}' ``` This should return a URI, likely ending in '.cluster.local'. If it doesn't, then @@ -179,7 +179,7 @@ We will verify that the two resources that the `chan` creates exist and are `chan` creates a K8s `Service`. ```shell -kubectl --namespace knative-debug get service -l provisioner=in-memory-channel,channel=chan +kubectl --namespace knative-debug get service -l messaging.knative.dev/role=in-memory-channel ``` It's spec is completely unimportant, as Istio will ignore it. It just needs to @@ -187,40 +187,19 @@ exist so that `src` can send events to it. If it doesn't exist, it implies that something went wrong during `chan` reconciliation. See [Channel Controller](#channel-controller). -###### `VirtualService` - -`chan` creates a `VirtualService` which redirects its hostname to the -`in-memory-channel` dispatcher. - -```shell -kubectl --namespace knative-debug get virtualservice -l provisioner=in-memory-channel,channel=chan -o custom-columns='HOST:.spec.hosts[0],DESTINATION:.spec.http[0].route[0].destination.host' -``` - -Verify that - -1. `HOST` is the same as the hostname returned by in `chan`'s - `status.address.hostname`. -1. `DESTINATION` is - `in-memory-channel-dispatcher.knative-eventing.svc.cluster.local`. - -If either of those is not accurate, then it implies that something went wrong -during `chan` reconciliation. See [Channel Controller](#channel-controller). - ##### `src` `src` is a -[`KubernetesEventSource`](https://github.com/knative/eventing-contrib/blob/master/pkg/apis/sources/v1alpha1/kuberneteseventsource_types.go), -which creates an underlying -[`ContainerSource`](https://github.com/knative/eventing/blob/master/pkg/apis/sources/v1alpha1/containersource_types.go). +[`ApiServerSource`](https://github.com/knative/eventing/blob/master/pkg/apis/sources/v1alpha1/apiserver_types.go). First we will verify that `src` is writing to `chan`. ```shell -kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.spec.sink}' +kubectl --namespace knative-debug get apiserversource src -o jsonpath='{.spec.sink}' ``` Which should return -`map[apiVersion:eventing.knative.dev/v1alpha1 kind:Channel name:chan]`. If it +`map[apiVersion:messaging.knative.dev/v1alpha1 kind:Channel name:chan]`. If it doesn't, then `src` was setup incorrectly and its `spec` needs to be fixed. Fixing should be as simple as updating its `spec` to have the correct `sink` (see [example.yaml](example.yaml)). @@ -228,63 +207,9 @@ Fixing should be as simple as updating its `spec` to have the correct `sink` Now that we know `src` is sending to `chan`, let's verify that it is `Ready`. ```shell -kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.status.conditions[?(.type == "Ready")].status}' +kubectl --namespace knative-debug get apiserversource src -o jsonpath='{.status.conditions[?(.type == "Ready")].status}' ``` -This should return `True`. If it doesn't, then we need to investigate why. First -we will look at the owned `ContainerSource` that underlies `src`, and if that is -not fruitful, look at the [Source Controller](#source-controller). - -##### ContainerSource - -`src` is backed by a `ContainerSource` resource. - -Is the `ContainerSource` `Ready`? - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[0].uid == '$srcUID')].status.conditions[?(.type == 'Ready')].status}" -``` - -That should be `True`. If it is, but `src` is not `Ready`, then that implies the -problem is in the [Source Controller](#source-controller). - -If `ContainerSource` is not `Ready`, then we need to look at its entire -`status`: - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -containerSourceName=$(kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$srcUID')].metadata.name}") -kubectl --namespace knative-debug get containersource $containerSourceName --output yaml -``` - -The most useful condition (when `Ready` is not `True`), is `Deployed`. - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -containerSourceName=$(kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$srcUID')].metadata.name}") -kubectl --namespace knative-debug get containersource $containerSourceName -o jsonpath='{.status.conditions[?(.type == "Deployed")].message}' -``` - -You should see something like `Updated deployment src-xz59f-hmtkp`. Let's see -the health of the `Deployment` that `ContainerSource` created (named in the -message, but we will get it directly in the following command): - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -containerSourceUID=$(kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$srcUID')].metadata.uid}") -deploymentName=$(kubectl --namespace knative-debug get deployment -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$containerSourceUID')].metadata.name}") -kubectl --namespace knative-debug get deployment $deploymentName --output yaml -``` - -If this is unhealthy, then it should tell you why. E.g. -`'pods "src-xz59f-hmtkp-7bd4bc6964-" is forbidden: error looking up service account knative-debug/events-sa: serviceaccount "events-sa" not found'`. -Fix any errors so that it the `Deployment` is healthy. - -If the `Deployment` is healthy, but the `ContainerSource` isn't, that implies -something went wrong in -[ContainerSource Controller](#containersource-controller). - #### `sub` `sub` is a `Subscription` from `chan` to `fn`. @@ -319,18 +244,18 @@ document. ##### Channel Controller -There is not a single `Channel` Controller. Instead, there is a single -Controller for each `ClusterChannelProvisioner`. `chan` uses the -`in-memory-channel` `ClusterChannelProvisioner`, whose Controller is: +There is not a single `Channel` Controller. Instead, there is one +Controller for each Channel CRD. `chan` uses the +`InMemoryChannel` `Channel CRD`, whose Controller is: ```shell -kubectl --namespace knative-eventing get pod -l clusterChannelProvisioner=in-memory-channel,role=controller --output yaml +kubectl --namespace knative-eventing get pod -l messaging.knative.dev/channel=in-memory-channel,messaging.knative.dev/role=controller --output yaml ``` See its logs with: ```shell -kubectl --namespace knative-eventing logs -l clusterChannelProvisioner=in-memory-channel,role=controller +kubectl --namespace knative-eventing logs -l messaging.knative.dev/channel=in-memory-channel,messaging.knative.dev/role=controller ``` Pay particular attention to any lines that have a logging level of `warning` or @@ -338,39 +263,29 @@ Pay particular attention to any lines that have a logging level of `warning` or ##### Source Controller -Each Source will have its own Controller. `src` is a `KubernetesEventSource`, so +Each Source will have its own Controller. `src` is a `ApiServerSource`, so its Controller is: -```shell -kubectl --namespace knative-sources get pod -l control-plane=controller-manager -``` - -This is actually a single binary that runs multiple Source Controllers, -importantly including [ContainerSource Controller](#containersource-controller). - -The `KubernetesEventSource` is fairly simple, as it delegates all functionality -to an underlying [ContainerSource](#containersource), so there is likely no -useful information in its logs. Instead more useful information is likely in the -[ContainerSource Controller](#containersource-controller)'s logs. If you want to -look at `KubernetesEventSource` Controller's logs anyway, they can be see with: - -```shell -kubectl --namespace knative-sources logs -l control-plane=controller-manager -``` - -###### ContainerSource Controller - -The `ContainerSource` Controller is run in the same binary as some other Source -Controllers from Eventing. It is: - ```shell kubectl --namespace knative-eventing get pod -l app=sources-controller ``` +This is actually a single binary that runs multiple Source Controllers, +importantly including [ApiServerSource Controller](#apiserversource-controller). + +###### ApiServerSource Controller + +The `ApiServerSource` Controller is run in the same binary as some other Source +Controllers from Eventing. It is: + +```shell +kubectl --namespace knative-debug get pod -l eventing.knative.dev/sourceName=src,eventing.knative.dev/source=apiserver-source-controller +``` + View its logs with: ```shell -kubectl --namespace knative-eventing logs -l app=sources-controller +kubectl --namespace knative-debug logs -l eventing.knative.dev/sourceName=src,eventing.knative.dev/source=apiserver-source-controller ``` Pay particular attention to any lines that have a logging level of `warning` or @@ -409,13 +324,7 @@ The Knative event takes the following path: (from nothing). 1. `src` is POSTing the event to `chan`'s address, - `chan-channel-45k5h.knative-debug.svc.cluster.local`. - -1. `src`'s Istio proxy is intercepting the request, seeing that the Host matches - a `VirtualService`. The request's Host is rewritten to - `chan.knative-debug.channels.cluster.local` and sent to the - [Channel Dispatcher](#channel-dispatcher), - `in-memory-channel-dispatcher.knative-eventing.svc.cluster.local`. + `http://chan-kn-channel.knative-debug.svc.cluster.local`. 1. The Channel Dispatcher receives the request and introspects the Host header to determine which `Channel` it corresponds to. It sees that it corresponds @@ -426,106 +335,6 @@ The Knative event takes the following path: We will investigate components in the order in which events should travel. -#### `src` - -Events should be generated at `src`. First let's look at the `Pod`s logs: - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -containerSourceName=$(kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$srcUID')].metadata.name}") -kubectl --namespace knative-debug logs -l source=$containerSourceName -c source -``` - -Note that a few log lines within the first ~15 seconds of the `Pod` starting -like the following are fine. They represent the time waiting for the Istio proxy -to start. If you see these more than a few seconds after the `Pod` starts, then -something is wrong. - -```shell -E0116 23:59:40.033667 1 reflector.go:205] github.com/knative/eventing-contrib/pkg/adapter/kubernetesevents/adapter.go:73: Failed to list *v1.Event: Get https://10.51.240.1:443/api/v1/namespaces/kna tive-debug/events?limit=500&resourceVersion=0: dial tcp 10.51.240.1:443: connect: connection refused -E0116 23:59:41.034572 1 reflector.go:205] github.com/knative/eventing-contrib/pkg/adapter/kubernetesevents/adapter.go:73: Failed to list *v1.Event: Get https://10.51.240.1:443/api/v1/namespaces/kna tive-debug/events?limit=500&resourceVersion=0: dial tcp 10.51.240.1:443: connect: connection refused -``` - -The success message is `debug` level, so we don't expect to see anything. If you -see lines with a logging level of `error`, look at their `msg`. For example: - -```shell -"msg":"[404] unexpected response \"\"" -``` - -Which means that `src` correctly got the Kubernetes `Event` and tried to send it -to `chan`, but failed to do so. In this case, the response code was a 404. We -will look at the Istio proxy's logs to see if we can get any further -information: - -```shell -srcUID=$(kubectl --namespace knative-debug get kuberneteseventsource src -o jsonpath='{.metadata.uid}') -containerSourceName=$(kubectl --namespace knative-debug get containersource -o jsonpath="{.items[?(.metadata.ownerReferences[*].uid == '$srcUID')].metadata.name}") -kubectl --namespace knative-debug logs -l source=$containerSourceName -c istio-proxy -``` - -We see lines like: - -```shell -[2019-01-17T17:16:11.898Z] "POST / HTTP/1.1" 404 NR 0 0 0 - "-" "Go-http-client/1.1" "4702a818-11e3-9e15-b523-277b94598101" "chan-channel-45k5h.knative-debug.svc.cluster.local" "-" -``` - -These are lines emitted by [Envoy](https://www.envoyproxy.io). The line is -documented as Envoy's -[Access Logging](https://www.envoyproxy.io/docs/envoy/latest/configuration/access_log). -That's odd, we already verified that there is a -[`VirtualService`](#virtualservice) for `chan`. In fact, we don't expect to see -`chan-channel-45k5h.knative-debug.svc.cluster.local` at all, it should be -replaced with `chan.knative-debug.channels.cluster.local`. We keep looking in -the same Istio proxy logs and see: - -```shell - [2019-01-16 23:59:41.408][23][warning][config] bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_mux_subscription_lib/common/config/grpc_mux_subscription_impl.h:70] gRPC config for type.googleapis.com/envoy.api.v2.RouteConfiguration rejected: Only unique values for domains are permitted. Duplicate entry of domain chan.knative-debug.channels.cluster.local -``` - -This shows that the [`VirtualService`](#virtualservice) created for `chan`, -which tries to map two hosts, -`chan-channel-45k5h.knative-debug.svc.cluster.local` and -`chan.knative-debug.channels.cluster.local`, is not working. The most likely -cause is duplicate `VirtualService`s that all try to rewrite those hosts. Look -at all the `VirtualService`s in the namespace and see what hosts they rewrite: - -```shell -kubectl --namespace knative-debug get virtualservice -o custom-columns='NAME:.metadata.name,HOST:.spec.hosts[*]' -``` - -In this case, we see: - -```shell -NAME HOST -chan-channel-38x5a chan-channel-45k5h.knative-debug.svc.cluster.local,chan.knative-debug.channels.cluster.local -chan-channel-8dc2x chan-channel-45k5h.knative-debug.svc.cluster.local,chan.knative-debug.channels.cluster.local -``` - -``` -Note: This shouldn't happen normally. It only happened here because I had local edits to the Channel controller and created a bug. If you see this with any released Channel Controllers, open a bug with all relevant information (Channel Controller info and YAML of all the VirtualServices). -``` - -Both are owned by `chan`. Deleting both, causes the -[Channel Controller](#channel-controller) to recreate the correct one. After -deleting both, a single new one is created (same command as above): - -```shell -NAME HOST -chan-channel-9kbr8 chan-channel-45k5h.knative-debug.svc.cluster.local,chan.knative-debug.channels.cluster.local -``` - -After [forcing a Kubernetes event to occur](#triggering-events), the Istio proxy -logs now have: - -```shell -[2019-01-17T18:04:07.571Z] "POST / HTTP/1.1" 202 - 795 0 1 1 "-" "Go-http-client/1.1" "ba36be7e-4fc4-9f26-83bd-b1438db730b0" "chan.knative-debug.channels.cluster.local" "10.48.1.94:8080" -``` - -Which looks correct. Most importantly, the return code is now 202 Accepted. In -addition, the request's Host is being correctly rewritten to -`chan.knative-debug.channels.cluster.local`. - #### Channel Dispatcher The Channel Dispatcher is the component that receives POSTs pushing events into @@ -537,14 +346,15 @@ binary that handles both the receiving and dispatching sides for all First we will inspect the Dispatcher's logs to see if it is anything obvious: ```shell -kubectl --namespace knative-eventing logs -l clusterChannelProvisioner=in-memory-channel,role=dispatcher -c dispatcher +kubectl --namespace knative-eventing logs -l messaging.knative.dev/channel=in-memory-channel,messaging.knative.dev/role=dispatcher -c dispatcher ``` Ideally we will see lines like: ```shell -{"level":"info","ts":1547752472.9581263,"caller":"provisioners/message_receiver.go:116","msg":"Received request for chan.knative-debug.channels.cluster.local"} -{"level":"info","ts":1547752472.9582398,"caller":"provisioners/message_dispatcher.go:106","msg":"Dispatching message to http://svc.knative-debug.svc.cluster.local/"} +{"level":"info","ts":"2019-08-16T13:50:55.424Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_receiver.go:147","msg":"Request mapped to channel: knative-debug/chan-kn-channel","knative.dev/controller":"in-memory-channel-dispatcher"} +{"level":"info","ts":"2019-08-16T13:50:55.425Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_dispatcher.go:112","msg":"Dispatching message to http://svc.knative-debug.svc.cluster.local/","knative.dev/controller":"in-memory-channel-dispatcher"} +{"level":"info","ts":"2019-08-16T13:50:55.981Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_receiver.go:140","msg":"Received request for chan-kn-channel.knative-debug.svc.cluster.local","knative.dev/controller":"in-memory-channel-dispatcher"} ``` Which shows that the request is being received and then sent to `svc`, which is @@ -552,10 +362,15 @@ returning a 2XX response code (likely 200, 202, or 204). However if we see something like: + ```shell -{"level":"info","ts":1547752478.5898774,"caller":"provisioners/message_receiver.go:116","msg":"Received request for chan.knative-debug.channels.cluster.local"} -{"level":"info","ts":1547752478.58999,"caller":"provisioners/message_dispatcher.go:106","msg":"Dispatching message to http://svc.knative-debug.svc.cluster.local/"} -{"level":"error","ts":1547752478.6035335,"caller":"fanout/fanout_handler.go:108","msg":"Fanout had an error","error":"Unable to complete request Post http://svc.knative-debug.svc.cluster.local/: EOF","stacktrace":"github.com/knative/eventing/pkg/sidecar/fanout.(*Handler).dispatch\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/sidecar/fanout/fanout_handler.go:108\ngithub.com/knative/eventing/pkg/sidecar/fanout.createReceiverFunction.func1\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/sidecar/fanout/fanout_handler.go:86\ngithub.com/knative/eventing/pkg/provisioners.(*MessageReceiver).HandleRequest\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/provisioners/message_receiver.go:132\ngithub.com/knative/eventing/pkg/sidecar/fanout.(*Handler).ServeHTTP\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/sidecar/fanout/fanout_handler.go:91\ngithub.com/knative/eventing/pkg/sidecar/multichannelfanout.(*Handler).ServeHTTP\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/sidecar/multichannelfanout/multi_channel_fanout_handler.go:128\ngithub.com/knative/eventing/pkg/sidecar/swappable.(*Handler).ServeHTTP\n\t/usr/local/google/home/harwayne/go/src/github.com/knative/eventing/pkg/sidecar/swappable/swappable.go:105\nnet/http.serverHandler.ServeHTTP\n\t/usr/lib/google-golang/src/net/http/server.go:2740\nnet/http.(*conn).serve\n\t/usr/lib/google-golang/src/net/http/server.go:1846"} +{"level":"info","ts":"2019-08-16T16:10:16.859Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_receiver.go:140","msg":"Received request for chan-kn-channel.knative-debug.svc.cluster.local","knative.dev/controller":"in-memory-channel-dispatcher"} +{"level":"info","ts":"2019-08-16T16:10:16.859Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_receiver.go:147","msg":"Request mapped to channel: knative-debug/chan-kn-channel","knative.dev/controller":"in-memory-channel-dispatcher"} +{"level":"info","ts":"2019-08-16T16:10:16.859Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"provisioners/message_dispatcher.go:112","msg":"Dispatching message to http://svc.knative-debug.svc.cluster.local/","knative.dev/controller":"in-memory-channel-dispatcher"} +{"level":"error","ts":"2019-08-16T16:10:38.169Z","logger":"inmemorychannel-dispatcher.in-memory-channel-dispatcher","caller":"fanout/fanout_handler.go:121","msg":"Fanout had an error","knative.dev/controller":"in-memory-channel-dispatcher","error":"Unable to complete request Post http://svc.knative-debug.svc.cluster.local/: dial tcp 10.4.44.156:80: i/o timeout","stacktrace":"knative.dev/eventing/pkg/provisioners/fanout.(*Handler).dispatch\n\t/Users/xxxxxx/go/src/knative.dev/eventing/pkg/provisioners/fanout/fanout_handler.go:121\nknative.dev/eventing/pkg/provisioners/fanout.createReceiverFunction.func1.1\n\t/Users/i512777/go/src/knative.dev/eventing/pkg/provisioners/fanout/fanout_handler.go:95"} ``` Then we know there was a problem posting to @@ -568,10 +383,4 @@ events about failures. TODO Fill in this section. -See `fn`'s Istio proxy logs: - -```shell -kubectl --namespace knative-debug logs -l app=fn -c istio-proxy -``` - # TODO Finish the guide. diff --git a/docs/eventing/debugging/example.yaml b/docs/eventing/debugging/example.yaml index de63a4aed..3f4c91be1 100644 --- a/docs/eventing/debugging/example.yaml +++ b/docs/eventing/debugging/example.yaml @@ -12,36 +12,33 @@ metadata: # The event source. apiVersion: sources.eventing.knative.dev/v1alpha1 -kind: KubernetesEventSource +kind: ApiServerSource metadata: name: src namespace: knative-debug spec: - namespace: knative-debug serviceAccountName: service-account + mode: Resource + resources: + - apiVersion: v1 + kind: Event sink: - apiVersion: eventing.knative.dev/v1alpha1 - kind: Channel + apiVersion: messaging.knative.dev/v1alpha1 + kind: InMemoryChannel name: chan --- # The Channel events are sent to. -apiVersion: eventing.knative.dev/v1alpha1 -kind: Channel +apiVersion: messaging.knative.dev/v1alpha1 +kind: InMemoryChannel metadata: name: chan namespace: knative-debug -spec: - provisioner: - apiVersion: eventing.knative.dev/v1alpha1 - kind: ClusterChannelProvisioner - name: in-memory-channel - --- -# The Subscription to the Channel. +# The Subscription to the InMemoryChannel. apiVersion: eventing.knative.dev/v1alpha1 kind: Subscription @@ -50,8 +47,8 @@ metadata: namespace: knative-debug spec: channel: - apiVersion: eventing.knative.dev/v1alpha1 - kind: Channel + apiVersion: messaging.knative.dev/v1alpha1 + kind: InMemoryChannel name: chan subscriber: ref: @@ -92,8 +89,6 @@ spec: app: fn template: metadata: - annotations: - sidecar.istio.io/inject: "true" labels: app: fn spec: diff --git a/docs/eventing/samples/apache-camel-source/README.md b/docs/eventing/samples/apache-camel-source/README.md index be848ceda..eed12a53d 100644 --- a/docs/eventing/samples/apache-camel-source/README.md +++ b/docs/eventing/samples/apache-camel-source/README.md @@ -118,7 +118,7 @@ kubectl delete camelsource camel-timer-source Install the [telegram CamelSource](source_telegram.yaml) from source: ```shell -kunbectl apply -f source_telegram.yaml +kubectl apply -f source_telegram.yaml ``` Start `kail` again and keep it open on the event display: diff --git a/docs/eventing/samples/apache-camel-source/display_resources.yaml b/docs/eventing/samples/apache-camel-source/display_resources.yaml index 539feaf8f..6dd13fd14 100644 --- a/docs/eventing/samples/apache-camel-source/display_resources.yaml +++ b/docs/eventing/samples/apache-camel-source/display_resources.yaml @@ -1,14 +1,13 @@ # Channel for testing events. -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: messaging.knative.dev/v1alpha1 kind: Channel metadata: name: camel-test spec: - provisioner: - apiVersion: eventing.knative.dev/v1alpha1 - kind: ClusterChannelProvisioner - name: in-memory-channel + channelTemplate: + apiVersion: messaging.knative.dev/v1alpha1 + kind: InMemoryChannel --- @@ -20,7 +19,7 @@ metadata: name: camel-source-display spec: channel: - apiVersion: eventing.knative.dev/v1alpha1 + apiVersion: messaging.knative.dev/v1alpha1 kind: Channel name: camel-test subscriber: diff --git a/docs/eventing/samples/apache-camel-source/source_camel_k.yaml b/docs/eventing/samples/apache-camel-source/source_camel_k.yaml index 13d0401b8..0faade924 100644 --- a/docs/eventing/samples/apache-camel-source/source_camel_k.yaml +++ b/docs/eventing/samples/apache-camel-source/source_camel_k.yaml @@ -19,6 +19,6 @@ spec: .setHeader("Content-Type").constant("application/json") .to("knative:endpoint/sink?cloudEventsType=org.apache.camel.quote") sink: - apiVersion: eventing.knative.dev/v1alpha1 + apiVersion: messaging.knative.dev/v1alpha1 kind: Channel name: camel-test diff --git a/docs/eventing/samples/apache-camel-source/source_telegram.yaml b/docs/eventing/samples/apache-camel-source/source_telegram.yaml index 7a54993d3..f040fb57f 100644 --- a/docs/eventing/samples/apache-camel-source/source_telegram.yaml +++ b/docs/eventing/samples/apache-camel-source/source_telegram.yaml @@ -19,6 +19,6 @@ spec: # Camel K option to enable serialization of the component output camel.component.knative.jsonSerializationEnabled: "true" sink: - apiVersion: eventing.knative.dev/v1alpha1 + apiVersion: messaging.knative.dev/v1alpha1 kind: Channel name: camel-test diff --git a/docs/eventing/samples/apache-camel-source/source_timer.yaml b/docs/eventing/samples/apache-camel-source/source_timer.yaml index b0c347b4b..9ef339130 100644 --- a/docs/eventing/samples/apache-camel-source/source_timer.yaml +++ b/docs/eventing/samples/apache-camel-source/source_timer.yaml @@ -14,6 +14,6 @@ spec: # Using 'period' URI option (see component documentation) uri: timer:tick?period=3s sink: - apiVersion: eventing.knative.dev/v1alpha1 + apiVersion: messaging.knative.dev/v1alpha1 kind: Channel name: camel-test diff --git a/docs/install/Knative-custom-install.md b/docs/install/Knative-custom-install.md index 101670097..fa5a66f7e 100644 --- a/docs/install/Knative-custom-install.md +++ b/docs/install/Knative-custom-install.md @@ -115,17 +115,17 @@ files from the Knative repositories: | **knative/eventing** | | | | [`release.yaml`][4.1]† | Installs the Eventing component. Includes [ContainerSource](../eventing#containersource), [CronJobSource][6.2], InMemoryChannel. | | | [`eventing.yaml`][4.2] | Installs the Eventing component. Includes [ContainerSource](../eventing#containersource) and [CronJobSource][6.2]. Does not include any Channel. | | -| [`in-memory-channel-crd.yaml`][4.3] | Installs only the InMemoryChannel. | Eventing component | -| [`natss.yaml`][4.5] | Installs only the NATSS channel provisioner. | Eventing component | -| [`gcp-pubsub.yaml`][4.6] | Installs only the GCP PubSub channel provisioner. | Eventing component | +| [`in-memory-channel-crd.yaml`][4.3] | Installs only the InMemoryChannel. | Eventing component | | **knative/eventing-contrib** | | | | [`github.yaml`][5.10]† | Installs the [GitHub][6.10] source. | Eventing component | | [`camel.yaml`][5.40] | Installs the Apache Camel source. | Eventing component | -| [`gcppubsub.yaml`][5.20] | Installs the [GCP PubSub source][6.30] | Eventing component | -| [`kafka.yaml`][5.50] | Installs the Apache Kafka source. | Eventing component | -| [`kafka-channel.yaml`][5.60] | Installs the KafkaChannel. | Eventing component | +| [`kafka-importer.yaml`][5.50] | Installs the Apache Kafka source. | Eventing component | +| [`kafka-channel.yaml`][5.60] | Installs the KafkaChannel. | Eventing component | | [`awssqs.yaml`][5.70] | Installs the AWS SQS source. | Eventing component | | [`event-display.yaml`][5.30] | Installs a Knative Service that logs events received for use in samples and debugging. | Serving component, Eventing component | +| [`natss-channel.yaml`][5.80] | Installs the NATS streaming channel implementation. | Eventing component | +| **knative/google/knative-gcp** | | | +| [`cloud-run-events.yaml`][7.10] | Installs the GCP PubSub channel implementation | | _\*_ See [Installing logging, metrics, and traces](../serving/installing-logging-metrics-traces.md) @@ -165,28 +165,26 @@ for details about installing the various supported observability plugins. [4.30]: https://github.com/knative/eventing/releases/download/{{< version >}}/in-memory-channel-crd.yaml [4.40]: https://github.com/knative/eventing/releases/download/{{< version >}}/kafka.yaml -[4.50]: https://github.com/knative/eventing/releases/download/{{< version >}}/natss.yaml -[4.60]: - https://github.com/knative/eventing/releases/download/{{< version >}}/gcp-pubsub.yaml [5.0]: https://github.com/knative/eventing-contrib/releases/tag/{{< version >}} [5.10]: https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/github.yaml -[5.20]: - https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/gcppubsub.yaml [5.30]: https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/event-display.yaml [5.40]: https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/camel.yaml [5.50]: - https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/kafka.yaml + https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/kafka-importer.yaml [5.60]: https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/kafka-channel.yaml [5.70]: https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/awssqs.yaml +[5.80]: + https://github.com/knative/eventing-contrib/releases/download/{{< version >}}/natss.yaml [6.10]: https://developer.github.com/v3/activity/events/types/ [6.20]: https://github.com/knative/eventing-contrib/blob/master/samples/cronjob-source/README.md -[6.30]: https://cloud.google.com/pubsub/ +[7.0]: https://github.com/google/knative-gcp/releases/tag/{{< version >}} +[7.10]: https://github.com/google/knative-gcp/releases/download/{{< version >}}/cloud-run-events.yaml ### Installing Knative