diff --git a/serving/accessing-logs.md b/serving/accessing-logs.md index 8901f5eae..5bee22f0c 100644 --- a/serving/accessing-logs.md +++ b/serving/accessing-logs.md @@ -35,6 +35,23 @@ up the necessary components first. +### Accessing stdout/stderr logs + +To find the logs sent to `stdout/stderr` from your application in the Kibana UI: + +1. Click `Discover` on the left side bar. +1. Choose `logstash-*` index pattern on the left top. +1. Input `tag: kubernetes*` in the top search bar then search. + +### Accessing request logs + +To find the request logs of your application in the Kibana UI : + +1. Click `Discover` on the left side bar. +1. Choose `logstash-*` index pattern on the left top. +1. Input `tag: "requestlog.logentry.istio-system"` in the top search bar then + search. + ### Accessing configuration and revision logs To access the logs for a configuration: diff --git a/serving/debugging-application-issues.md b/serving/debugging-application-issues.md index c544e6550..9d3cb35c9 100644 --- a/serving/debugging-application-issues.md +++ b/serving/debugging-application-issues.md @@ -41,6 +41,72 @@ details, see Knative [Error Conditions and Reporting](https://github.com/knative/serving/blob/master/docs/spec/errors.md)(currently some of them are not implemented yet). +### Check ClusterIngress/Istio routing + +Run the following command to list all the cluster ingress, with their labels + +```shell +kubectl get clusteringress -o=custom-columns='NAME:.metadata.name,LABELS:.metadata.labels' +NAME LABELS +helloworld-go-h5kd4 map[serving.knative.dev/route:helloworld-go serving.knative.dev/routeNamespace:default] +``` + +The labels `serving.knative.dev/route` and `serving.knative.dev/routeNamespace` +will tell exactly which Route a ClusterIngress is a child resource of. Find the +one corresponding to your Route. If a ClusterIngress does not exist, the route +controller believes that the Revisions targeted by your Route/Service isn't +ready. Please proceed to later sections to diagnose Revision readiness status. + +Otherwise, run the following command to look at the ClusterIngress created for +your Route + +``` +kubectl get clusteringress -o yaml +``` + +particularly, look at the `status:` section. If the ClusterIngress is working +correctly, we should see the condition with `type=Ready` to have `status=True`. +Otherwise, there will be error messages. + +Now, if ClusterIngress shows status Ready, there must be a corresponding +VirtualService. Run the following command: + +```shell +kubectl get virtualservice -n knative-serving -o yaml +``` + +the network configuration in VirtualService must match that of ClusterIngress +and Route. VirtualService currently doesn't expose a Status field, so if one +exists and have matching configurations with ClusterIngress and Route, you may +want to wait a little bit for those settings to propagate. + +If you are familar with Istio and `istioctl`, you may try using `istioctl` to +look deeper using Istio +[guide](https://istio.io/help/ops/traffic-management/proxy-cmd/). + +### Check Ingress status + +Before Knative 0.3 we use a LoadBalancer service call `knative-ingressgateway` +to handle ingress. Since Knative 0.3 we now use `istio-ingressgateway` Service. + +To check the IP address of your Ingress, use + +```shell +kubectl get svc -n istio-system istio-ingressgateway +``` + +Or replace that with `knative-ingressgateway` if you are using Knative release +older than 0.3. + +If there is no external IP address, use + +```shell +kubectl describe svc istio-ingressgateway -n istio-system +``` + +to see a reason why IP addresses weren't provisioned. Most likely it is due to a +quota issue. + ## Check Revision status If you configure your `Route` with `Configuration`, run the following command to @@ -73,6 +139,7 @@ If you see this condition, check the following to continue debugging: - [Check Pod status](#check-pod-status) - [Check application logs](#check-application-logs) +- [Check Istio routing](#check-clusteringressistio-routing) If you see other conditions, to debug further: