Commit Graph

110 Commits

Author SHA1 Message Date
Victor Agababov acd3f31714
nits (#1529) 2020-07-20 18:44:35 -07:00
jjzeng-seattle 03151bfc84
Add component name as the metric prefix for opencensus exporter (#1493)
* Add component name as the metric prefix for opencensus exporter

Read from all meters instead of just he default one

Add component name as the metric prefix for opencensus exporter

* Update codegen

* Add domain as metric prefix

* Use path.Join to simply the code
2020-07-20 14:10:36 -07:00
Victor Agababov 25f2aa6443
micro-Nit in the test: don't make a private type public (#1510) 2020-07-16 10:41:33 -07:00
Markus Thömmes cb729b8ae6
Assorted linting fixes. (#1507)
- Remove unused code.
- Remove unnecessary conversion.
- Prevent copies of structs that contain sync.Mutex.
2020-07-16 05:59:33 -07:00
Evan Anderson 9f3fb07346
Remove delay-based flakiness from workqueue_test.go (#1497) 2020-07-15 10:20:33 -07:00
Evan Anderson 723b71fa47
Update Stackdriver custom exporter to support RecordBatch where some records are special and others are not. (#1495) 2020-07-15 10:05:33 -07:00
jjzeng-seattle 858f1a7b0d
Read from all meters instead of just he default one (#1491) 2020-07-14 14:44:32 -07:00
Evan Anderson e7bd895fbd
Introduce `pkg/metrics` support for multiple Resources (#1392)
* Update OpenCensus to include #1196.

* Extend OpenCensus to track per-Resource stats.

* Add generic ResourceView mechanics, part 1. TODO: wire newFactory, tests

* Move factory creation to where it can be customized per-backend

* Integrate opencensus export with Resource-based exporters

* Add stackdriver support for resource views

* address feedbacks

* add test cases

* make KnativeRevision a map

* Update go modules to point to latest opencensus which includes `view.Meter`.

* Add support for Resources to Prometheus Exporter.

This involved copying/improving much of the contrib prometheus exporter.

* add resource_view tests and fix minor issues

* add recorder tests for meter

* Update resource_view_test with an e2e export test.

* Fix prometheus e2e resource_view_test.

Found a fun circular lock.

Found a fun double-registration in metricproducer/Meter.

Found some unnecessary extra code.

Fixed prometheus string sanitization.

Removed a layer of Prometheus wrapping which made debugging more challenging.

* Add test for standard view exports and non-resourced views to e2e test.

* Add UnregisterResourceView to fix tests

* Add OpenCensus e2e resource export test.

Still a little flaky, so work around that with "<=".

* Fix tests for OpenCensus export.

* Fix flakiness / timing issues in OpenCensus exporter e2e test.

* Update to latest ocagent to fix `Resources` bug.

* Don't swallow errors from stackdriver init.

* Get Stackdriver export mostly working in e2e tests

Tests still fail because stackdriver converts custom metrics to a namespaced format and seems to lose Resource information.

* Add support for Stackdriver resource export.

* ./hack/update-codegen.sh

* Fix data race with graceful shutdown.

* Fix issues noticed by @vagababov

* Address mattmoor-bot comments.

* Remove MonitoredResource and apply suggestions from yanweiguo

* More cleanup to stackdriver_exporter_test.go

* Update tests (particularly config_test.go) for Stackdriver recorder changes which sometimes record with additional resources.

* Avoid setting two `view.WithRecorder` options on record

* Add stackdriver tests

* Address comments

* Switch Prometheus port due to possible conflict in test env

* Fix missing :=

* FIx data races around *view.Data shared across Meters.

Cleanup resource_view_test.

* Remove unneeded monitored_resources.go

* Use sync.WaitGroup to avoid data races on closing channel.

* Sort the map for ResourceAsString

* Apply lint and vagababov changes.

* Fix typos

* refactor ResourceAsString

* Update stackdriver exporter version

* Update metrics/resource_view_test.go

Remove accidental paste

* Fix go.sum

* Address yanweiguo comments on comments

* Fix autogen/go mod

* Handling resource extraction when custom metrics are allowed

* Add tests for AllowCustomMetrics set to true

* Address vagababov and yanweiguo comments.

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

* address comments

* Renaming

* Reduce some condition nesting

* go fmt

* Sleep for 1ms to allow metrics being sent (#9)

* Address yanweiguo comments

Co-authored-by: JJ Zeng <jjzeng@google.com>
Co-authored-by: jjzeng-seattle <56568376+jjzeng-seattle@users.noreply.github.com>
2020-07-14 10:46:19 -07:00
Victor Agababov eb05e8dd5b
Various cleanups (#1446)
Thanks @markusthoemmes for the tool :)
2020-06-24 14:04:28 -07:00
Evan Anderson b9a30ee123
Update config_test.go and metricstest to support multiple Meters. (#1421) 2020-06-18 10:50:25 -07:00
Victor Agababov 34c17cc2d5
Make things private that should be so (#1417)
* Make things private that should be so

Make some constants and enums private.
The ones I found to be used elsewhere, I kept public.

* sort
2020-06-17 09:26:24 -07:00
Antoine Cotten ffb929374a
Make Prometheus exporter port configurable via env (#1411) 2020-06-16 16:26:24 -07:00
Antoine Cotten 0d967ff0d8
Clean metrics config tests (#1412)
* Use constants in metrics config tests

* Normalize test cases indentation
2020-06-16 11:09:24 -07:00
jjzeng-seattle e366b8278b
Add a passthrough method RegisterResourceView (#1410)
* Add a passthrough method RegisterResourceView

* gofmt

* Update metrics/exporter.go

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>

* Address comments

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>
2020-06-16 10:58:24 -07:00
Victor Agababov fef70cc261
Unexport some constants (#1393)
* fix the space issue

* merge shenanigans

* nit and tests

* Unexport some constants that don't seem to be used anywhere
2020-06-09 11:40:32 -07:00
Evan Anderson c87d420af5
Don't swallow errors from stackdriver init. (#1367)
* Don't swallow errors from stackdriver init.

* Also check error from StartMetricsExporter.
2020-06-01 10:11:03 -07:00
Dave Protasowski 2d1a04d1ff
bump k8s packages to 1.17.6 (#1285)
* include script to bump k8s deps

* ran ./hack/update-k8s-deps.sh

* fix client-go breaking changes

* refactor update-k8s-deps.sh

* bump to k8s v0.17.6
2020-05-27 10:37:59 -07:00
Markus Thömmes 60f4ae1dbe
Use config map parsing lib and enhance it further. (#1335)
* Use config map parsing lib and enhance it further.

* Unexport test struct values.

* Migrate tracing config too.

* Use sample-rate.
2020-05-18 10:42:06 -07:00
Victor Agababov 1ceb875570
Fix \" escaping and other nits in the pkg (#1339) 2020-05-16 09:36:02 -07:00
Yanwei Guo f1c51d6830
Delete nit check log (#1340)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* delete
2020-05-16 09:00:02 -07:00
Yanwei Guo 16d7b96341
Print nil check log only once (#1334)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* delete

* log once

* remove space
2020-05-14 17:25:00 -07:00
Lionel Villard 40ed633032
add source metrics to report general errors and timeout (#1283)
* add source metrics to report general errors and timeout

* some renaming...

* reintroduce constant
2020-05-13 10:29:58 -07:00
Yanwei Guo de5c590700
Don't record data points when metric config is not initialized yet (#1323)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* dont record

* add comment
2020-05-11 15:34:46 -07:00
Yanwei Guo 19b1d7b64d
Add a helper func to set a default metric config for unit tests (#1263)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* add a comment

* fix typo

* fix tests

* revert

* revert tests

* revert

* fix conflicts

* one more test file
2020-05-07 21:11:45 -07:00
Annie Fu 099343d49e
Make updating metrics config and metrics exporter atomic (#1271)
* Make updating metricsConfig and updating metricsExporter atomic

This requires doing anything with a kubeclient before obtaining the
metricsMux lock.

* Address PR
2020-05-01 15:35:42 -07:00
Markus Thömmes 1b903ad8cc
Fix malformed license headers. (#1258) 2020-04-29 08:38:42 -07:00
Victor Agababov 5ba6c1d101
Change %v to %w in errors and other nits (#1252)
* Change %v to %w in errors and other nits
Other are things I noticed when fixing the main %v->%w conversion

* fold
2020-04-27 15:04:51 -07:00
Victor Agababov fdb7691d92
Brush up the observability config (#1204)
- add proper defaulting
- cleanup test (was pretty shaggy with wrong variable names and such)
- raised coverage a bit
2020-04-07 16:50:00 -07:00
Markus Thömmes c09083a601
Fix a few simple linter warnings: unused code, govet warnings etc. (#1191) 2020-04-06 08:21:20 -07:00
Markus Thömmes 92cdec5b35
Actually use the subtest names in recorder test. (#1193) 2020-04-04 11:17:34 -07:00
Markus Thömmes 9b70d057b1
Allow a 'None' metrics backend, to basically do nothing. (#1190) 2020-04-04 11:17:27 -07:00
Paul Morie d166581448
Mutate `-` to `_` in component names for metrics (#1145) 2020-03-06 14:56:27 -08:00
Victor Agababov 2fe8db3000
Add standard pod and container name keys (#1135)
Those are used in many places and thus deserve to be named constants.
First class citizenship for pod name!
2020-02-27 11:38:51 -08:00
Matt Moore 3925d38991
Format markdown (#1128)
Produced via:
  `prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github)`
/assign n3wscott
/cc n3wscott
2020-02-26 06:55:50 -08:00
Evan Anderson 7af3fab62c
Roll forward #1045 (#1109)
* Revert "Add support for client TLS to pkg/metrics (#1045)"

This reverts commit 945b556708.

* Roll forward "Add support for client TLS to pkg/metrics (#1045)"

Adds support for client TLS certs for opencensus export

* Switch sharedmain to only pass a getter rather than an all-namespace lister.

* Add a TODO about using a cached copy if this generates undue load

* Update deps per build failure.

* Refactor per @anniefu suggestion
2020-02-25 11:42:51 -08:00
Evan Anderson 1ad2b11321
Revert "Add support for client TLS to pkg/metrics (#1045)" (#1106)
This reverts commit 945b556708.
2020-02-19 08:20:06 -08:00
Evan Anderson 945b556708
Add support for client TLS to pkg/metrics (#1045)
* Document the plan for metrics

* Update with issue links

* Update with other @mattmoor comments.

* Address feedback from @anniefu

* Update export address since `localhost` won't work for a DaemonSet.

* Add an explicit action item to migrate Prometheus

* Fix possible (ambiguous) link vs checkbox.

* Fix typo in metrics/README.md

* Add support for client TLS to pkg/metrics

* Address comments from @vagababov and @anniefu

* Finish removing Resource code

* Update name to be shorter and not require a double migration.

* Add tests for opencensus exporting with TLS
2020-02-18 11:04:58 -08:00
Victor Agababov 91a86d6aec
Add a new tester for distribution (#1092)
that only checks count, since sometimes we care about number of things being
reported and less about their values
2020-02-13 16:56:40 -08:00
Matt Moore 2320af4964
golang format tools (#1042)
Produced via:
  `gofmt -s -w $(find -path './vendor' -prune -o -path './third_party' -prune -o -type f -name '*.go' -print)`
  `goimports -w $(find -name '*.go' | grep -v vendor | grep -v third_party)`
/assign n3wscott
/cc n3wscott
2020-02-04 08:20:36 -08:00
Victor Agababov 7ddd3b6a42
Make sure we actually check the last value. (#1041)
Currently the function aptly named CheckLastValueData does not only check that, but
also checks that it's the _single_ entry, which is not necessarily true,
especially if run as part of a bigger test suite.
So change it to actually verify the last row and remove the check for the singular entry.
Given that all the tests pass, this probably was not intented anyway.
2020-02-03 17:45:30 -08:00
Victor Agababov 92c9752a4c
Add RecordN method to permit recording of N measurements simultaneously. (#1025)
* Add RecordN method to permit recording of N measurements simultaneously.

In serving we have many places where we try to record several metrics at the same time.
Besides very repetetive code, this also brings inefficiences, since metrics code has to do _some_ work
for every call, which is completely avoidable.

This way we can record N measurements with one hit.

* tests and nits

* review items per discussion with Evan:

- remove ros
- make measurements ...

* review
2020-01-31 11:26:28 -08:00
cshou 62801f1d34 Allow to specify default exporter option when updating from ConfigMap (#1006)
* Allow to specify default exporter option when updating from ConfigMap

* renaming

* rename test
2020-01-22 15:34:23 -08:00
Nima Kaviani bb0f16f214 Add default metrics backend to observability config (#994) 2020-01-14 23:08:06 -08:00
Matt Moore b8dc5fbc6d golang format tools (#990)
Produced via:
  `gofmt -s -w $(find -path './vendor' -prune -o -path './third_party' -prune -o -type f -name '*.go' -print)`
  `goimports -w $(find -name '*.go' | grep -v vendor | grep -v third_party)`
/assign n3wscott
/cc n3wscott
2020-01-13 10:25:02 -08:00
Evan Anderson 8c11c24324 Add support for OpenCensus Agent as a destination for Knative metrics. (#978)
* Add support for OpenCensus Agent as a destination for Knative metrics.

* Update with comments from @mattmoor and @anniefu

* Remember to register exporter.

Fix tests.

* Fix naming per discussion with @anniefu
2020-01-10 18:42:58 -08:00
Matt Moore 8f763fa65a Format markdown (#981)
Produced via:
  `prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github)`
/assign n3wscott
/cc n3wscott
2020-01-08 07:19:53 -08:00
Scott Nichols 298f22bea6 update boilerplate to use 2020 (#979) 2020-01-07 16:04:51 -08:00
Evan Anderson e53a03271f Document the plan for metrics (#953)
* Document the plan for metrics

* Update with issue links

* Update with other @mattmoor comments.

* Address feedback from @anniefu

* Update export address since `localhost` won't work for a DaemonSet.

* Add an explicit action item to migrate Prometheus

* Fix possible (ambiguous) link vs checkbox.
2020-01-07 15:55:52 -08:00
Zhimin Xiang a43659c656 Add metrics for Knative Certificate creation count (#976) 2020-01-06 15:57:49 -08:00
Ignacio (Nacho) Cano daef119f73 adding knative_broker metrics (#944) 2020-01-06 15:21:49 -08:00