Commit Graph

124 Commits

Author SHA1 Message Date
Scott Nichols 7dec722e26
found another way to get repo name in gh actions (#1622)
* foiund another way to get repo name

* fix vet errors

* did not mean 14
2020-08-14 18:48:06 -07:00
Markus Thömmes c903170f41
Completely skip metric reporting if backend is 'none'. (#1599) 2020-08-10 10:22:30 -07:00
Evan Anderson e487a47312
Use real synchronization to make sure we get four exports in the OpenCensus test (#1591)
* Use real synchronization to make sure we get four exports in the OpenCensus test.

* Fix a few bugs in the metrics implementation:

1. We could create two OpenCensus exporters for the empty resource, which would cause double-reporting
2. Add a mechanism to clear out background Meter threads to to prevent them exporting in subsequent tests.
2020-08-07 11:51:29 -07:00
Yanwei Guo ec67865ad3
timeout (#1593) 2020-08-06 15:02:28 -07:00
Stavros Kontopoulos 23ef76f986
Add field to observability config for enabling request logging (#1577)
* add field for enabling request logging

* fixes

* update logic

* add unit tests

* add comment
2020-08-05 09:19:28 -07:00
jjzeng-seattle bac0302109
Set reporting period for all meters (#1575)
* Set reporting period for all meters

* using switch statement to simplify

* Change 60 seconds to a minute

* Add a test case for openCencus customed reporting period
2020-08-04 15:06:28 -07:00
Victor Agababov f1e8ecbf78
Disable GC to reduce test flakiness (#1581)
At least twice I saw this report +1 over what we expect, which I attribute
to actual GC being collected.
E.g. https://prow.knative.dev/view/gs/knative-prow/pr-logs/pull/knative_pkg/1580/pull-knative-pkg-unit-tests/1290650655098146816
So disable GC for the duration of this test
2020-08-04 12:12:28 -07:00
Ian Milligan c3c869a344
Remove resourceToKey map (#1579)
There is currently no mechanism to resolve a resource to a unique pointer causing this map to leak
memory.
2020-08-03 22:12:27 -07:00
jjzeng-seattle 68b3eeeaab
Add helper functions for metric tests in serving (#1564)
* Add helper functions for metric tests in seving

* typo

* Add WithResource to simply the code
2020-07-29 03:08:59 -07:00
Victor Agababov 7ea21a17b9
Make sure metrics tests are re-entrant (#1566)
* Make sure metrics tests are re-entrant
Currently they are not. 'go test -count=2' will miserably fail.
So ensure they are. Also add a helper to return current value.

* review
2020-07-28 14:52:59 -07:00
Evan Anderson deb6b33d2a
Make EnsureRecorded part of metricstest (#1554)
* Make EnsureRecorded part of metricstest; call it automatically in the "Assert" methods.

Increase test coverage.

* Address yanweiguo comments

* Add a barrier to ensure all metrics are recorded before scraping prometheus.
2020-07-27 08:45:58 -07:00
Evan Anderson c36f46cc8c
Wait longer for metrics collection to settle for OpenCensus (#1555) 2020-07-23 22:55:57 -07:00
Victor Agababov 6757bb169b
Some nits in the tests in effort to fix race (#1553)
which panics in a strange place (sync.waitgroup that disables race detection)
and other nits
2020-07-23 15:17:57 -07:00
Evan Anderson bc4dc000cb
Add utilities for accessing metrics via metricproducer (#1526)
* Add utilities for accessing metrics via metricproducer.

* Fix tag comparison, bump up coverage.

* Convert workqueue_test (one of the main users of metricstest) to the new methods.

* Force event propagation sync one more place.

* Boost test coverage again.

* Address vagababov comments
2020-07-22 18:31:57 -07:00
Victor Agababov acd3f31714
nits (#1529) 2020-07-20 18:44:35 -07:00
jjzeng-seattle 03151bfc84
Add component name as the metric prefix for opencensus exporter (#1493)
* Add component name as the metric prefix for opencensus exporter

Read from all meters instead of just he default one

Add component name as the metric prefix for opencensus exporter

* Update codegen

* Add domain as metric prefix

* Use path.Join to simply the code
2020-07-20 14:10:36 -07:00
Victor Agababov 25f2aa6443
micro-Nit in the test: don't make a private type public (#1510) 2020-07-16 10:41:33 -07:00
Markus Thömmes cb729b8ae6
Assorted linting fixes. (#1507)
- Remove unused code.
- Remove unnecessary conversion.
- Prevent copies of structs that contain sync.Mutex.
2020-07-16 05:59:33 -07:00
Evan Anderson 9f3fb07346
Remove delay-based flakiness from workqueue_test.go (#1497) 2020-07-15 10:20:33 -07:00
Evan Anderson 723b71fa47
Update Stackdriver custom exporter to support RecordBatch where some records are special and others are not. (#1495) 2020-07-15 10:05:33 -07:00
jjzeng-seattle 858f1a7b0d
Read from all meters instead of just he default one (#1491) 2020-07-14 14:44:32 -07:00
Evan Anderson e7bd895fbd
Introduce `pkg/metrics` support for multiple Resources (#1392)
* Update OpenCensus to include #1196.

* Extend OpenCensus to track per-Resource stats.

* Add generic ResourceView mechanics, part 1. TODO: wire newFactory, tests

* Move factory creation to where it can be customized per-backend

* Integrate opencensus export with Resource-based exporters

* Add stackdriver support for resource views

* address feedbacks

* add test cases

* make KnativeRevision a map

* Update go modules to point to latest opencensus which includes `view.Meter`.

* Add support for Resources to Prometheus Exporter.

This involved copying/improving much of the contrib prometheus exporter.

* add resource_view tests and fix minor issues

* add recorder tests for meter

* Update resource_view_test with an e2e export test.

* Fix prometheus e2e resource_view_test.

Found a fun circular lock.

Found a fun double-registration in metricproducer/Meter.

Found some unnecessary extra code.

Fixed prometheus string sanitization.

Removed a layer of Prometheus wrapping which made debugging more challenging.

* Add test for standard view exports and non-resourced views to e2e test.

* Add UnregisterResourceView to fix tests

* Add OpenCensus e2e resource export test.

Still a little flaky, so work around that with "<=".

* Fix tests for OpenCensus export.

* Fix flakiness / timing issues in OpenCensus exporter e2e test.

* Update to latest ocagent to fix `Resources` bug.

* Don't swallow errors from stackdriver init.

* Get Stackdriver export mostly working in e2e tests

Tests still fail because stackdriver converts custom metrics to a namespaced format and seems to lose Resource information.

* Add support for Stackdriver resource export.

* ./hack/update-codegen.sh

* Fix data race with graceful shutdown.

* Fix issues noticed by @vagababov

* Address mattmoor-bot comments.

* Remove MonitoredResource and apply suggestions from yanweiguo

* More cleanup to stackdriver_exporter_test.go

* Update tests (particularly config_test.go) for Stackdriver recorder changes which sometimes record with additional resources.

* Avoid setting two `view.WithRecorder` options on record

* Add stackdriver tests

* Address comments

* Switch Prometheus port due to possible conflict in test env

* Fix missing :=

* FIx data races around *view.Data shared across Meters.

Cleanup resource_view_test.

* Remove unneeded monitored_resources.go

* Use sync.WaitGroup to avoid data races on closing channel.

* Sort the map for ResourceAsString

* Apply lint and vagababov changes.

* Fix typos

* refactor ResourceAsString

* Update stackdriver exporter version

* Update metrics/resource_view_test.go

Remove accidental paste

* Fix go.sum

* Address yanweiguo comments on comments

* Fix autogen/go mod

* Handling resource extraction when custom metrics are allowed

* Add tests for AllowCustomMetrics set to true

* Address vagababov and yanweiguo comments.

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

* address comments

* Renaming

* Reduce some condition nesting

* go fmt

* Sleep for 1ms to allow metrics being sent (#9)

* Address yanweiguo comments

Co-authored-by: JJ Zeng <jjzeng@google.com>
Co-authored-by: jjzeng-seattle <56568376+jjzeng-seattle@users.noreply.github.com>
2020-07-14 10:46:19 -07:00
Victor Agababov eb05e8dd5b
Various cleanups (#1446)
Thanks @markusthoemmes for the tool :)
2020-06-24 14:04:28 -07:00
Evan Anderson b9a30ee123
Update config_test.go and metricstest to support multiple Meters. (#1421) 2020-06-18 10:50:25 -07:00
Victor Agababov 34c17cc2d5
Make things private that should be so (#1417)
* Make things private that should be so

Make some constants and enums private.
The ones I found to be used elsewhere, I kept public.

* sort
2020-06-17 09:26:24 -07:00
Antoine Cotten ffb929374a
Make Prometheus exporter port configurable via env (#1411) 2020-06-16 16:26:24 -07:00
Antoine Cotten 0d967ff0d8
Clean metrics config tests (#1412)
* Use constants in metrics config tests

* Normalize test cases indentation
2020-06-16 11:09:24 -07:00
jjzeng-seattle e366b8278b
Add a passthrough method RegisterResourceView (#1410)
* Add a passthrough method RegisterResourceView

* gofmt

* Update metrics/exporter.go

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>

* Address comments

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>
2020-06-16 10:58:24 -07:00
Victor Agababov fef70cc261
Unexport some constants (#1393)
* fix the space issue

* merge shenanigans

* nit and tests

* Unexport some constants that don't seem to be used anywhere
2020-06-09 11:40:32 -07:00
Evan Anderson c87d420af5
Don't swallow errors from stackdriver init. (#1367)
* Don't swallow errors from stackdriver init.

* Also check error from StartMetricsExporter.
2020-06-01 10:11:03 -07:00
Dave Protasowski 2d1a04d1ff
bump k8s packages to 1.17.6 (#1285)
* include script to bump k8s deps

* ran ./hack/update-k8s-deps.sh

* fix client-go breaking changes

* refactor update-k8s-deps.sh

* bump to k8s v0.17.6
2020-05-27 10:37:59 -07:00
Markus Thömmes 60f4ae1dbe
Use config map parsing lib and enhance it further. (#1335)
* Use config map parsing lib and enhance it further.

* Unexport test struct values.

* Migrate tracing config too.

* Use sample-rate.
2020-05-18 10:42:06 -07:00
Victor Agababov 1ceb875570
Fix \" escaping and other nits in the pkg (#1339) 2020-05-16 09:36:02 -07:00
Yanwei Guo f1c51d6830
Delete nit check log (#1340)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* delete
2020-05-16 09:00:02 -07:00
Yanwei Guo 16d7b96341
Print nil check log only once (#1334)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* delete

* log once

* remove space
2020-05-14 17:25:00 -07:00
Lionel Villard 40ed633032
add source metrics to report general errors and timeout (#1283)
* add source metrics to report general errors and timeout

* some renaming...

* reintroduce constant
2020-05-13 10:29:58 -07:00
Yanwei Guo de5c590700
Don't record data points when metric config is not initialized yet (#1323)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* dont record

* add comment
2020-05-11 15:34:46 -07:00
Yanwei Guo 19b1d7b64d
Add a helper func to set a default metric config for unit tests (#1263)
* do not record for empty metric config

* Revert "do not record for empty metric config"

This reverts commit 539a5e4dbb.

* add a comment

* fix typo

* fix tests

* revert

* revert tests

* revert

* fix conflicts

* one more test file
2020-05-07 21:11:45 -07:00
Annie Fu 099343d49e
Make updating metrics config and metrics exporter atomic (#1271)
* Make updating metricsConfig and updating metricsExporter atomic

This requires doing anything with a kubeclient before obtaining the
metricsMux lock.

* Address PR
2020-05-01 15:35:42 -07:00
Markus Thömmes 1b903ad8cc
Fix malformed license headers. (#1258) 2020-04-29 08:38:42 -07:00
Victor Agababov 5ba6c1d101
Change %v to %w in errors and other nits (#1252)
* Change %v to %w in errors and other nits
Other are things I noticed when fixing the main %v->%w conversion

* fold
2020-04-27 15:04:51 -07:00
Victor Agababov fdb7691d92
Brush up the observability config (#1204)
- add proper defaulting
- cleanup test (was pretty shaggy with wrong variable names and such)
- raised coverage a bit
2020-04-07 16:50:00 -07:00
Markus Thömmes c09083a601
Fix a few simple linter warnings: unused code, govet warnings etc. (#1191) 2020-04-06 08:21:20 -07:00
Markus Thömmes 92cdec5b35
Actually use the subtest names in recorder test. (#1193) 2020-04-04 11:17:34 -07:00
Markus Thömmes 9b70d057b1
Allow a 'None' metrics backend, to basically do nothing. (#1190) 2020-04-04 11:17:27 -07:00
Paul Morie d166581448
Mutate `-` to `_` in component names for metrics (#1145) 2020-03-06 14:56:27 -08:00
Victor Agababov 2fe8db3000
Add standard pod and container name keys (#1135)
Those are used in many places and thus deserve to be named constants.
First class citizenship for pod name!
2020-02-27 11:38:51 -08:00
Matt Moore 3925d38991
Format markdown (#1128)
Produced via:
  `prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github)`
/assign n3wscott
/cc n3wscott
2020-02-26 06:55:50 -08:00
Evan Anderson 7af3fab62c
Roll forward #1045 (#1109)
* Revert "Add support for client TLS to pkg/metrics (#1045)"

This reverts commit 945b556708.

* Roll forward "Add support for client TLS to pkg/metrics (#1045)"

Adds support for client TLS certs for opencensus export

* Switch sharedmain to only pass a getter rather than an all-namespace lister.

* Add a TODO about using a cached copy if this generates undue load

* Update deps per build failure.

* Refactor per @anniefu suggestion
2020-02-25 11:42:51 -08:00
Evan Anderson 1ad2b11321
Revert "Add support for client TLS to pkg/metrics (#1045)" (#1106)
This reverts commit 945b556708.
2020-02-19 08:20:06 -08:00