Commit Graph

25 Commits

Author SHA1 Message Date
jjzeng-seattle bac0302109
Set reporting period for all meters (#1575)
* Set reporting period for all meters

* using switch statement to simplify

* Change 60 seconds to a minute

* Add a test case for openCencus customed reporting period
2020-08-04 15:06:28 -07:00
Evan Anderson e7bd895fbd
Introduce `pkg/metrics` support for multiple Resources (#1392)
* Update OpenCensus to include #1196.

* Extend OpenCensus to track per-Resource stats.

* Add generic ResourceView mechanics, part 1. TODO: wire newFactory, tests

* Move factory creation to where it can be customized per-backend

* Integrate opencensus export with Resource-based exporters

* Add stackdriver support for resource views

* address feedbacks

* add test cases

* make KnativeRevision a map

* Update go modules to point to latest opencensus which includes `view.Meter`.

* Add support for Resources to Prometheus Exporter.

This involved copying/improving much of the contrib prometheus exporter.

* add resource_view tests and fix minor issues

* add recorder tests for meter

* Update resource_view_test with an e2e export test.

* Fix prometheus e2e resource_view_test.

Found a fun circular lock.

Found a fun double-registration in metricproducer/Meter.

Found some unnecessary extra code.

Fixed prometheus string sanitization.

Removed a layer of Prometheus wrapping which made debugging more challenging.

* Add test for standard view exports and non-resourced views to e2e test.

* Add UnregisterResourceView to fix tests

* Add OpenCensus e2e resource export test.

Still a little flaky, so work around that with "<=".

* Fix tests for OpenCensus export.

* Fix flakiness / timing issues in OpenCensus exporter e2e test.

* Update to latest ocagent to fix `Resources` bug.

* Don't swallow errors from stackdriver init.

* Get Stackdriver export mostly working in e2e tests

Tests still fail because stackdriver converts custom metrics to a namespaced format and seems to lose Resource information.

* Add support for Stackdriver resource export.

* ./hack/update-codegen.sh

* Fix data race with graceful shutdown.

* Fix issues noticed by @vagababov

* Address mattmoor-bot comments.

* Remove MonitoredResource and apply suggestions from yanweiguo

* More cleanup to stackdriver_exporter_test.go

* Update tests (particularly config_test.go) for Stackdriver recorder changes which sometimes record with additional resources.

* Avoid setting two `view.WithRecorder` options on record

* Add stackdriver tests

* Address comments

* Switch Prometheus port due to possible conflict in test env

* Fix missing :=

* FIx data races around *view.Data shared across Meters.

Cleanup resource_view_test.

* Remove unneeded monitored_resources.go

* Use sync.WaitGroup to avoid data races on closing channel.

* Sort the map for ResourceAsString

* Apply lint and vagababov changes.

* Fix typos

* refactor ResourceAsString

* Update stackdriver exporter version

* Update metrics/resource_view_test.go

Remove accidental paste

* Fix go.sum

* Address yanweiguo comments on comments

* Fix autogen/go mod

* Handling resource extraction when custom metrics are allowed

* Add tests for AllowCustomMetrics set to true

* Address vagababov and yanweiguo comments.

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

* address comments

* Renaming

* Reduce some condition nesting

* go fmt

* Sleep for 1ms to allow metrics being sent (#9)

* Address yanweiguo comments

Co-authored-by: JJ Zeng <jjzeng@google.com>
Co-authored-by: jjzeng-seattle <56568376+jjzeng-seattle@users.noreply.github.com>
2020-07-14 10:46:19 -07:00
Victor Agababov 34c17cc2d5
Make things private that should be so (#1417)
* Make things private that should be so

Make some constants and enums private.
The ones I found to be used elsewhere, I kept public.

* sort
2020-06-17 09:26:24 -07:00
jjzeng-seattle e366b8278b
Add a passthrough method RegisterResourceView (#1410)
* Add a passthrough method RegisterResourceView

* gofmt

* Update metrics/exporter.go

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>

* Address comments

Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com>
2020-06-16 10:58:24 -07:00
Annie Fu 099343d49e
Make updating metrics config and metrics exporter atomic (#1271)
* Make updating metricsConfig and updating metricsExporter atomic

This requires doing anything with a kubeclient before obtaining the
metricsMux lock.

* Address PR
2020-05-01 15:35:42 -07:00
Markus Thömmes 1b903ad8cc
Fix malformed license headers. (#1258) 2020-04-29 08:38:42 -07:00
Markus Thömmes 9b70d057b1
Allow a 'None' metrics backend, to basically do nothing. (#1190) 2020-04-04 11:17:27 -07:00
Paul Morie d166581448
Mutate `-` to `_` in component names for metrics (#1145) 2020-03-06 14:56:27 -08:00
Evan Anderson 7af3fab62c
Roll forward #1045 (#1109)
* Revert "Add support for client TLS to pkg/metrics (#1045)"

This reverts commit 945b556708.

* Roll forward "Add support for client TLS to pkg/metrics (#1045)"

Adds support for client TLS certs for opencensus export

* Switch sharedmain to only pass a getter rather than an all-namespace lister.

* Add a TODO about using a cached copy if this generates undue load

* Update deps per build failure.

* Refactor per @anniefu suggestion
2020-02-25 11:42:51 -08:00
Evan Anderson 1ad2b11321
Revert "Add support for client TLS to pkg/metrics (#1045)" (#1106)
This reverts commit 945b556708.
2020-02-19 08:20:06 -08:00
Evan Anderson 945b556708
Add support for client TLS to pkg/metrics (#1045)
* Document the plan for metrics

* Update with issue links

* Update with other @mattmoor comments.

* Address feedback from @anniefu

* Update export address since `localhost` won't work for a DaemonSet.

* Add an explicit action item to migrate Prometheus

* Fix possible (ambiguous) link vs checkbox.

* Fix typo in metrics/README.md

* Add support for client TLS to pkg/metrics

* Address comments from @vagababov and @anniefu

* Finish removing Resource code

* Update name to be shorter and not require a double migration.

* Add tests for opencensus exporting with TLS
2020-02-18 11:04:58 -08:00
cshou 62801f1d34 Allow to specify default exporter option when updating from ConfigMap (#1006)
* Allow to specify default exporter option when updating from ConfigMap

* renaming

* rename test
2020-01-22 15:34:23 -08:00
Evan Anderson 8c11c24324 Add support for OpenCensus Agent as a destination for Knative metrics. (#978)
* Add support for OpenCensus Agent as a destination for Knative metrics.

* Update with comments from @mattmoor and @anniefu

* Remember to register exporter.

Fix tests.

* Fix naming per discussion with @anniefu
2020-01-10 18:42:58 -08:00
Yanwei Guo 94a34e416c Upgrade OpenCensus Stackdriver exporter (#865)
* upgrade

* change unit tests

* upgrade os sd again

* change unit tests

* add comment

* add comment

* address comments
2019-12-03 14:12:37 -08:00
Annie Fu f684a1078a Add field to metrics config for configuring Stackdriver exporter client (#794)
* Add field to metrics config for configuring Stackdriver exporter client
options

* Fix test setting global state and interfering with other tests

* Use correct GCP metadata values & cleanup

* Some code cleanup

* Remove as much test-specific function stubbing as possible

* Remove more stubbed out test code

* Make more descriptive error when getting secret

* Use former locking pattern to avoid deadlock

There is a cyclical dependency between the metrics pkg and client-go
kubeclient where the kubeclient will attempt to read the metricsConfig
to record metrics about kubeclient requests. This prevents using a kubeclient within the metricsMux lock.
2019-10-25 10:28:36 -07:00
Annie Fu 66c91bcfa6 Fix race condition when updating Stackdriver exporter (#769)
* Remove race condition between reading updates to config-observability
and updating stackdriver exporter

* Simply isNewExporterRequired if statement

* Remove unused thread-safe versions of functions

* Use a reader-writer lock & add function comment

* Make internal unlocked function names consistent
2019-10-18 13:48:19 -07:00
Victor Agababov 74c5d67ea0 Fix the issues pointed out by staticcheck (#541)
* Fix the issues pointed out by staticcheck

* review fix
2019-07-23 13:13:36 -07:00
Mustafa Demirhan 3c8c4a9354 Add support to flush the metrics exporter (#391)
* Add support to flush the metrics exporter.

* Updates to the comments.
2019-04-18 20:12:27 -07:00
Yanwei Guo b4a3bde045 Decide whether to record metrics according to current metrics config (#292)
* split and update

* bug fixed

* change the copyright year

* use fake metadata in tests

* exclude test coverage for gcp_metadata

* add record

* add tests

* revert 2019 and add a new var

* change documents

* address comment

* change line
2019-03-07 10:26:42 -08:00
Yanwei Guo 1494b24d7f Update OpenCensus Stackdriver exporter version (#280)
* split and update

* bug fixed

* change the copyright year

* use fake metadata in tests

* exclude test coverage for gcp_metadata

* revert 2019 change on existing files
2019-02-20 14:18:46 -08:00
Victor Agababov 267197692a Use the k8s sets for the code that is actually set of strings (#261) 2019-02-04 14:35:41 -08:00
Victor Agababov 225d11cc1a Some fixes to the spoof.go and exporter.go (#223)
* Some fixes to the spoof.go and exporter.go

While reviewing some other CL, I saw some avenues for improving
spoof.go, to log the URL that's being fetched, which would help in test
debugging and to use switch construct, rather than nested if's.

While testing the change, I noticed some shifty loggin from the
exporter, so I fixed that as well while I was there.

* Continuation of the previous cleanups.
2019-01-11 12:13:29 -08:00
Yanwei Guo 08febf8f13 Change Stackdriver resource type for metrics (#210)
* change resource type

* copied PR #188

* add unit tests

* change Fatal back to Error

* solve comments

* change to use map

* modify tests
2019-01-08 13:14:42 -08:00
Mustafa Demirhan 53b1235c2a Use different reporting periods based on the metrics backend (#182)
* Use different reporting periods based on the metrics backend. Original implementation was setting this value to 1 min to accommodate the minimum allowed value for stackdriver, but that is a sub optimal behavior for Prometheus. The change also allows operator to override the default values.

* Fix the units of reconciler latency from nano seconds to milliseconds.

* Address PR comments.

* Add better logging for reconciliation.

* Remove unnecessary lock in the UT.

* Address PR comments.
2018-12-04 15:16:25 -08:00
akyyy 10ceeb6163 Add stackdriver exporter support in pkg. (#126)
* update dep golang.org/x/net

* Add stackdriver exporter support in pkg

* fix race and tests

* Use thread safe newMetricsExporter in tests

* address cr comments

* minor fix

* checkpoint

* Address cr comments

* add test for UpdateExporterFromConfigMap

* Move lock position

* add a test case for stackdriver error check

* minor change

* removing one test case since it seems app default creds exists for presubmit tests

* replace promSrvChan with promSrv as cr suggsted

* use a local var server for ListenAndServe

* add getter and setter for shared variables

* add func isMetricsConfigChanged and test

* address cr comments

* Update opencensus-go-exporter-stackdriver to v0.7.0

* Revert "Update opencensus-go-exporter-stackdriver to v0.7.0"

This reverts commit 9825f641f8.

* run update-deps.sh

* merge conflict

* run update-deps.sh

* Update Gopkg.toml so it doesn't have stackdriver exporter

* remove regex for metrics domain at pkg repo per cr comments

* merge conflict
2018-10-24 15:23:31 -07:00