Commit Graph

29 Commits

Author SHA1 Message Date
Annie Fu 099343d49e
Make updating metrics config and metrics exporter atomic (#1271)
* Make updating metricsConfig and updating metricsExporter atomic

This requires doing anything with a kubeclient before obtaining the
metricsMux lock.

* Address PR
2020-05-01 15:35:42 -07:00
Markus Thömmes 9b70d057b1
Allow a 'None' metrics backend, to basically do nothing. (#1190) 2020-04-04 11:17:27 -07:00
Evan Anderson 7af3fab62c
Roll forward #1045 (#1109)
* Revert "Add support for client TLS to pkg/metrics (#1045)"

This reverts commit 945b556708.

* Roll forward "Add support for client TLS to pkg/metrics (#1045)"

Adds support for client TLS certs for opencensus export

* Switch sharedmain to only pass a getter rather than an all-namespace lister.

* Add a TODO about using a cached copy if this generates undue load

* Update deps per build failure.

* Refactor per @anniefu suggestion
2020-02-25 11:42:51 -08:00
Evan Anderson 1ad2b11321
Revert "Add support for client TLS to pkg/metrics (#1045)" (#1106)
This reverts commit 945b556708.
2020-02-19 08:20:06 -08:00
Evan Anderson 945b556708
Add support for client TLS to pkg/metrics (#1045)
* Document the plan for metrics

* Update with issue links

* Update with other @mattmoor comments.

* Address feedback from @anniefu

* Update export address since `localhost` won't work for a DaemonSet.

* Add an explicit action item to migrate Prometheus

* Fix possible (ambiguous) link vs checkbox.

* Fix typo in metrics/README.md

* Add support for client TLS to pkg/metrics

* Address comments from @vagababov and @anniefu

* Finish removing Resource code

* Update name to be shorter and not require a double migration.

* Add tests for opencensus exporting with TLS
2020-02-18 11:04:58 -08:00
Victor Agababov 92c9752a4c
Add RecordN method to permit recording of N measurements simultaneously. (#1025)
* Add RecordN method to permit recording of N measurements simultaneously.

In serving we have many places where we try to record several metrics at the same time.
Besides very repetetive code, this also brings inefficiences, since metrics code has to do _some_ work
for every call, which is completely avoidable.

This way we can record N measurements with one hit.

* tests and nits

* review items per discussion with Evan:

- remove ros
- make measurements ...

* review
2020-01-31 11:26:28 -08:00
Evan Anderson 8c11c24324 Add support for OpenCensus Agent as a destination for Knative metrics. (#978)
* Add support for OpenCensus Agent as a destination for Knative metrics.

* Update with comments from @mattmoor and @anniefu

* Remember to register exporter.

Fix tests.

* Fix naming per discussion with @anniefu
2020-01-10 18:42:58 -08:00
Ignacio (Nacho) Cano daef119f73 adding knative_broker metrics (#944) 2020-01-06 15:21:49 -08:00
Matt Moore e96196e347 golang format tools (#876)
Produced via:
  `gofmt -s -w $(find -path './vendor' -prune -o -type f -name '*.go' -print))`
  `goimports -w $(find -name '*.go' | grep -v vendor)`
/assign n3wscott
2019-11-13 15:49:28 -08:00
Nima Kaviani 4deb5d83d2 move metrics config to pkg (#786)
the observability config is used in other packages including
the sample controller. This commit pulls observability config
into knative.dev/pkg similar to logging config for better reusability.
2019-11-13 06:52:28 -08:00
Evan Anderson a9bdd123b7 Start moving Stackdriver and Prometheus-specific code to a strategy pattern (#812)
* Start moving Stackdriver and Prometheus-specific code to a strategy pattern.

* Respond to review feedback, add test.

* gofmt
2019-11-04 16:05:21 -08:00
Annie Fu 4d39b058e2 Restrict metric pkg Stackdriver Secret to controlled name & namespace (#824)
* Restrict Stackdriver Secret to controlled name & namespace

* Address PR comments

* Lock when reading secret name & namespace

* Export StackdriverClientConfig struct so consumers don't have to
redefine it

* Ensure metrics pkg consumers call SetStackdriverSecretLocation to set
Secret name and namespace

* Export StackdriverClientConfig consts & helper function

* Add TODO for secret watcher
2019-11-04 11:57:19 -08:00
Annie Fu f684a1078a Add field to metrics config for configuring Stackdriver exporter client (#794)
* Add field to metrics config for configuring Stackdriver exporter client
options

* Fix test setting global state and interfering with other tests

* Use correct GCP metadata values & cleanup

* Some code cleanup

* Remove as much test-specific function stubbing as possible

* Remove more stubbed out test code

* Make more descriptive error when getting secret

* Use former locking pattern to avoid deadlock

There is a cyclical dependency between the metrics pkg and client-go
kubeclient where the kubeclient will attempt to read the metricsConfig
to record metrics about kubeclient requests. This prevents using a kubeclient within the metricsMux lock.
2019-10-25 10:28:36 -07:00
Annie Fu 66c91bcfa6 Fix race condition when updating Stackdriver exporter (#769)
* Remove race condition between reading updates to config-observability
and updating stackdriver exporter

* Simply isNewExporterRequired if statement

* Remove unused thread-safe versions of functions

* Use a reader-writer lock & add function comment

* Make internal unlocked function names consistent
2019-10-18 13:48:19 -07:00
Yanwei Guo 9a71140445 add comment (#761) 2019-10-14 15:29:32 -07:00
Ignacio Cano 6b1c5cf09a StackDriver CustomMetrics with Custom Domain Prefix (#676)
* renaming source_name and source_resource_group to name and resource_group,
as they will be used in other resources

* updating

* updates based on code review

* update

* fix
2019-09-13 15:50:35 -07:00
Ignacio Cano c56cedc7e7 Serializing/Deserializing Logging and Metrics configs (#656)
* source stats reporter... should be used by well-behaved sources

* public fields

* moving serialization/deserialization of metrics and logging maps to pkg

* nits

* adding UT

* same order

* updates

* sock-puppet

* unregistration problem?

* removing base64 encoding

* removing TODO

* comments after review

* removing the other constants
2019-09-10 12:36:29 -07:00
Matt Moore 222dd25986 Migrate pkg to use the knative.dev/pkg import path (#489)
* Manual changes.

* scripted changes.
2019-06-26 13:02:06 -07:00
Matt Moore 523d91657b Add a testing package to setup metrics.Domain() (#429)
This mirrors what we do for system.Namespace() today.
2019-05-28 08:01:41 -07:00
Matt Moore 2923ca42d9 Share the configmap name logic for Logging and Metrics. (#425)
This also adds a way to fetch the metrics "domain" following a similar pattern to `system.Namespace()`.
2019-05-27 18:41:29 -07:00
Mustafa Demirhan 3c8c4a9354 Add support to flush the metrics exporter (#391)
* Add support to flush the metrics exporter.

* Updates to the comments.
2019-04-18 20:12:27 -07:00
Yanwei Guo 5323eaf18b Expose UpdateExporter function to accept map as parameter (#347)
* expose func

* use different metric name

* remove stackdriver change

* change comment

* return err
2019-03-29 16:38:54 -07:00
Yanwei Guo b4a3bde045 Decide whether to record metrics according to current metrics config (#292)
* split and update

* bug fixed

* change the copyright year

* use fake metadata in tests

* exclude test coverage for gcp_metadata

* add record

* add tests

* revert 2019 and add a new var

* change documents

* address comment

* change line
2019-03-07 10:26:42 -08:00
Victor Agababov c100323403 Fix the sugared logger usage (#300)
* Fix the sugared logger usage

Without `w` the logged messages are quite useless and annoying.

* some more magic
2019-02-28 18:17:38 -08:00
Yanwei Guo 1aca435944 Read metrics backend from environment variable (#218)
* add env to set backend

* join two lines
2019-01-10 11:03:42 -08:00
jonjohnsonjr a8c40a9e3b Default metrics.backend-destination to Prometheus (#213)
Fixes #212
2019-01-08 09:54:43 -08:00
Mustafa Demirhan 53b1235c2a Use different reporting periods based on the metrics backend (#182)
* Use different reporting periods based on the metrics backend. Original implementation was setting this value to 1 min to accommodate the minimum allowed value for stackdriver, but that is a sub optimal behavior for Prometheus. The change also allows operator to override the default values.

* Fix the units of reconciler latency from nano seconds to milliseconds.

* Address PR comments.

* Add better logging for reconciliation.

* Remove unnecessary lock in the UT.

* Address PR comments.
2018-12-04 15:16:25 -08:00
akyyy 02b354e3a4 Relax the requirement for stackdriver project id for metrics exporter. (#147)
* fix flaky test

* relax the requirement for stackdriver project id

* improve log

* add comment for stackdriverProjectIDKey not provided case

* minor format

* simplify the map access
2018-10-30 16:20:34 -07:00
akyyy 10ceeb6163 Add stackdriver exporter support in pkg. (#126)
* update dep golang.org/x/net

* Add stackdriver exporter support in pkg

* fix race and tests

* Use thread safe newMetricsExporter in tests

* address cr comments

* minor fix

* checkpoint

* Address cr comments

* add test for UpdateExporterFromConfigMap

* Move lock position

* add a test case for stackdriver error check

* minor change

* removing one test case since it seems app default creds exists for presubmit tests

* replace promSrvChan with promSrv as cr suggsted

* use a local var server for ListenAndServe

* add getter and setter for shared variables

* add func isMetricsConfigChanged and test

* address cr comments

* Update opencensus-go-exporter-stackdriver to v0.7.0

* Revert "Update opencensus-go-exporter-stackdriver to v0.7.0"

This reverts commit 9825f641f8.

* run update-deps.sh

* merge conflict

* run update-deps.sh

* Update Gopkg.toml so it doesn't have stackdriver exporter

* remove regex for metrics domain at pkg repo per cr comments

* merge conflict
2018-10-24 15:23:31 -07:00