Commit Graph

39 Commits

Author SHA1 Message Date
Douglas Reid 4f4173257d
stackdriver(logging): add support for cel logging filters (#3630)
* stackdriver(logging): add support for cel logging filters

* check if filter expression exists in init
2022-01-06 12:31:52 -08:00
Douglas Reid ff775f5605
Add support for configurability of stackdriver metrics (#3505)
* add support for configurability of stackdriver metrics

* add tag validity checks and support apiName and apiVersion tags

* cleanup

* address review comments

* more const refs

* update config doc

* update const refs

* push tagkey registration into configure()
2021-09-30 22:17:14 -07:00
Pengyuan Bian f1bacaa7dd
Remove Edge reporting. (#3379)
* remove edge

* build.

* fix

* update

* update.

* update.
2021-06-18 10:59:36 -07:00
Nupur Garg 45ef14104a
Add customized access logging support for stackdriver access logs (#3047)
* Add customized access logging support for stackdriver access logs

TODO: add support for removal of tags and customization of TCP access logs

Improve comment

Remove proto refactoring
Enable it for TCP

fix extra semi colon

add cleanupExpressions and cache for TCP

fix test

fix comments

* fix build

* fix comments

* Update doc
2020-10-20 17:59:42 -07:00
Pengyuan Bian 58f07a7722
Parse workload and service from resource labels (#3051)
* update sha

* parse workload and service from resource labels

* add test

* clean up

* fix

* clean up

* skip looking at localhost endpoint

* fix

* fix

* update cluster name
2020-10-15 17:55:12 -07:00
Nupur Garg a5ef9f7daf
Minor changes to Access Logs (#3000)
1) Made batch size and interval for reporting configurable
2) Don't add labels if they are empty-> this is done for labels that are known to empty sometimes
3) Add upstream_transport_failure_reason label to logs

Signed-off-by: gargnupur <gargnupur@google.com>

Added a default timer for ticker

Signed-off-by: gargnupur <gargnupur@google.com>

Fixed feedback

Signed-off-by: gargnupur <gargnupur@google.com>

Updated

Signed-off-by: gargnupur <gargnupur@google.com>
2020-10-08 08:59:47 -07:00
Kuat 938a9485a4
update envoy-wasm (#3032)
* export

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix build

Signed-off-by: Kuat Yessenov <kuat@google.com>

* add an empty test

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix build

Signed-off-by: Kuat Yessenov <kuat@google.com>

* egregious logging

Signed-off-by: Kuat Yessenov <kuat@google.com>

* use detached buffer

Signed-off-by: Kuat Yessenov <kuat@google.com>

* cleanup

Signed-off-by: Kuat Yessenov <kuat@google.com>

* review

Signed-off-by: Kuat Yessenov <kuat@google.com>
2020-10-06 14:25:16 -07:00
David Raskin 40c3904827
Add audit log stream to stackdriver filter (#2970)
* Initial auditEntry implementation

Signed-off-by: davidraskin <draskin@google.com>

* Initial audit log implementation

Signed-off-by: davidraskin <draskin@google.com>

* Format

Signed-off-by: davidraskin <draskin@google.com>

* Use LogEntryType and map for WriteLogEntriesRequest

Signed-off-by: davidraskin <draskin@google.com>

* Add function to initialize logEntriesRequests

Signed-off-by: davidraskin <draskin@google.com>

* Format

Signed-off-by: davidraskin <draskin@google.com>

* Switched to using bool for outbound and bool for audit

Signed-off-by: davidraskin <draskin@google.com>

* Format and remove uneeded functions

Signed-off-by: davidraskin <draskin@google.com>

* Undo stats modification for non cpp20

Signed-off-by: davidraskin <draskin@google.com>

* Takeout double setting of label

Signed-off-by: davidraskin <draskin@google.com>

* Update extensions/common/context.h

Co-authored-by: Nupur Garg <37600866+gargnupur@users.noreply.github.com>

* Update extensions/common/context.h

Co-authored-by: Nupur Garg <37600866+gargnupur@users.noreply.github.com>

* Remove redundant request_info population

Signed-off-by: davidraskin <draskin@google.com>

* Addressed comments

Signed-off-by: davidraskin <draskin@google.com>

* Update comments + change when audit entry is added

Signed-off-by: davidraskin <draskin@google.com>

* Update comments. remove unnecessary include

Signed-off-by: davidraskin <draskin@google.com>

* format

Signed-off-by: davidraskin <draskin@google.com>

* Fix tcp audit logging

Signed-off-by: davidraskin <draskin@google.com>

* Documentation of config

Signed-off-by: davidraskin <draskin@google.com>

* Integration test + address comments

Signed-off-by: davidraskin <draskin@google.com>

* Format

Signed-off-by: davidraskin <draskin@google.com>

* test multiple auditentries + format comment

Signed-off-by: davidraskin <draskin@google.com>

* Remove test case list + add test to inventory

Signed-off-by: davidraskin <draskin@google.com>

Co-authored-by: Nupur Garg <37600866+gargnupur@users.noreply.github.com>
2020-08-20 10:37:12 -07:00
Nupur Garg 3b0c7f243f
Enable Client Side Access Logs for SD (#2955)
* Enable Client Side Access Logs for SD

Fix fmt

Fix fmt

Fix test

Added config options and test for the same

Fixed after rebase

Fixed config and added another test case

Run fmt

Change from ERR_ONLY to ERR_ONLY_ON_NO_MX

Fixed based on feedback

Updated config

Updated config

Fixed based on feedback

* Fixed based on feedback

* Fix lint

* change client error access log
2020-08-11 12:05:47 -07:00
Piotr Sikora 4bf5d18e2e
Update Envoy-WASM SHA to latest. (#2956)
* Update Envoy-WASM SHA to latest.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: update SHA.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2020-08-08 21:35:48 -07:00
Nupur Garg 47d38c13e1
Add TCP Access Log on Open (#2912)
* Add TCP Access Log on Open

Signed-off-by: gargnupur <gargnupur@google.com>

fixed formatting

Signed-off-by: gargnupur <gargnupur@google.com>

Adding to see exact thing in PR for my own review

Signed-off-by: gargnupur <gargnupur@google.com>

Adding to see exact thing in PR for my own review -> fix formatting

Signed-off-by: gargnupur <gargnupur@google.com>

Fix tests

Signed-off-by: gargnupur <gargnupur@google.com>

Fix format

Signed-off-by: gargnupur <gargnupur@google.com>

Fix test

Signed-off-by: gargnupur <gargnupur@google.com>

* make tests parallel

Signed-off-by: gargnupur <gargnupur@google.com>

* fix lint error

Signed-off-by: gargnupur <gargnupur@google.com>

* fix test and based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* Add log_sampled after rebase

Signed-off-by: gargnupur <gargnupur@google.com>

* Log metrics too on timeout

Signed-off-by: gargnupur <gargnupur@google.com>

* Fix based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>
2020-07-20 21:33:29 -07:00
Nupur Garg 5f16bfd2dc
Add filtered label for sampled logs (#2918)
* Add filtered label for sampled logs

Signed-off-by: gargnupur <gargnupur@google.com>

* Fix unit test

Signed-off-by: gargnupur <gargnupur@google.com>

* Fix unit test

Signed-off-by: gargnupur <gargnupur@google.com>

* Fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* Fix based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>
2020-07-17 18:54:34 -07:00
Nupur Garg ea0ce55278
Add TCP Access Logging (#2852)
* Add TCP Metrics in Stackdriver

Signed-off-by: gargnupur <gargnupur@google.com>

fix

Signed-off-by: gargnupur <gargnupur@google.com>

changes

Signed-off-by: gargnupur <gargnupur@google.com>

refactor

Signed-off-by: gargnupur <gargnupur@google.com>

fix test

Add TCP Metrics in Stackdriver

Signed-off-by: gargnupur <gargnupur@google.com>

fix

Signed-off-by: gargnupur <gargnupur@google.com>

Add TCP access logging

changes

Signed-off-by: gargnupur <gargnupur@google.com>

Remove local path from Makefile

changes

Signed-off-by: gargnupur <gargnupur@google.com>

Add functions for queue

fix build

fix test

remove debug time

Signed-off-by: gargnupur <gargnupur@google.com>

remove debug time

Signed-off-by: gargnupur <gargnupur@google.com>

Add protocol in http accesslog

* fix lint

Signed-off-by: gargnupur <gargnupur@google.com>

* fix merge from master

Signed-off-by: gargnupur <gargnupur@google.com>

* fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* fix test

* fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* add comment
2020-06-08 11:56:57 -07:00
Nupur Garg 46babbff42
Add TCP Metrics in Stackdriver (#2843)
* Add TCP Metrics in Stackdriver

Signed-off-by: gargnupur <gargnupur@google.com>

fix

Signed-off-by: gargnupur <gargnupur@google.com>

changes

Signed-off-by: gargnupur <gargnupur@google.com>

refactor

Signed-off-by: gargnupur <gargnupur@google.com>

fix test

Fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* Fix yaml files

Signed-off-by: gargnupur <gargnupur@google.com>

* Fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* Fixed based on feedback

Signed-off-by: gargnupur <gargnupur@google.com>

* Set timer always

Signed-off-by: gargnupur <gargnupur@google.com>

* remove access logger for TCP

Signed-off-by: gargnupur <gargnupur@google.com>
2020-06-04 10:15:56 -07:00
Pengyuan Bian 93937b6861
Switch back envoy upstream to envoy-wasm (#2862)
* switch back to envoy wasm

* remove envoy path override in check_wasm job

* fix release build

* fix release build

* fix wasm build target

* disable rbe

* fix build and restore rbe

* clean up

* format

* add bazel config rel

* fix wasm path again

* add resource class to circle macos flow

* remove large resource class and test step
2020-05-27 07:55:55 -07:00
mandarjog 2379bf3da9
ensure that telemetry plugins do not return config errors (#2857)
* ensure that telemetry plugin do not return errors

* initialize to false
2020-05-19 10:48:07 -07:00
Kuat 910639ad62
replace protobuffers with flatbuffers at runtime (#2794)
* implement flatbuffer read and measure

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix all tests

Signed-off-by: Kuat Yessenov <kuat@google.com>

* add cache

Signed-off-by: Kuat Yessenov <kuat@google.com>

* finish

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix build?

Signed-off-by: Kuat Yessenov <kuat@google.com>

* maybe?

Signed-off-by: Kuat Yessenov <kuat@google.com>

* maybe?

Signed-off-by: Kuat Yessenov <kuat@google.com>

* simplify

Signed-off-by: Kuat Yessenov <kuat@google.com>

* feedback

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix wasm build

Signed-off-by: Kuat Yessenov <kuat@google.com>

* set peer size by default

Signed-off-by: Kuat Yessenov <kuat@google.com>

* split inbound and outbound peer caches

Signed-off-by: Kuat Yessenov <kuat@google.com>

* merge fix

Signed-off-by: Kuat Yessenov <kuat@google.com>
2020-04-08 11:22:55 -07:00
Pengyuan Bian d7d2ad8100
Flush log entries in on done (#2780)
* flush log entries in on done

* add a comment about logger

* clean up logging stats since it is already tracked by envoy grpc stats

* simplify vm reload test
2020-03-24 14:31:55 -07:00
Douglas Reid 748ac4cf51
feat(edges): add support for reporting two batches (new vs full) (#2669)
* feat(edges): add support for reporting two batches (new vs full)

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* cleanup

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix comment in test

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* address review feedback

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>
2020-02-13 10:24:44 -08:00
John Plevyak 1d997c17e2 Merge envoy-wasm for v1.13.0. (#2631)
* Merge envoy-wasm for v1.13.0.

Signed-off-by: John Plevyak <jplevyak@gmail.com>

* Update .wasm files.

Signed-off-by: John Plevyak <jplevyak@gmail.com>

* WASM SDK v2 -> v3.

Signed-off-by: John Plevyak <jplevyak@gmail.com>

* Address comments.

Signed-off-by: John Plevyak <jplevyak@gmail.com>

* Bump bazelversion.

Signed-off-by: John Plevyak <jplevyak@gmail.com>
2020-01-24 13:53:13 -08:00
Nupur Garg 072630f6f0 Add support for Access Log Policy Filter (#2602)
* Add Access Log Sampling Filter

Format

Make it just mvp ready

Format

Fix extra line

Fix go-lint errors

Fix fmt

change to using absl::Time and absl::Duration instead of long int for time calculations

Fixed after feedback

Fix lint error

* Debug test failure in circleci

* Debug test failure in circleci

* Debug test failure in circleci

* Debug test failure in circleci

* Add lock for the cache

* Debug circle ci

* Fix test
2020-01-06 18:26:28 -08:00
Pengyuan Bian 5a66e29530 Extract service name from request host and cluster name (#2548)
* support extracing service name from request host

* service_name:port

* s/namespace_pos/name_pos

* update

* lint

* add initialization to service auth policy

* format

* use absl time

* update test

* address comment
2019-12-14 00:24:47 -08:00
Lizan Zhou 73b18cae6f Update envoy-wasm sha (#2559)
* Update envoy-wasm sha

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* fix due to ABI change

* remove compile db

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* fix for envoyproxy/envoy-wasm#216

Signed-off-by: Lizan Zhou <lizan@tetrate.io>

* fix ci

* roll forward

Signed-off-by: Lizan Zhou <lizan@tetrate.io>
2019-11-15 10:43:46 -08:00
mandarjog 0ace1cc059 v2: Add ability to control host header fallback (#2546)
* WIP

* WIP2

* add test

* WIP5

* WIP7

* Update testdata

* fix go error

* Use dynamic client port
2019-11-10 21:07:53 -08:00
Pengyuan Bian f998652f28 Move node_info_cache to common (#2454)
* move node_info_cache to common

* add a new method to also fetch peer metadata id.

* get peer node info inside root context

* add capability to disabel node cache

* make node pointer return const

* fix test
2019-10-30 11:37:36 -07:00
Douglas Reid 787a566773 fix(edges reporting): fix error in reporting logic (#2486)
* fix(edges reporting): fix error in reporting logic

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* add missing files

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* better error messages in test

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix mesh_id handling

* fix mesh uid in stackdriver integration test

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix mesh id logic

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>
2019-10-29 10:51:35 -07:00
Douglas Reid 41007f92fb feat(edges): enable edge reporting in stackdriver extension (#2468)
* feat(edges): enable edge reporting in stackdriver extension

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix(edges): add stats_prefix and change reporting duration

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>

* fix: remove unused constexpr

Signed-off-by: Douglas Reid <douglas-reid@users.noreply.github.com>
2019-10-23 09:39:10 -07:00
Pengyuan Bian 08cec447df Fix SD filter root context and update e2e test (#2466)
* root id and vm id and test

* format

* fix test

* update error message
2019-10-16 16:30:14 -07:00
Pengyuan Bian cd63c8174b Add usage of sd logger and an e2e test (#2458)
* add usage of stackdriver logger and test

* add stat prefix

* add metrics and only export at server

* remove timeseries.go

* fix test due to merge conflict
2019-10-15 18:12:38 -07:00
Kuat e383776139 wasm: use filter state instead of metadata (#2421)
* fixing the plugins

Signed-off-by: Kuat Yessenov <kuat@google.com>

* ammend

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update to use filter state

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fixes

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update to use simpler API

Signed-off-by: Kuat Yessenov <kuat@google.com>

* updates

Signed-off-by: Kuat Yessenov <kuat@google.com>

* forgot to initialize ints

Signed-off-by: Kuat Yessenov <kuat@google.com>

* second attempt

Signed-off-by: Kuat Yessenov <kuat@google.com>

* simplify

Signed-off-by: Kuat Yessenov <kuat@google.com>

* pull latest

Signed-off-by: Kuat Yessenov <kuat@google.com>

* fix bug

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update formatting

Signed-off-by: Kuat Yessenov <kuat@google.com>

* merge fix

Signed-off-by: Kuat Yessenov <kuat@google.com>

* build fix

Signed-off-by: Kuat Yessenov <kuat@google.com>

* copy traffic direction

Signed-off-by: Kuat Yessenov <kuat@google.com>

* update envoy SHA

Signed-off-by: Kuat Yessenov <kuat@google.com>
2019-09-25 16:47:41 -07:00
Pengyuan Bian a8723bc94d Record SD metric and add an integration test (#2359)
* metric record and integration test

* make test run

* make circle pass

* update go path

* try fix prow

* update sd filter and test

* clean

* remove direction from stackdriver config

* cleanup comment

* address comment

* fix build

* recover fake sd server

* add test prefix to field name

* update test config
2019-09-13 15:43:03 -07:00
Piotr Sikora 47e4559b8e Update Envoy-WASM SHA to latest with security fixes. (#2355)
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2019-08-13 13:15:55 -07:00
Pengyuan Bian 154dfbe83e migrate to extension common (#2349) 2019-08-08 15:08:49 -07:00
Pengyuan Bian 536638dd9e Add function to record SD metrics (#2322)
* Add metric recorder

* fix test

* version proto

* format
2019-08-08 12:04:37 -07:00
Piotr Sikora 048890f7e0 Update Envoy-WASM SHA to latest. (#2346)
Signed-off-by: Piotr Sikora <piotrsikora@google.com>
2019-08-07 10:25:35 -07:00
Pengyuan Bian 113d7e2b4f Opencensus registry (#2317) 2019-07-23 17:32:37 -07:00
Pengyuan Bian 23511889de Add NodeInfo and RequestInfo into Stackdriver plugin (#2315)
* request and node info

* new line

* remove unused key

* change node info to protobuf.

* comment

* change text to use json string

* format

* address comment

* format

* address comment

* static_cast->dynamic_cast

* move all data extraction into onLog
2019-07-19 15:20:26 -07:00
Pengyuan Bian dc9081baf8 Add Stackdriver plugin config (#2314)
* Add Stackdriver plugin config.

* format

* lower case function name
2019-07-18 16:27:26 -07:00
Pengyuan Bian a0528f8eb0 Add scaffolding code for stackdriver wasm native plugin (#2297)
* stackdriver opencensus

* license and copyright

* fix opencensus.bzl

* format

* format

* address comment

* cleanup

* clean up build file

* remove -I

* address comment

* new line

* address comment

* fix
2019-07-16 11:39:16 -07:00