Commit Graph

3 Commits

Author SHA1 Message Date
Jorge Turrado Ferrero b5c41c8c2a
chore: add golangci-lint (#584) 2023-01-27 08:52:15 +01:00
Aaron Schlesinger 6b8d4ab8aa
Refactoring the interceptor (#105)
* Refactoring the interceptor

- Adding more testing to the proxy
- Refactoring the proxy to use standard net/http (gives it more control over the response lifecycle)
- Using the standard library test framework, because stretchr/testify/suite was not adding much (stretchr/testify/require is still in use though)

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding test for holding the connection

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing dead code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing dead code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* using the getter/watcher logic in the proxy handler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* progress on fast getter/watcher implementation:

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding deployment cache

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding a backoff for the network connection

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding test targets to the magefile

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Fixing and adding more tests to the proxy

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding test to ensure proxy handler holds until >0 replicas on target deployment

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* refactoring in progress and more tests

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* todo test

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing hanging issue

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding test for slow origin, and a TODO in the proxy tests

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* TODO tests

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more work on connection retry and backoff

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* logging nested errors

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* better test checks and explanatory comments

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* moving sumExp utility function to top

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding tests for DialContextWithRetry

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* refactoring backoff etc...

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* refactoring the dialer

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* refactoring if/else multiple return statements

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing addition from num steps

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing dead code and updating knative attribution comment

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing net error type check

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding one more test

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* implementing TODO tests and cleaning up unused and logging code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding wait func timeout

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding waitFunc timeout logic

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* starting tests for the deployment cache

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding deployment cache watch test

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding timing checks to the deployment cache test

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Fixing the last broken test

Turns out, don't run a handler in a goroutine because (net/http/httptest).ResponseRecorder
is not concurrency-safe. Thanks to @asw101 and @khaosdoctor for help on this!

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Tidying up interceptor config and making operator set more config vars

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing tests and compile errs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-04-12 12:48:07 -07:00
Aaron Schlesinger 7b75262476
Starting to move components from prototype repo (#2)
* starting the scaler and interceptor

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding interceptor

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* link to issue

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* operator and CLI start

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* starting on the dockerfiles

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Improving the operator Dockerfile

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fix

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* note

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding go based operator

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* a bit of progress

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* moving rust operator to old

will delete this shortly

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* moving goperator to operator

forgot to do this last commit

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* moving go.mod to root

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* marking deprecated

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* it builds
gs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* progress on the operator

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more progress

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* All my changes and implementations (#2)

* Fix error with existing variable

* Rename ScaledObject to HTTPScaledObject

* Add draft implementation for status spec

* Rename files to match new naming

* Remove scaffolding comments

* Change requeue time to 50s

* Variable renaming to better read

* Add new spec field to make polling configurable

* Remove previously added objects in case of error

* Set Statuses

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Fix Typos (#4)

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Add RBAC rules for services, pods and deployments (#5)

* Fix Typos

* Add RBAC rules for services, pods and deployments

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Bunch of changes (#6)

* Fix Typos

* Implement object deletion

* Add prefix for logging

* Remove unused status interface

* Add status check for "Ready"

* Golint

* Generate CRD Manifests

* Reorder RBAC Roles

* Change file namings

* Remove CLI part for now

* Add internal build context for operator

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Charts, docs, readability and other changes (#8)

* Fix Typos

* Implement object deletion

* Add prefix for logging

* Remove unused status interface

* Add status check for "Ready"

* Golint

* Generate CRD Manifests

* Reorder RBAC Roles

* Change file namings

* Remove CLI part for now

* Add internal build context for operator

* Remove Rust-related things

* Add comment docs and reorder packages

* Change variable name and add package docs

* Add package docs

* Add package docs

* Rename var for readability

* Adds package docs

* Remove commented code

* Remove unused admin echo server and rename current one

* Add more logging and docs

* Fix doc

* Update readme on cli

* Add scaffold to operator chart

* Add docs

* Golint

* Update name in kustomization

* Add descriptions to fields

* Update default image name

* Change resource names

* Add version tag to kustomization

* Add make command for charts

* Create base values file

* Add patch for image name

* Add environment variable to manager

* Update charts/operator/Chart.yaml

Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>

* Update charts/operator/Chart.yaml

Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>

* Correct dir name and reuse step

* Rename dir to keda-http-operator

* Add envs to deployment pkg

* Add new statuses for extended objetct

* Fixing interceptor listening port

* Include new "Terminating" status

* Add namespaces for logs

* Change variable names

* Add new statuses

* Code reorder

* Create interceptor

* Add missing return

* Remove comment

* Fix link

* Remove all references to external scaler address

This is because we'll use one external scaler for each new application, so there's no point in having a single environment variable controlling the
address of the external scaler.

The address will be defined by the operator for each new app and a new scaler will be created.

* Fixing external scaler port

* Code reorder and creation of external scaler

* Improve code readability

* Golint

* Add colons

* Further clean the code

* Move functions to other file

* Rename files

* Add more meaninful names

* Rename method

* Update roles and enums

* Make yamls

* Add next TODO

* Add todo

Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Fixing the operator Dockerfile

also adding env vars to allow users to customize docker image names

Fixes https://github.com/kedacore/http-add-on/issues/5

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Updating to the KEDA 2.0 external scaler gRPC interface

The notable change here is that there is an added StreamIsActive
RPC call. The New and Close RPCs are removed.

Fixes https://github.com/kedacore/http-add-on/issues/3

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Making the scaler talk to interceptors to get pending HTTP queue sizes

Fixes https://github.com/kedacore/http-add-on/issues/4

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Initial install docs

Includes various tweaks needed to make install work

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more progress

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* proper port and path

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* splitting up helm chart

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fix chart, example yaml, and fix panic in operator logging

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing log errors

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* minor changes

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* gracefully moving on when deleting kube resources

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* creating dynamic client

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* using proper group for scaled objects

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding scaledobject to ClusterRole

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Helm installation in the Makefile

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more loggign

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* properly specifying namespaces etc...

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* setting operator deployment pull policy to always

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding more logs to resource creation

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing namespace problems and simplifying code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* proper API version

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* aking image names and ports for external scaler and interceptor configurable

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding docs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* proper env var names

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Specifying proper external scaler ports

Also moving the application config in the operator into operator/config

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* ports

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* updated ScaledObject spec

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* proper port formatting for interceptor

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* more logging and not requeueing

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* cscaler => kedahttp

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* trying something out

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding finalizer, simplifying get code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* finalizers

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* ignoring more stuff

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing the right things

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* updating deletion & error logic

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* logging status updates

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* check for already exists inside specific creator logic

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Fiddling with status

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* upgrading CRD version

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding instructions on microk8s installs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Restoring the status file

Don't know why I deleted it in 021b5f1b76

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* myapp => xkcd

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* service type

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* logging

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* configuring services properly

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* readme

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

Co-authored-by: Lucas Santos <lhs.santoss@gmail.com>
Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>
2021-01-27 11:41:52 -08:00