Commit Graph

8 Commits

Author SHA1 Message Date
Aaron Schlesinger c211da9bd1
Multi-tenant interceptor and scaler (#206)
* multi-tenant interceptor and scaler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* specifying host in XKCD ingress

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* routing the xkcd chart to the interceptor properly

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* check host header first

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* sending true active response in stream

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* passing target pending requests through to the underlying ScaledObject (so the scaler can read it later)

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* removing broken target pending requests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* using getHost in proxy handler

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding integration test

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding more tests to the integration test

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* splitting up integration tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* more checks

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* mark new test TODO

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* expanding interceptor integration tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* error messages

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* refactor test

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* more test improvements

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* rolling back target pending requests in ScaledObject

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* removing target metric error. it's not used anymore

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* improving waitFunc test

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* Refactoring the deployment cache to add better error handing and resilience.

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding doc comment

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* refactoring deploy cache and adding tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* Using interfaces for deployment watch & list

this makes tests easier

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding more deploy cache tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* Fixing up TestK8sDeploymentCacheRewatch

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* shutting down everything else when one thing errors, and adding a deployments cache endpoint

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* removing commented code

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* clarifying deployment cache JSON output, and simplifying deployment watch function

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding TODO tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* error logs and restoring the count middleware

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* using consistent net/http package name throughout main.go

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* Refactoring deployment cache deployment storage

Also, running go mod tidy and adding new TODO (i.e. failing) tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* using deployment.Status.ReadyReplicas, instead of just replicas

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* integration_tets ==> proxy_handlers_integration_test

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding some resilience to tests

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding deployment cache endpoint documentation

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* running the global test target with sh.RunV

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* adding timeout to magefile test target

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>

* finishing one TODO test and adding issue for the rest:

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2021-09-03 10:19:20 +02:00
Aaron Schlesinger 22c912071b
Fixing app creation timing issue (#116)
* upgrading all dockerfile Go versions to 1.16

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* creating new targets for the example/test HTTPScaledObject

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* moving external scaler generated files

the scaler uses the server stubs but the operator now needs to use the client, so it needs to move out of the scaler binary

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* progress on code to wait for scaler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* checking deployment replicas:

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding health check endpoints to the scaler

Also centralizing config for the scaler with envconfig

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Closing both the gRPC server and HTTP healthcheck server on context cancel/timeout

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding util functions for adding liveness/readiness probes

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing var name

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding readiness and liveness probes, and tests for it

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* printing output for scaler tests, even when they fail

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* simplifying scaler health check code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding health check test code to scaler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-05-18 18:55:47 +00:00
Lucas Santos d6a2265041
Use embedded scaledobject templates (#106)
* Tidy mod file

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Remove KEDA unstructured types

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Update tests to use go 1.16

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Add missing check and return errors

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Add error handling

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2021-03-24 14:59:28 -07:00
Aaron Schlesinger 2b395d1548
Scaling up the interceptor (#32)
* Scaling up the interceptor

Fixes https://github.com/kedacore/http-add-on/issues/30

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Specifying min and max replicas on ScaledObjects

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* respecting the min and max replica count

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding new enum types to kubebuilder definition

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* re-generating CRD

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding check for interceptor scaled object to tests

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* generalizing unstructured parsing code in test

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-03-22 12:06:35 -07:00
Aaron Schlesinger fd4cdfe718
Not creating app deployments (#78)
* Allowing users to scale an existing deployment

Fixes https://github.com/kedacore/http-add-on/issues/35

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* checking custom deployment info

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding tests for new deployment logic

Also generalizing test "infrastructure" code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* regenerating code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Remove functionality to auto-create deployments and services

Users provide a scale target ref, which is the name of the
deployment to scale and the service to route to. They are required
to have already deployed these things already

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding more docs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* regenerating with 0.5.0 controller-gen

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing compile err

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-03-19 12:58:29 -07:00
Lucas Santos 269d17c410
Add min and max replicas (#85)
* Add min and max replicas

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Go fmt

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Update example

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Fix wrong value key

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Using internal struct for replicas

Signed-off-by: Lucas Santos <hello@lsantos.dev>

* Update manifests

Signed-off-by: Lucas Santos <hello@lsantos.dev>

* Fix wrong naming

Signed-off-by: Lucas Santos <hello@lsantos.dev>

* Create tests

Signed-off-by: Lucas Santos <hello@lsantos.dev>

Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2021-03-15 16:26:36 -07:00
Lucas Santos ebef98e514
Improve status conditions (#47)
* Improve status conditions

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* refactoring typed Kubernetes clients out

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* finishing delete logic

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing now-unused code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing commented unused code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* fixing lint error

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Delete test file

PR #43 will address it

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Update deployment client

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Update scaled object client

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Add fix notation

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Move FIX to TODO

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Add context to requestCounts

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Adding an FAQ (#71)

Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Remove unused code

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Use two strings as opposed to interface

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Revert "Delete test file"

This reverts commit 46f4de7c0bf59ea9aa8d9a345762ccb8b652ea33.

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Restore pkg/k8s/client.go

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Add context to queue pinger

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

* Adding an FAQ (#71)

Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>

Signed-off-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Revert "Delete test file"

This reverts commit 46f4de7c0bf59ea9aa8d9a345762ccb8b652ea33.

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Restore pkg/k8s/client.go

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>
Sign-off-executed-by: Lucas Santos <lhs.santoss@gmail.com>
Approved-at: Aaron Schlesinger <aaron@ecomaz.net>

* Add context to queue pinger

Signed-off-by: Lucas Santos <lhs.santoss@gmail.com>

Co-authored-by: Aaron Schlesinger <aaron@ecomaz.net>
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2021-02-26 10:57:59 -08:00
Aaron Schlesinger 698a5ee0c2
Creating 2 interceptor services (#16)
* logging

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Adding helm delete functionality to makefile

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Splitting up functionality, logs and more

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* checking error fetching services

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Using the proper service name

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* enabling RBAC on endpoints so the scaler can access interceptor queue sizes

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* logs and TODOs

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* using IPs instead of hostnames in external scaler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* backing off polling interval

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* logging on external scaler startup

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing superfluous logging in scaler

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* port and updating namespaces

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* the right namespace

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding grpc reflection

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Makefile targets for creating/deleting example HTTPScaledObject

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Removing namespace from example HTTPScaledObject file

it is specified when you add it from the make target

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* adding keda creation logic to Makefile (and install doc)

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* helm delete keda target

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* Passing queue pinger around as a pointer

Otherwise the queue lengths are always reported as 0

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-01-29 16:46:31 -08:00