kubeflow
/
spark-operator
mirror of https://github.com/kubeflow/spark-operator.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,100 Commits 14 Branches 100 Tags 62 MiB
Commit Graph

17 Commits

Author SHA1 Message Date
Yi Chen 85b209ec20
Remove v1beta1 API (#2516) 
Signed-off-by: Yi Chen <github@chenyicn.net>
 2025-05-03 00:00:04 +00:00
Jacob Salway 25ca90cb07
Support Kubernetes 1.32 (#2416) 
Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
Signed-off-by: Jacob Salway <jacob.salway@rokt.com>
 2025-02-12 12:02:29 +00:00
Yi Chen 2375a306f9
Move sparkctl to cmd directory (#2347) 
* Move spark-operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Move sparkctl to cmd directory

Signed-off-by: Yi Chen <github@chenyicn.net>

* Remove unnecessary app package/directory

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
 2024-12-06 05:08:00 +00:00
Thomas Newton d815e78c21
Robustness to driver pod taking time to create (#2315) 
* Retry after driver pod now found if recent submission

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Add a test

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Make grace period configurable

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Update test

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Add an extra test with the driver pod

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Separate context to create and delete the driver pod

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Tidy

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Autoformat

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Update error message

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Add helm paramater

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Update internal/controller/sparkapplication/controller.go

Co-authored-by: Yi Chen <github@chenyicn.net>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

* Newlines between helm tests

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>

---------

Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Co-authored-by: Yi Chen <github@chenyicn.net>
 2024-12-04 12:58:59 +00:00
C. H. Afzal a261523144
The webhook-key-name command-line param isn't taking effect (#2344) 
Signed-off-by: C. H. Afzal <c-h-afzal@outlook.com>
 2024-12-04 09:18:01 +00:00
Sébastien Maintrot a8b5d644b5
implement an upper bound limit to the number of tracked executor (#2181) 
* implement an upper bound limit to the number of tracked executor

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

* add upper bound limit to the number of tracked executor to helm chart

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

---------

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
 2024-10-11 05:54:10 +00:00
Yi Chen 143b16ee75
Update integration test workflow and add golangci lint check (#2197) 
* Update integration test workflow

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update golangci lint config

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
 2024-10-08 02:06:07 +00:00
Jacob Salway 56b4974310
Fix ingress capability discovery (#2201) 
Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
 2024-09-29 07:07:37 +00:00
Sébastien Maintrot d37a0e938a
FEATURE: add cli argument to modify controller workqueue ratelimiter (#2186) 
* add cli argument to modify controller workqueue ratelimiter

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

* add cli argument to modify controller workqueue ratelimiter support to helm chart

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

---------

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
 2024-09-29 01:47:37 +00:00
Sébastien Maintrot e2cc295204
FEATURE: build operator image as non-root (#2171) 
Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
 2024-09-23 03:18:31 +00:00
Sébastien Maintrot 75b926652b
Feature: Add pprof endpoint (#2164) 
* add pprof support to the operator Controller Manager

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

* add pprof support to helm chart

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>

---------

Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>
 2024-09-16 10:53:25 +00:00
Yi Chen e8d3de9e1a
Support extended kube-scheduler as batch scheduler (#2136) 
* Support coscheduling with kube-scheduler plugins

Signed-off-by: Yi Chen <github@chenyicn.net>

* Add example for using kube-schulder coscheduling

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
 2024-09-03 03:23:13 +00:00
Yi Chen c93b0ec0e7
Adding support for setting spark job namespaces to all namespaces (#2123) 
Signed-off-by: Yi Chen <github@chenyicn.net>
 2024-09-03 02:55:14 +00:00
Jacob Salway 9cc1c02c64
Add default batch scheduler argument (#2143) 
* Add default batch scheduler argument

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add helm unit test

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
 2024-08-28 02:53:03 +00:00
Neo 52f818d535
fix: Add default values for namespaces to match usage descriptions (#2128) 
* fix: Add default values for namespaces to match usage descriptions

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

* fix: remove incorrect cache settings

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>

---------

Signed-off-by: pengfei4.li <pengfei4.li@ly.com>
Co-authored-by: pengfei4.li <pengfei4.li@ly.com>
 2024-08-22 16:24:58 +00:00
Jacob Salway 8fcda12657
Support gang scheduling with Yunikorn (#2107) 
* Add Yunikorn scheduler and example

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add test cases

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add code comments

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Add license comment

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Inline mergeNodeSelector

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

* Fix initial number implementation

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>

---------

Signed-off-by: Jacob Salway <jacob.salway@gmail.com>
 2024-08-22 04:15:57 +00:00
Yi Chen 0dc641bd1d
Use controller-runtime to reconsturct spark operator (#2072) 
* Use controller-runtime to reconstruct spark operator

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update helm charts

Signed-off-by: Yi Chen <github@chenyicn.net>

* Update examples

Signed-off-by: Yi Chen <github@chenyicn.net>

---------

Signed-off-by: Yi Chen <github@chenyicn.net>
 2024-08-01 12:29:06 +00:00