spark-operator

Commit Graph

Author	SHA1	Message	Date
Shingo Omura	6fc3696bbd	Implement Suspend/Resume reconciliation in SparkApplication controller and its unit tests Signed-off-by: Shingo Omura <everpeace@gmail.com>	2025-04-09 20:12:41 +09:00
Thomas Newton	d815e78c21	Robustness to driver pod taking time to create (#2315 ) * Retry after driver pod now found if recent submission Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add a test Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Make grace period configurable Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Update test Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add an extra test with the driver pod Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Separate context to create and delete the driver pod Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Tidy Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Autoformat Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Update error message Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add helm paramater Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Update internal/controller/sparkapplication/controller.go Co-authored-by: Yi Chen <github@chenyicn.net> Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Newlines between helm tests Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> --------- Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> Co-authored-by: Yi Chen <github@chenyicn.net>	2024-12-04 12:58:59 +00:00
Thomas Newton	5ff8dcf350	`omitempty` corrections (#2255 ) * Still working on tests Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Maybe progress Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * First working validation Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Lots of cleanup needed but it actually reproduced Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Working but ugly get schema from CRD Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Satisfactory test Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add missing omitempty for optional values Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Remove omitempty on required fields Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Run update-crd Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Remove temp schema config Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Tidy Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * go import Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Cover more test cases Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add tests that spec and metadata are required Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add tests against error content Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * `go mod tidy` Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Fix lint Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Remove test - hopefully we can add a better test as a follow up Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Make `mainApplicationFile` required Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Regenerated api-docs Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> * Add `MainApplicationFile` in tests Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com> --------- Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>	2024-10-18 11:06:20 +00:00
Yi Chen	3acd0f1a90	remove redundant test.sh file (#2243 ) * Fix go lint error Signed-off-by: Yi Chen <github@chenyicn.net> * Remove test.sh Signed-off-by: Yi Chen <github@chenyicn.net> --------- Signed-off-by: Yi Chen <github@chenyicn.net>	2024-10-14 01:23:16 +00:00
Jacob Salway	718e2444a4	Upgrade to Spark 3.5.3 (#2202 ) * Upgrade to Spark 3.5.3 Signed-off-by: Jacob Salway <jacob.salway@gmail.com> * Check result of err Signed-off-by: Jacob Salway <jacob.salway@gmail.com> --------- Signed-off-by: Jacob Salway <jacob.salway@gmail.com>	2024-10-13 15:43:15 +00:00
Sébastien Maintrot	a8b5d644b5	implement an upper bound limit to the number of tracked executor (#2181 ) * implement an upper bound limit to the number of tracked executor Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com> * add upper bound limit to the number of tracked executor to helm chart Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com> --------- Signed-off-by: ImpSy <3097030+ImpSy@users.noreply.github.com>	2024-10-11 05:54:10 +00:00
Yi Chen	0dc641bd1d	Use controller-runtime to reconsturct spark operator (#2072 ) * Use controller-runtime to reconstruct spark operator Signed-off-by: Yi Chen <github@chenyicn.net> * Update helm charts Signed-off-by: Yi Chen <github@chenyicn.net> * Update examples Signed-off-by: Yi Chen <github@chenyicn.net> --------- Signed-off-by: Yi Chen <github@chenyicn.net>	2024-08-01 12:29:06 +00:00

7 Commits