* feat(frontend): make container name configurable
Add an UIConfig param to specify the container name in a pod
where logs are retrieved. The default container name for
the Argo is `main`. However, in Tekton, the main container name
contains the `step-` prefix. Make this configurable to support
other orchestrations.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* fix test case error
add `undefined` as the expected `logContainerName` param
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* address the comment
use default value `main` in `getPodLogsStreamFromK8s`
same as `getPodLogs`
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* only pass podLogContainerName
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
---------
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* feat: customizable tensorboard image and env vars
* feat: sample pipeline using tensorboard visualization with minio
* change podtemplatespec format to be JSON in mlpipeline-ui-metadata
* fix default value
* update test config
* increase test timeout
* fix test
* fix args
* fix
* address comments
* improve component logging
* support local file storage type for local volume mount path, refer: https://github.com/kubeflow/pipelines/issues/4208
* add todo comment to support directory and filePath include wildcards '*', detail refer: https://github.com/kubeflow/pipelines/issues/4208
* revert old code indent
* run 'npm run format' to format code
* support tensorboard viewer and other visualize Results using volume mount path, modify 'file' schema to 'volume':
1. source schema: volume://volume-name/relative/path/from/volume/xxx.csv
2. for tensorboard(also support Series1:volume://volume-name/path_to_model_dir_1,Series2:volume://volume-name/path_to_model_dir_2):
* check volume-name was specified in podTemplateSpec( which was inject by VIEWER_TENSORBOARD_POD_TEMPLATE_SPEC_PATH env)
* check /relative/path/from/volume/xxx file path was prefix-mounted in podTemplateSpec
3. for others:
* check volume-name was specified at ml-pipeline-ui pod
* check /relative/path/from/volume/xxx.csv file path exist
* fix test and add more tests
* change error message not found to not exist.
* fix tensorboard create test
* combining volume mount path and key as artifacts path
* extra complex code to a function and add more test
* use ml-pipeline-ui container name to find server container instead of use containers[0]
* fix review suggestion: https://github.com/kubeflow/pipelines/pull/4236
* format code
* extract how to find file path on a pod volume to a common function, and optimize error message
* fix k8s-helper.test error
* add more documentation and fix mistake: volumeMountPath to filePathInVolume
* fix test error
* Update k8s-helper.test.ts
* format error message
Co-authored-by: Yuan (Bob) Gong <gongyuan94@gmail.com>
* Authorization service proto
* implement auth service
* Add unit tests
* Generate auth api client
* Authorization checks for tensorboard apis
* UI Server authorization checks
* Clean up error parsing
* Revert changes
* Fix portable-fetch not found bug
* Fix unit test
* Include portable-fetch required by api client
* Fix portable-fetch module import error
* Fix portable-fetch again
* Add unit tests
* Address CR comments
* add unit test for header
* Update readme
* [UI Server] Add namespace argument for tensorboard endpoints
* Allow local node server to talk to minio in cluster
* Use tensorboard namespace in UI
* Add unit tests for tensorboard UI server
* Fix tests
* Fix tensorboard proxy url
* Fix tensorboard proxy failure
* Fix tests
* Remove unecessary encodeURIComponent
* Add old comment back
* [UI Server] Pod info handler
* [UI] Pod info tab in run details page
* Change pod info preview to use yaml editor
* Fix namespace
* Adds error handling for PodInfo
* Adjust to warning message
* [UI] Pod events in RunDetails page
* Adjust error message
* Refactor k8s helper to get rid of in cluster limit
* Tests for pod info handler
* Tests for pod event list handler
* Move pod yaml viewer related components to separate file.
* Unit tests for PodYaml component
* Fix react unit tests
* Fix error message
* Address CR comments
* Add permission to ui role
* Parametrize the image used for the Tensorboard viewer
Right now, the image is hard-coded and it's not possible to override it.
This is a problem when running Kubeflow Pipelines in a cluster without
access to the internet or public repositories (e.g.
https://github.com/kubeflow/pipelines/issues/3232). This solves the
issue by parametrizing the image name through an environment
variable.
* Fix formatting issues
* Address review comments
* Add unit tests for aws-helper and minio-helper.
* Break up server.ts into app.ts and handlers/*.ts so that unit test can be written more easily.
* update @types/node because of https://github.com/microsoft/TypeScript/issues/32333
* upgrade kubernete-client version to be compatible with new @types/node version
* fix bug: AWSInstanceProfileCredentials.profile() returns a promise not a string
* remove I prefix from interfaces. minor fixes on lint errors.
* fix format errors
* Added more unit tests, and minor fixes.
* Fix comment: GKS should be GKE
* use jest.resetAllMocks instead of reset each mock individually
* fix format for helper test
* Support select tensorflow image for tensorboard
* modify test for tensorflow version select
* delete not available image entry
* Support tensorflow image selection to run tensorboard
* format code with prettier
* use HasPrefix instead of regexp
* delete
* modified tensorboard test
* delete tensorboard
* modify typo
* test tensorboard
* tensorboard test
* fuck
* fuck2
* modify test
* modify typo in tensorboard hint
* npm run format
* modify tensorboard snapshot
* compatible with previous kfp version. Allow vacant tensorflowImage field.
* add 2 tests for dialog
* modify default tensorflow image to 1.13.2
* merge get version and get tensorboard; let --bind_all support tensorboard3.x
* modify reconciler.go
* reconciler rollback
* modify corresponding test for --bind_all
* modify requested chances 12/23
* formControl sorted alphabetically
* select sorted alphabetically
* modify details from PR request 12/24
* moidfy format
* modify details 12/23
* modify snapshot
* retest
* retest
* Format other frontend code using prettier
* Regenerate frontend api clients
* Pass namespace to CreateRun api request
* Fetch logs from pod namespace
* Fix log handler cannot work with pod name only query bug
* Fix and refactor existing tests
* Unit test adding namespace to create run api call.
* Remove unneeded snapshot
* Fix lint errors
* Consistently use resource reference id for namespace
* Script to set up workload identity for standalone deployment
* Migrate tests to run on standalone + workload identity
* Fix test script
* Switch to static GSAs for testing, because they have name length limit
* Add workload identity binding for argo
* Fix argo workload identity bindings
* Remove user-gcp-sa from tests
* Remove use_gcp_secret from xgboost sample
* Allow debugging tests locally
* Wait for policies to take effect
* Update deploy-pipeline-lite.sh
* Update deploy-pipeline-lite.sh
* [WIP] test gcloud auth list with test-runner sa
* Add namespace
* test again
* Use new image builder
* test again
* Remove debug code
* Remove usages of use_gcp_secret
* Fix unit test and tensorboard pod template
* Add debug code again to test
* Try waiting until workload identity bindings are ready
* Fix some other samples
* Fix parameterized tfx oss sample
* Add retry to image building
* Try fixing tfx oss sample
* Fix compiled tfx oss sample
* Update all google/cloud-sdk to latest
* Try fixing parameterized tfx oss sample again
* Also verify pipeline-runner ksa is working
* Fix parameterized_tfx_oss sample
* Update gcp-workload-identity-setup.sh
* Revert unneeded change
* Pin to new google/cloud-sdk
* Remove wrongly commited binaries
* Without version bump
* fix the delete caller
* return after delete
* reconciler removes old viewer crd file that misses image specification
* add frontend comment
* remove accidental changes that are irrelevant
* Revise log message
* Add error handling
* add test
* tensorflow image check only applies to viewer tensorboard type and thus put it after the type check.
* Use of default image instead of validation
* Retrieve pod logs from argo archive
* Added aws instance profile iam credential support for minio client. Read workflow status for argo archive location for pod logs.
* fix minor typo, and enforce typing for minio client options
* Update helm chart for pipelines ui role with permission to access secret and workflow crd
* remove unnecessary type cast
* Fix bug: s3client should be a callable, so that iam token is refreshed
* Add a helper function to create a tensorboard crd.
* Add a helper function to create a tensorboard crd.
* Remove optional parameter "pretty".
* Reuse constants when possible
* createTensorboardInstance, getTensorboardInstance, waitForTensorboardInstance
* Revise getTensorboardHandler and createTensorboardHandler to use getTensorboardInstance
and createTensorboardInstance
* remove trailing spaces
* Fix comment
* Ajust paremeter values.
* stringfy error objects; change 'plural' to 'viewer'
* Change 'plural' to 'viewers'
* Use derived namespace (instead of hardcoded kubeflow) to create viewer
CRD. Derive viewer CRD's name from log dir.
* Adjust local dns
* Add GCP credentials for viewer resource.
* Address pr reviews
* auto formatting
* 1. viewer controller propagates credentials when creating pod
2. set credentials in viewer crd
* Put credentials info in PodTemplateSpec
* Cleanup not-to-be-included changes.
* Empty line
* Formatting
* Remove a duplicate include
* add crypto-js dependency
* Add path prefix
* Update k8s-helper.ts
* (1) set viewer service port be 6006 (2) trailing / is required for tensorboard path
* Remove url address encoding
* Remove an accidental change
* Formatting
* Add comment and TODO
* Change 6006 back to 80
* Remove deprecated functions from k8shelper. Revise some comments.
* Use getNamespacedCustomobject
* Try to fix unit test for the tensorboard.tsx change
* Re-arrange func order
* Polish comments
* When there is no existing tensorboard, GetNamespacedCustomObject's promise is reject, and in case of that reject, use ''
Yihong Wang
Yuan (Bob) Gong
Alexey Volkov
haibingzhao
Yuan (Bob) Gong
Rafael Barreto
Eterna2
dldaisy
jingzhang36
Riley Bauer
Yasser Elsayed
Yang Pan
Pascal Vicaire