* fix(frontend): Introduce ALLOWED_ARTIFACT_DOMAIN_REGEX flag to prevent accessing undesired domains. Remove user input string from server response.
1. Remove user query parameter string from server response.
2. Introduce ALLOWED_ARTIFACT_DOMAIN_REGEX to allow configuration of allowed domains querying.
3. By default, ALLOWED_ARTIFACT_DOMAIN_REGEX is match all. Default configuration is meant to be demo purpose and not for production.
4. Users can provide ALLOWED_ARTIFACT_DOMAIN_REGEX environment variable to restrict object storage endpoint querying domains.
* add test
* feat(frontend): make container name configurable
Add an UIConfig param to specify the container name in a pod
where logs are retrieved. The default container name for
the Argo is `main`. However, in Tekton, the main container name
contains the `step-` prefix. Make this configurable to support
other orchestrations.
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* fix test case error
add `undefined` as the expected `logContainerName` param
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* address the comment
use default value `main` in `getPodLogsStreamFromK8s`
same as `getPodLogs`
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* only pass podLogContainerName
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
---------
Signed-off-by: Yihong Wang <yh.wang@ibm.com>
* Add apiVersion2 and apiVersion2Prefix in configs.ts
Support v2 request in frontend server.
* Fix incorrect prefix for v2.
* Fix incorrect reference for apiVersion2Prefix
Add v2 healthz handler().
* Add v2 pod / logs handler
Add v2 conditions for sending 403 to endpoints.
Add proxyMiddleware for v2 prefix.
* Add v2apis script in package.json
* Generate typescript v2 api client.
* Change api client codegen section in README
Make it more clear: put the jar file in frontend folder.
* Add v2 api codegen script in README.
* Remove beta1 text to avoid confusion in README.
* Format
* Rename folder as apisv2beta1.
Add apisv2beta1 in .eslintrc.yaml file
* Change script from v2apis to apis:v2beta1
Remove unnecessary folder v2generated (move files to
generated/apisv2beta1/
* Add the command for exporting jar file path to $PATH in README file.
* use aws js sdk for credentials concerning aws
* format code
* catch exception if no credentials are found after using credentialchain
* revert changes and add section for specific aws changes
* revert deleting aws-helper mock
* add privatelink test for isS3endpiont
* fix try block
* fix(frontend): Replace license-checker with yarn licenses.
* grant license sh permission. Upgrade caniuse-lite. Migrate to media for darkMode in tailwind css.
* Upgrade @types/node and note-fetch in frontend server
* npm ci in server
* improve readability
* WIP: Support Protobuf.Value in V2 IR
* chore(frontend): Convert to ts-proto for Protobuf.Value in PipelineSpec
* remove ValueType
* npm run format
* Add license for ts-proto-descriptors
* feat(frontend): Allow setting AWS_S3_ENDPOINT
Allows setting the S3 endpoint used by minio to allow regions other than us-east-1
* feat(frontend): allow to configure s3 bucket region for artifacts
* fix formatting
* add integration test for non-default s3 bucket region
* start adding description to version
* add description to resource manager
* remove description from pipeline default version
* update pipeline_store
* add some tests to ensure its being passed around
* update frontend generated api
* run auto formatter
* add comment to frontend readme about running npm run format
* add missing api converter step
* feat: customizable tensorboard image and env vars
* feat: sample pipeline using tensorboard visualization with minio
* change podtemplatespec format to be JSON in mlpipeline-ui-metadata
* fix default value
* update test config
* increase test timeout
* fix test
* fix args
* fix
* address comments
* improve component logging
* Set current namespace in local KFP context if running from notebook
* Create "~/.config/kfp/" instead of ".config/kfp/"
At first it was assumed the `get_user_namespace` command would be executed from the home directory.
* Create local context file if it doesn't exist during set_user_namespace
* Grab path from LOCAL_KFP_CONTEXT when creating folder
Instead of harcoding the os.mkdirs path to `~/.config/kfp` it now grabs it from the LOCAL_KFP_CONTEXT. Also, removed path creation in `get_user_namespace` as that is now handled in `set_user_namespace`. Also, it now checks if the path exists rather than the local_context_file to remove the situation where it tries to create ~/.config/kfp/ because the context.json doesn't exist when the path does.
* add multi-user setting to healthz api
* Add http prefix to health api url
* move healtz api call to own function and fix multi_user boolean
* Fix HEALTH_PATH declaration
* Move check to Client __init__ and change get_kfp_healthz to avoid breaking in case of old apiserver image
* Add multi_user to frontend healthz
* Expose multi_user in frontend and add integration test
* Fix integration test
* Fix host hardcoding and error handling
* Handle empty API response, check if API up to date
* Fix response return
* remove API check due to empty response
* retry API call if first response empty
* retry getting healthz api if no response
* change health_api to https
The healthz_api has been returning empty responses which might be caused by sending an http request to an https endpoint. Although requests handles redirects, this commit is to test if this solves the issue.
* Add some debug info to healthz exception
* add url to debug and lower retries to 1
* Use api_client to get healthz data
* Debug info for API response
* Follow API redirect history
* Fix indentation
* Add healthz proto
* Try getting healthz api with new python backend
* Add installation of kfp_server_api in tests
* Fix incorrect setup location
* Replace old .get with new http backend .multi_user
* Code clean up
* Small fixes and TimeOutError for retries healthz api
* Remove changes to go dependencies
* Send empty proto request and fix exception client
* Remove unused commit_sha and tag_name
* New server API: read run log
- The new server API endpoint (/apis/v1beta1/runs/{run_id}/nodes/{node_id}/log) to fetch run log
- `ARCHIVE_LOG_FILE_NAME` and `ARCHIVE_LOG_PATH_PREFIX` options allows to control archive log path
- UI Server fetches logs from server API or directly from k8s depending on `STREAM_LOGS_FROM_SERVER_API` option
* New server API: read run log
- ml-pipeline rbac update: allow for access to log
* Read run log: enhanced error handling
- log message on Pod access errors
* Read run log: enhanced log archive options
* Code format
* Test update after getPodLogs signature change
* Updated comments after review
* `follow` query parameter in GET /apis/v1beta1/runs/{run_id}/nodes/{node_id}/log
* Env variable friendly config names & comments
- Config options: ARCHIVE_CONFIG_LOG_FILE_NAME, ARCHIVE_CONFIG_LOG_PATH_PREFIX
- Copyright message update
- New endpoint as `v1alpha1`
* Licence updates
- fluent-bit licence inlined
- copyright message updates
* Master merge
- dependency conflicts
* support local file storage type for local volume mount path, refer: https://github.com/kubeflow/pipelines/issues/4208
* add todo comment to support directory and filePath include wildcards '*', detail refer: https://github.com/kubeflow/pipelines/issues/4208
* revert old code indent
* run 'npm run format' to format code
* support tensorboard viewer and other visualize Results using volume mount path, modify 'file' schema to 'volume':
1. source schema: volume://volume-name/relative/path/from/volume/xxx.csv
2. for tensorboard(also support Series1:volume://volume-name/path_to_model_dir_1,Series2:volume://volume-name/path_to_model_dir_2):
* check volume-name was specified in podTemplateSpec( which was inject by VIEWER_TENSORBOARD_POD_TEMPLATE_SPEC_PATH env)
* check /relative/path/from/volume/xxx file path was prefix-mounted in podTemplateSpec
3. for others:
* check volume-name was specified at ml-pipeline-ui pod
* check /relative/path/from/volume/xxx.csv file path exist
* fix test and add more tests
* change error message not found to not exist.
* fix tensorboard create test
* combining volume mount path and key as artifacts path
* extra complex code to a function and add more test
* use ml-pipeline-ui container name to find server container instead of use containers[0]
* fix review suggestion: https://github.com/kubeflow/pipelines/pull/4236
* format code
* extract how to find file path on a pod volume to a common function, and optimize error message
* fix k8s-helper.test error
* add more documentation and fix mistake: volumeMountPath to filePathInVolume
* fix test error
* Update k8s-helper.test.ts
* format error message
Co-authored-by: Yuan (Bob) Gong <gongyuan94@gmail.com>
* Show version tag in UI
* Add new arguments to test cloudbuild configuration
* backend cloudbuild should use commit_sha as argument
* Fix minor bug during dev
* Authorization service proto
* implement auth service
* Add unit tests
* Generate auth api client
* Authorization checks for tensorboard apis
* UI Server authorization checks
* Clean up error parsing
* Revert changes
* Fix portable-fetch not found bug
* Fix unit test
* Include portable-fetch required by api client
* Fix portable-fetch module import error
* Fix portable-fetch again
* Add unit tests
* Address CR comments
* add unit test for header
* Update readme
* WorkflowParser->loadNodeOutputPaths
source: s3.endpoint === 's3.amazonaws.com' ? StorageService.S3 : StorageService.MINIO
* Use isS3Endpoint (server/aws-helper.ts) to identify artifact source
* npm run format
* created src/lib/AwsHelper.ts (for frontend code), because frontend client and frontend server do not share code for now.
dependabot[bot]
nagar-ajay
James Liu
Chen Sun
Yihong Wang
Joe Li
ryansteakley
rrrkharse
Neer Friedman
Gerard Casas Saez
Yuan (Bob) Gong
Alexey Volkov
algs
StefanoFioravanzo
DavidSpek
Rafał Bigaj
Yuan (Bob) Gong
haibingzhao
Jonas De Beukelaer
Dmitry B