Commit Graph

19 Commits

Author SHA1 Message Date
Yuan (Bob) Gong cadcac08bd
feat: customizable tensorboard image and env + sample pipeline with minio. Fixes #5471, fixes #4714, fixes #5449, fixes #699, fixes #4290 (#5515)
* feat: customizable tensorboard image and env vars

* feat: sample pipeline using tensorboard visualization with minio

* change podtemplatespec format to be JSON in mlpipeline-ui-metadata

* fix default value

* update test config

* increase test timeout

* fix test

* fix args

* fix

* address comments

* improve component logging
2021-05-11 00:45:42 -07:00
Alexey Volkov cc83e1089b
Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
StefanoFioravanzo 0c93397135
feat(frontend): Hide sidebar in Kubeflow mode. Fixes #5199 (#5200)
* Introduce HIDE_SIDENAV env var

Signed-off-by: Stefano Fioravanzo <stefano@arrikto.com>

* Hide sidebar when HIDE_SIDEBAR is true

Signed-off-by: Stefano Fioravanzo <stefano@arrikto.com>

* Run prettier

Signed-off-by: Stefano Fioravanzo <stefano@arrikto.com>
2021-03-09 07:02:24 -08:00
DavidSpek 0df9473bba
feat: Set current namespace for in-cluster SDK in multi-user mode and add healthz endpoint to API backends (#4638)
* Set current namespace in local KFP context if running from notebook

* Create "~/.config/kfp/" instead of ".config/kfp/"

At first it was assumed the `get_user_namespace` command would be executed from the home directory.

* Create local context file if it doesn't exist during set_user_namespace

* Grab path from LOCAL_KFP_CONTEXT when creating folder

Instead of harcoding the os.mkdirs path to `~/.config/kfp` it now grabs it from the LOCAL_KFP_CONTEXT. Also, removed path creation in `get_user_namespace` as that is now handled in `set_user_namespace`. Also, it now checks if the path exists rather than the local_context_file to remove the situation where it tries to create ~/.config/kfp/ because the context.json doesn't exist when the path does.

* add multi-user setting to healthz api

* Add http prefix to health api url

* move healtz api call to own function and fix multi_user boolean

* Fix HEALTH_PATH declaration

* Move check to Client __init__ and change get_kfp_healthz to avoid breaking in case of old apiserver image

* Add multi_user to frontend healthz

* Expose multi_user in frontend and add integration test

* Fix integration test

* Fix host hardcoding and error handling

* Handle empty API response, check if API up to date

* Fix response return

* remove API check due to empty response

* retry API call if first response empty

* retry getting healthz api if no response

* change health_api to https

The healthz_api has been returning empty responses which might be caused by sending an http request to an https endpoint. Although requests handles redirects, this commit is to test if this solves the issue.

* Add some debug info to healthz exception

* add url to debug and lower retries to 1

* Use api_client to get healthz data

* Debug info for API response

* Follow API redirect history

* Fix indentation

* Add healthz proto

* Try getting healthz api with new python backend

* Add installation of kfp_server_api in tests

* Fix incorrect setup location

* Replace old .get with new http backend .multi_user

* Code clean up

* Small fixes and TimeOutError for retries healthz api

* Remove changes to go dependencies

* Send empty proto request and fix exception client

* Remove unused commit_sha and tag_name
2020-11-24 15:36:39 -08:00
Yuan (Bob) Gong e3992faf83
feat(frontend): UX change to support downloading directory artifacts. Fixes #3667 (#4696)
* feat(frontend): change artifact link to download without extracting. Fixes #3667

* update artifact preview UX

* Fix tests

* add frontend unit tests

* fix

* move artifact integration tests to its own file

* fixed flaky aws-helper test

* refactor and add ui server integration tests

* Update UX according to feedback

* update snapshots

* Update minio-helper.ts

* update
2020-11-03 05:36:15 -08:00
haibingzhao 7f77e8bc7f
feat(frontend): support tensorboard viewer and other visualize Results using volume mount path. Part of #4208 (#4236)
* support local file storage type for local volume mount path, refer: https://github.com/kubeflow/pipelines/issues/4208

* add todo comment to support directory and filePath include wildcards '*', detail refer: https://github.com/kubeflow/pipelines/issues/4208

* revert old code indent

* run 'npm run format' to format code

* support tensorboard viewer and other visualize Results using volume mount path, modify 'file' schema to 'volume':

1. source schema: volume://volume-name/relative/path/from/volume/xxx.csv
2. for tensorboard(also support Series1:volume://volume-name/path_to_model_dir_1,Series2:volume://volume-name/path_to_model_dir_2):
* check volume-name was specified in podTemplateSpec( which was inject by VIEWER_TENSORBOARD_POD_TEMPLATE_SPEC_PATH env)
* check /relative/path/from/volume/xxx file path was prefix-mounted in podTemplateSpec
3. for others:
* check volume-name was specified at ml-pipeline-ui pod
* check /relative/path/from/volume/xxx.csv file path exist

* fix test and add more tests

* change error message not found to not exist.

* fix tensorboard create test

* combining volume mount path and key as artifacts path

* extra complex code to a function and add more test

* use ml-pipeline-ui container name to find server container instead of use containers[0]

* fix review suggestion: https://github.com/kubeflow/pipelines/pull/4236

* format code

* extract how to find file path on a pod volume to a common function, and optimize error message

* fix k8s-helper.test error

* add more documentation and fix mistake: volumeMountPath to filePathInVolume

* fix test error

* Update k8s-helper.test.ts

* format error message

Co-authored-by: Yuan (Bob) Gong <gongyuan94@gmail.com>
2020-08-06 01:06:55 -07:00
Yuan (Bob) Gong 5435e8724f
[UI Server] Refactor for configurable auth header (#3753)
* [UI] Make auth header configurable

* Refactor authorizeFn to move side effects out

* Refactor tests to reduce duplication
2020-05-12 22:06:25 -07:00
Yuan (Bob) Gong e8356bc37a
Show version tag in UI (#3743)
* Show version tag in UI

* Add new arguments to test cloudbuild configuration

* backend cloudbuild should use commit_sha as argument

* Fix minor bug during dev
2020-05-12 11:02:21 -07:00
Yuan (Bob) Gong cb71eed5f9
[UI] authorize tensorboard actions (#3639)
* Authorization service proto

* implement auth service

* Add unit tests

* Generate auth api client

* Authorization checks for tensorboard apis

* UI Server authorization checks

* Clean up error parsing

* Revert changes

* Fix portable-fetch not found bug

* Fix unit test

* Include portable-fetch required by api client

* Fix portable-fetch module import error

* Fix portable-fetch again

* Add unit tests

* Address CR comments

* add unit test for header

* Update readme
2020-04-28 15:26:24 -07:00
Yuan (Bob) Gong b7706c0134
[UI Server] Enable strict type checking and fix errors (#3593)
* wip

* Fix typing

* Fix build error

* Add type checking to tests

* Fix server typing

* Clean up

* Fix server typing
2020-04-23 09:25:11 -07:00
Yuan (Bob) Gong 47cde0b6b9
[UI] Tensorboard support for multi user (#3355)
* [UI Server] Add namespace argument for tensorboard endpoints

* Allow local node server to talk to minio in cluster

* Use tensorboard namespace in UI

* Add unit tests for tensorboard UI server

* Fix tests

* Fix tensorboard proxy url

* Fix tensorboard proxy failure

* Fix tests

* Remove unecessary encodeURIComponent

* Add old comment back
2020-03-26 19:50:22 -07:00
Yuan (Bob) Gong 0ef074239d
[UI Server] Blocks non public KFP report APIs (#3334)
* [UI Server] Blocks reportMetrics KFP api

* Also reject report workflow endpoint

* Also block report swf endpoint
2020-03-23 23:34:47 -07:00
Eterna2 a7606a12de
kfp UI node server support preview and handles gzip, tarball, and raw artifacts in a consistent manner. (#2992) 2020-03-23 22:30:47 -07:00
Yuan (Bob) Gong f882f361cc
[UI] Show step pod yaml and events in RunDetails page (#3304)
* [UI Server] Pod info handler

* [UI] Pod info tab in run details page

* Change pod info preview to use yaml editor

* Fix namespace

* Adds error handling for PodInfo

* Adjust to warning message

* [UI] Pod events in RunDetails page

* Adjust error message

* Refactor k8s helper to get rid of in cluster limit

* Tests for pod info handler

* Tests for pod event list handler

* Move pod yaml viewer related components to separate file.

* Unit tests for PodYaml component

* Fix react unit tests

* Fix error message

* Address CR comments

* Add permission to ui role
2020-03-20 02:42:38 -07:00
Kubernetes Prow Robot 498b9832a6
[Frontend] GKE metadata endpoint should fail when fetched response is not ok (#3218)
* [Frontend] Add mock data for gke metadata

* Fix GKE metadata endpoint when request is not ok
2020-03-05 00:18:36 -08:00
Yuan (Bob) Gong 3b072b2ff7
[UIServer] Add DISABLE_GKE_METADATA env flag to skip metadata retrieval. (#3118)
* Add DISABLE_GKE_METADATA env flag to skip metadata retrieval.

* node server build step fix and cleanup

* Update frontend dockerfile to use npm ci
2020-02-20 03:35:02 -08:00
Yuan (Bob) Gong 4709c6b42f
[UI] Getting started page for hosted pipelines (#2935)
* Implement getting started page.

* Add feature flag to only show getting started page on hosted pipelines

* Add tests

* Fix format

* Implement requested layout in getting started page

* Minor adjust layout

* Fix tests

* Fix snapshots

* Update page title
2020-01-31 12:35:20 -08:00
Yuan (Bob) Gong 7e19d7fa6c [UI] fix artifact handler query parameter key (#2809)
* Fix node server typing problems

* Fix artifact handler query name
2020-01-07 19:53:40 -08:00
Eterna2 2ec2fd23c5 [Frontend] unit tests for node server (#2745)
* Add unit tests for aws-helper and minio-helper.

* Break up server.ts into app.ts and handlers/*.ts so that unit test can be written more easily.

* update @types/node because of https://github.com/microsoft/TypeScript/issues/32333

* upgrade kubernete-client version to be compatible with new @types/node version

* fix bug: AWSInstanceProfileCredentials.profile() returns a promise not a string

* remove I prefix from interfaces. minor fixes on lint errors.

* fix format errors

* Added more unit tests, and minor fixes.

* Fix comment: GKS should be GKE

* use jest.resetAllMocks instead of reset each mock individually

* fix format for helper test
2020-01-05 20:43:40 -08:00