Commit Graph

88 Commits

Author SHA1 Message Date
Jiyong Jung e07ba451af
fix(sdk): fixes typo of 'desc'. (#4938) 2020-12-24 00:38:26 -08:00
Yuan (Bob) Gong 44fcda7dca
fix(sdk): make healthz exception visible in logs by default (#4904)
* fix(sdk): make healthz exception visible in logs by default

* only catch exceptions, but not keyboard interrupt

* address comments
2020-12-16 20:29:08 -08:00
capri-xiyue 2b718f1bbd
docs(sdk): update comments of _create_job_config (#4899)
* update comments of _create_job_config

* cleand up comments

* fixed comments
2020-12-16 06:16:22 -08:00
DavidSpek 0df9473bba
feat: Set current namespace for in-cluster SDK in multi-user mode and add healthz endpoint to API backends (#4638)
* Set current namespace in local KFP context if running from notebook

* Create "~/.config/kfp/" instead of ".config/kfp/"

At first it was assumed the `get_user_namespace` command would be executed from the home directory.

* Create local context file if it doesn't exist during set_user_namespace

* Grab path from LOCAL_KFP_CONTEXT when creating folder

Instead of harcoding the os.mkdirs path to `~/.config/kfp` it now grabs it from the LOCAL_KFP_CONTEXT. Also, removed path creation in `get_user_namespace` as that is now handled in `set_user_namespace`. Also, it now checks if the path exists rather than the local_context_file to remove the situation where it tries to create ~/.config/kfp/ because the context.json doesn't exist when the path does.

* add multi-user setting to healthz api

* Add http prefix to health api url

* move healtz api call to own function and fix multi_user boolean

* Fix HEALTH_PATH declaration

* Move check to Client __init__ and change get_kfp_healthz to avoid breaking in case of old apiserver image

* Add multi_user to frontend healthz

* Expose multi_user in frontend and add integration test

* Fix integration test

* Fix host hardcoding and error handling

* Handle empty API response, check if API up to date

* Fix response return

* remove API check due to empty response

* retry API call if first response empty

* retry getting healthz api if no response

* change health_api to https

The healthz_api has been returning empty responses which might be caused by sending an http request to an https endpoint. Although requests handles redirects, this commit is to test if this solves the issue.

* Add some debug info to healthz exception

* add url to debug and lower retries to 1

* Use api_client to get healthz data

* Debug info for API response

* Follow API redirect history

* Fix indentation

* Add healthz proto

* Try getting healthz api with new python backend

* Add installation of kfp_server_api in tests

* Fix incorrect setup location

* Replace old .get with new http backend .multi_user

* Code clean up

* Small fixes and TimeOutError for retries healthz api

* Remove changes to go dependencies

* Send empty proto request and fix exception client

* Remove unused commit_sha and tag_name
2020-11-24 15:36:39 -08:00
Alexey Volkov 5c1b919793
fix(sdk): Client - Fixed wait_for_run_completion (#4808)
The default timeout value is of type timedelta which cannot be compared with a plain number later in the code.
Also replaced time_delta.seconds with time_delta.total_seconds() which gives out correct value.
2020-11-22 15:19:34 -08:00
Tanguy Compagnon 950fae4921
fix(sdk): fixes too broad except when retrieving experiments. (#4765) 2020-11-12 15:08:25 -08:00
Marcin Owsiany a840ce75e3
fix(sdk): use a meaningful link text in ipython (#4570) 2020-10-07 04:30:42 -07:00
Padarn Wilson aed37c7792
feat(sdk): optional context in kubenetes client (#4401)
* adding optional context to client

* update naming of context -> kube_context
2020-09-29 14:33:24 -07:00
Yihong Wang c8b4599622
feat(sdk/cli): add extra header for cli (#4400)
In some cases, it may need extra header to handle the API calls.
Directly add `header_name` and `header_value` props in
`.config/kfp/context.json` and pass the name/value pair
to APIClient.

Here is one of the use cases:
if the service is protected by istio RBAC and need JWT header
for authentication, you can specify the JWT id-token in
the .config/kfp/context.json with these two new props.
The id-token would be carried in the specified header name.
Then the API call can be properly authentiecated and checked
if the user has the permission to access the service.
2020-09-22 11:06:42 -07:00
Jiaxiao Zheng f98252f3d7
chore: Corrects doc string of two client methods. (#4442)
* clean up doc string for two methods

* add missing arg doc

* add missing type hints.
2020-09-08 13:57:52 -07:00
Lars Schöning d7e5894912
fix(sdk): TypeError in Client.get_pipeline_id() when pipeline does not exist (#4319) 2020-08-05 07:40:28 -07:00
Alex Latchford 704c8c7660
chore: Clean up KFP SDK docstrings, make formatting a little more consistent (#4218)
* Prepare SDK docs environment so its easier to understand how to build the docs locally so theyre consistent with ReadTheDocs.

* Clean up docstrings for kfp.Client

* Add in updates to the docs for compiler and components

* Update components area to add in code references and make formatting a little more consistent.

* Clean up containers, add in custom CSS to ensure we do not overflow on inline code blocks

* Clean up containers, add in custom CSS to ensure we do not overflow on inline code blocks

* Remove unused kfp.notebook package links

* Clean up a few more errant references

* Clean up the DSL docs some more

* Update SDK docs for KFP extensions to follow Sphinx guidelines

* Clean up formatting of docstrings after Ark-Kuns comments
2020-08-04 00:33:47 +08:00
Renmin cc78bd1a4f
fix(sdk/client): reserve the host protocal (http or https) so that http host can work. Fixes #4277 (#4285) 2020-07-29 20:38:30 -07:00
Renmin ac66626a2c
feat(sdk): support HTTP/S PROXY for SDK client (#4215)
* done

* done

* reserve https

* done
2020-07-22 01:42:05 -07:00
Tom Hastjarjanto 45c796e112
feat(sdk): Expose upload pipeline version (#4146)
* feat(SDK): Expose upload pipeline version

* feat(sdk): Improve parameter name

* feat: Add option to specify either pipeline_id or pipeline_name

* Update sdk/python/kfp/_client.py

fix: Missing import

Co-authored-by: Niklas Hansson <niklas.sven.hansson@gmail.com>

* Fix: Optionals, incorrect var reference

* Fix: import orders

* chore: PR feedback

Co-authored-by: Niklas Hansson <niklas.sven.hansson@gmail.com>
2020-07-17 09:43:01 -07:00
Paulo Roberto de Oliveira Castro d91ac0f217
feat(sdk/client): add cookie configuration to kfp client (#2679)
* Add cookie configuration to kfp client

* Change cookie->cookies and add docs
2020-07-14 23:08:37 -07:00
Jiaxiao Zheng d3ef0dcbd0
fix(sdk): fixes the default value for _is_refresh_token. Fixes #4189 (#4195) 2020-07-09 16:56:34 -07:00
Ilias Katsakioris 749538d031
fix(sdk): Fix get_experiment when there are no experiments. Fixes #4176 (#4177)
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
2020-07-08 13:25:21 -07:00
Niklas Hansson aae1eba096
feat(sdk): CLI - Allow upload_version by the pipeline's name. Fixes #3901 (#4087)
* Update _client.py

* Allow for passing name instead of only pipeline_id

* fixed old rebase issue

* protobuf fix

* raise error and remove f-string

* moved file

* updated python proto packages versions

* Update sdk/python/kfp/_client.py

* restructured and added to build

* changes structure of import

* Updated the files

* further updates of client and filter.proto

* alternative

* clean up

* clean up

* remove helperfiles

* futher clean up

* clean up or eaither name or id

* update doc strings

* remove page_size

* Update sdk/python/kfp/_client.py

Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>

* Update sdk/python/kfp/cli/pipeline.py

Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>

* updated to classical string formattin

Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
2020-07-08 03:37:10 -07:00
Pierre Tholoniat 50fd45fed9
Fix typo in error message (#3369)
"The package_file '+ package_file + ' should ends with..." replaced by "The package_file '+ package_file + ' should end with..."
2020-06-29 20:49:58 -07:00
Paul Selden a802dbbed0
[SDK] Add list_pipeline_versions to kfp.Client (#3340)
Allows the user to list available pipeline versions for a given pipeline id.
2020-06-19 01:43:58 -07:00
Francis Laforge c9f621ad0c
feat: add kpf pipeline list-versions command (#3928) 2020-06-17 13:48:04 -07:00
Jiaxiao Zheng c04b9f0a2c
update refreshing mechanism (#3981) 2020-06-16 15:54:03 -07:00
Amy 5d302b6edb
fix typo on sort order: 'des' --> 'desc' (#3893) 2020-06-02 18:00:17 -07:00
Alexey Volkov cd8a913a31
SDK - Client - Use temporary directory context for pipeline package (#3878)
Maybe fixes https://github.com/kubeflow/pipelines/issues/3835#issuecomment-635195462
2020-06-01 03:49:55 -07:00
Alexey Volkov e52481a164
Client - Allow specifying pipeline description when uploading (#3828)
* Client - Allow specifying pipeline description when uploading

Fixes https://github.com/kubeflow/pipelines/issues/3825

* Implemented review feedback
2020-05-23 18:13:11 -07:00
faweis 165a0a9f04
kfp_client: fix wrong check (#3652) 2020-04-29 13:30:18 -07:00
faweis 97756e5577
Add method to schedule a recurring run to python client (#2978)
* python_kfp_client: add method to create recurring run

* client: add list_recurring_runs, get_recurring_run

* kfp_client: swap _create_job_config <-> run_pipeline

* kfp_client: mk propper trigger
2020-04-23 08:43:11 -07:00
Chen Sun 6bfe6ccc73
Fix list_run bug (#3539) 2020-04-17 16:53:35 -07:00
jingzhang36 2621b677c7
Revise run_pipeline comment (as expected) (#3506)
* revise run_pipeline comment (as expected)

* add the explanation of behavior if old appoarch is used

* add periods to the end of the sentences
2020-04-17 15:35:47 +08:00
Paul Selden 3713410022
[SDK/CLI] Add version param to run_pipeline (#3339)
* [SDK/CLI] Add version param to run_pipeline

* Set PIPELINE_VERSION relationship to CREATOR

Also adds a note about pipeline_id taking precedence over version_id
2020-04-06 08:21:48 -07:00
Chen Sun 213bf328d1
[SDK][Multi-user] refine sdk for multi-user support (#3417)
* Allow writing/reading user namespace to/from local context file

* update docstring

* Move LOCAL_KFP_CONTEXT into Client class

* Fix docstring

Signed-off-by: Chen Sun <chensun@users.noreply.github.com>

* Make context_setting an instance variable and load from file during init.
2020-04-03 15:25:46 -07:00
Chen Sun bd858030ec
Skip host name preprocess for the IAP case (#3427) 2020-04-02 17:19:46 -07:00
Samuel Ngahane 046c2f8afb
Fix lstrip + regex bug in the KFP client (#3396) 2020-03-30 17:34:41 -07:00
Renmin a92d522425
pass token outside of SDK for server-to-server case (#3363)
* pass token outside of SDK for server-to-server case

* add more docs

* fix merge issue

* fix merge issue

Co-authored-by: Renmin Gu <renming@google.com>
2020-03-30 02:27:55 -07:00
Chen Sun 1b0c91a091
Add namespace to experiment SDK calls (#3272) 2020-03-23 11:35:25 -07:00
Jiaxiao Zheng 256bc2e9a3
add warning and auto-correction. (#3287) 2020-03-16 22:16:22 -07:00
sh-tatsuno 64fd0ced17
SDK/CLI: Implement CLI delete pipeline command (#2879)
* feat: add delete pipeline command

* fix: docstring
2020-03-09 04:47:34 -07:00
Kubernetes Prow Robot 75b5fd72d2
[SDK] New fields for schedule pipeline (#3201)
* [SDK] New fields for schedule pipeline

* Update according to CR comments

* Update snapshot
2020-03-04 14:08:38 -08:00
Niklas Hansson 74e05935aa
Update to fix bug in kip.client for scheduling (#3173)
* update to fetch remote

* missed to add the description
2020-02-27 09:05:28 -08:00
Niklas Hansson adc81dc389
Updated with scheduling capability for kfp.Client (#2981)
* updated with scheduling capability

* added digits for the name

* bad import
2020-02-24 05:36:48 -08:00
Chen Sun 587292fbae
Deduce proxy type from the presence of client_id (#3003)
* Deduce proxy type from presence of client_id

* handle error in get_gcp_access_token()

* restore the logic to detect inverse proxy host
2020-02-10 14:43:54 -08:00
Ning e1190bd186 Add ns in sdk (#2665)
* add namespace to some run APIs

* update only the create run api

* add resourcereference for namespace runs

* add variables in const

* add namespace to run_pipeline

* add types to toModel func

* minor fix

* bug fix

* strip the namespace resource reference when mapping to the db model

* add unit tests

* use gofmt

* replace belonging relationshipreference to owner

* put a todo for further investigation of using namespace or uuid

* apply gofmt

* revert minor change

* use owner relationship

* add docstring for namespace

* add namespace to all create run func

* added todos for wrapping the configrations for different auth methods; default namespaces
2019-12-05 16:32:50 -08:00
hongye-sun 1b04983d96
update server and sdk version to 0.1.37 (#2701)
* update server and sdk version to 0.1.37

* update api package version to latest

* fix test error
2019-12-05 17:05:07 -05:00
Jiaxiao Zheng 790fe99aca [SDK] Relax k8s sanitization (#2634)
* update

* add allow_capital

* fix

* fix volume_ops sample

* fix pipeline name sanitization

* fix unittests

* fix sanitization in _client.py

* fix component output sanitization
2019-11-26 10:28:10 -08:00
Ning c74a9d8cd2 Add support for user account auth with IAP (#2626)
* Add support for user account auth with IAP

* add comments

* add the url in case open_new_tab does not work(AI Platform notebook).

* code refactor: move get_auth_token logic in auth.py

* store the credentials locally

* minior fix

* add support in kfp cli

* fix diagnose me bug

* add reference

* add todo for refresh token expiration

* address comments
2019-11-22 15:26:42 -08:00
Alexey Volkov 9b98f15849 SDK - Client - Fixed client on Windows (#2524)
URLs joining should not depend on OS path separator.
2019-11-04 01:05:40 -08:00
Alexey Volkov c063a050ed SDK - Client - Added API models to the generated API (#2418)
* SDK - Client - Added API models to the generated API

The API objects should now be available in the generated client API: `kfp.Client().jobs.models.ApiJob`.

* Simplified the change
2019-11-01 18:07:41 -07:00
IronPan 8032b5002a Fix inverse proxy matching regex (#2486)
The new format is now 
36a59b5371d6ab2-dot-us-west1.notebooks.googleusercontent.com
Remove the part that break the SDL to use a more generic matching string 
/assign @numerology @hongye-sun
2019-10-24 15:35:39 -07:00
Alexey Volkov 735e627a03 SDK - Refactoring - Split the K8sHelper class (#2333)
* SDK - Refactoring - Split the K8sHelper class

One part was only used by container builder and provided higher-level API over K8s Client.
Another was used by the compiler and did not use the kubernetes library.

* Updated the license year.
2019-10-21 14:57:22 -07:00