* Reformat sdk only using the new yapf config.
* Reformat docstrings using docformatter.
* update golden files to resolve diff caused by whitespaces
* fix some tests
* format .py files under sdk/python/tests using yapf
* additional docformatter
* fix some tests
* Introduce ServiceAccountTokenVolumeCredentials
Part of kubeflow/pipelines#5138
This is a subclass of TokenCredentials and implements the logic of
retrieving a service account token provided via a ProjectedVolume.
The 'get_token()' method reads and doesn't store the token as the
kubelet is refreshing it quite often.
Relevant docs:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection
* Attemp to use credentials when in-cluster
If the KFP client detects it's running inside a pod and the user hasn't
provided any credentials, it now attempts to set up credentials based on
a projected service account token.
* Move credentials to kfp.auth module
Also change some names and values to avoid "ml-pipeline" references.
* Import kfp.auth lazily during the Client init
* Have credentials handle errors internally
* Let ServiceAccountTokenVolumeCredentials handle OSErrors internally
* Have the default-creds-setter only check if credentials provide a
valid value
* Use Configuration's 'refresh_api_key_hook' instead of having duplicate
code
* Remove dead code
* Add kfp.auth in the setup.py
* Move abstraction to the 'refresh_api_key_hook' method
* Introduce TokenCredentials abstract class
As presented in kubeflow/pipelines#4683, in this commit we introduce a
TokenCredentials abstract class which encapsulates the retrieval and
refresh of token credentials.
The reason we are using a class for the credentials is the fact that
usually tokens are short-lived and the client needs to refresh them.
All subclasses should define a 'get_token()' method responsible for
fetching and refreshing (if needed) a token for authentication.
* Configure credentials when initializing KFP client
* Make cli.experiment.list_experiments use kfp.Client.list_experiments for multi-user environment
* Add delete_experiment method to kfp.Client and make cli.experiment.delete use this method
* Set current namespace in local KFP context if running from notebook
* Create "~/.config/kfp/" instead of ".config/kfp/"
At first it was assumed the `get_user_namespace` command would be executed from the home directory.
* Create local context file if it doesn't exist during set_user_namespace
* Grab path from LOCAL_KFP_CONTEXT when creating folder
Instead of harcoding the os.mkdirs path to `~/.config/kfp` it now grabs it from the LOCAL_KFP_CONTEXT. Also, removed path creation in `get_user_namespace` as that is now handled in `set_user_namespace`. Also, it now checks if the path exists rather than the local_context_file to remove the situation where it tries to create ~/.config/kfp/ because the context.json doesn't exist when the path does.
* add multi-user setting to healthz api
* Add http prefix to health api url
* move healtz api call to own function and fix multi_user boolean
* Fix HEALTH_PATH declaration
* Move check to Client __init__ and change get_kfp_healthz to avoid breaking in case of old apiserver image
* Add multi_user to frontend healthz
* Expose multi_user in frontend and add integration test
* Fix integration test
* Fix host hardcoding and error handling
* Handle empty API response, check if API up to date
* Fix response return
* remove API check due to empty response
* retry API call if first response empty
* retry getting healthz api if no response
* change health_api to https
The healthz_api has been returning empty responses which might be caused by sending an http request to an https endpoint. Although requests handles redirects, this commit is to test if this solves the issue.
* Add some debug info to healthz exception
* add url to debug and lower retries to 1
* Use api_client to get healthz data
* Debug info for API response
* Follow API redirect history
* Fix indentation
* Add healthz proto
* Try getting healthz api with new python backend
* Add installation of kfp_server_api in tests
* Fix incorrect setup location
* Replace old .get with new http backend .multi_user
* Code clean up
* Small fixes and TimeOutError for retries healthz api
* Remove changes to go dependencies
* Send empty proto request and fix exception client
* Remove unused commit_sha and tag_name
The default timeout value is of type timedelta which cannot be compared with a plain number later in the code.
Also replaced time_delta.seconds with time_delta.total_seconds() which gives out correct value.
In some cases, it may need extra header to handle the API calls.
Directly add `header_name` and `header_value` props in
`.config/kfp/context.json` and pass the name/value pair
to APIClient.
Here is one of the use cases:
if the service is protected by istio RBAC and need JWT header
for authentication, you can specify the JWT id-token in
the .config/kfp/context.json with these two new props.
The id-token would be carried in the specified header name.
Then the API call can be properly authentiecated and checked
if the user has the permission to access the service.
* Prepare SDK docs environment so its easier to understand how to build the docs locally so theyre consistent with ReadTheDocs.
* Clean up docstrings for kfp.Client
* Add in updates to the docs for compiler and components
* Update components area to add in code references and make formatting a little more consistent.
* Clean up containers, add in custom CSS to ensure we do not overflow on inline code blocks
* Clean up containers, add in custom CSS to ensure we do not overflow on inline code blocks
* Remove unused kfp.notebook package links
* Clean up a few more errant references
* Clean up the DSL docs some more
* Update SDK docs for KFP extensions to follow Sphinx guidelines
* Clean up formatting of docstrings after Ark-Kuns comments
* Update _client.py
* Allow for passing name instead of only pipeline_id
* fixed old rebase issue
* protobuf fix
* raise error and remove f-string
* moved file
* updated python proto packages versions
* Update sdk/python/kfp/_client.py
* restructured and added to build
* changes structure of import
* Updated the files
* further updates of client and filter.proto
* alternative
* clean up
* clean up
* remove helperfiles
* futher clean up
* clean up or eaither name or id
* update doc strings
* remove page_size
* Update sdk/python/kfp/_client.py
Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
* Update sdk/python/kfp/cli/pipeline.py
Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
* updated to classical string formattin
Co-authored-by: Jiaxiao Zheng <jxzheng@google.com>
* [SDK/CLI] Add version param to run_pipeline
* Set PIPELINE_VERSION relationship to CREATOR
Also adds a note about pipeline_id taking precedence over version_id
* Allow writing/reading user namespace to/from local context file
* update docstring
* Move LOCAL_KFP_CONTEXT into Client class
* Fix docstring
Signed-off-by: Chen Sun <chensun@users.noreply.github.com>
* Make context_setting an instance variable and load from file during init.
Chen Sun