Commit Graph

1082 Commits

Author SHA1 Message Date
Chen Sun f84b51e72d
chore(sdk): Support loading v1 yaml with {executorInput: } placeholder (#7870)
* Support {executorInput: } placeholder

* merge

* use single quotes for consistency
2022-06-14 13:14:31 -07:00
Connor McCarthy 12d5cc3284
chore(sdk): use pytest instead of unittest for test execution (#7869)
* fix compiler test data path

* condense and add development requirements

* use pytest instead of unittest

* update contributing guidelines
2022-06-10 05:49:19 +00:00
Connor McCarthy 684f477a88
chore(sdk): fix component compilation test path (#7868) 2022-06-09 15:28:00 +00:00
Connor McCarthy de0b824be1
feat(sdk): add support for IfPresentPlaceholder and ConcatPlaceholder strings (#7795)
* move placeholders to placeholders module; implement if-present and concat placeholders

* update module name and method names throughout codebase

* rename placeholder abc

* add back compat support for CEL-style concat string

* add copyright

* add docstrings; clean up

* remove duplicate code
2022-06-09 07:42:00 +00:00
Chen Sun 0ada48b55a
chore(sdk): move pipeline test samples closer to compiler unit tests. (#7849)
* chore(sdk): move pipeline test samples closer to compiler unit tests.

* explicitly list test files

* remove dead code
2022-06-08 18:35:17 +00:00
johnmacnamararseg 9ffcb2e9db
feat(sdk): Add function to sdk client for terminating run (#7835)
* Add function to sdk client for terminating run

* RELEASE notes
2022-06-08 17:08:16 +00:00
Connor McCarthy 340318625c
chore(sdk): use markdown for long description content type (#7825) 2022-06-02 23:37:25 +00:00
Connor McCarthy c491f331d3
chore(sdk): bump kfp version to 2.0.0a5 (#7814)
* bump version

* update release notes
2022-06-02 22:34:26 +00:00
Connor McCarthy 0cf817364c
fix(sdk): fix load v1 if present placeholder (#7765)
* fix load v1 if present placeholder

* add test for load v1 if present placeholder
2022-06-02 16:49:53 +00:00
Connor McCarthy 2c27475fbc
add README with project description (#7782) 2022-05-26 14:59:12 -06:00
chongyouquan 281c86d41e
chore(sdk): 2.0.0-alpha.4 release (#7771)
* add release notes

* bump version to alpha.4

* fix release notes

* fix release notes

* fix release notes
2022-05-24 16:49:12 -07:00
Connor McCarthy a6ec449273
feat(sdk): only use IR when saving component (#7690)
* move function into compiler module

* remove v2 readable yaml write path

* remove v2 readable yaml test data; remove duplicate tests
2022-05-24 22:23:33 +00:00
Connor McCarthy cbc2ac5266
feat(sdk): enable read in component using IR (#7689)
* enable read in primitive components

* enable read in via YAML component

* convert OutputValuePlaceholder to OutputParameterPlaceholder

* add docstring
2022-05-24 18:58:57 +00:00
chongyouquan eaa8ec7807
fix(sdk): Fix corner cases and implement validation (#7763)
* Handle corner case when list is empty

* add validation for tags and versions

* add tests for empty lists

* add tests for version and tag validation

* fix formatting and optional arguments

* fix type annotation
2022-05-23 17:47:02 +00:00
Connor McCarthy b0db428165
feat(sdk)!: make CLI output consistent, readable, and usable (#7739)
* fix cli upload pipeline version

* organize requirements.in

* generate requirements.txt

* add pip-tools to requirements-dev

* improve CLI output

* update release notes
2022-05-20 00:03:23 +00:00
Connor McCarthy 6698801fc0
refactor(sdk): move placeholder logic to structures (#7687)
* move placeholder formatting logic to structures

* add placeholder formatting logic tests

* update placeholder references throughout codebase

* remove ABC meta inheritance
2022-05-19 07:45:56 +00:00
Connor McCarthy 101d243f48
feat(sdk): validate pipeline resource name before submission (#7713)
* add pipeline resource name validation

* add pipeline name validation tests

* update downstream pipeline name logic in cli
2022-05-13 20:12:41 +00:00
Connor McCarthy 62972eccf9
feat(sdk): enable compilation of primitive components (#7580)
* clean up relevant files

* use context manager for temporary directory cleanup

* add component compilation methods

* add component compilation tests

* add more complete pipeline_func argument handling

* use context manager for handling type check mode

* improve checks on inputs for component compilation

* correct pipeline name for primitive component IR

* simplify component compilation logic

* extract compiler methods

* move compiler helpers to helpers file to avoid circular imports

* remove underscore prefix from functions in hidden module

* move helpers tests to helper_test module

* put to_pipeline_spec on ComponentSpec

* fix inputs reference

* use same naming regex for pipelines and components

* permit compilation of all BaseComponent subclasses

* clean up docstrings

* merge compiler.helpers into pipeline_spec_builder

* modify docstrings; refactor

* modify docstring
2022-05-13 20:12:34 +00:00
Connor McCarthy 49cdb42c12
fix cli upload pipeline version (#7722) 2022-05-13 12:13:25 -06:00
Connor McCarthy 2570922a7f
feat(sdk): compile JSON with formatting (#7712)
* clean files

* implement formatted write

* add tests
2022-05-13 17:34:34 +00:00
Connor McCarthy 6b2f015672
chore(sdk): use unittest.mock instead of mock (#7694) 2022-05-10 22:55:33 +00:00
Connor McCarthy 1ae6e11bab
feat(sdk): make client return values consistent (#7659)
* clean up client.py

* update client method return values

* update other types

* use preferred path to kfp_server_api classes

* fix incorrect module name

* other type annotation cleanup

* make docstrings consistent
2022-05-10 14:35:12 -06:00
chongyouquan 25e4c58820
feat(sdk): Implement Registry client (#7597)
* Implement registry client

* Update registry client code

* Add test skeleton

* Add some tests

* Update code

* add tests

* update tests

* update tests

* Rename Client -> RegistryClient

* Update wrt comments

* add type annotations

* fix renaming in __init__.py

* remove unused imports

* extract host variable in test

* format using yapf

* remove locals and use arg keywords

* remove json conversion

* fix header

* write bytes when downloading file

* fix create_tag; fix tests

* fix request_body for update_tag and create_tag using json.dumps

* simply return json for delete_tag

* rename files

* format files

* update return types and format double quotes

* add comments and format files

* add todos

* update credentials and change open to use context

* format using yapf

* move request into context

* Update comments

* Update release notes

* Update release notes
2022-05-10 06:53:06 +00:00
Connor McCarthy 49ccb4f61f
chore(sdk): 2.0.0-alpha.3 release (#7675)
* reorganize release notes

* finalize alpha.3 notes

* bump version to alpha.3

* add one more release note
2022-05-06 16:45:28 -06:00
Connor McCarthy 18f0842ee0
fix(sdk): some cli fixes (#7668)
* use correct type when option is path

* fix options on create recurring-run

* make output logging more consistent

* show deprecation warning for dsl-compile on invocation

* fix unarchive experiment

* fix broken f-string

* convert camelcase methods to snake case
2022-05-05 12:47:18 -06:00
Connor McCarthy 5da3826bb5
feat(sdk): use custom basemodel and remove pydantic (#7639)
* fix discovered bug

* update tests

* implement custom BaseModel

* use basemodel for structures

* remove pydantic dependency

* assorted code cleanup
2022-05-04 12:56:32 -06:00
Mai Nakagawa 1118f4859b
feat(sdk): Accept newer `typing-extensions` versions (#7632)
* Accept newer typing-extensions versions

* Add a description to the release notes.
2022-04-30 00:01:25 +00:00
Connor McCarthy 652ec3b936
feat(sdk): rename CLI methods to 'create' (#7607)
* add deprecated alias group

* add deprecated alias group tests

* implement deprecated alias group

* clean up other alias tests

* clean up docstring

* fix type annotations / mypy

* format with yapf
2022-04-29 18:52:58 +00:00
Connor McCarthy 6cecfd948e
add deprecation warning to dsl-compile (#7628) 2022-04-29 11:34:48 -06:00
Connor McCarthy ec0081372a
feat(sdk): improve cli help text (#7618)
* add docstring parsing utils

* implement docstring parsing for help text

* implement smoke test of all cli commands

* clean up use of optional in docstring

* make command descriptions more consistent

* fix disable_type_check

* update release notes
2022-04-29 17:20:57 +00:00
Connor McCarthy 027ac3274f
feat(sdk): add client methods to cli (#7606) 2022-04-27 13:21:13 -06:00
Connor McCarthy e12ac39ba4
feat(sdk): use dsl noun group for compile CLI commands (#7603)
* use kfp dsl compile for compilation commands

* create and move tests
2022-04-26 15:05:31 -06:00
Connor McCarthy fbfeadd4a4
feat(sdk): add autocomplete and version options to kfp cli (#7567)
* add helpful options to cli

* add tests
2022-04-25 14:01:02 -06:00
Alex 2636727141
feat(sdk): add archive, unarchive, and delete run methods and tidy docstrings (#7562) 2022-04-25 19:05:11 +00:00
Connor McCarthy e077d01cf9
feat(sdk): add noun aliasing to cli (#7569)
* allow both singular and plural versions of nouns

* use command table for client creation

* cleanup

* simplify docker mock

* add docker to test requirements; clean up existing requirements
2022-04-25 12:24:57 -06:00
Connor McCarthy 2db431b5f4
use click for components commands (#7559) 2022-04-20 14:59:11 -06:00
Connor McCarthy a5548440a3
use click for dsl-compile command (#7560) 2022-04-20 14:57:10 -06:00
Connor McCarthy b58a31b307
chore(sdk): clean up v2 CLI (#7558)
* remove linting comments

* move global variable to local

* simplify diagnose_me command

* use f-strings

* idiomatic python refactorings

* remove unused import

* add typestub library

* silence unresolvable mypy error

* update type information

* remove unused variable

* remove unused import

* use __main__.py for cli entrypoint

* expand imports
2022-04-20 10:40:38 -06:00
Connor McCarthy 2494447578
feat(sdk): add .list_pipeline_versions and .unarchive_experiment methods to Client (#7563)
* add client methods

* update release notes
2022-04-14 21:28:38 +00:00
Connor McCarthy d46fafe4ce
feat(sdk): port cli code from v1 to v2 (#7547)
* port cli

* fix docker mock
2022-04-13 20:19:52 +00:00
Connor McCarthy 94960cf5ab
chore(sdk): use setuptools.find_packages (#7548) 2022-04-12 21:52:51 +00:00
Connor McCarthy f70413ad7c
chore(sdk): stop using pydantic for serialization (#7536)
* add ir_utils

* add ir_utils tests

* use ir_utils to write ir to file

* cast posix path to string in `kfp components build` command

* add copyright

* remove unused import

* update setup.py packages
2022-04-12 17:03:55 +00:00
Connor McCarthy c98f14abb4
chore(sdk): make field order in compile deterministic (#7520)
* apply docformatter

* use sort_keys=True in compiler

* update golden snapshots

* set other sort_keys=True

* update structures_test yamls
2022-04-06 21:46:13 +00:00
Connor McCarthy 4eeb420905
feat(sdk)!: promote v2 compiler command and deprecate v1 command (#7508) 2022-04-05 18:07:37 +00:00
Chen Sun 9db6e74556
chore(sdk): release KFP sdk 2.0.0-alpha.2 (#7519) 2022-04-05 00:08:35 -07:00
Connor McCarthy d74d7b979b
add typestub libraries (#7451) 2022-04-04 18:04:33 -06:00
Chen Sun 9e708b9208
fix(sdk): fix yaml support `create_run_from_pipeline_func` (#7500) 2022-04-01 19:18:25 +00:00
Hao Xin 48644edf57
chore(deps): Bump google-cloud-storage to 2.2.1 (#7493) 2022-04-01 18:33:25 +00:00
Chen Sun 2019a46114
chore(sdk): Update proto link for PipelineTaskFinalStatus (#7496) 2022-03-31 20:12:30 +00:00
Connor McCarthy 1c3e2768e6
chore(sdk): modify deprecated tests (#7446) 2022-03-25 21:10:11 +00:00