e50f40305d
fix(sdk): fix upload_pipeline when no pipeline name is provided ( #8695 )
2023-01-31 13:45:39 -08:00
edfb267790
bump version; update release notes; add docs ( #8745 )
2023-01-20 12:56:56 -08:00
1c7ab4c5f5
pin kfp-pipeline-spec version exactly ( #8737 )
2023-01-20 12:49:36 -08:00
78d31dc8a3
chore(sdk): remove v1 dependencies from SDK v2 ( #8668 )
...
* split out deprecated requirements
* pip install deprecated requirements for sample tests
* remove superfluous comments in requirements.in
* update release notes
* ignore deprecated directory in unit tests
* remove unused argo compiler
* ignore deprecated path
* update run sample yaml
* install deprecated requirements for e2e tests
* add todos
2023-01-19 12:53:27 -08:00
07615eb88a
chore(sdk): Improve CLI utility function ( #8686 )
2023-01-17 14:51:50 -08:00
5adc7f1b50
chore(sdk): clean up client.py ( #8656 )
2023-01-17 11:04:46 -08:00
6040334e84
feat(sdk): Accepts higher PyYAML versions ( #8665 )
...
* Accepts higher PyYAML versions
* Update the release note
2023-01-17 10:46:42 -08:00
fdf3ee7b68
chore(sdk): release KFP SDK 2.0.0b10 ( #8638 )
...
* bump version
* update release notes
* add docs version
2022-12-29 18:05:36 -05:00
84241d6460
fix(sdk): fix deserializing v1 component yaml defaults ( #8639 )
...
* fix deserializing v1 defaults
* update test cases
* update type annotation
* add unit tests
2022-12-29 17:48:58 -05:00
151e8a3c6e
feat(sdk): support optional artifact inputs ( #8623 )
...
* add compiler tests
* support optional artifact inputs
* update executor test
* update existing golden snapshot
* add new compiler test/snapshot
* add release note
* bump kfp-pipeline-spec dependency
2022-12-28 13:40:12 -05:00
5491f6f3db
fix(sdk): support writing/reading isOptional for input parameters ( #8612 )
...
* add tests
* support writing/reading isOptional for parameters
* update golden snapshots
* temporarily maintain workaround for v1 optional artifact inputs
* add release note
2022-12-21 14:46:59 -08:00
ecfdc665c4
chore(sdk): release KFP SDK 2.0.0b9 ( #8609 )
2022-12-21 00:55:12 +00:00
f64348f29e
fix(sdk): make executor output writer more permissive [sdk v2] ( #8607 )
2022-12-21 00:23:13 +00:00
2eef97cfc7
fix(sdk): add special handling for PipelineTaskFinalStatus to get_parameter_type ( #8587 )
2022-12-19 18:48:46 +00:00
08b2408d3d
feat(sdk): add support for lists of artifacts in Python components [support lists of artifacts pt. 3] ( #8465 )
...
* remove unused codepath
* rename to avoid confusing variable names
* implement support in executor
* add comment
* make executor code safer
2022-12-15 21:34:21 +00:00
47eb1cad92
chore(sdk): update golden snapshots with pipeline interface comments ( #8575 )
...
* update path in readme
* update golden snapshots
2022-12-15 01:32:21 +00:00
d4b896d3e3
fix(sdk): new fix for multiple workers writing with gcsfuse bug [KFP SDK v2] ( #8544 )
...
* fix and test
* update release notes
* extract constants
2022-12-13 18:01:29 +00:00
f4588f3112
chore(sdk): idiomatic Python pure refactorings ( #8537 )
...
* use f-strings
* inline immediately returned variables
* simplify comprehension
* use list literals
* lift return statements into control flow blocks
* other pure refactorings
* remove passthrough function
* use a ternary when it improves readability
* use __getitem__ for regex match groups; introduced in Python 3.6
* fix changed comments
* revert one change
2022-12-06 23:44:36 +00:00
49db63c916
feat(sdk): Add comments to IR YAML file ( #8467 )
...
* base
* add tests
* fix bug
* adress comments
* address comments 2
* sort comments
* sort signatures
* add indempotent test
* add indempotent test2
* support multiline docstring
* review
* docformatter presubmit exclude
* docformatter presubmit exclude
* docformatter
* docformatter
* merge 1
* update readme
* nit .items()
* remove reduntant test
2022-12-05 14:43:14 -08:00
1d6a52b9ff
fix(sdk): print max_attempts instead of sleep_duration ( #8523 )
2022-12-02 19:32:16 +00:00
00c6b01040
fix(sdk): fix bug when referencing component.pipeline_spec in a pipeline body ( #8514 )
...
* add tests
* implement fix
* add TODO
2022-12-02 09:35:16 +00:00
614c231ae6
fix(sdk): unblock valid topology. ( #8416 )
...
* unblock valid topology
* add more tests
* handle git fork
* sample_test_cases
* main
* restore to master
* resolve comments on PR
* resolve conflicts
* resolve conflict 2
* revert conflict fix
* fix changes
* address comments
* review
* docformatter presubmit
* revert docformatter
* update release.md
2022-12-02 00:15:15 +00:00
b26dd100e1
test(sdk): implement small tests for Client class ( #8517 )
...
Co-authored-by: droctothorpe <mythicalsunlight@gmail.com>
Co-authored-by: droctothorpe <mythicalsunlight@gmail.com>
2022-12-01 19:39:45 +00:00
e7c82c0593
feat(sdk): support list of artifacts annotations [list of artifacts support pt. 1] ( #8464 )
...
* update get_io_artifact_class function
* update various type annotation tests
* support list of artifacts in component interfaces
* move helper function
* clarify function name
* respond to review feedback
2022-11-30 05:24:15 +00:00
249e7af384
chore(sdk): release KFP SDK 2.0.0b8 ( #8508 )
...
* update release notes
* bump version
* add beta 8 to docs; remove beta 7
2022-11-30 05:04:15 +00:00
834bfbe90a
fix(sdk): revert premature client changes ( #8507 )
...
* revert _v1 client changes
* do not install kfp_server_api from source
2022-11-29 20:39:23 -08:00
1043a88842
chore(sdk): release KFP SDK 2.0.0b7 ( #8503 )
...
* update release notes
* bump version
* add docs version to dropdown
2022-11-28 22:58:42 +00:00
b829e4a5da
feat(sdk): enable use of primitive placeholders in f-string ( #8494 )
...
* support primitive placeholders in fstring
* add read/write test component
2022-11-28 14:45:48 -08:00
4bb57e6723
fix(sdk): fix multiple workers writing with gcsfuse bug [KFP SDK v2] ( #8455 )
...
* fix(sdk): fix multiple workers writing with gcsfuse bug [KFP SDK v1]
* add comment for context
2022-11-23 00:38:48 +00:00
e97fe8e04f
feat(sdk): add pipeline job placholders for scheduled runs ( #8447 )
...
* add pipeline job placeholders
* simplify example code in docs
* add _PLACEHOLDER suffix to variable names
2022-11-23 00:38:42 +00:00
be766dc352
docs(sdk): update pipeline task .set_gpu_limit reference docs ( #8477 )
2022-11-18 17:58:22 +00:00
6b3b4df3d2
refactor(sdk): simplify InputSpec ( #8422 )
...
* simplify usage of inputspec
* remove underscore from optional
* add error message and test
* remove optional
2022-11-14 18:39:26 +00:00
9e9e1081c6
fix(sdk): fix boolean default value serialization bug ( #8444 )
...
* add test cases
* fix bug
* add v1 component yaml back compat changes
* update golden snapshot
2022-11-11 15:42:33 -08:00
3f2e20d14f
chore(backend): regenerate v1beta1 api clients ( #8403 )
...
* add v2 experiment.proto
* regenerate v1beta1 client
* remove api version from combined swagger file
* Revert "regenerate v1beta1 client"
This reverts commit 02769d779bc24fcfbed9
2022-11-10 19:18:00 +00:00
24e17f7352
chore(sdk): quiet noisy SDK unit test output ( #8434 )
2022-11-09 17:47:39 +00:00
d33f359797
fix(sdk): fix nested placeholders and block illegal IfPresent form in Concat ( #8414 )
...
* support single element IfPresentPlaceholders that contain a primitive placeholder
* block list then and else_ inside concat
* support single element IfPresent in v1
* clean up typing
* skip loading batch_predict_job yaml
2022-11-04 18:20:16 +00:00
08a8b1df97
refactor(sdk): remove dead BaseModel code ( #8415 )
...
* remove dead basemodel code
* remove unused deserialization logic
* remove more dead code
* remove basemodel entirely
2022-11-03 21:58:15 +00:00
4c6abe5afc
refactor(sdk): simplify placeholders and logic ( #8402 )
...
* simplify placeholders
* update placeholders_test.py
* update structures to use placeholders correctly
* update use of placeholders elsewhere in codebase
* clean up
* move placeholder type checking logic to PipelineTask to prevent v1 load breaks
* clean up
* update docstrings
* make to_dict and to_string private methods
* revert unnecessary change to tests
* improve type safety of code
* prefix module-level constant with underscore
2022-11-02 00:31:45 +00:00
ca6ad27020
chore(sdk): remove unused linter comments ( #8405 )
2022-11-01 18:31:46 +00:00
5b75d75ad7
feat(sdk): add --build-image option to 'kfp components build' to allow users to skip docker build. Fixes #8382 for 2.0 ( #8387 )
...
* feat: added --build-image/--no-build-image flags to skip docker build
* fix: logic error for warnings/errors
2022-10-26 15:58:31 +00:00
8df22db757
chore(release): release sdk 2.0.0b6 ( #8373 )
...
* release sdk 2.0.0b6
* address review comments
2022-10-18 00:30:27 +00:00
ac02a0e191
fix(sdk): Throw exception for component parameter named Output ( #8367 )
...
* fix(sdk): Throw exception for component parameter named Output
* Add unit test
* lint fix
* nit fixes
* nit
* nit
* format fixes
2022-10-17 23:40:27 +00:00
30669e46dc
chore(sdk): improve KFP SDK reference documentation ( #8337 )
...
* use autodoc default options
* sort symbol order by typical usage patterns
* include summary of submodule symbols
* expose Input[] and Output[] type generics
* include tables of contents on API reference docs pages
* expand global TOC for discoverability
* fix missing reference / circular import problem
2022-10-17 19:58:53 +00:00
54922b32c6
fix(sdk): fix `kfp component build` only produces empty requirements.txt ( #8372 )
...
* fix CUJ2 always writes empty requirements.txt
* sort dependencies
2022-10-17 19:22:53 +00:00
4cc0e80183
fix(sdk): Allow using cli `kfp dsl compile` to compile components ( #8371 )
...
* Allow using cli `kfp dsl compile` to compile components
* improve error message
* address review comments
2022-10-17 16:06:52 +00:00
83e0f16af1
fix(sdk): Fix "No KFP components found in file" error. Fixes #8353 ( #8359 )
...
* Update component.py
* Update component.py
* Update component.py
2022-10-13 21:04:03 +00:00
764140153f
fix(sdk): Fix PyPi typo in cli/component docstring ( #8361 )
2022-10-13 20:28:03 +00:00
26f1fe4dfa
fix(sdk): fix NamedTuple output with Dict/List bug ( #8316 )
2022-09-28 02:25:10 +00:00
19b8709ca3
chore(release): release sdk 2.0.0b5 ( #8309 )
...
* bump reference docs
* update release notes; modify structure
* bump version
* implement review feedback
2022-09-26 21:38:35 +00:00
8ab690f3ed
feat(sdk): add workaround for google artifact (support custom artifact types pt. 4) ( #8279 )
2022-09-19 22:32:38 +00:00
0cb0d99d69
fix(sdk): fix InputPath error in executor ( #8273 )
...
* fix inputpath error
* add tests
* fix nit
2022-09-19 22:32:31 +00:00
c353245baf
feat(sdk): enable dynamic importer metadata ( #7660 )
...
* support placeholder in metadata dict
* add unit tests
* add compilation test
* fix tests
* update compiler test
* use 'metadata' as input key
* update compiler test and golden snapshot
* add int input to compiler test
* change constant name
* use util function
* support f-strings in dynamic import metadata; add and update tests
* update method name after rebase
* make compile consistent across python versions
2022-09-19 21:58:31 +00:00
6443fc17dc
chore(sdk): hide private methods in placeholders for simpler doc ( #8271 )
...
* hide private methods in ConcatPlaceholders and IfPresentPlaceholder by renaming
* rename transform and validate methods
2022-09-16 16:16:30 -06:00
65f2b644df
fix(sdk): Replace OOB authentication with loopback flow ( #8262 )
...
* Replace oob authentication with loopback flow. Fixes #7474 . Closes #7526 .
* Replace oob authentication with loopback flow. Fixes #7474 . Closes #7526 .
* Improve error messages
* Resolve comments in the PR
* Fix refresh/access token misconception
* Fix type hint annotation
2022-09-15 23:27:24 +00:00
f54f0b9341
fix v1 output deserialization ( #8272 )
2022-09-15 15:51:23 -06:00
166d6bb917
feat(sdk): add runtime logic for custom artifact types (support for custom artifact types pt. 3) ( #8233 )
...
* add runtime artifact instance creation logic
* refactor executor
* add executor tests
* add custom artifact type import handling and tests
* fix artifact class construction
* fix custom artifact type in tests
* add typing extensions dependency for all python versions
* use mock google namespace artifact for tests
* remove print statement
* update google artifact golden snapshot
* resolve some review feedback
* remove handling for OutputPath and InputPath custom artifact types; update function names and tests
* clarify named tuple tests
* update executor tests
* add artifact return and named tuple support; refactor; clean tests
* implement review feedback; clean up artifact names
* move test method
2022-09-15 01:00:40 +00:00
231c5079f0
fix(sdk): make sdk installable with poetry ( #8263 )
...
* Make kfp package installable with poetry
The poetry package manager executes `setup.py` differently than pip, which means that `__file__` is initialized to just `setup.py`, which means that `where` would be set to the empty string. The proposed change has been tested with both pip and poetry.
* apply yapf formatting
2022-09-14 21:08:08 +00:00
5dff29a071
fix(sdk): keeping placeholder strings in command and args even if default value is set ( #8259 )
...
* return string representation of placeholder in pipeline_task
* return string representation of placeholder in pipeline_task
2022-09-13 23:44:31 +00:00
e371fa3795
feat(sdk): add compilation logic for google artifact types (support custom artifact types pt. 2) ( #8232 )
...
* remove dependency of deprecated on v2
* use new artifact identification logic
* add custom artifact type compiler test
* compile schema_version from non-kfp artifacts
* update tests
* add importer to compiler test
* update golden snapshots
* fix type checking code
* undo executor_test diff
* remove gcpc types compiler test
* add compilation test comment
* update google artifact handling
* update compiler test
* only permit system. and google. artifact types; add tests
* update compiler test
* simplify type checking logic
* use bundled artifact types in PipelineArtifactChannel
* raise error on unknown v1 input or output
* add type utils and move functions
* add type utils tests
* remove schema_version from inputspec and outputspec
* update artifact proto creation and tests
* propogate bundled artifact type string changes throughout sdk
* update golden snapshot
* remove unused v1 compat and tests
* update type checking and tests to validate schema_version using semantic versioning
* update importer sample
* update test data config
* update test_data config
* clean up sample
* clean up sample
2022-09-12 14:00:02 -06:00
16c7d6658e
test(sdk): create SDK execution test suite and restructure read/write tests ( #8245 )
...
* update client for graph components
* use yaml for config
* simplify read write tests
* add test script
* restructure yaml and read/write tests
* add execution test file
* move test_data; update paths; update readme
* update kfp endpoint
* update pipeline with env
* update logging
* update config
* update kfp endpoint and timeout
* add kfp as requirement
* mock kfp_package_path for lightweight component execution
* revert pipeline name changes
* update config with name
* update read write tests to use explicit name
* update execution tests to use explicit name
2022-09-12 18:33:06 +00:00
92a1e1fe26
feature(sdk): print links outside of ipython. Fixes #8141 ( #8142 )
2022-09-09 00:00:27 +00:00
e14a784327
feat(sdk): migrate v1 auth to v2 ( #7789 )
...
* initial auth migration (@ji-yaqi)
* clean up files
* update imports
* add type annotations
* update docstrings
* add tests
2022-09-06 17:54:29 -06:00
eb453d10a9
fix(sdk): fix dsl control flow docstrings ( #8183 )
...
* fix dsl control flow docstrings
* update docstring
2022-09-01 16:33:07 -06:00
2a566181bf
chore(sdk): bump kfp version to 2.0.0-beta.4 ( #8229 )
2022-09-01 06:41:55 +00:00
25cd45d772
chore(sdk): add script to check for unused imports ( #8221 )
...
* add dev requirement
* add unused import check to isort script
2022-08-31 22:53:51 -07:00
d88358201d
feat(sdk): Support using pipeline in exit handlers ( #8220 )
...
* Support using pipeline in exit handlers
* release note
* remove dead code
* fix component spec merging flakiness
* address comments in PR#8209
* address review comments
* fix bad merge in release note
2022-09-01 01:45:54 +00:00
c34e5e4edd
chore(sdk): Update upper bound for kubernetes dependency ( #8173 )
...
* Update upper bound for kubernetes dependency
* Add in release note for change
2022-08-31 22:47:54 +00:00
889637ade1
fix(sdk): fix constructor signature on dsl.Condition ( #8182 )
2022-08-31 21:35:54 +00:00
410e99c604
feat(sdk): Support loading pipeline from yaml ( #8209 )
...
* support loading pipeline from yaml
* release note
* cleanup
* maintain read/write consistency with component/pipeline compilation.
2022-08-31 12:05:55 +00:00
f5277d3078
Fix the example pipeline in sdk/python/README.md ( #8206 )
2022-08-30 14:55:45 -06:00
48574dc648
feat(sdk): Support pipeline outputs ( #8204 )
...
* Support pipeline outputs
* release note
2022-08-29 20:26:53 +00:00
72c1d10a33
feat(sdk): rename kfp artifact attributes (support custom artifact types pt. 1) ( #8191 )
2022-08-27 13:20:50 +00:00
5ce4954d6b
feat(sdk)!: Implement Graph Component ( #8179 )
...
* Implement Graph Component
* isort imports
* release note
* address review comments
* address review comments
2022-08-26 09:28:49 +00:00
121ddcbc04
fix(sdk): fix v2 sample tests for cuj3 ( #8168 )
...
* fix v2 sample tests for cuj3
* separate sh -c
* fix two step pipeline
* fix piping error
* fix nit
* change path to uri to correct cat command
* correct bash script
* remove display name
* change directory reference in commands
* remove old files
2022-08-25 15:30:22 -07:00
459e5358e6
chore(sdk): Bump KFP Python SDK to support Python v3.10 ( #8186 )
...
* Update upper bound for kubernetes dependency
* Bump KFP Python SDK to support Py v3.10
* Reset requirements.* changes to isolate out Python v3.10 changes
* Add in release note
2022-08-24 21:34:08 +00:00
5ccf53a251
fix(sdk): fix output type of importer ( #8172 )
...
* change output type of importer to artifact_class.TYPE_NAME
* use the same logic as component_factory._annotation_to_type_struct for determining type_name
* format using yapf
* fix type_name
* format using yapf
* add yaml file
* changes with regards to PR comments
* format pipeline_with_importer_and_gcpc_types.py
* remove unused imports
* change name of test component
* add pipeline_with_importer_and_gcpc_types to config
* update yaml file
2022-08-24 20:15:07 +00:00
88a1b314c9
feat(sdk): add support for metadata placeholders ( #8151 )
2022-08-17 23:10:06 +00:00
51bea09833
feat(sdk): Support Parallelism in ParallelFor in KFP SDK V2 ( #8146 )
...
* feat(sdk): add support for ParallelFor parallelism setting
* add parallelism value check
* Add unit tests
* Adding a compiler test
* used None as default parallelism input signature, other minor fixes on format and tests
* fix import statements
* add release.md message
* update ParallelFor docstring
* fixed docstring comments
* removed 'optional' in docstrings
2022-08-17 21:06:01 +00:00
41b69fd90d
chore(sdk): bump kfp version to 2.0.0-beta.3 ( #8149 )
...
* update release notes
* bump version
2022-08-17 14:21:52 +00:00
c7a90a3fa8
feat(sdk): expose IfPresentPlaceholder and ConcatPlaceholder to kfp.dsl ( #8145 )
2022-08-15 23:22:06 +00:00
35fccb1fae
feat(sdk): expose new ContainerSpec API ( #8144 )
...
* rename ContainerSpec to ContainerSpecImplementation for internal usage
* fix nit
2022-08-15 20:33:34 +00:00
6137a236a5
chore(sdk): bump kfp sdk version to 2.0.0b2 ( #8135 )
2022-08-11 00:08:03 +00:00
d3c0537c8b
chore(sdk): Add test for if and concat placeholder and validate artifact channel usage ( #8120 )
...
* test if and concat placeholder and validate artifact channel usage
* fix nit in read write test config
* resolve comments
* resolve comments and move test cases
* resolve comments and add placeholder validation
* remove unused imports
* update release file
2022-08-10 23:35:03 +00:00
bdff332ac6
feat(sdk): support more than one exit handler per pipeline ( #8088 )
...
* add compiler test pipeline with multiple exit handlers
* remove blocker of multiple exit handlers
* move exit handler builder logic to pipeline_spec_builder
* build all exit handlers per pipeline
* add compiler test with IR inspection
* prevent usage of cross-pipeline after
* test cross-pipeline after is prevented
* update existing task dependency logic and tests
* add v2 sample test
* remove cross-pipeline .after
* prevent cross-dag data dependency for dsl features
* add compiler test pipeline with nested exit handlers
* add support for nested exit handlers
* clean up pipeline with nested exit handlers
* remove sample with multiple exit handlers
* remove compiler test with nested exit handlers
* add compilation guard against nested exit handlers in subdag
* update release notes
2022-08-10 16:56:21 +00:00
e728d0871b
chore(sdk): add pre-commit hook to remove unused imports ( #8123 )
...
* add pycln pre-commit hook to remove unused imports
* remove existing unused imports
2022-08-10 08:17:21 +00:00
e47a784035
docs(sdk): clean up kfp v2 cli reference docs ( #8115 )
...
* use expanded "Command Line Interface" for title
* remove alpha feature stage in help text
* hide deprecated argument
* use compile as command name
2022-08-09 18:40:39 +00:00
04c827fc3a
feat(sdk): supporting bring your own container for arbitrary input and outputs ( #8066 )
...
* support container_component decorator for function with no inputs
* resolve review comments
* add sample tests for milestone 1
* modify compiler test data
* resolve reviews
* resolve reviews
* WIP
* implementation of function of no inputs
* fixed sample test
* re-fix sample test
* fix rebase merge conflict
* resolve formatting
* resolve isort error for test data
* resolve comments
* fix nit
* resolve nit
* add implementation for placeholders i/o, sample and compiler tests
* resolve comments and merge logic for constructing container component
* resolve comments
* resolve comments
* fix assertion messages
* add error handling for accessing artifact by itself
* add test for raising error for accessing artifact by itself
2022-08-08 21:00:53 +00:00
45f3bacc0d
fix(sdk): fix compiling components with artifact inputs ( #8106 )
2022-08-05 17:36:15 +00:00
257f8dbe0a
chore(sdk): change "PipelienSpec" to "PipelineSpec" ( #8102 )
2022-08-04 16:55:06 +00:00
f221d855b1
chore(sdk): remove the unreachable validation code ( #8082 )
2022-08-01 21:03:47 +00:00
1dd06cd8e8
feat(sdk): Allow artifact inputs in pipeline definition. ( #8044 )
...
* Allow artifact inputs in pipeline definition.
* release note
* isort fix
* remove unused imports
* Delete flycheck_compiler.py
* Delete flycheck_pipeline_spec_builder.py
* Delete flycheck_structures.py
* address cr comment
2022-07-28 00:17:53 +00:00
c6a5e387dd
docs(sdk): refresh SDK public API docstrings ( #8057 )
...
* add module-level docstrings
* update compiler docstrings
* update registry module docstrings
* add BaseComponent and children to public api, but discourage use
* update artifact docstrings and type annotations
* update dsl docstrings
* update client docstrings
* clean up kfp.__init__
* add dsl placeholder docstrings
* many more docstring updates
* document type aliases in dsl module
2022-07-21 19:51:18 +00:00
d8b9439ef9
add kfp v2 api reference docs ( #8042 )
...
* use __all__ in top-level modules to record public api
* add index and source files
* add kubeflow assets to _static/
* add and pin requirements
* use block quote instead of header for readme notice
* update conf.py
* delete old files
2022-07-19 15:53:54 -06:00
e6c9411ddf
fix(sdk): fixes kfp.components module type annotations, docstrings, and string/bytes processing ( #8040 )
...
* clean up components types, docstrings, and processing
* update tests
* use real request to get component yaml
* move docstring below copyright
2022-07-19 21:02:49 +00:00
13736b85b6
chore(sdk): clean up kfp code style ( #8027 )
...
* remove unused imports
* use google as isort profile
* sort imports
* format with yapf
* clean end of file new line, trailing whitespace, double quoted strings
2022-07-15 00:06:34 +00:00
be5ff7215f
chore(sdk): update contributing guidelines/tests ( #7802 )
...
* add presubmit checks, contributing guidelines, and pre-commit file
* delete old unused formatting script
* modify scripts
2022-07-12 20:35:29 -06:00
b95c5aba85
feat(sdk/client): implements overriding caching options at submission ( #7912 )
...
* implement override caching options for client
* use absolute path in test for overriding caching
* format code and add unit tests from pipeline
* delete typo
* add two-step pipeline test and review changes
* fix formatting
* fix typo on deprecated code
* update release file
* resolve comments on docstring and testing
* update release file
2022-07-12 16:49:30 +00:00
850a750496
feat(sdk): add retry policy support to kfp v2 ( #7867 )
...
* depend on kfp-pipeline-spec>=0.1.16
* implement RetryPolicy structure
* implement RetryPolicy structure tests
* implement .set_retry method on PipelineTask
* implement retry policy compilation logic
* implement compiler tests for retry
* implement pipeline with retry compilation test
* add release note
* clean up
2022-07-11 16:44:52 +00:00
d3592b9bbd
chore(sdk): bump kfp version to 2.0.0b1 ( #7993 )
...
* bump version
* update release notes
2022-07-07 11:30:01 -06:00
7253203239
add registry json to package data ( #7987 )
2022-07-07 10:48:40 -06:00
aab7fda89b
feat(sdk): Add default registry context ( #7948 )
...
* Add default registry context
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
* Address review comments
* Reinstate `_is_ar_host()` check
* Remove `view=FULL` flag for default list and get
version URLs
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
* Change outdated test name
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
2022-06-30 13:51:05 -06:00
42600a796d
chore(sdk): bump kfp version to 2.0.0b0 ( #7952 )
2022-06-27 18:11:59 -06:00
f877a1ffc8
fix(sdk): fix ifpresent and concat placeholder compile ( #7930 )
...
* fix ifpresent and concat placeholder expansion
* update golden snapshots
* uncomment v1 ifpresent component test case
2022-06-27 16:59:34 -06:00
ad458d535c
fix(sdk): fix placeholder read/write logic ( #7928 )
...
* fix ExecutorInputPlaceholder() usage
* fix OutputPath and OutputParameter placeholder task creation logic
* fix v1 component yaml deserialization logic
* fix typo in test
* ignore description when comparing ComponentSpec
* uncomment fixed test cases
2022-06-27 15:06:38 -06:00
ecb55bde74
feat(sdk): throw informative exception on uncalled @pipeline decorator ( #7913 )
...
* add informative error message for uncalled @pipeline
* add test for exception
2022-06-27 12:29:48 -06:00
a020c9b01c
fix(sdk): fix extract docstring in load component ( #7921 )
...
* fix extract docstring in load component
* implement review feedback
2022-06-24 12:12:35 -06:00
281151c051
chore(sdk): improve IR read/write testing ( #7920 )
...
* add more load component test cases
* add read write equality tests
* add copyright
* remove duplicative structures tests
* add if __name__ == '__main__' block
2022-06-24 09:45:15 -06:00
58f7ab8b49
chore(sdk): Registry Client - default to read full spec of versions ( #7936 )
...
* default to read full spec of versions
* update tests
* fix formatting
2022-06-24 09:43:51 -06:00
4f01b7e602
feat(kfp): implement config file support for registry client ( #7908 )
...
* implement config file support
* update tests
* update PR in response to comments
2022-06-21 15:54:09 -06:00
c6125ffc44
feat(sdk): enable component compilation via cli ( #7649 )
...
* enable component compilation via cli
* reorganize compiler/test_data dir
* clean up pipeline samples
* add component compilation test cases
* update compiler test_data README
* add compiler tests
* test component deserialization is lossless
2022-06-17 05:49:10 +00:00
6935a47e72
feat(sdk): enable component compilation via component decorator ( #7554 )
...
* enable component compilation via decorator
* add release notes
* use decorator syntactic sugar in tests
2022-06-17 00:23:10 +00:00
f84b51e72d
chore(sdk): Support loading v1 yaml with {executorInput: } placeholder ( #7870 )
...
* Support {executorInput: } placeholder
* merge
* use single quotes for consistency
2022-06-14 13:14:31 -07:00
12d5cc3284
chore(sdk): use pytest instead of unittest for test execution ( #7869 )
...
* fix compiler test data path
* condense and add development requirements
* use pytest instead of unittest
* update contributing guidelines
2022-06-10 05:49:19 +00:00
684f477a88
chore(sdk): fix component compilation test path ( #7868 )
2022-06-09 15:28:00 +00:00
de0b824be1
feat(sdk): add support for IfPresentPlaceholder and ConcatPlaceholder strings ( #7795 )
...
* move placeholders to placeholders module; implement if-present and concat placeholders
* update module name and method names throughout codebase
* rename placeholder abc
* add back compat support for CEL-style concat string
* add copyright
* add docstrings; clean up
* remove duplicate code
2022-06-09 07:42:00 +00:00
0ada48b55a
chore(sdk): move pipeline test samples closer to compiler unit tests. ( #7849 )
...
* chore(sdk): move pipeline test samples closer to compiler unit tests.
* explicitly list test files
* remove dead code
2022-06-08 18:35:17 +00:00
9ffcb2e9db
feat(sdk): Add function to sdk client for terminating run ( #7835 )
...
* Add function to sdk client for terminating run
* RELEASE notes
2022-06-08 17:08:16 +00:00
340318625c
chore(sdk): use markdown for long description content type ( #7825 )
2022-06-02 23:37:25 +00:00
c491f331d3
chore(sdk): bump kfp version to 2.0.0a5 ( #7814 )
...
* bump version
* update release notes
2022-06-02 22:34:26 +00:00
0cf817364c
fix(sdk): fix load v1 if present placeholder ( #7765 )
...
* fix load v1 if present placeholder
* add test for load v1 if present placeholder
2022-06-02 16:49:53 +00:00
2c27475fbc
add README with project description ( #7782 )
2022-05-26 14:59:12 -06:00
281c86d41e
chore(sdk): 2.0.0-alpha.4 release ( #7771 )
...
* add release notes
* bump version to alpha.4
* fix release notes
* fix release notes
* fix release notes
2022-05-24 16:49:12 -07:00
a6ec449273
feat(sdk): only use IR when saving component ( #7690 )
...
* move function into compiler module
* remove v2 readable yaml write path
* remove v2 readable yaml test data; remove duplicate tests
2022-05-24 22:23:33 +00:00
cbc2ac5266
feat(sdk): enable read in component using IR ( #7689 )
...
* enable read in primitive components
* enable read in via YAML component
* convert OutputValuePlaceholder to OutputParameterPlaceholder
* add docstring
2022-05-24 18:58:57 +00:00
eaa8ec7807
fix(sdk): Fix corner cases and implement validation ( #7763 )
...
* Handle corner case when list is empty
* add validation for tags and versions
* add tests for empty lists
* add tests for version and tag validation
* fix formatting and optional arguments
* fix type annotation
2022-05-23 17:47:02 +00:00
b0db428165
feat(sdk)!: make CLI output consistent, readable, and usable ( #7739 )
...
* fix cli upload pipeline version
* organize requirements.in
* generate requirements.txt
* add pip-tools to requirements-dev
* improve CLI output
* update release notes
2022-05-20 00:03:23 +00:00
6698801fc0
refactor(sdk): move placeholder logic to structures ( #7687 )
...
* move placeholder formatting logic to structures
* add placeholder formatting logic tests
* update placeholder references throughout codebase
* remove ABC meta inheritance
2022-05-19 07:45:56 +00:00
101d243f48
feat(sdk): validate pipeline resource name before submission ( #7713 )
...
* add pipeline resource name validation
* add pipeline name validation tests
* update downstream pipeline name logic in cli
2022-05-13 20:12:41 +00:00
62972eccf9
feat(sdk): enable compilation of primitive components ( #7580 )
...
* clean up relevant files
* use context manager for temporary directory cleanup
* add component compilation methods
* add component compilation tests
* add more complete pipeline_func argument handling
* use context manager for handling type check mode
* improve checks on inputs for component compilation
* correct pipeline name for primitive component IR
* simplify component compilation logic
* extract compiler methods
* move compiler helpers to helpers file to avoid circular imports
* remove underscore prefix from functions in hidden module
* move helpers tests to helper_test module
* put to_pipeline_spec on ComponentSpec
* fix inputs reference
* use same naming regex for pipelines and components
* permit compilation of all BaseComponent subclasses
* clean up docstrings
* merge compiler.helpers into pipeline_spec_builder
* modify docstrings; refactor
* modify docstring
2022-05-13 20:12:34 +00:00
49cdb42c12
fix cli upload pipeline version ( #7722 )
2022-05-13 12:13:25 -06:00
2570922a7f
feat(sdk): compile JSON with formatting ( #7712 )
...
* clean files
* implement formatted write
* add tests
2022-05-13 17:34:34 +00:00
6b2f015672
chore(sdk): use unittest.mock instead of mock ( #7694 )
2022-05-10 22:55:33 +00:00
1ae6e11bab
feat(sdk): make client return values consistent ( #7659 )
...
* clean up client.py
* update client method return values
* update other types
* use preferred path to kfp_server_api classes
* fix incorrect module name
* other type annotation cleanup
* make docstrings consistent
2022-05-10 14:35:12 -06:00
25e4c58820
feat(sdk): Implement Registry client ( #7597 )
...
* Implement registry client
* Update registry client code
* Add test skeleton
* Add some tests
* Update code
* add tests
* update tests
* update tests
* Rename Client -> RegistryClient
* Update wrt comments
* add type annotations
* fix renaming in __init__.py
* remove unused imports
* extract host variable in test
* format using yapf
* remove locals and use arg keywords
* remove json conversion
* fix header
* write bytes when downloading file
* fix create_tag; fix tests
* fix request_body for update_tag and create_tag using json.dumps
* simply return json for delete_tag
* rename files
* format files
* update return types and format double quotes
* add comments and format files
* add todos
* update credentials and change open to use context
* format using yapf
* move request into context
* Update comments
* Update release notes
* Update release notes
2022-05-10 06:53:06 +00:00
49ccb4f61f
chore(sdk): 2.0.0-alpha.3 release ( #7675 )
...
* reorganize release notes
* finalize alpha.3 notes
* bump version to alpha.3
* add one more release note
2022-05-06 16:45:28 -06:00
18f0842ee0
fix(sdk): some cli fixes ( #7668 )
...
* use correct type when option is path
* fix options on create recurring-run
* make output logging more consistent
* show deprecation warning for dsl-compile on invocation
* fix unarchive experiment
* fix broken f-string
* convert camelcase methods to snake case
2022-05-05 12:47:18 -06:00
5da3826bb5
feat(sdk): use custom basemodel and remove pydantic ( #7639 )
...
* fix discovered bug
* update tests
* implement custom BaseModel
* use basemodel for structures
* remove pydantic dependency
* assorted code cleanup
2022-05-04 12:56:32 -06:00
1118f4859b
feat(sdk): Accept newer `typing-extensions` versions ( #7632 )
...
* Accept newer typing-extensions versions
* Add a description to the release notes.
2022-04-30 00:01:25 +00:00
652ec3b936
feat(sdk): rename CLI methods to 'create' ( #7607 )
...
* add deprecated alias group
* add deprecated alias group tests
* implement deprecated alias group
* clean up other alias tests
* clean up docstring
* fix type annotations / mypy
* format with yapf
2022-04-29 18:52:58 +00:00
6cecfd948e
add deprecation warning to dsl-compile ( #7628 )
2022-04-29 11:34:48 -06:00
ec0081372a
feat(sdk): improve cli help text ( #7618 )
...
* add docstring parsing utils
* implement docstring parsing for help text
* implement smoke test of all cli commands
* clean up use of optional in docstring
* make command descriptions more consistent
* fix disable_type_check
* update release notes
2022-04-29 17:20:57 +00:00
027ac3274f
feat(sdk): add client methods to cli ( #7606 )
2022-04-27 13:21:13 -06:00
e12ac39ba4
feat(sdk): use dsl noun group for compile CLI commands ( #7603 )
...
* use kfp dsl compile for compilation commands
* create and move tests
2022-04-26 15:05:31 -06:00
fbfeadd4a4
feat(sdk): add autocomplete and version options to kfp cli ( #7567 )
...
* add helpful options to cli
* add tests
2022-04-25 14:01:02 -06:00
2636727141
feat(sdk): add archive, unarchive, and delete run methods and tidy docstrings ( #7562 )
2022-04-25 19:05:11 +00:00
e077d01cf9
feat(sdk): add noun aliasing to cli ( #7569 )
...
* allow both singular and plural versions of nouns
* use command table for client creation
* cleanup
* simplify docker mock
* add docker to test requirements; clean up existing requirements
2022-04-25 12:24:57 -06:00
2db431b5f4
use click for components commands ( #7559 )
2022-04-20 14:59:11 -06:00
a5548440a3
use click for dsl-compile command ( #7560 )
2022-04-20 14:57:10 -06:00
b58a31b307
chore(sdk): clean up v2 CLI ( #7558 )
...
* remove linting comments
* move global variable to local
* simplify diagnose_me command
* use f-strings
* idiomatic python refactorings
* remove unused import
* add typestub library
* silence unresolvable mypy error
* update type information
* remove unused variable
* remove unused import
* use __main__.py for cli entrypoint
* expand imports
2022-04-20 10:40:38 -06:00
2494447578
feat(sdk): add .list_pipeline_versions and .unarchive_experiment methods to Client ( #7563 )
...
* add client methods
* update release notes
2022-04-14 21:28:38 +00:00
d46fafe4ce
feat(sdk): port cli code from v1 to v2 ( #7547 )
...
* port cli
* fix docker mock
2022-04-13 20:19:52 +00:00
94960cf5ab
chore(sdk): use setuptools.find_packages ( #7548 )
2022-04-12 21:52:51 +00:00
f70413ad7c
chore(sdk): stop using pydantic for serialization ( #7536 )
...
* add ir_utils
* add ir_utils tests
* use ir_utils to write ir to file
* cast posix path to string in `kfp components build` command
* add copyright
* remove unused import
* update setup.py packages
2022-04-12 17:03:55 +00:00
c98f14abb4
chore(sdk): make field order in compile deterministic ( #7520 )
...
* apply docformatter
* use sort_keys=True in compiler
* update golden snapshots
* set other sort_keys=True
* update structures_test yamls
2022-04-06 21:46:13 +00:00
4eeb420905
feat(sdk)!: promote v2 compiler command and deprecate v1 command ( #7508 )
2022-04-05 18:07:37 +00:00
9db6e74556
chore(sdk): release KFP sdk 2.0.0-alpha.2 ( #7519 )
2022-04-05 00:08:35 -07:00
d74d7b979b
add typestub libraries ( #7451 )
2022-04-04 18:04:33 -06:00
9e708b9208
fix(sdk): fix yaml support `create_run_from_pipeline_func` ( #7500 )
2022-04-01 19:18:25 +00:00
48644edf57
chore(deps): Bump google-cloud-storage to 2.2.1 ( #7493 )
2022-04-01 18:33:25 +00:00
2019a46114
chore(sdk): Update proto link for PipelineTaskFinalStatus ( #7496 )
2022-03-31 20:12:30 +00:00
1c3e2768e6
chore(sdk): modify deprecated tests ( #7446 )
2022-03-25 21:10:11 +00:00
d6e4a34c50
feat(sdk): Add pipeline_task_name to PipelineTaskFinalStatus ( #7464 )
...
* Add pipeline_task_name to PipelineTaskFinalStatus
* release note
2022-03-25 20:48:11 +00:00
9f413706df
feat(sdk): enable pip install from custom index url ( #7453 )
...
* add pip_index_urls to component constructor
* add unit tests
* add compiler cli tests for pip_index_urls
* update golden snapshots
* add release notes
* include test file in compiler_cli_tests
* change spacing in command
* revert golden snapshots
* rename test method
* fix diff
* improve quote consistency
* extract string template from function
* fix component_op
* remove json golden
* update compiler_cli_tests
2022-03-24 20:41:47 +00:00
dfc85b0a0d
feat(sdk)!: Support submitting pipeline IR in yaml format via `kfp.client` ( #7458 )
...
* support yaml payload for pipeline manifest
* release note
2022-03-23 22:48:46 +00:00
04123280a4
fix(sdk): add default value for inputs ( #7405 )
...
* fix(sdk): add default value for inputs
* merge conflict
* release
* fix sample
2022-03-23 18:59:20 +00:00
41caca8850
chore(sdk): remove python 3.6 dead code ( #7440 )
...
* migrate modules to >= python 3.7
* update test images to greatest supported python version
* remove unused import
* update test data
* clean comment
* undo 3.7 image change
* revert 3.9 -> 3.7
* revert OrderedDict to Dict
2022-03-21 22:15:30 +00:00
c7301c449c
feat(sdk)!: use yaml for IR package ( #7431 )
...
* change serialization format from json to yaml
* refactor: use guard clause for clarity
* convert test data from json to yaml
* add and update tests
* permit deprecated compile to json
* re-add json test data
* update compiler tests
* refactor Compiler._write_pipeline_spec_file
* update golden snapshots
* update sample output extensions
* test file extension exceptions and warnings
* update release notes
* remove json golden snapshots
2022-03-21 21:45:31 +00:00
d6affa5afe
docs(sdk): update contributing guidelines ( #7436 )
...
* update SDK contributing guidelines
* add requirements-dev.txt
* update shell script instructions
* update shell script instructions
* add mypy to code style guidelines
* fix path
* Update README.md
2022-03-18 23:41:06 +00:00
bfffa83fba
chore(sdk): apply docformatter ( #7438 )
2022-03-18 17:02:40 +00:00
2c0b38162f
chore(sdk): make kfp v2 hermetic ( #7428 )
...
* move v1 files to v2
* format with yapf
* remove unused imports
* clean up
* update copyright
* move imports to module top level
* apply some pylint changes
* loosen pylintrc
* remove v2_yaml_utils.py
2022-03-18 03:00:40 +00:00
c03701b82a
fix(sdk): fix wrong kfp import causes wrong sdk_version being set in pipeline_spec. ( #7433 )
...
* fix wrong kfp import in compiler
* release note
2022-03-18 00:05:39 +00:00
02a52a4185
fix(sdk): no longer require KFP client for kfp components build ( #7430 )
...
* fix(sdk): no longer require KFP client for kfp components build
* docs(sdk): added release note for kfp components build fix
2022-03-17 22:25:39 +00:00
a5a8c750b5
chore: fix ref link ( #7434 )
2022-03-17 16:03:11 +00:00
481b108319
fix(sdk): update kfp-server-api dependencies and bump sdk version to 2.0.0-alpha.1 ( #7427 )
...
* update kfp-server-api dependencies
* release note
* bump sdk version to 2.0.0-alpha.1
* fix
2022-03-16 23:30:09 +00:00
40e9610711
fix(sdk): include requirements.in in sdk manifest ( #7429 )
2022-03-16 15:55:14 -07:00
4621aec146
chore(sdk): apply yapf formatting ( #7414 )
2022-03-16 20:34:36 +00:00
b741926bb4
feat(sdk): client to support KFP v2 API. ( #7411 )
...
* copy _client.py and _auth.py from deprecated folder
* Make kfp.Client support v2 API (IR).
* fix test golden
* address comments
* copyright year
* chore(backend): clean up pipelinespec.Value usage (#7407 )
* chore(sdk): clean, dedup, reconcile, organize requirements (#7400 )
* chore(sdk): clean, dedup, reconcile, organize requirements
* apply yapf formatting
* correct copyright
* chore(sdk): fix test missing tempdir cleanup (#7403 )
* chore(sdk): fix test missing tempdir cleanup
* fix test path
* fix test path
* clean up setUp method
* change string placeholder from s to test_dir
* release note
Co-authored-by: Connor McCarthy <mccarthy.connor.james@gmail.com>
2022-03-15 21:45:37 +00:00
5fb126a715
feat(sdk): alias v2 imports and provide DeprecationWarning ( #7398 )
...
* feat(sdk): alias v2 imports and provide DeprecationWarning
* simplify tests
* enable import of v2 namespace
* assert DeprecationWarning is raised
* update copyright
* silence pylint
* move v2 alias tests
* use assertWarnsRegex instead of assertWarns
* fix tests
* remove test for deprecation warning
* update copyright
2022-03-15 03:28:39 +00:00
247672c75c
chore(sdk): fix test missing tempdir cleanup ( #7403 )
...
* chore(sdk): fix test missing tempdir cleanup
* fix test path
* fix test path
* clean up setUp method
* change string placeholder from s to test_dir
2022-03-14 21:25:40 +00:00
7613cf9a1f
chore(sdk): clean, dedup, reconcile, organize requirements ( #7400 )
...
* chore(sdk): clean, dedup, reconcile, organize requirements
* apply yapf formatting
* correct copyright
2022-03-14 18:27:57 +00:00
5dcc6d9910
chore(sdk): rename PipelineTask arguments parameter to args ( #7392 )
2022-03-10 21:42:54 +00:00
5ca4cb78d3
chore(sdk): remove unused imports and variables ( #7393 )
2022-03-09 21:42:11 +00:00
82d17ff616
feat(sdk)!: rename commands, arguments parameters to command, args ( #7391 )
2022-03-09 05:39:11 +00:00
d695ec556a
feat(sdk)!: throw error for output_component_file ( #7390 )
...
* feat(sdk)!: throw error for output_component_file
* Update readme
2022-03-09 01:05:10 +00:00
8bb0d0e025
feat(sdk)!: move v2 to main namespace ( #7376 )
...
* feat(sdk)!: move v2 to main namespace
* edit samples
2022-03-04 21:56:51 +00:00
7f67130c6e
fix(sdk): require google-api-core>=1.31.5, >=2.3.2 ( #7377 )
...
* require google-api-core>=1.31.5, >=2.3.2
* add release note
2022-03-04 19:56:59 +00:00
b693909a08
feat(sdk)!: move v1 to deprecated folder ( #7291 )
...
* chore(sdk): move v1 to deprecated
* fix testsg
* fix testsg
* fix setup.py
* fix test
* s'
* fix tests
* fix test
* retore v2 test changes
* fix py3.6 test
* fix py3.6 test
* fix py3.6 import fallback error
* remove deprecated
* fix samples test
* sample test
* fix samples
* add readme
* restroe test
* python require
* remove path
* fix tests
* inteegration tests
* remove tfx tests for dependency with kfp v1
* fix e2e
* fix e2e
* fix integration tests
* fix sampe
* move client down
* change to kfp
* add import alias
* fix
* runid
* fix dsl
* only use kfp for function
* revert train_until_good
* tfx test
* kfp
* try import
* onprem
2022-03-04 00:52:59 +00:00
913539728b
chore(sdk): Fix a minor bug in task final status test sample ( #7314 )
2022-02-17 00:27:34 +00:00
de2a4a45f2
chore(sdk): bump version to 2.0.0-alpha.0 ( #7313 )
...
* chore(sdk): bump version to 2.0.0-alpha.0
* relax regex to match sdk version in tests
2022-02-15 20:07:41 +00:00
54d5789bb4
feat(sdk)!: drop support for python 3.6 ( #7303 )
...
* chore(sdk): drop support for python 3.6
* empty
* address comments
2022-02-15 10:54:46 -08:00
ab805f886c
feat(sdk): Support getting pipeline status in exit handler. ( #7309 )
...
* Support PipelineTaskFinalStatus in exit handler.
* release note
2022-02-15 17:58:41 +00:00
9be1720f55
chore(sdk): Relax Typing Extensions Constraint ( #7288 )
...
* Relax Typing Extensions Constraint
* Update requirements.txt for typing extensions
Co-authored-by: Mehdi Drissi <mdrissi@snapchat.com>
2022-02-15 10:32:40 +00:00
e07643453c
feat(sdk): Add `enable_job` method to client ( #7239 )
...
* feat(sdk): implement enable_job method
* Update RELEASE.md
Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
2022-02-03 18:29:22 -08:00
060588e1d9
fix(sdk): add verify_ssl to kfp client, fixes #4569 ( #7174 )
...
* adding verify_ssl
* adding code formatter changes
* retrigger checks
* updating release.md
* retrigger checks
* retrigger checks
* updating release.md
2022-02-03 18:28:16 -08:00
808ff5d8ed
fix(sdk): Add in filter to list_pipeline_versions SDK method ( #7223 )
...
* AIP-5661: Add in filter to list_pipeline_versions SDK method
* AIP-5661: Update type hints + docstrings to conform to standard. Include message in release notes
2022-02-03 16:52:45 -08:00
1acb90fc94
fix(sdk.v2): Fix importer not using correct output artifact type ( #7235 )
...
* fix importer output type
* release note
2022-01-31 14:34:08 -08:00
7f178900ae
chore(sdk): Bump SDK version to 1.8.11 ( #7202 )
2022-01-24 22:42:19 +00:00
577902ca46
fix(sdk): Support per workflow TTL (ttl_seconds_after_finished) with new format of Argo workflow manifest ( #7141 )
...
* Support per workflow TTL (ttl_seconds_after_finished) with new format of Argo workflow manifest
* Update test for TTL
* Declare fix in release note of SDK
* update syntaxe
* Update RELEASE.md
Add Braking change due to incompatibility with KFP pre 1.7 due to Argo 2.X
2022-01-20 00:43:52 +00:00
22a8fdfe31
fix(sdk): bumps up absl version to 1.x for compatibility with TF 2.8+. Fixes #7171 ( #7172 )
...
* Bumps up absl version to 1.x for compatibility with TF 2.8+.
* Adds a release note for absl version.
2022-01-19 22:14:52 +00:00
83d54d3cb7
fix(sdk): Fix create_runtime_artifact not covering all types. ( #7168 )
...
* Fix create_runtime_artifact not covering all types.
* release note
2022-01-19 04:16:19 +00:00
fc2d05f90d
fix(sdk): Respect namespace if set on client ( #7056 )
...
* respect namespace if set on client
* make sure is_multi_user
* overwrite directly context_setting
* revert non-related changes
* revert overwrite_namespace
* expand text
* add release notes
* add PR link
2022-01-11 06:04:42 +00:00
5a29d7f95b
feat(sdk): add metadata field for importer ( #7112 )
...
* feat(sdk): add metadata
* metadata
* release notes
* address comments
2022-01-06 00:41:35 +00:00
ca6e05591d
test: update README of v2 sample test ( #7121 )
...
* test: update README of v2 sample test
* address review comments by @zijianjoy
* use absolute paths, suggested by @zijianjoy
* Update README.md
2021-12-30 16:22:54 +08:00
2460aaa653
test: migrate sample integration test to v2 sample test infra ( #7115 )
...
* test: migrate sample integration test to v2 sample test infra
* test: configure v2 integration test timeout
2021-12-26 04:17:44 +00:00
44228c242b
test(v2): v2 sample test supports notebooks ( #7109 )
...
* test(v2): v2 sample test supports notebooks
* fix
* fix relative path problem
2021-12-24 02:37:42 +00:00
6dfaeebd92
fix(sdk.v2)!: Block task dependency referencing tasks inside a sibling condition or loop group. ( #7050 )
2021-12-13 19:08:37 +00:00
85d74337d6
fix(sdk): fix input/outputspec and positional arguments ( #6980 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
* fix(sdk): fix resource spec test
* release notes
* test for ontology
* fix(sdk): fix input/outputspec
* fix reuse
* fix xgboost sample
* release
* fix tests
* address comments
* fix test
* restore tests
* address comments
2021-12-13 18:12:37 +00:00
1a0bef7102
fix(sdk.v2): fix boolean error for inputs ( #7041 )
2021-12-11 17:25:35 +00:00
6ca35800c7
chore: SDK release note for 1.8.10 ( #7017 )
...
* Update RELEASE.md
* Update __init__.py
2021-12-08 00:05:59 +00:00
a71fce0841
fix(sdk): fix bug in checking values in _param_values. Fixes #6799 ( #6965 )
2021-12-03 18:14:57 +00:00
bc7aad8b75
chore(sdk): fix error 'ApiResourceReference' object has no attribute 'id' ( #6997 )
...
* chore(sdk): fix error 'ApiResourceReference' object has no attribute 'id'
* fix more
2021-12-03 01:28:58 +00:00
c1b67154a2
fix(sdk.v2): fix regression on nested loops ( #6990 )
...
* fix regression on nested loops
* release note
2021-12-01 22:34:05 +00:00
ddbfcde16f
feat(sdk): Improve CLI experience for archiving experiments, managing recurring runs and listing resources ( #6934 )
2021-12-01 18:07:13 +00:00
485a07719f
fix(sdk): fix resource spec test ( #6979 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
* fix(sdk): fix resource spec test
* release notes
* test for ontology
2021-11-30 19:25:31 +00:00
0842934bae
fix(sdk): fix load_test ( #6978 )
...
* fix(sdk): fix load_test
* concat placeholder
* release notes
2021-11-29 23:00:30 +00:00
8aee62142a
feat(sdk)!: Deprecate V2 compatible mode in v1 compiler ( #6958 )
...
* Deprecate V2 compatible mode in v1 compiler
* release note
* fix tests
2021-11-24 22:11:34 +00:00
59dfffebea
fix(sdk.v2): Avoid pydantic union bug ( #6957 )
...
* avoid pydantic union bug
* release note
2021-11-24 21:48:34 +00:00
cd20373310
feat(sdk): add set_env_variable for Pipeline task ( #6919 )
...
* feat(sdk): add set_env_variable
* release notes
* enable test
* fix test
* fix test
* address comments
2021-11-23 21:46:47 +00:00
4484ccd0b3
fix(sdk): change otherwise to else in yaml ( #6952 )
...
* fix(sdk): change otherwise to else in yaml
* release notes
2021-11-23 21:31:49 +00:00
6d55e262b4
fix(sdk): refresh access token only when it expires. Fixes #6883 ( #6941 )
...
* refresh access token whenerver it expires
* tight the condition when refreshing the access token
2021-11-23 19:35:07 +00:00
8064383cf1
chore(sdk): allow google-auth < 3 ( #6939 )
...
* Allow google-auth < 3
google-auth recently published a 2.0.0 release which removed support for Python 2.7. google-auth now requires Python >=3.6. No other breaking changes were made. You can see the full list of changes [here](googleapis/google-auth-library-python@560cf1e ).
I am opening PRs to expand google-auth version ranges for packages that meet either of the following criteria:
* Package has >10,000 monthly downloads as of June 2021
* Package is owned by a Google team
**Motivation:**
google-auth is a dependency of many different libraries that interact with Google APIs. Increasing the time and number of packages with compatible pins on google-auth lowers the chance end developers who use multiple libraries will see dependency conflicts.
If possible, please also retain compatibility with 1.x.x versions of google-auth until this issue is resolved
googleapis/google-cloud-python#10566 , as that will further reduce the likelihood of diamond dependency conflicts.
Googlers, see [this doc](https://docs.google.com/document/d/1euAvUsia_4zf98lNvpwA3K0o2b4y5Xr9xAkyzCnIMzQ/edit ) for more information.
* chore: also update requirements.in
* chore: update RELEASE.md
2021-11-23 18:47:06 +00:00
228d37ec5c
chore(sdk.v2): Add test sample for runtime placeholders ( #6946 )
2021-11-23 17:23:06 +00:00
b6406b02f4
chore(sdk): Fix a warning message ( #6911 )
...
* Fix(sdk): Enable importing `kfp.v2.components.create_component_from_func`
* Revert "Fix(sdk): Enable importing `kfp.v2.components.create_component_from_func`"
This reverts commit b5d3f13720
2021-11-22 16:17:05 -08:00
2910d10cc3
fix(sdk.v2): Fix bug on executor input with default value. ( #6937 )
...
* fix bug on executor input
* release note
* fix test on empty message
* fix part of the test
2021-11-22 21:09:06 +00:00
c4d8dfd8c6
feat(sdk): ImporterSpec v2 ( #6917 )
...
* importer
* compiler
* check diff
* fix format
* remove v1 node test
* remove importer v2 compat test
* release note
* address comments
2021-11-18 23:11:32 +00:00
b482ba83d8
fix(sdk): visualizations and metrics do not work with data_passing_methods ( #6882 )
...
* Update _data_passing_using_volume.py
* Update _data_passing_using_volume.py
* Update _data_passing_using_volume.py
* Update _data_passing_using_volume.py
* Update _data_passing_using_volume.py
* Update artifact_passing_using_volume.yaml
* Update artifact_passing_using_volume.py
* Update _data_passing_using_volume.py
2021-11-16 01:33:12 +00:00
9ee4534aef
fix(sdk.v2): Fix regression on optional inputs ( #6905 )
...
* Fix regression on optional inputs
* add release note
2021-11-15 18:43:43 +00:00
7921946b07
feat(sdk.v2)!: Merge v2 experimental change back to v2 namespace ( #6890 )
...
* merge v2 experimental change back to v2 namespace
* release note
* fix tests
* fix cli
* fix py3.6 test
* skip v2 compatible mode tests
* fix tests
* skip v2 engine tests
2021-11-11 00:25:42 +00:00
b7fd47f70b
chore(sdk): Remove sdk/python/kfp/v2/google directory for v2 ( #6886 )
...
* Delete sdk/python/kfp/v2/google directory
* Update RELEASE.md
* Update MANIFEST.in
* Update requirements-test.txt
* Update requirements.in
* fix setup
* remove test
2021-11-10 01:18:04 -08:00
3bbe81a463
chore(sdk): Remove redundant check in set_gpu_limit ( #6866 )
...
* remove redundant check in set_gpu_limit
* release note
2021-11-08 10:35:02 -08:00
bb9db6a19d
chore(sdk): Release KFP SDK 1.8.9 off `sdk/release-1.8` branch ( #6864 )
...
* Update RELEASE.md
* Update __init__.py
2021-11-03 23:00:21 -07:00
dea0823fe8
fix(sdk): Make `Artifact` type be compatible with any sub-artifact types bidirectionally ( #6859 )
2021-11-03 16:34:21 -07:00
334307d6c3
chore(sdk): add deprecation warnings for v2 ( #6851 )
2021-11-02 16:26:06 -07:00
b2414f3eb8
fix(sdk): fix cloud scheduler's job name ( #6844 )
2021-11-02 13:46:06 -07:00
2f2ebf8962
fix(sdk): Fix invalid doc example of python_component ( #6841 )
2021-11-02 11:41:06 -07:00
0f2cab9b39
fix(sdk.v2): Fix display name support for groups ( #6832 )
...
* fix display name for experimental tasks_group
* release note
2021-10-29 00:30:51 -07:00
f1bb852852
feat(sdk): add load_component_from_* ( #6822 )
...
* feat(sdk): add load_component_from_*
* address comments'
:
* update release notes
2021-10-28 22:30:34 -07:00
8c6843fdb8
fix(sdk.v2): Fix importer ignoring `reimport` setting, and switch to Protobuf.Value for import uri. ( #6827 )
...
* fix importer
* release note
* remove error usage in test sample
* disable importer test
2021-10-28 20:40:51 -07:00
ea2e5be81a
feat(sdk.v2): Implement experimental `@component` component ( #6825 )
...
* Implement experimental `@component` component
* release note
2021-10-28 18:29:51 -07:00
0be57c3880
feat(sdk): Use google.protobuf.Value in v2 for passing parameters. ( #6804 )
...
* Use google.protobuf.Value in v2 for passing parameters.
* retest samples.
* Fix tests.
* Update release, more cleanup.
* Use github.com/kubeflow/pipelines/api from same repo.
* Run go mod tidy
* chore: go mod tidy
* fix v2 compile error and clean up unused code
* pr comments.
* update goldens
* Fix metadata recording.
* Update kfp mlmd client.
* fix test again
* another try.
* chore: migrate v2 DAG driver input parameters to protobuf.Value + small refactorings
* fix v2 launcher + clean up
* fix a compile error
* fix a few more tests
* fix number parsing
* clean up
* disable cache_v2 test.
Co-authored-by: Yuan Gong <gongyuan94@gmail.com>
2021-10-28 14:25:51 -07:00
0c40154f6d
chore: Release KFP SDK and v2 launcher 1.8.7 ( #6823 )
...
* Update RELEASE.md
* Update __init__.py
* Update v2_compat.py
2021-10-28 10:53:26 -07:00
3e6c776360
feat(sdk): Add load_component_from_spec. Fixes #5708 #3748 ( #6690 )
...
* add load_component_from_spec
* update release.md
* update RELEASE.md
* update RELEASE.md
2021-10-28 10:27:57 -07:00
41275b4228
fix(sdk): fixes the specified 'mlpipeline-ui-metadata','mlpipeline-metrics' path is overrided by default value ( #6796 )
...
* fix the issue:
the 'mlpipeline-ui-metadata','mlpipeline-metrics' path is replaced by the default path as '_components._generate_output_file_name(output.name)'
* reformat code
* Update _container_op.py
* Update RELEASE.md
2021-10-28 01:33:00 -07:00
37d3234ce5
feat(sdk): Add filters to python client. ( #6748 )
...
* Add filters to python client.
* Add filter docstrings.
h/t @chensun
Co-authored-by: Chen Sun <chensun@users.noreply.github.com>
2021-10-28 01:15:50 -07:00
ec4ab2dc4c
fix(sdk): Add missing retry policy. ( #6808 )
2021-10-27 23:41:30 -07:00
7768f28de3
fix(sdk.v2): Fix missing experimental modules in setup.py ( #6817 )
...
* fix missing experimental modules
* fix typo
* fix test
2021-10-27 10:58:39 -07:00
fca4c1c2fd
Fix missing experimental type module in setup.py. ( #6816 )
2021-10-27 09:17:03 -07:00
02b96f0a63
feat(sdk.v2): Implement v2 experimental compiler. ( #6803 )
...
* Implement v2 experimental compiler.
* add release note
* Support protobuf.Value in Condition operand
2021-10-26 14:12:37 -07:00
5dfae1d8cc
feat(sdk): Add version description optionally to Py SDK. Part of #6256 ( #6472 )
...
* add description optionally
* add release notes
* add new line
* capture if description is not accepted by server
* add todo comment
* add better description api missing exception
2021-10-24 21:24:07 -07:00
61fea6ba1f
fix(sdk.v2): Fix InputPathPlaceholder changed to InputValuePlaceholder ( #6794 )
...
* fix InputPathPlaceholder changed to InputValuePlaceholder
* release note
2021-10-24 17:10:07 -07:00
Tara Tufano
Connor McCarthy
Owen
Alex
Mai Nakagawa
Sune Debel
JOCSTAA
Alex
andreafehrman
Lingqing Gan
Jonny Browning (Datatonic)
Chen Sun
Scott_Xu
gkcalat
Alex Latchford
Yusuke Nishioka
chongyouquan
Bo
Deepyaman Datta
Christian Kadner
johnmacnamararseg
Mai Nakagawa
Hao Xin
Yaqi Ji
Mehdi Drissi
vmallya-123
Seb
Jiyong Jung
Gerard Casas Saez
Yuan (Bob) Gong
michaelliqx
hieuhc
Bu Sun Kim
juliusvonkohout
Trs
Toshiya Kawasaki
Ajay Gopinathan
Omar Marzouk
dinimicky
Joshua Carp