kubeflow
/
pipelines
mirror of https://github.com/kubeflow/pipelines.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,601 Commits 301 Branches 176 Tags 466 MiB
Commit Graph

23 Commits

Author SHA1 Message Date
Connor McCarthy 08a8b1df97
refactor(sdk): remove dead BaseModel code (#8415) 
* remove dead basemodel code

* remove unused deserialization logic

* remove more dead code

* remove basemodel entirely
 2022-11-03 21:58:15 +00:00
Connor McCarthy 4c6abe5afc
refactor(sdk): simplify placeholders and logic (#8402) 
* simplify placeholders

* update placeholders_test.py

* update structures to use placeholders correctly

* update use of placeholders elsewhere in codebase

* clean up

* move placeholder type checking logic to PipelineTask to prevent v1 load breaks

* clean up

* update docstrings

* make to_dict and to_string private methods

* revert unnecessary change to tests

* improve type safety of code

* prefix module-level constant with underscore
 2022-11-02 00:31:45 +00:00
Connor McCarthy e371fa3795
feat(sdk): add compilation logic for google artifact types (support custom artifact types pt. 2) (#8232) 
* remove dependency of deprecated on v2

* use new artifact identification logic

* add custom artifact type compiler test

* compile schema_version from non-kfp artifacts

* update tests

* add importer to compiler test

* update golden snapshots

* fix type checking code

* undo executor_test diff

* remove gcpc types compiler test

* add compilation test comment

* update google artifact handling

* update compiler test

* only permit system. and google. artifact types; add tests

* update compiler test

* simplify type checking logic

* use bundled artifact types in PipelineArtifactChannel

* raise error on unknown v1 input or output

* add type utils and move functions

* add type utils tests

* remove schema_version from inputspec and outputspec

* update artifact proto creation and tests

* propogate bundled artifact type string changes throughout sdk

* update golden snapshot

* remove unused v1 compat and tests

* update type checking and tests to validate schema_version using semantic versioning

* update importer sample

* update test data config

* update test_data config

* clean up sample

* clean up sample
 2022-09-12 14:00:02 -06:00
Chen Sun 410e99c604
feat(sdk): Support loading pipeline from yaml (#8209) 
* support loading pipeline from yaml

* release note

* cleanup

* maintain read/write consistency with component/pipeline compilation.
 2022-08-31 12:05:55 +00:00
Scott_Xu c7a90a3fa8
feat(sdk): expose IfPresentPlaceholder and ConcatPlaceholder to kfp.dsl (#8145) 2022-08-15 23:22:06 +00:00
Scott_Xu 35fccb1fae
feat(sdk): expose new ContainerSpec API (#8144) 
* rename ContainerSpec to ContainerSpecImplementation for internal usage

* fix nit
 2022-08-15 20:33:34 +00:00
Scott_Xu d3c0537c8b
chore(sdk): Add test for if and concat placeholder and validate artifact channel usage (#8120) 
* test if and concat placeholder and validate artifact channel usage

* fix nit in read write test config

* resolve comments

* resolve comments and move test cases

* resolve comments and add placeholder validation

* remove unused imports

* update release file
 2022-08-10 23:35:03 +00:00
Connor McCarthy 850a750496
feat(sdk): add retry policy support to kfp v2 (#7867) 
* depend on kfp-pipeline-spec>=0.1.16

* implement RetryPolicy structure

* implement RetryPolicy structure tests

* implement .set_retry method on PipelineTask

* implement retry policy compilation logic

* implement compiler tests for retry

* implement pipeline with retry compilation test

* add release note

* clean up
 2022-07-11 16:44:52 +00:00
Connor McCarthy ad458d535c
fix(sdk): fix placeholder read/write logic (#7928) 
* fix ExecutorInputPlaceholder() usage

* fix OutputPath and OutputParameter placeholder task creation logic

* fix v1 component yaml deserialization logic

* fix typo in test

* ignore description when comparing ComponentSpec

* uncomment fixed test cases
 2022-06-27 15:06:38 -06:00
Connor McCarthy 281151c051
chore(sdk): improve IR read/write testing (#7920) 
* add more load component test cases

* add read write equality tests

* add copyright

* remove duplicative structures tests

* add if __name__ == '__main__' block
 2022-06-24 09:45:15 -06:00
Connor McCarthy c6125ffc44
feat(sdk): enable component compilation via cli (#7649) 
* enable component compilation via cli

* reorganize compiler/test_data dir

* clean up pipeline samples

* add component compilation test cases

* update compiler test_data README

* add compiler tests

* test component deserialization is lossless
 2022-06-17 05:49:10 +00:00
Chen Sun f84b51e72d
chore(sdk): Support loading v1 yaml with {executorInput: } placeholder (#7870) 
* Support {executorInput: } placeholder

* merge

* use single quotes for consistency
 2022-06-14 13:14:31 -07:00
Connor McCarthy 684f477a88
chore(sdk): fix component compilation test path (#7868) 2022-06-09 15:28:00 +00:00
Connor McCarthy de0b824be1
feat(sdk): add support for IfPresentPlaceholder and ConcatPlaceholder strings (#7795) 
* move placeholders to placeholders module; implement if-present and concat placeholders

* update module name and method names throughout codebase

* rename placeholder abc

* add back compat support for CEL-style concat string

* add copyright

* add docstrings; clean up

* remove duplicate code
 2022-06-09 07:42:00 +00:00
Connor McCarthy 0cf817364c
fix(sdk): fix load v1 if present placeholder (#7765) 
* fix load v1 if present placeholder

* add test for load v1 if present placeholder
 2022-06-02 16:49:53 +00:00
Connor McCarthy a6ec449273
feat(sdk): only use IR when saving component (#7690) 
* move function into compiler module

* remove v2 readable yaml write path

* remove v2 readable yaml test data; remove duplicate tests
 2022-05-24 22:23:33 +00:00
Connor McCarthy cbc2ac5266
feat(sdk): enable read in component using IR (#7689) 
* enable read in primitive components

* enable read in via YAML component

* convert OutputValuePlaceholder to OutputParameterPlaceholder

* add docstring
 2022-05-24 18:58:57 +00:00
Connor McCarthy 6698801fc0
refactor(sdk): move placeholder logic to structures (#7687) 
* move placeholder formatting logic to structures

* add placeholder formatting logic tests

* update placeholder references throughout codebase

* remove ABC meta inheritance
 2022-05-19 07:45:56 +00:00
Connor McCarthy 5da3826bb5
feat(sdk): use custom basemodel and remove pydantic (#7639) 
* fix discovered bug

* update tests

* implement custom BaseModel

* use basemodel for structures

* remove pydantic dependency

* assorted code cleanup
 2022-05-04 12:56:32 -06:00
Connor McCarthy f70413ad7c
chore(sdk): stop using pydantic for serialization (#7536) 
* add ir_utils

* add ir_utils tests

* use ir_utils to write ir to file

* cast posix path to string in `kfp components build` command

* add copyright

* remove unused import

* update setup.py packages
 2022-04-12 17:03:55 +00:00
Connor McCarthy c98f14abb4
chore(sdk): make field order in compile deterministic (#7520) 
* apply docformatter

* use sort_keys=True in compiler

* update golden snapshots

* set other sort_keys=True

* update structures_test yamls
 2022-04-06 21:46:13 +00:00
Connor McCarthy 82d17ff616
feat(sdk)!: rename commands, arguments parameters to command, args (#7391) 2022-03-09 05:39:11 +00:00
Yaqi Ji 8bb0d0e025
feat(sdk)!: move v2 to main namespace (#7376) 
* feat(sdk)!: move v2 to main namespace

* edit samples
 2022-03-04 21:56:51 +00:00