This is needed for the future storage system based on volume mounts:
If outputs were written to files in the same dir (e.g. /outputs/out1.txt and /outputs/out2.txt), then we cannot separate them and mount to the downstream task containers independently.
* Reworked the Component structures.
Rewrote parsing, type checking and serialization code.
Improved the graph component structures.
Added most of the needed k8s structures.
Added model validation (input/output existence etc).
Added task cycle detection and topological sorting to GraphSpec.
All container component tests now work.
Added some graph component tests.
* Fixed incompatibilities with python <3.7
* Added __init__.py to make the Travis tests work.
* Adding kubernetes structures to setup.py
* Addressed PR feedback: Renamed _original_names to _serialized_names
* Addressed PR feedback: Reduced indentation.
* Added descriptions for all component structures.
* Fixed a bug in ComponentSpec._post_init()
* Added documentation for ModelBase class and functions.
* Added __eq__/__ne__ and improved __repr__
* Added ModelBase tests
* add comments
* relocate functions in compiler to aggregate similar functions; move _build_conventional_artifact as a nested function
* reduce sanitize functions into one in the dsl.
* more comments
* move all sanitization(op name, param name) from dsl to compiler
* sanitize pipelineparam name and op_name; remove format check in pipelineparam
* remove unit test for pipelineparam op_name format checking
* fix bug: correctly replace input in the argument list
* fix bug: replace arguments with found ones
* Sanitize the file_output keys, Matches the param in the args/cmds with the whole serialized param str, Verify both param name and container name
* loosen the containerop and param name restrictions
* Expose that the python API is Python3.5+ only
Adding these classifiers (many are the same as e.g. TensorFlow) makes sure pip/pypi index pick these up as being 3.5+ only, so that developers on Python 2 are not suprised when it fails on Python 3 syntax (e.g. typehints are only py 3.5+).
* Update setup.py
* Update setup.py
* Update setup.py
* Support replacable arguments in command as well (besides arguments) in container op.
* Fix components builder.
* Fix tests.
* Follow up CR comments.
* Fix test.
* Update sample notebook to clean up deployed models.
Update SDK client to return correct links in local Jupyter with user's own proxy connection.
* Fix sample tests.
* First step to bring back CMLE sample.
Set GCP credential for gcloud. Do not export TPU Model because a) Serving does not need TPU. b) Workaround a CMLE 1.9 issue.
* Follow up on CR comments.
* Add comments.
* SDK/Components - Made component search locations configurable
* Raise proper error on existing, but malformed components instead of saying the component was not found.
Component name must be non-empty.
Addressed PR feedback.
* add get_experiment_id and list_runs_by_experiment
* offer only one get_experiment function
* return experiment body instead of id
* simply codes
* simply code 2
* remove experiment_id check in the while loop
* minor bug
* add notebook sample tests for tfx
* parameterize component image tag
* parameterize base and target image tags
* install tensorflow package for the notebook tfx sample test
* bug fixes
* start debug mode
* fix bugs
* add namespace arg to check_notebook_results, copy test results to gcs, fix minor bugs
add CMLE model deletion
* install the correct KFP version in the notebook; parameterize deployer model name and version
* fix CMLE model name bug
* add notebook sample test in v2
* add gcp sa in notebook tfx sample and shutdown debug mode
* import kfp.gcp
* Now pipeline function takes direct default values rather than dsp.PipelineParam. It simplifies the sample code a lot.
* Remove extraneous parenthesis.
* Follow up CR comments.
* Change Dockerfile (not done).
* Fix dockerfile.
* Fix Dockerfile again.
* Remove unneeded installation of packages in Dockerfile.
* Add support for nvidia gpu limit
* Expose resource limits, requests and nodeSelector to ContainerOp
* Fix test data
* Add explicit set_gpu_limit function
* Fix logical bug
Alexey Volkov
Ning
Tim Zaman
qimingj
IronPan
hongye-sun
Chris Van Pelt