Commit Graph

11 Commits

Author SHA1 Message Date
Jiaxiao Zheng b099c6f5d3
chore: Rollback telemetry related changes (4088)
* Revert "fix length (#3934)"

This reverts commit 7fbb7cae

* Revert "[SDK] Add first party component label (#3861)"

This reverts commit 1e2b9d4e

* Revert "[SDK] Add pod labels for telemetry purpose. (#3578)"

This reverts commit aa8da64b
2020-06-27 15:46:14 -07:00
Jiaxiao Zheng aa8da64b4c
[SDK] Add pod labels for telemetry purpose. (#3578)
* add telemetry pod labels

* revert the id label

* update compiler tests

* update cli arg

* bypass tfx

* update docstring
2020-04-27 18:50:04 -07:00
Alexey Volkov ef63c653af SDK - Compiler - Fix large data passing (#2173)
* SDK - Compiler - Fix large data passing

Stop outputting parameters unless they're consumed as parameters downstream.
This prevents the situaltion when component outputs a big file, but DSL compiler instructs Argo to pick it up as parameter (parameters only hold few kilobytes of data).

As byproduct, this change fixes some minor compiler data passing bugs where some parameters were being passed around, but never consumed (happened with `ResourceOp`, `dsl.Condition` and recursion).

* Replaced ... with `raise AssertionError`

* Fixed small bug

* Removed unused variables

* Fixed names of the mark_upstream_ios_of_* functions

* Fixed detection of parameter output references

* Fixed handling of volumes
2019-09-20 15:05:27 -07:00
Alexey Volkov e3c72fc251 SDK - Persisting all output values (#2134)
Currently, the parameter output values are not saved to storage and their values are lost as soon as garbage collector removes the workflow object.
This change makes is so the parameter output values are persisted.
2019-09-16 19:44:24 -07:00
Alexey Volkov d83601d19a SDK - Compiler - Quoting the predicate operands (#2043)
Fixes https://github.com/kubeflow/pipelines/issues/1950
2019-09-06 17:05:21 -07:00
Alexey Volkov 5360f3fcab SDK - Compiler - Stopped adding mlpipeline artifacts to every compiled template (#2046)
* Explicitly added mlpipeline outputs to the components that actually produce them

* Updated samples

* SDK - DSL - Stopped adding mlpipeline artifacts to every compiled template
Fixes https://github.com/kubeflow/pipelines/issues/1421
Fixes https://github.com/kubeflow/pipelines/issues/1422

* Updated the Lighweight sample

* Updated the compiler tests

* Fixed the lightweight sample

* Reverted the change to one contrib/samples/openvino
The sample will still work fine as it is now.
I'll add the change to that file as a separate PR.
2019-09-05 17:56:57 -07:00
Alexey Volkov b496720d6d SDK - Skip attributes with missing values during PipelineMeta serialization (#1448)
* SDK - Skip attributes with missing values during PipelineMeta serialization

* Fixed the tests
2019-08-26 17:02:40 -07:00
Alexey Volkov d8eaeaad95
SDK - Preserving the pipeline input information in the compiled Workflow (#1381)
* SDK - Preserving the pipeline metadata in the compiled Workflow

* Stabilizing the DSL compiler tests
2019-08-15 17:25:59 -07:00
Alexey Volkov be4e56a247 SDK - Stopped hard-coding artifact storage configuration in the pipeline packages (#1297)
We should follow Argo's prefferred way to configure the artifact storage: https://github.com/argoproj/argo/blob/master/ARTIFACT_REPO.md#configure-the-default-artifact-repository and use a cluster-local configMap. This way the pipelines remain clean and portable: https://github.com/argoproj/argo/blob/master/examples/artifact-passing.yaml
Kubeflow deployer has already been pre-installing the configMap for several months: https://github.com/kubeflow/kubeflow/pull/2238
2019-05-13 17:29:08 -07:00
Alexey Volkov 6b0d4d13e7 Marking the UI-metadata and Metrics artifacts as optional (#1260)
* Marking the UI-metadata and Metrics artifacts as optional

* Updated the compiler tests

* Updates one more test.
2019-04-29 17:27:37 -07:00
Ning 1a04e86ed7 Recursion bug fix (#1061)
* remove the graph component output; add support for dependency on graph component

* fix bug; adjust unit tests

* add support for explicit dependency of graph component

* adjust unit test

* add a todo

* bug fixes for unit tests

* refactor condition_param code; fix bug when the inputs task name is None; need to remove the print later

* do not pass condition param as arguments to downstream ops, remove print logs; add unit tests

* add unit test golden yaml

* fix bug

* fix the sample
2019-04-02 09:49:19 -07:00