* kfp frontend API service can configure minio client params thru env vars
* minio endpoint is composed from host and namespace to support k8s yaml
* Added kustomize patch for pipeline-ui deploy
* Shows link to Stackdriver logs if logs retrieval fails and cluster is running in GKE
* Add additional link for new k8s stackdriver resource names
* Fix spacing
* Add s3 and local support for artifact viewer
* Remove local file support and use ViewerCRD instead
* Fix condition failed tests
* Keep using minio for metadata manifests
* Allows toggling between one-off and recurring runs in the new run page
* Clean up and adds tests
* Fix integration test - account for extra field in form
* Cleanup and PR comments
* SDK: Create BaseOp class
* BaseOp class is the base class for any Argo Template type
* ContainerOp derives from BaseOp
* Rename dependent_names to deps
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: In preparation for the new feature ResourceOps (#801)
* Add cops attributes to Pipeline. This is a dict having all the
ContainerOps of the pipeline.
* Set some processing in _op_to_template as ContainerOp specific
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the consumption of Volumes by ContainerOps
Add `pvolumes` argument and attribute to ContainerOp. It is a dict
having mount paths as keys and V1Volumes as values. These are added to
the pipeline and mounted by the container of the ContainerOp.
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Add ResourceOp
* ResourceOp is the SDK's equivalent for Argo's resource template
* Add rops attribute to Pipeline: Dictionary containing ResourceOps
* Extend _op_to_template to produce the template for ResourceOps
* Use processed_op instead of op everywhere in _op_to_template()
* Add samples/resourceop/resourceop_basic.py
* Add tests/dsl/resource_op_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the creation of PersistentVolumeClaim instances
* Add VolumeOp: A specified ResourceOp for PVC creation
* Add samples/resourceops/volumeop_basic.py
* Add tests/dsl/volume_op_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Emit a V1Volume as `.volume` from dsl.VolumeOp
* Extend VolumeOp so it outputs a `.volume` attribute ready to be
consumed by the `pvolumes` argument to ContainerOp's constructor
* Update samples/resourceop/volumeop_basic.py
* Extend tests/dsl/volume_op_tests.py
* Update tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Add PipelineVolume
* PipelineVolume inherits from V1Volume and it comes with its own set of
KFP-specific dependencies. It is aligned with how PipelineParam
instances are used. I.e. consuming a PipelineVolume leads to implicit
dependencies without the user having to call the `.after()` method on
a ContainerOp.
* PipelineVolume comes with its own `.after()` method, which can be used
to append extra dependencies to the instance.
* Extend ContainerOp to handle PipelineVolume deps
* Set `.volume` attribute of VolumeOp to be a PipelineVolume instead
* Add samples/resourceops/volumeop_{parallel,dag,sequential}.py
* Fix tests/dsl/volume_op_tests.py
* Add tests/dsl/pipeline_volume_tests.py
* Extend tests/compiler/compiler_tests.py
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* SDK: Simplify the creation of VolumeSnapshot instances
* VolumeSnapshotOp: A specified ResourceOp for VolumeSnapshot creation
* Add samples/resourceops/volume_snapshotop_{sequential,rokurl}.py
* Add tests/dsl/volume_snapshotop_tests.py
* Extend tests/compiler/compiler_tests.py
NOTE: VolumeSnapshots is an Alpha feature at the time of this commit.
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* Extend UI for the ResourceOp and Volumes feature of the Compiler
* Add VolumeMounts tab/entry (Run/Pipeline view)
* Add Manifest tab/entry (Run/Pipeline view)
* Add & Extend tests
* Update tests snapshot files
Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* Cleaning up the diff (before moving things back)
* Renamed op.deps back to op.dependent_names
* Moved Container, Sidecar and BaseOp classed back to _container_op.py
This way the diff is much smaller and more understandable. We can always split or refactor the file later. Refactorings should not be mixed with genuine changes.
* Adds 'Create run' button to experiment list / all runs page
* Add run without experiment and filtering to FE integration test
* Update snapshots
* Add refresh and wait to integration test
* Adjust
* Adjust
* Don't exit integration test early if npm test fails
* PR comments
* TEMP - take screenshots to debug integration test
* Store screenshots
* Remove create run without experiment integration test for now as it fails due to the default experiment being deleted at the end of the API initialization and integration test suites
* Fixes deletion of recurring runs and redirects after deleting pipeline from pipeline details page
* Add comment for deletion on recurring run and pipeline details pages
* Remove refreshOnComplete flag from buttons
From the ExperimentDetails and ExperimentList pages, calls to getRun and
getExperiment are removed.
From the RunList page, calls to getRun are removed.
* Update swagger definitions
* WIP - Adds ability to terminate runs to frontend
* Update snapshots
* Adds tests. Also changes warning message color to orange rather than red
* Remove refresh button from run details page
* Elaborate terminate confirmation message
* Minor fixes
* Remove references to refresh button from integration tests
* Adds support for rendering recursive pipelines to the StaticGraphParser
* Adds a recursive pipeline to the list of mocks
* Fixes recursive graph rendering under new styling
* Adds additional comments and surfaces pipeline yaml error to users
* Remove unnecessary comment
* Updates styling for the graphs
* Break all diagonal edges into vertical and horizontal components
* Checkpointing. A lot of minor adjustments being made
* Change graph rendering to only use horizontal and vertical lines for
edges. Previously diagonal edges are converted into two vertical edges
and one horiztonal
* Small cleanup
* Hovering over node now highlights all incoming and outgoing edges, as does selecting a node
More fixes, stop stacking starting edges
* Clean up
* Remove edge starting circle code
* Returns the DAG to using arbitrary angles rather than right angles
* Adds small vertical segment to end of all edges
* Significant clean up and updating tests
* A little more clean up and adding tests for the new Status util function
* One more test for Status
* Increase node font weight
* PR comments
* A little more cleanup in Graph
* Updates Typescript to 3.3.1 and updates code around CustomTable customRenderer to make better use of FunctionComponent typings
* Remove extra afterEach from merge
* Further merge clean-up
* update ts definitions from swagger
* fix case
* Update run.go
* storageState -> storage_state
* storageState -> storage_state
* wip mock backend changes, need to use filter for storagestate
* filter_by -> filter, remove options handlers
* show only available runs, mock middleware support
* add basic Archive page with runs
* fixes, added button to sidenav
* fix generated ts filter-related changes in definitions
* storageState -> storage_state
* fix after rebase
* cleanup
* use notequals archived instead of equals available
* augment runs request filter with storage state
* fix tests
* treat available runs as unarchived
* refactor buttons to separate file
* experiment details buttons, remove test console log
* cleanup
* added actions to Buttons module
* tests
* cleanup
* add messages to dialogs
* add archive/restore button to run details
* archive breadcrumb
* mention run will not stop when archiving
* pr comments
* Improve runtime graph starting and running experience
- displays spinner when no nodes have started
- adds info message to bottom of runtime graph indicating that it is
runtime and that the graph will update over time
- adds placeholder nodes at end of graph to indicate future progress
* Update test snapshots
* Fix bug with virtual nodes, and add transition to graph
* PR comments, updating tests
* Moves IconWithTooltip to atoms/ and adds tests
* Update copyright and add further tests
* Auto-refreshes the run details page
Auto-refresh is paused when the window loses focus (blur) and is resumed
upon re-focus. Autorefresh is permanently terminated if the run has
stopped due to failure, error, being skipped, or succeeding.
* Adds tests for Status.hasCompleted
* Clean up and PR comments
* Adds run (created_at) date to tooltip for experiment last 5 runs
* Adds start and end date to tooltip for status icons in runtime graph
* Add start date to RunDetails status tooltip
* Mock formatDateString util function to avoid mismatch between local and CI test environments
* Makes KF logo a button and adds tooltips to sidenav when collapsed
* Adds build version, date, and link to side nav. Still needs tests
* Cleanup and PR comments
* Adds simple filtering by 'name' to CustomTable
* Update tests
* Adds new tests for filtering the CustomTable
* Filter using 'is_substring' rather than 'equal'
* Clean up and some comments
* Add snapshot to handleFilterChange test
* Uses 'Create' for all actions that lead to creation flow, or result in a static object (experiment). 'Start' is used solely for initiating runs
* Update integration test
* Updates material-ui and react npm libraries
* Update failing BusyButton snapshots
* Reduce material-ui version to 3.7.1 to avoid known issue with 3.8.1
* pure components, no animations for roc curve thumbnails
* fix snapshot tests
* set state once when loading parameters
* manually check plot card pros/state to make it pure
* Make all ListXXX operations use POST instead of GET.
Generate new swagger definitions and use these to generate the frontend
APIs using `npm run apis`.
This is to support filtering in List requests, as the current
grpc-gateway swagger generator tool does not support repeated fields in
requests used in GET endpoints.
* Use base64-encoded JSON-stringified version of Filter instead.
This lets us keep filter as a simple parameter in the ListXXX requests,
and gets around having to use POST for List requests.
* refactor filter parsing to parseAPIFilter and add tests
* Hack to ensure correct Swagger definitions are generated for Filter.
* Fix merge conflicts with master after rebase
* fix indentation
* Fix hack so frontend apis compile.
* print failing experiments
* try print again.
* revert experiment_api_test
* Use StdEncoding for base64 encoding
* Fix nil pointer dereference error caused err variable shadowing
* Add 'new run' button to Pipeline details page
Needs test.
* Adds tests for the NewRun page. Still needs tests for the PipelineDetails page
* Adds tests for the PipelineDetails page
* Use fromRunId as query param for embedded pipelines instead of pipelineFromRun
* Refactors some NewRun tests and separates embedded pipeline handling in NewRun
* wip, showing outputs with no labels
* showing outputs with step labels
* wip fixing tests
* fix tests
* test new functionality
* global tree
* add empty message
* pr comments
* Adds an experiment selector to the new run page. Needs tests
* Adds an experiment selector to the new run page. Needs tests
* Adds tests for the new experiment selector in NewRun
* Rename PipelineSelector -> ResourceSelector since it handles experiments as well
* Makes ResourceSelector more abstract. No longer coupled to experiments and pipelines
* PR comments, NewRun clean-up
* Moves resourceToRow function into ResourceSelector
* Fix e2e test
* show 'View pipeline' button when no pipeline id is present
* wip pipeline details to show pipeline from run spec
* pipeline details working, still need to show breadcrumbs
* breadcrumbs fixed, needs testing
* fix tests
* add more tests to cover changes
* make sure only pipeline is defined
* pr comments
This can happen whenever a user opens up a page from a URL.
A common way to hit this is to create a run from a notebook and click the link to see it in the UI.
* Changes "Hide" button on Pipeline summary to fully hide the paper card
Now, when the summary is hidden, a "Show summary" button will appear at the bottom of the graph to reopen the card.
Additionally, added a small note reading "Static pipeline graph" to the bottom of the graph
* Clean up
* Fixed compilation of dsl.Conditional
The compiler no longer produced intermediate steps.
* Got rid of _create_new_groups
* Changed the sub_group.type check
* Update frontend handling of graphs (#293)
* Updates the frontend to correctly parse the new format of conditional pipelines
* WIP - Assume tasks and templates don't share names
* Greatly simplifies graphing of conditional and non-conditional pipelines
* Adds/updates StaticParser tests
* Give nodes unique names
* refactor extractMetricMetadata into RunUtils
* hard code CREATED_AT as initial sort key
* refactor run list to reuse DisplayRun computation code, tests
* pr comments
* use setStateSafe
* Adds tests for the NewRun page
* Add more tests
* Adds many more tests, cleans-up PipelineSelector a bit, and fixes bug in Utils.ts
* Update create run button ID in e2e test
* Test clean-up and PR comments
Riley Bauer
Eran Nussbaum
Eterna2
Jiaxin Shan
Ilias Katsakioris
Alexey Volkov
IronPan
Yasser Elsayed
hongye-sun
Elvira
Ajay Gopinathan
Chen Zhiwei
Chris Van Pelt
Ajay Alfred