kubeflow
/
pipelines
mirror of https://github.com/kubeflow/pipelines.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
pipelines/components
History
Alexey Volkov cd4aa7d71f
refactor(components): De-hardcoded local output paths. (#4495) 
* Components - De-hardcoded local output paths.

* pip install pathlib2

* Added component.yaml changes
 		2021-06-08 01:13:45 -07:00
..
CatBoost Components - CatBoost - Fixed small bug in sample (#4032) 2020-06-19 11:56:43 -07:00
PyTorch feat(components): Adding Visualization component for PyTorch - KFP (#5810) 2021-06-07 21:57:53 -07:00
XGBoost chore(components): Update XGBoost predict components output type. (#5491) 2021-04-16 01:30:13 -07:00
_converters feat(components): Converters - OnnxModel - to_TensorflowSavedModel (#5325) 2021-03-30 02:56:23 -07:00
arena chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
aws chore(components): AWS SageMaker update eks cluster version for tests (#5735) 2021-05-25 19:53:40 -07:00
azure <feat>[components]: Add pipeline component to schedule spark jobs in Azure Synapse (#5070) 2021-02-25 19:05:15 -08:00
basics/Calculate_hash feat(components): Added the Basics - Calculate data hash component (#4503) 2020-09-21 03:28:31 -07:00
dataset_manipulation/split_data_into_folds/in_CSV components: Added dataset_manipulation - split_data_into_folds component (#4200) 2020-08-11 02:12:17 -07:00
datasets/Chicago_Taxi_Trips feat(components): Made the Chicago Taxi Trips dataset component compatible with volume-based data passing (#5216) 2021-03-13 23:20:47 -08:00
deprecated refactor(components): De-hardcoded local output paths. (#4495) 2021-06-08 01:13:45 -07:00
diagnostics/diagnose_me Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
filesystem Components - Filesystem (#2659) 2019-11-27 11:43:03 -08:00
gcp chore(release): bumped version to 1.6.0 2021-05-24 13:36:51 +08:00
git/clone feat(sdk): DSL - Added support for volatile components (#4104) 2020-07-06 18:09:57 -07:00
google-cloud fix(components/google-cloud): update version to include google-cloud-aiplatform fix (#5721) 2021-05-27 11:16:59 -07:00
great-expectations/validate chore(components): Fix typos in Great Expectations component (#5326) 2021-04-03 19:58:21 -07:00
ibm-components chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
json feat(components): JSON - Added the "Build dict", "Build list" and "Combine lists" components (#4631) 2020-10-26 14:01:59 -07:00
keras/Train_classifier feat(components): Added the component to convert from KerasModelHdf5 to TensorflowSavedModel (#4524) 2020-09-21 02:40:41 -07:00
kubeflow refactor(components): De-hardcoded local output paths. (#4495) 2021-06-08 01:13:45 -07:00
kubernetes feat(components): Added Kubernetes CRUD operations (#5239) 2021-03-12 17:50:46 -08:00
local refactor(components): De-hardcoded local output paths. (#4495) 2021-06-08 01:13:45 -07:00
ml_metrics feat(components): Added Calculate classification metrics component (#4441) 2020-09-20 21:48:30 -07:00
notebooks feat(components): Added the Run notebook using papermill component. Fixes #497 (#4578) 2020-10-12 03:26:48 -07:00
nuclio add nuclio components (to build/deploy, delete, invoke functions) (#1295) 2019-05-08 01:58:33 -07:00
pandas/Transform_DataFrame Components - pandas - Transform DataFrame - Fixed transforms that replace dataframe (#4042) 2020-06-22 22:09:57 -07:00
presto/query chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
sample Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
tables/Remove_header Components - tables - Remove_header (#4044) 2020-07-11 18:31:21 -07:00
tensorflow/tensorboard/prepare_tensorboard feat: customizable tensorboard image and env + sample pipeline with minio. Fixes #5471, fixes #4714, fixes #5449, fixes #699, fixes #4290 (#5515) 2021-05-11 00:45:42 -07:00
web/Download fix(components): Web - Download data - Fixed the image which does not support volumes (#5644) 2021-05-19 20:30:46 -07:00
OWNERS chore: remove inactive reviewers (#4111) 2020-06-30 19:10:06 -07:00
README.md chore(release): bumped version to 1.6.0 2021-05-24 13:36:51 +08:00
build_image.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
generate_component_index.py doc(components): Added component index in the readme (#4762) 2020-11-12 13:46:25 -08:00
pipeline_component_repository.yaml Components - Added the pipeline component repository marker file (#3883) 2020-06-01 03:08:10 -07:00
release-in-place.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
release.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
test_load_all_components.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00

README.md

Components for Kubeflow Pipelines

Kubeflow Pipelines system orchestrates pipeline graphs of containerized command-line programs. Components are the building blocks of pipelines. Component definition files describe component interface (inputs and outputs) and implementation (how to call the containerized program). Users can load components with KFP SDK, instantiate the components and compose them in a pipeline graph.

Example components

See how to build your own components. Also see the tutorials for data passing for components based on python functions and components based on command-line programs.

Index of components

/ _converters / ApacheParquet / from_ApacheArrowFeather / Convert apache arrow feather to apache parquet

/ _converters / ApacheParquet / from_CSV / Convert csv to apache parquet

/ _converters / ApacheParquet / from_TSV / Convert tsv to apache parquet

/ _converters / ApacheParquet / to_ApacheArrowFeather / Convert apache parquet to apache arrow feather

/ _converters / ApacheParquet / to_CSV / Convert apache parquet to csv

/ _converters / ApacheParquet / to_TSV / Convert apache parquet to tsv

/ _converters / KerasModelHdf5 / to_TensorflowSavedModel / Keras convert hdf5 model to tf saved model

/ _converters / OnnxModel / from_KerasModelHdf5 / To ONNX from Keras HDF5 model

/ _converters / OnnxModel / from_TensorflowSavedModel / To ONNX from Tensorflow SavedModel

/ _converters / TensorflowJSGraphModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS GraphModel

/ _converters / TensorflowJSGraphModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow JS GraphModel

/ _converters / TensorflowJSLayersModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS LayersModel

/ _converters / TensorflowJSLayersModel / from_TensorflowSavedModel / Convert Keras SavedModel to Tensorflow JS LayersModel

/ _converters / TensorflowLiteModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow Lite model

/ _converters / TensorflowLiteModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow Lite model

/ aws / athena / query / Athena Query

/ aws / emr / create_cluster / emr_create_cluster

/ aws / emr / delete_cluster / emr_delete_cluster

/ aws / emr / submit_pyspark_job / emr_submit_pyspark_job

/ aws / emr / submit_spark_job / emr_submit_spark_job

/ aws / sagemaker / batch_transform / SageMaker - Batch Transformation

/ aws / sagemaker / create_simulation_app / RoboMaker - Create Simulation Application

/ aws / sagemaker / delete_simulation_app / RoboMaker - Delete Simulation Application

/ aws / sagemaker / deploy / SageMaker - Deploy Model

/ aws / sagemaker / ground_truth / SageMaker - Ground Truth

/ aws / sagemaker / hyperparameter_tuning / SageMaker - Hyperparameter Tuning

/ aws / sagemaker / model / SageMaker - Create Model

/ aws / sagemaker / process / SageMaker - Processing Job

/ aws / sagemaker / rlestimator / SageMaker - RLEstimator Training Job

/ aws / sagemaker / simulation_job / RoboMaker - Create Simulation Job

/ aws / sagemaker / simulation_job_batch / RoboMaker - Create Simulation Job Batch

/ aws / sagemaker / train / SageMaker - Training Job

/ aws / sagemaker / workteam / SageMaker - Private Workforce

/ azure / azuredevops / queue-pipeline / Queue Azure Pipeline

/ azure / azureml / aml-deploy-model / Azure ML Deploy Model

/ azure / azureml / aml-register-model / Azure ML Register Model

/ azure / azuresynapse / runsparkjob / Azure Synapse Run Spark Job

/ basics / Calculate_hash / Calculate data hash

/ CatBoost / convert_CatBoostModel_to_AppleCoreMLModel / Convert CatBoostModel to AppleCoreMLModel

/ CatBoost / convert_CatBoostModel_to_ONNX / Convert CatBoostModel to ONNX

/ CatBoost / Predict_class_probabilities / from_CSV / Catboost predict class probabilities

/ CatBoost / Predict_classes / from_CSV / Catboost predict classes

/ CatBoost / Predict_values / from_CSV / Catboost predict values

/ CatBoost / Train_classifier / from_CSV / Catboost train classifier

/ CatBoost / Train_regression / from_CSV / Catboost train regression

/ dataset_manipulation / split_data_into_folds / in_CSV / Split table into folds

/ datasets / Chicago_Taxi_Trips / Chicago Taxi Trips dataset

/ deprecated / dataflow / predict / Predict using TF on Dataflow

/ deprecated / dataflow / tfdv / TFX - Data Validation

/ deprecated / dataflow / tfma / TFX - Analyze model

/ deprecated / dataflow / tft / Transform using TF on Dataflow

/ deprecated / tfx / Evaluator / Evaluator

/ deprecated / tfx / Evaluator / with_URI_IO / Evaluator

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / with_URI_IO / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / CsvExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / with_URI_IO / CsvExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / ImportExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / with_URI_IO / ImportExampleGen

/ deprecated / tfx / ExampleValidator / ExampleValidator

/ deprecated / tfx / ExampleValidator / with_URI_IO / ExampleValidator

/ deprecated / tfx / SchemaGen / SchemaGen

/ deprecated / tfx / SchemaGen / with_URI_IO / SchemaGen

/ deprecated / tfx / StatisticsGen / StatisticsGen

/ deprecated / tfx / StatisticsGen / with_URI_IO / StatisticsGen

/ deprecated / tfx / Trainer / Trainer

/ deprecated / tfx / Trainer / with_URI_IO / Trainer

/ deprecated / tfx / Transform / Transform

/ deprecated / tfx / Transform / with_URI_IO / Transform

/ diagnostics / diagnose_me / Run diagnose me

/ filesystem / get_file / Get file

/ filesystem / get_subdirectory / Get subdirectory

/ filesystem / list_items / List items

/ gcp / automl / create_dataset_for_tables / Automl create dataset for tables

/ gcp / automl / create_model_for_tables / Automl create model for tables

/ gcp / automl / deploy_model / Automl deploy model

/ gcp / automl / export_data_to_gcs / Automl export data to gcs

/ gcp / automl / export_model_to_gcs / Automl export model to gcs

/ gcp / automl / import_data_from_bigquery / Automl import data from bigquery

/ gcp / automl / import_data_from_gcs / Automl import data from gcs

/ gcp / automl / prediction_service_batch_predict / Automl prediction service batch predict

/ gcp / automl / split_dataset_table_column_names / Automl split dataset table column names

/ gcp / bigquery / query / to_CSV / Bigquery - Query

/ gcp / bigquery / query / to_gcs / Bigquery - Query

/ gcp / bigquery / query / to_table / Bigquery - Query

/ gcp / dataflow / launch_python / Launch Python

/ gcp / dataflow / launch_template / Launch Dataflow Template

/ gcp / dataproc / create_cluster / dataproc_create_cluster

/ gcp / dataproc / delete_cluster / dataproc_delete_cluster

/ gcp / dataproc / submit_hadoop_job / dataproc_submit_hadoop_job

/ gcp / dataproc / submit_hive_job / dataproc_submit_hive_job

/ gcp / dataproc / submit_pig_job / dataproc_submit_pig_job

/ gcp / dataproc / submit_pyspark_job / dataproc_submit_pyspark_job

/ gcp / dataproc / submit_spark_job / dataproc_submit_spark_job

/ gcp / dataproc / submit_sparksql_job / dataproc_submit_sparksql_job

/ gcp / ml_engine / batch_predict / Batch predict against a model with Cloud ML Engine

/ gcp / ml_engine / deploy / Deploying a trained model to Cloud Machine Learning Engine

/ gcp / ml_engine / train / Submitting a Cloud ML training job as a pipeline step

/ git / clone / Git clone

/ google-cloud / AutoML / Tables / Create_dataset / from_CSV / Automl create tables dataset from csv

/ google-cloud / Optimizer / Add_measurement_for_trial / Add measurement for trial in gcp ai platform optimizer

/ google-cloud / Optimizer / Create_study / Create study in gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_parameter_sets_based_on_measurements / Suggest parameter sets from measurements using gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_trials / Suggest trials in gcp ai platform optimizer

/ google-cloud / storage / download / Download from GCS

/ google-cloud / storage / download_blob / Download from GCS

/ google-cloud / storage / download_dir / Download from GCS

/ google-cloud / storage / list / List blobs

/ google-cloud / storage / upload_to_explicit_uri / Upload to GCS

/ google-cloud / storage / upload_to_unique_uri / Upload to GCS

/ ibm-components / commons / config / Create Secret - Kubernetes Cluster

/ ibm-components / ffdl / serve / Serve PyTorch Model - Seldon Core

/ ibm-components / ffdl / train / Train Model - FfDL

/ ibm-components / spark / data_preprocess_spark / Preprocess Data using Spark - IBM Cloud

/ ibm-components / spark / store_spark_model / Store Spark Model - Watson Machine Learning

/ ibm-components / spark / train_spark / Train Spark Model - IBM Cloud

/ ibm-components / watson / deploy / Deploy Model - Watson Machine Learning

/ ibm-components / watson / manage / monitor_fairness / Monitor Fairness - Watson OpenScale

/ ibm-components / watson / manage / monitor_quality / Monitor quality - Watson OpenScale

/ ibm-components / watson / manage / subscribe / Subscribe - Watson OpenScale

/ ibm-components / watson / store / Store model - Watson Machine Learning

/ ibm-components / watson / train / Train Model - Watson Machine Learning

/ json / Build_dict / Build dict

/ json / Build_list / Build list

/ json / Combine_lists / Combine lists

/ json / Get_element_by_index / Get element by index from JSON

/ json / Get_element_by_key / Get element by key from JSON

/ json / Query / Query JSON using JQ

/ keras / Train_classifier / from_CSV / Keras train classifier from csv

/ kubeflow / deployer / Kubeflow - Serve TF model

/ kubeflow / dnntrainer / Train FC DNN using TF

/ kubeflow / katib-launcher / Katib - Launch Experiment

/ kubeflow / kfserving / Kubeflow - Serve Model using KFServing

/ kubeflow / launcher / Kubeflow - Launch TFJob

/ local / confusion_matrix / Confusion matrix

/ local / roc / ROC curve

/ ml_metrics / Aggregate_regression_metrics / Aggregate regression metrics

/ ml_metrics / Calculate_classification_metrics / from_CSV / Calculate classification metrics from csv

/ ml_metrics / Calculate_regression_metrics / from_CSV / Calculate regression metrics from csv

/ notebooks / Run_notebook_using_papermill / Run notebook using papermill

/ nuclio / delete / nuclio delete

/ nuclio / deploy / nuclio deploy

/ nuclio / invoker / nuclio invoker

/ pandas / Transform_DataFrame / in_ApacheParquet_format / Pandas Transform DataFrame in ApacheParquet format

/ pandas / Transform_DataFrame / in_CSV_format / Pandas Transform DataFrame in CSV format

/ presto / query / Presto Query

/ sample / C%23_script / Filter text

/ sample / keras / train_classifier / Keras - Train classifier

/ sample / R_script / Filter text

/ sample / Shell_script / Filter text using shell and grep

/ tables / Remove_header / Remove header

/ tensorflow / tensorboard / prepare_tensorboard / Create Tensorboard visualization

/ web / Download / Download data

/ XGBoost / Cross_validation_for_regression / from_CSV / Xgboost 5 fold cross validation for regression

/ XGBoost / Predict / Xgboost predict

/ XGBoost / Predict / from_ApacheParquet / Xgboost predict

/ XGBoost / Train / Xgboost train

/ XGBoost / Train / from_ApacheParquet / Xgboost train

/ XGBoost / Train_and_cross-validate_regression / from_CSV / Xgboost train and cv regression on csv

/ XGBoost / Train_regression_and_calculate_metrics / from_CSV / Xgboost train regression and calculate metrics on csv