kubeflow
/
pipelines
mirror of https://github.com/kubeflow/pipelines.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
pipelines/components
History
sina chavoshi ae9956ce32
chore(components/google-cloud): Post release v0.1.4 clean up (#6275) 
* google-cloud-pipeline-components-0.1.4

* update release docs

* Update version for custom_job docker

* fixing module path

* temporary fixed for 0.1.4 released module

* add init for remote dir

* post release clean up

* point custom job to latest

* revert docker tag version
 		2021-08-09 23:30:24 -07:00
..
CatBoost chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
PyTorch fix(components/pytorch): PyTorch Samples - Generating component.yaml using templates (#6231) 2021-08-08 23:34:25 -07:00
XGBoost chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
_converters chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
arena chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
aws chore: Updates argoproj/argo URLs to argoproj/argo-workflows (#5969) 2021-07-06 21:52:20 -07:00
azure <feat>[components]: Add pipeline component to schedule spark jobs in Azure Synapse (#5070) 2021-02-25 19:05:15 -08:00
basics/Calculate_hash chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
dataset_manipulation/split_data_into_folds/in_CSV chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
datasets chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
deprecated chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
diagnostics/diagnose_me Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
filesystem chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
gcp chore(release): bumped version to 1.7.0-rc.3 2021-08-06 07:13:16 +00:00
git/clone chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
google-cloud chore(components/google-cloud): Post release v0.1.4 clean up (#6275) 2021-08-09 23:30:24 -07:00
great-expectations/validate chore(components): Fix typos in Great Expectations component (#5326) 2021-04-03 19:58:21 -07:00
ibm-components chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
json feat(components/pytorch): Example to estimate best parameters using AX (#6041) 2021-07-16 04:43:38 -07:00
keras/Train_classifier chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
kfp/Run_component chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
kubeflow chore(release): bumped version to 1.7.0-rc.3 2021-08-06 07:13:16 +00:00
kubernetes chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
local chore(release): bumped version to 1.7.0-rc.3 2021-08-06 07:13:16 +00:00
ml_metrics chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
notebooks chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
nuclio add nuclio components (to build/deploy, delete, invoke functions) (#1295) 2019-05-08 01:58:33 -07:00
pandas/Transform_DataFrame chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
presto/query chore: Remove pip requirements from docker files (#5064) 2021-02-26 02:31:15 -08:00
sample chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
tables/Remove_header chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
tensorflow/tensorboard/prepare_tensorboard chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
web/Download chore(components): Added canonical location for some components (#6060) 2021-07-15 21:28:12 -07:00
OWNERS chore: update component owner file (#6239) 2021-08-05 08:43:18 -07:00
README.md chore(release): bumped version to 1.7.0-rc.3 2021-08-06 07:13:16 +00:00
build_image.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
generate_component_index.py doc(components): Added component index in the readme (#4762) 2020-11-12 13:46:25 -08:00
pipeline_component_repository.yaml Components - Added the pipeline component repository marker file (#3883) 2020-06-01 03:08:10 -07:00
release-in-place.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
release.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00
test_load_all_components.sh Assigned copyright to the project authors (#5587) 2021-05-05 13:53:22 +08:00

README.md

Components for Kubeflow Pipelines

Kubeflow Pipelines system orchestrates pipeline graphs of containerized command-line programs. Components are the building blocks of pipelines. Component definition files describe component interface (inputs and outputs) and implementation (how to call the containerized program). Users can load components with KFP SDK, instantiate the components and compose them in a pipeline graph.

Example components

See how to build your own components. Also see the tutorials for data passing for components based on python functions and components based on command-line programs.

Index of components

/ _converters / ApacheParquet / from_ApacheArrowFeather / Convert apache arrow feather to apache parquet

/ _converters / ApacheParquet / from_CSV / Convert csv to apache parquet

/ _converters / ApacheParquet / from_TSV / Convert tsv to apache parquet

/ _converters / ApacheParquet / to_ApacheArrowFeather / Convert apache parquet to apache arrow feather

/ _converters / ApacheParquet / to_CSV / Convert apache parquet to csv

/ _converters / ApacheParquet / to_TSV / Convert apache parquet to tsv

/ _converters / KerasModelHdf5 / to_TensorflowSavedModel / Keras convert hdf5 model to tf saved model

/ _converters / OnnxModel / from_KerasModelHdf5 / To ONNX from Keras HDF5 model

/ _converters / OnnxModel / from_TensorflowSavedModel / To ONNX from Tensorflow SavedModel

/ _converters / TensorflowJSGraphModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS GraphModel

/ _converters / TensorflowJSGraphModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow JS GraphModel

/ _converters / TensorflowJSLayersModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow JS LayersModel

/ _converters / TensorflowJSLayersModel / from_TensorflowSavedModel / Convert Keras SavedModel to Tensorflow JS LayersModel

/ _converters / TensorflowLiteModel / from_KerasModelHdf5 / Convert Keras HDF5 model to Tensorflow Lite model

/ _converters / TensorflowLiteModel / from_TensorflowSavedModel / Convert Tensorflow SavedModel to Tensorflow Lite model

/ aws / athena / query / Athena Query

/ aws / emr / create_cluster / emr_create_cluster

/ aws / emr / delete_cluster / emr_delete_cluster

/ aws / emr / submit_pyspark_job / emr_submit_pyspark_job

/ aws / emr / submit_spark_job / emr_submit_spark_job

/ aws / sagemaker / batch_transform / SageMaker - Batch Transformation

/ aws / sagemaker / create_simulation_app / RoboMaker - Create Simulation Application

/ aws / sagemaker / delete_simulation_app / RoboMaker - Delete Simulation Application

/ aws / sagemaker / deploy / SageMaker - Deploy Model

/ aws / sagemaker / ground_truth / SageMaker - Ground Truth

/ aws / sagemaker / hyperparameter_tuning / SageMaker - Hyperparameter Tuning

/ aws / sagemaker / model / SageMaker - Create Model

/ aws / sagemaker / process / SageMaker - Processing Job

/ aws / sagemaker / rlestimator / SageMaker - RLEstimator Training Job

/ aws / sagemaker / simulation_job / RoboMaker - Create Simulation Job

/ aws / sagemaker / simulation_job_batch / RoboMaker - Create Simulation Job Batch

/ aws / sagemaker / train / SageMaker - Training Job

/ aws / sagemaker / workteam / SageMaker - Private Workforce

/ azure / azuredevops / queue-pipeline / Queue Azure Pipeline

/ azure / azureml / aml-deploy-model / Azure ML Deploy Model

/ azure / azureml / aml-register-model / Azure ML Register Model

/ azure / azuresynapse / runsparkjob / Azure Synapse Run Spark Job

/ basics / Calculate_hash / Calculate data hash

/ CatBoost / convert_CatBoostModel_to_AppleCoreMLModel / Convert CatBoostModel to AppleCoreMLModel

/ CatBoost / convert_CatBoostModel_to_ONNX / Convert CatBoostModel to ONNX

/ CatBoost / Predict_class_probabilities / from_CSV / Catboost predict class probabilities

/ CatBoost / Predict_classes / from_CSV / Catboost predict classes

/ CatBoost / Predict_values / from_CSV / Catboost predict values

/ CatBoost / Train_classifier / from_CSV / Catboost train classifier

/ CatBoost / Train_regression / from_CSV / Catboost train regression

/ dataset_manipulation / split_data_into_folds / in_CSV / Split table into folds

/ datasets / Chicago_Taxi_Trips / Chicago Taxi Trips dataset

/ deprecated / dataflow / predict / Predict using TF on Dataflow

/ deprecated / dataflow / tfdv / TFX - Data Validation

/ deprecated / dataflow / tfma / TFX - Analyze model

/ deprecated / dataflow / tft / Transform using TF on Dataflow

/ deprecated / tfx / Evaluator / Evaluator

/ deprecated / tfx / Evaluator / with_URI_IO / Evaluator

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / BigQueryExampleGen / with_URI_IO / BigQueryExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / CsvExampleGen

/ deprecated / tfx / ExampleGen / CsvExampleGen / with_URI_IO / CsvExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / ImportExampleGen

/ deprecated / tfx / ExampleGen / ImportExampleGen / with_URI_IO / ImportExampleGen

/ deprecated / tfx / ExampleValidator / ExampleValidator

/ deprecated / tfx / ExampleValidator / with_URI_IO / ExampleValidator

/ deprecated / tfx / SchemaGen / SchemaGen

/ deprecated / tfx / SchemaGen / with_URI_IO / SchemaGen

/ deprecated / tfx / StatisticsGen / StatisticsGen

/ deprecated / tfx / StatisticsGen / with_URI_IO / StatisticsGen

/ deprecated / tfx / Trainer / Trainer

/ deprecated / tfx / Trainer / with_URI_IO / Trainer

/ deprecated / tfx / Transform / Transform

/ deprecated / tfx / Transform / with_URI_IO / Transform

/ diagnostics / diagnose_me / Run diagnose me

/ filesystem / get_file / Get file

/ filesystem / get_subdirectory / Get subdirectory

/ filesystem / list_items / List items

/ gcp / automl / create_dataset_for_tables / Automl create dataset for tables

/ gcp / automl / create_model_for_tables / Automl create model for tables

/ gcp / automl / deploy_model / Automl deploy model

/ gcp / automl / export_data_to_gcs / Automl export data to gcs

/ gcp / automl / export_model_to_gcs / Automl export model to gcs

/ gcp / automl / import_data_from_bigquery / Automl import data from bigquery

/ gcp / automl / import_data_from_gcs / Automl import data from gcs

/ gcp / automl / prediction_service_batch_predict / Automl prediction service batch predict

/ gcp / automl / split_dataset_table_column_names / Automl split dataset table column names

/ gcp / bigquery / query / to_CSV / Bigquery - Query

/ gcp / bigquery / query / to_gcs / Bigquery - Query

/ gcp / bigquery / query / to_table / Bigquery - Query

/ gcp / dataflow / launch_python / Launch Python

/ gcp / dataflow / launch_template / Launch Dataflow Template

/ gcp / dataproc / create_cluster / dataproc_create_cluster

/ gcp / dataproc / delete_cluster / dataproc_delete_cluster

/ gcp / dataproc / submit_hadoop_job / dataproc_submit_hadoop_job

/ gcp / dataproc / submit_hive_job / dataproc_submit_hive_job

/ gcp / dataproc / submit_pig_job / dataproc_submit_pig_job

/ gcp / dataproc / submit_pyspark_job / dataproc_submit_pyspark_job

/ gcp / dataproc / submit_spark_job / dataproc_submit_spark_job

/ gcp / dataproc / submit_sparksql_job / dataproc_submit_sparksql_job

/ gcp / ml_engine / batch_predict / Batch predict against a model with Cloud ML Engine

/ gcp / ml_engine / deploy / Deploying a trained model to Cloud Machine Learning Engine

/ gcp / ml_engine / train / Submitting a Cloud ML training job as a pipeline step

/ git / clone / Git clone

/ google-cloud / AutoML / Tables / Create_dataset / from_CSV / Automl create tables dataset from csv

/ google-cloud / Optimizer / Add_measurement_for_trial / Add measurement for trial in gcp ai platform optimizer

/ google-cloud / Optimizer / Create_study / Create study in gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_parameter_sets_based_on_measurements / Suggest parameter sets from measurements using gcp ai platform optimizer

/ google-cloud / Optimizer / Suggest_trials / Suggest trials in gcp ai platform optimizer

/ google-cloud / storage / download / Download from GCS

/ google-cloud / storage / download_blob / Download from GCS

/ google-cloud / storage / download_dir / Download from GCS

/ google-cloud / storage / list / List blobs

/ google-cloud / storage / upload_to_explicit_uri / Upload to GCS

/ google-cloud / storage / upload_to_unique_uri / Upload to GCS

/ ibm-components / commons / config / Create Secret - Kubernetes Cluster

/ ibm-components / ffdl / serve / Serve PyTorch Model - Seldon Core

/ ibm-components / ffdl / train / Train Model - FfDL

/ ibm-components / spark / data_preprocess_spark / Preprocess Data using Spark - IBM Cloud

/ ibm-components / spark / store_spark_model / Store Spark Model - Watson Machine Learning

/ ibm-components / spark / train_spark / Train Spark Model - IBM Cloud

/ ibm-components / watson / deploy / Deploy Model - Watson Machine Learning

/ ibm-components / watson / manage / monitor_fairness / Monitor Fairness - Watson OpenScale

/ ibm-components / watson / manage / monitor_quality / Monitor quality - Watson OpenScale

/ ibm-components / watson / manage / subscribe / Subscribe - Watson OpenScale

/ ibm-components / watson / store / Store model - Watson Machine Learning

/ ibm-components / watson / train / Train Model - Watson Machine Learning

/ json / Build_dict / Build dict

/ json / Build_list / Build list

/ json / Combine_lists / Combine lists

/ json / Get_element_by_index / Get element by index from JSON

/ json / Get_element_by_key / Get element by key from JSON

/ json / Query / Query JSON using JQ

/ keras / Train_classifier / from_CSV / Keras train classifier from csv

/ kubeflow / deployer / Kubeflow - Serve TF model

/ kubeflow / dnntrainer / Train FC DNN using TF

/ kubeflow / katib-launcher / Katib - Launch Experiment

/ kubeflow / kfserving / Kubeflow - Serve Model using KFServing

/ kubeflow / launcher / Kubeflow - Launch TFJob

/ local / confusion_matrix / Confusion matrix

/ local / roc / ROC curve

/ ml_metrics / Aggregate_regression_metrics / Aggregate regression metrics

/ ml_metrics / Calculate_classification_metrics / from_CSV / Calculate classification metrics from csv

/ ml_metrics / Calculate_regression_metrics / from_CSV / Calculate regression metrics from csv

/ notebooks / Run_notebook_using_papermill / Run notebook using papermill

/ nuclio / delete / nuclio delete

/ nuclio / deploy / nuclio deploy

/ nuclio / invoker / nuclio invoker

/ pandas / Transform_DataFrame / in_ApacheParquet_format / Pandas Transform DataFrame in ApacheParquet format

/ pandas / Transform_DataFrame / in_CSV_format / Pandas Transform DataFrame in CSV format

/ presto / query / Presto Query

/ sample / C%23_script / Filter text

/ sample / keras / train_classifier / Keras - Train classifier

/ sample / R_script / Filter text

/ sample / Shell_script / Filter text using shell and grep

/ tables / Remove_header / Remove header

/ tensorflow / tensorboard / prepare_tensorboard / Create Tensorboard visualization

/ web / Download / Download data

/ XGBoost / Cross_validation_for_regression / from_CSV / Xgboost 5 fold cross validation for regression

/ XGBoost / Predict / Xgboost predict

/ XGBoost / Predict / from_ApacheParquet / Xgboost predict

/ XGBoost / Train / Xgboost train

/ XGBoost / Train / from_ApacheParquet / Xgboost train

/ XGBoost / Train_and_cross-validate_regression / from_CSV / Xgboost train and cv regression on csv

/ XGBoost / Train_regression_and_calculate_metrics / from_CSV / Xgboost train regression and calculate metrics on csv