Commit Graph

16 Commits

Author SHA1 Message Date
ananth102 09d5151542
chore(components): Update Sagemaker sample to use v2 Hosting components (#9398)
* Update mnist sample with Hosting

* updated sample with new directory

* Updating sample to use output rather than after

* Address comments
2023-05-12 22:27:28 +00:00
ryansteakley 4c097171c1
chore(components): Add sample for blended pipeline using v1 and v2 AWS Sagemaker components (#8284)
* blended pipeline

* add sample to samples folder

* update pinned version of TrainingJob

* update samples instructions

* update bucket and sagemaker role sections

* update broken link and fix indentation:
2022-11-01 23:55:45 +00:00
ryansteakley cc8e3fdec1
chore: add missing imports for aws samples data scripts (#7877)
* update source for input data

* add imports for samples

* use aws legal approved s3 buckets

* change bucket in pipeline input
2022-07-07 22:33:51 +00:00
ryansteakley 77ab758a4e
chore: update source for aws samples input data (#7772) 2022-06-07 17:04:30 +00:00
Nicholas Thomson d81c8095d0
refactor(components): AWS SageMaker - Full component refactoring (#4336)
* Temporary rebase commit

* Add yaml compiler

* Add compiler CLI

* Update Dockerfile to copy all files

* Add validate input list vs dict

* Add unit test for new train

* Add minor bug fixes

* Override tag when generating specs

* Update pydocs with formatter

* Add contributing doc

* Add formatters to CONTRIBUTING

* Add working generic logic applied to train

* Update component input and output to inherit

* Downgrade to Python 3.7

* Update add outputValue to arg list

* Updated outputValue to outputPath

* Add empty string default to not-required inputs

* Update path to component relative to root

* Update faulty False-y condition

* Update outputs to write to file

* Update doc formatting

* Update docstrings to match structure

* Add unit tests for component and compiler

* Add unit tests for component

* Add spec unit tests

* Add training unit tests

* Update unit test automation

* Add sample formatting checks

* Remove extra flake8 check in integ tests

* Add unit test black check

* Update black formatting for all files

* Update include black formatting

* Add batch component

* Remove old transform components

* Update region input description

* Add all component specs

* Add deploy component

* Add ground truth component

* Add HPO component

* Add create model component

* Add processing component

* Add workteam component

* Add spec unit tests

* Add deploy unit tests

* Add ground truth unit tests

* Add tuning component unit tests

* Add create model component unit test

* Add process component unit tests

* Add workteam component unit tests

* Remove output_path from required_args

* Remove old component implementations

* Update black formatting

* Add assume role feature

* Compiled all components

* Update doc formatting

* Fix process terminate syntax error

* Update compiler to use kfp structures

* Update nits

* Update unified requirements

* Rebase on debugging commit

* Add debugger unit tests

* Update formatting

* Update component YAML

* Fix unit test Dockerfile relative directory

* Update unit test context to root

* Update Batch to auto-generate name

* Update minor docs and formatting changes

* Update deploy name to common autogenerated

* Add f-strings to logs

* Add update support

* Add Amazon license header

* Update autogen and autoformat

* Rename SpecValidator to SpecInputParser

* Split requirements by dev and prod

* Support for checking generated specs

* Update minor changes

* Update deploy component output description

* Update components to beta repository

* Update fix unit test requirements

* Update unit test build spec for new results path

* Update deploy wait for endpoint complete

* Update component configure AWS clients in new method

* Update boto3 retry method

* Update license version

* Update component YAML versions

* Add new version to Changelog

* Update component spec types

* Update deploy config ignore overwrite

* Update component for debugging

* Update images back to 1.0.0

* Remove coverage from components
2020-10-27 14:17:57 -07:00
Suraj Kota 6860681122
feat(components): [AWS SageMaker] Minimize inputs for mnist classification pipeline (#4192)
* minimize parameters and bug fixes

* endpoint_name variable fix

* only add necessary inputs

* remove run_time

* trim down v2

* Update example with S3 bucket as runtime input

* nit fix

* fix variable names
2020-07-17 13:30:50 -07:00
Nicholas Thomson 7619a01546
[AWS SageMaker] Processing job sample (#4009)
* Added preprocessing step to sample pipeline

* Add processing to components README

* Remove unused import

* Add data set prerequisite to training component

* Remove simple HPO pipeline

* Update MNIST header in README

* Remove simple HPO sample integration test

* Empty commit to trigger google bot
2020-06-30 16:28:06 -07:00
Suraj Kota bc6aed9cac
[AWS Sagemaker] aws-samples kmeans-hpo pipeline test (#3905)
* aws-samples kmeans-hpo pipeline test
	- code clean up
	- removed unused args

* add test dependency

* Trigger Build
2020-06-16 10:26:04 -07:00
Kartik Kalamadi d18ad7a563
AWS SageMaker : Use IAM Roles for Service Account (#3719)
* don't use aws-secret and update readme for sample pipelines

* Addressed comments on PR and few more readme changes

* small changes to readme

* nit change

* Address comments
2020-05-21 10:36:14 -07:00
Nicholas Thomson bd8c1ddd38
[AWS SageMaker] Specify component input types (#3683)
* Replace all string types with Python types

* Update HPO yaml

* Update Batch YAML

* Update Deploy YAML

* Update GroundTruth YAML

* Update Model YAML

* Update Train YAML

* Update WorkTeam YAML

* Updated samples to remove strings

* Update to temporary image

* Remove unnecessary imports

* Update image to newer image

* Update components to python3

* Update bool parser type

* Remove empty ContentType in samples

* Update to temporary image

* Update to version 0.3.1

* Update deploy to login

* Update deploy load config path

* Fix export environment variable in deploy

* Fix env name

* Update deploy reflow env paths

* Add debug config line

* Use username and password directly

* Updated to 0.3.1

* Update field types to JsonObject and JsonArray
2020-05-11 22:06:21 -07:00
Kartik Kalamadi 0259fe50b3
AWS Sagemaker : Use json.dumps() to better organize the input and remove data_locations (#3518)
* construct channel input using json.dumps()

* remover data_location parameters

* add changelog

* Update version in license file and small changes to readme
2020-04-23 12:14:07 -07:00
Kartik Kalamadi 060cabf911
AWS Sagemaker : Updated documents (#3440)
* Initial readme for Train component

* example input

* add train pipeline

* added simple_train_pipeline

* Updated readme to include kmeans-hpo-pipeline.py

* Updated train component readme

* fix typo

* Update details about how to get sample data for Train component

* update comment and give a defaault path for output

* change s3 bucket to match other sample pipelines
2020-04-07 11:17:44 -07:00
Redback 2fe8c0de61 [Component] Add VPC Interface Endpoint Support for SageMaker (#2299)
* Added Private Link Components

* Updated Component Dockerfile

* Added endpoint_url to Samples
2019-10-03 18:11:56 -07:00
Redback 12dde375b8 [Component] Add Managed Spot Training Support for SageMaker (#2219)
* Added Spot Instance Support

* Fixed missing output configuration

* Added spot instance support to example pipelines

* Updated image to new repository
2019-10-03 12:11:56 -07:00
carolynwang 351f4562a4 Refactor to match new samples folder structure (#1741) 2019-08-06 01:23:56 -07:00
Ning e8a6feb229
Restructure samples (#1710)
* restructure samples
* update apiserver and sample test for the new location
2019-08-01 17:31:37 -07:00