Commit Graph

19 Commits

Author SHA1 Message Date
Kartik Kalamadi b3d8e04e1e
[AWS SageMaker] Print SageMaker job logs in kfp UI (#3954)
* Print logs for AWS SM Componenets on KFP UI

* address comments

* update version number to 0.5.0

* update yaml to version 0.5.0

* update changelog
2020-06-19 00:33:58 -07:00
Nicholas Thomson bea63652e1
[AWS SageMaker] Processing job component (#3944)
* Add TDD processing definition

* Update README

* Update temporary image

* Update component entrypoint

* Add WORKDIR to fix Docker 18 support

* integration test for processing job

* Remove job links

* Add container outputs and tests

* Update default properties

* Remove max_run_time if none provided

* Update integration readme steps

* Updated README with more resources

* Add CloudWatch link back to logs

* Update input and output config to arrays

* Update processing integration test

* Update process README

* Update unit tests

* Updated license version

* Update component image versions

* Update changelog

Co-authored-by: Suraj Kota <surakota@amazon.com>
2020-06-17 11:29:24 -07:00
Kartik Kalamadi b50305069b
[AWS SageMaker] Add more unit tests (#3783)
* add more tests for deploy and ground_truth components

* add more tests for workteam component

* add unit tests for model component

* add more unit tests for batchTransform component

* add more tests

* add 'request' function tests

* add more unit tests for ground truth
2020-05-27 10:04:10 -07:00
Gautam Kumar bbe598db26
Adding HPO unit test (#3791)
* Adding HPO unit test

* Adding best training job

* Addressing comment
2020-05-22 22:57:10 -07:00
Nicholas Thomson ddd1969b34
[AWS SageMaker] Unit tests for Training component (#3722)
* Added additional training unit tests

* Add main training function tests

* Add full training test coverage

* Fix import sys

* Fix poorly named test
2020-05-13 16:14:22 -07:00
Nicholas Thomson bd8c1ddd38
[AWS SageMaker] Specify component input types (#3683)
* Replace all string types with Python types

* Update HPO yaml

* Update Batch YAML

* Update Deploy YAML

* Update GroundTruth YAML

* Update Model YAML

* Update Train YAML

* Update WorkTeam YAML

* Updated samples to remove strings

* Update to temporary image

* Remove unnecessary imports

* Update image to newer image

* Update components to python3

* Update bool parser type

* Remove empty ContentType in samples

* Update to temporary image

* Update to version 0.3.1

* Update deploy to login

* Update deploy load config path

* Fix export environment variable in deploy

* Fix env name

* Update deploy reflow env paths

* Add debug config line

* Use username and password directly

* Updated to 0.3.1

* Update field types to JsonObject and JsonArray
2020-05-11 22:06:21 -07:00
Kartik Kalamadi 2f4eafb031
AWS Sagemaker : Add unit tests (#3642)
* Initial changes

* add one test for each component

* Add readme for unit tests

* add empty string test and dockerfile

* added dockerfile

* use python3 in dockerfile

* add coverage report to unit tests

* update readme for PR

* small changes to resolve git comments

* copy requirements.txt separately in dockerfile

* small changes

* pin pip package versions in unit_tests
2020-04-30 01:32:18 -07:00
Kartik Kalamadi 0259fe50b3
AWS Sagemaker : Use json.dumps() to better organize the input and remove data_locations (#3518)
* construct channel input using json.dumps()

* remover data_location parameters

* add changelog

* Update version in license file and small changes to readme
2020-04-23 12:14:07 -07:00
Suraj Kota fbed280e55
add user agent header to boto3 client for aws components (#3487)
* add user agent header to boto client

* add component version according to license file

* fetch version from license file at runtime
2020-04-15 11:25:46 -07:00
Kartik Kalamadi f041b08190
AWS sagemaker: fixed a bug in ground_truth and updated all components to use images from new docker hub repo (#3474)
* Don't leave active_learning_model_arn.txt empty

* updated readme for ground_truth_pipeline_demo

* update docker repo

* Small changes to readme of ground truth sample pipeline
2020-04-14 10:26:13 -07:00
Kartik Kalamadi 942be78bfe
Make endpoint_url None (#3374) 2020-04-07 13:19:43 -07:00
Redback 2fe8c0de61 [Component] Add VPC Interface Endpoint Support for SageMaker (#2299)
* Added Private Link Components

* Updated Component Dockerfile

* Added endpoint_url to Samples
2019-10-03 18:11:56 -07:00
Redback 12dde375b8 [Component] Add Managed Spot Training Support for SageMaker (#2219)
* Added Spot Instance Support

* Fixed missing output configuration

* Added spot instance support to example pipelines

* Updated image to new repository
2019-10-03 12:11:56 -07:00
carolynwang 69ca3c7e4b Update images, bug fixes, clean up code (#1778)
* Update docker images and minor refactoring

* Update image tag, bug fixes, remove unneeded imports

* Revert to using image version, use origin batch transform output method

* Forgot to save 2 changes
2019-08-09 15:25:13 -07:00
carolynwang 3f8bcffaa7 Add SageMaker create workteam and Ground Truth components, sample demo pipeline, other minor updates (#1716)
* Add components for workteam and Ground Truth, minor update for HPO and train, add sample pipeline demo for workteam and GT, update images

* Minor style fixes

* Address PR comments

* Refactor for new folder structure
2019-08-05 15:33:50 -07:00
carolynwang 81b0f08a84 Update SageMaker components and sample pipeline (#1682)
* Update HPO, train, batch transform components, add MNIST kmeans with HPO pipeline

* Minor bug fixes

* Minor bug fixes, update Dockerfiles

* Update docker images and sample pipeline

* Update all components and sample pipeline

* Delete Dockerfiles for individual component

* Typo fix in Dockerfile
2019-07-30 12:47:51 -07:00
carolynwang 2778632ba2 Add SageMaker HPO component and sample usage in a pipeline (#1628)
* add HPO component and sample pipeline usage

* Update Dockerfile to include HPO component

* Update docker image used in hpo component

* Update HPO readme, make HPO job name required, allow empty string for int params, reintro some default values

* Resolve a couple todos

* Add Dockerfile for HPO and update docker image used in HPO component

* Add Dockerfile for HPO
2019-07-21 19:06:51 -07:00
Jiaxin Shan d5147b9776 Add HyperParameters back to SageMaker training job (#1377) 2019-05-30 19:24:21 -07:00
Jiaxin Shan 5374b6b2b4 Add SageMaker components and example pipeline (#1276)
* Add SageMaker components and example pipeline

* Address review feedbacks

* Expose more training job configs

* Update components/aws/sagemaker/batch_transform/component.yaml

Update components descriptions

Co-Authored-By: Jeffwan <seedjeffwan@gmail.com>
2019-05-03 14:55:37 -07:00