Commit Graph

61 Commits

Author SHA1 Message Date
Nicholas Thomson bea63652e1
[AWS SageMaker] Processing job component (#3944)
* Add TDD processing definition

* Update README

* Update temporary image

* Update component entrypoint

* Add WORKDIR to fix Docker 18 support

* integration test for processing job

* Remove job links

* Add container outputs and tests

* Update default properties

* Remove max_run_time if none provided

* Update integration readme steps

* Updated README with more resources

* Add CloudWatch link back to logs

* Update input and output config to arrays

* Update processing integration test

* Update process README

* Update unit tests

* Updated license version

* Update component image versions

* Update changelog

Co-authored-by: Suraj Kota <surakota@amazon.com>
2020-06-17 11:29:24 -07:00
Suraj Kota bc6aed9cac
[AWS Sagemaker] aws-samples kmeans-hpo pipeline test (#3905)
* aws-samples kmeans-hpo pipeline test
	- code clean up
	- removed unused args

* add test dependency

* Trigger Build
2020-06-16 10:26:04 -07:00
Kartik Kalamadi 35019eb3ea
[AWS SageMaker] Add integration test for sample pipeline train (#3876)
* add integ test for sample pipeline train

* change docker build context integ test

* add spot test and use train ccomponent test for sample train pipeline

* small changes and ran flake8 and black

* address comments
2020-06-12 12:11:55 -07:00
Nicholas Thomson d6920ca2ad
[AWS SageMaker] Update GroundTruth integration test timeout (#3973)
Update the timeout for the GroundTruth integration test from 10 seconds to 1200 seconds.
2020-06-11 17:07:57 -07:00
Gautam Kumar a7be049b6d
Move the minio artifact download under try block (#3889)
If minio artifact download fails the workteam is not being deleted.
2020-06-01 19:06:15 -07:00
Nicholas Thomson 37a63638c7
[AWS SageMaker] Add working FSx setup and test (#3831)
* Add working FSx setup and test

* Removed duplicate test function

* Replaced failure return with exit

* Update parallel methods to export

* Update EKS cluster name outside parallel task

* Add SKIP_FSX_TEST in buildspec

* Add revoke security group ingress

* Add default pytest FSx values
2020-05-29 11:01:15 -07:00
Gautam Kumar 58ff65f330
fixing case when status is None (#3865)
* fixing case when status is None

* Fixing status update
2020-05-28 22:05:14 -07:00
Meghna Baijal fb549531f1
[AWS SageMaker] Integration Test for AWS SageMaker GroundTruth Component (#3830)
* Integration Test for AWS SageMaker GroundTruth Component

* Unfix already fixed bug

* Fix the README I overwrote by mistake

* Remove use of aws-secret for OIDC

* Rev 2: Fix linting errors
2020-05-26 15:44:40 -07:00
Nicholas Thomson f2a860b84c
[AWS SageMaker] Integration tests automation (#3768)
* # This is a combination of 5 commits.
# This is the 1st commit message:

Add initial scripts

# This is the commit message #2:

Add working pytest script

# This is the commit message #3:

Add initial scripts

# This is the commit message #4:

Add environment variable files

# This is the commit message #5:

Remove old cluster script

* Add initial scripts

Add working pytest script

Add initial scripts

Add environment variable files

Remove old cluster script

Update pipeline credentials to OIDC

Add initial scripts

Add working pytest script

Add initial scripts

Add working pytest script

* Remove debugging mark

* Update example EKS cluster name

* Remove quiet from Docker build

* Manually pass env

* Update env list vars as string

* Update use array directly

* Update variable array to export

* Update to using read for splitting

* Move to helper script

* Update export from CodeBuild

* Add wait for minio

* Update kubectl wait timeout

* Update minor changes for PR

* Update integration test buildspec to quiet build

* Add region to delete EKS

* Add wait for pods

* Updated README

* Add fixed interval wait

* Fix CodeBuild step order

* Add file lock for experiment ID

* Fix missing pytest parameter

* Update run create only once

* Add filelock to conda env

* Update experiment name ensuring creation each time

* Add try/catch with create experiment

* Remove caching from KFP deployment

* Remove disable KFP caching

* Move .gitignore changes to inside component

* Add blank line to default .gitignore
2020-05-20 14:18:19 -07:00
Suraj Kota bff83921d7
AWS Sagemaker Components - enhance integration test coverage (#3720)
* AWS Sagemaker Components - enhance integration test coverage
	- Add tests for create endpoint, hpo job and batch transform
	- Minor bug fixes and documentation

* rev2: Address comments and clean up generated artifacts

* rev3: address more comments

* rev4: add canary test marker

* Trigger Build
2020-05-15 10:21:36 -07:00
Suraj Kota 6beab2251d
Integration tests for AWS SageMaker Components (#3654)
* integration tests for aws sagemaker components with comment

* address comment related to S3 dataset creation

* rev3: bug fix in conda env yaml and resuse sagemaker method to get image URI

* Add createModel test

	- reduce code duplication
	- add some utility methods
2020-05-06 22:19:09 -07:00