Commit Graph

11 Commits

Author SHA1 Message Date
rd-pong 07e67bb0ca
feat(components): SageMaker V2 model monitor component and testing (#9253)
* Add model monitor component and integration tests

* Generate model monitor using updated generator

* Add sleep for monitoring schedule

* Update requirements v2

* Change model monitor image url

* minor fix

* minor fix

* minor fix

* Add unit testing for MonitoringSchedule

* Delete assume-role.json

* Add doc and sample pipeline for Monitoring Schedule

* Regenerate using the latest code generator.
Make parameter description 1 sentence long.

* Revert "Add doc and sample pipeline for Monitoring Schedule"

This reverts commit 6b3b7cc6f5.

* Delete print statements

* Update component with new generator

* address comments

* Add retry for _delete_resource

* Add try catch protection for _get_resource and _delete_resource

* Add integration tests for monitoring job definition components

* Update is_endpoint_deleted for new _get_resource

* Add integration test for updating monitoring schedule

* Remove update from canary

* Add doc and sample pipeline for Monitoring Schedule

* Add doc for monitoring job definition
Update doc for monitoring schedule

* Remove sample for monitoring schedule

* Address comments

* Address comments

* Address comment for unit test

* Address doc comments

* Address test comments
2023-05-09 19:42:33 +00:00
ananth102 d4aaa03035
refactor(components): Open sourcing v2 AWS TrainingJob component. (#8258)
* added training source code

* added commonv2

* added v2 dockerfiles

* took out redudant installations

* fixed image

* fixed dockerignore

* updated docker description

* updated file name

* updated license

* removed unused dependencies

* removed redundant code

* updated dockerignore and licenses

* added v1 licesne to dockerignore

* temp change

* applied black

* decrease paralell tests

* updated documentation

* updated changelog and image

* increase paralell test runs

* updated dockerignore
2022-09-16 22:07:45 +00:00
ananth102 916777e62f
test(components): Added integration test for Sagemaker TrainingJob component v2. (#8247)
* Integration tests for sagemaker training v2

* removed redundant check

* removed redundant print

* added safety check

* pr changes

* updated python and kubernetes

* reverting dependency versions

* Revert "updated python and kubernetes"

This reverts commit e92034d5f9.

* added linting
2022-09-14 17:42:04 -07:00
Mauricio Scheffer bb9fbc80c9
chore: fix link to pytest-xdist docs (#7760) 2022-06-30 20:05:17 +00:00
Leonard O' Sullivan 4aa11c3c7f
feat(components) Adds RoboMaker and SageMaker RLEstimator components (#4813)
* Adds RoboMaker and SageMaker RLEstimator components

* Genericise samples

* Genericise samples

* Adds better logging and updates shim component in samples

* Adds fixes for PR comments. Updates tests accordingly

* Adds docker image reference for integration tests. Allows for setting job_name for RLEstimator training jobs

* Separate RM and SM execution roles

* Remove README reference to VPC config items

* Adds more reliable integration test for RoboMaker Simulation Job

* Simplifies integration tests

* Reverted test container entrypoints

* Update black formatting

* Update components for redbackthomson repo

* Prefix RLEstimator job name

* Add RoboMakerFullAccess to generated roles

* Update version to official 1.1.0

* Formatting int test file

* Add PassRole IAM permission to OIDC

* Adds ROBOMAKER_EXECUTION_ROLE_ARN to build vars

Co-authored-by: Nicholas Thomson <nithomso@amazon.com>
2020-12-11 13:27:27 -08:00
jkuruba 7c349f3f82
feat(components): AWS SageMaker - Changes for updating an existing endpoint (#4424)
* Changes for updating existing endpoint

* Review comments addressed

* Review comments addressed

* Review comments addressed

* Changed awscli and boto3 version. Ran black to format integration tests

* Removing temporarily to debug integration failures

* Adding back integration tests

* Control the number of parallel integration tests to 10

* Third Party License updated

* Version changed to 0.9.0

* Fixed a typo in Changelog
2020-09-18 16:00:28 -07:00
Nicholas Thomson 8014a44229
feat(components): AWS SageMaker - Support for assuming a role (#4212)
* Add client assume role functionality

* Add assume_role to component.yaml files

* Update image to personal

* Update input to force NoneType on empty

* Update integration test setup with assumed role

* Add assume role integration test

* Update boto session to use refreshing credentials

* Update assume role relax trust relationship

* Add check for defined assumed role name

* Add processing assume integ test

* Add assume role unit test for main methods

* Add assume_role to all READMEs

* Update session to use AssumeRoleProvider

* Remove region from child calls to session

* Fix extra region_name in test

* Update assume role processing integ test name

* Add processing integ test to list

* Update assumed role to remain if not generated

* Update license version

* Update image tag to new version

* Add new version to Changelog
2020-08-03 10:53:43 -07:00
Nicholas Thomson 37a63638c7
[AWS SageMaker] Add working FSx setup and test (#3831)
* Add working FSx setup and test

* Removed duplicate test function

* Replaced failure return with exit

* Update parallel methods to export

* Update EKS cluster name outside parallel task

* Add SKIP_FSX_TEST in buildspec

* Add revoke security group ingress

* Add default pytest FSx values
2020-05-29 11:01:15 -07:00
Meghna Baijal fb549531f1
[AWS SageMaker] Integration Test for AWS SageMaker GroundTruth Component (#3830)
* Integration Test for AWS SageMaker GroundTruth Component

* Unfix already fixed bug

* Fix the README I overwrote by mistake

* Remove use of aws-secret for OIDC

* Rev 2: Fix linting errors
2020-05-26 15:44:40 -07:00
Nicholas Thomson f2a860b84c
[AWS SageMaker] Integration tests automation (#3768)
* # This is a combination of 5 commits.
# This is the 1st commit message:

Add initial scripts

# This is the commit message #2:

Add working pytest script

# This is the commit message #3:

Add initial scripts

# This is the commit message #4:

Add environment variable files

# This is the commit message #5:

Remove old cluster script

* Add initial scripts

Add working pytest script

Add initial scripts

Add environment variable files

Remove old cluster script

Update pipeline credentials to OIDC

Add initial scripts

Add working pytest script

Add initial scripts

Add working pytest script

* Remove debugging mark

* Update example EKS cluster name

* Remove quiet from Docker build

* Manually pass env

* Update env list vars as string

* Update use array directly

* Update variable array to export

* Update to using read for splitting

* Move to helper script

* Update export from CodeBuild

* Add wait for minio

* Update kubectl wait timeout

* Update minor changes for PR

* Update integration test buildspec to quiet build

* Add region to delete EKS

* Add wait for pods

* Updated README

* Add fixed interval wait

* Fix CodeBuild step order

* Add file lock for experiment ID

* Fix missing pytest parameter

* Update run create only once

* Add filelock to conda env

* Update experiment name ensuring creation each time

* Add try/catch with create experiment

* Remove caching from KFP deployment

* Remove disable KFP caching

* Move .gitignore changes to inside component

* Add blank line to default .gitignore
2020-05-20 14:18:19 -07:00
Suraj Kota 6beab2251d
Integration tests for AWS SageMaker Components (#3654)
* integration tests for aws sagemaker components with comment

* address comment related to S3 dataset creation

* rev3: bug fix in conda env yaml and resuse sagemaker method to get image URI

* Add createModel test

	- reduce code duplication
	- add some utility methods
2020-05-06 22:19:09 -07:00