* Adds RoboMaker and SageMaker RLEstimator components
* Genericise samples
* Genericise samples
* Adds better logging and updates shim component in samples
* Adds fixes for PR comments. Updates tests accordingly
* Adds docker image reference for integration tests. Allows for setting job_name for RLEstimator training jobs
* Separate RM and SM execution roles
* Remove README reference to VPC config items
* Adds more reliable integration test for RoboMaker Simulation Job
* Simplifies integration tests
* Reverted test container entrypoints
* Update black formatting
* Update components for redbackthomson repo
* Prefix RLEstimator job name
* Add RoboMakerFullAccess to generated roles
* Update version to official 1.1.0
* Formatting int test file
* Add PassRole IAM permission to OIDC
* Adds ROBOMAKER_EXECUTION_ROLE_ARN to build vars
Co-authored-by: Nicholas Thomson <nithomso@amazon.com>
* Temporary rebase commit
* Add yaml compiler
* Add compiler CLI
* Update Dockerfile to copy all files
* Add validate input list vs dict
* Add unit test for new train
* Add minor bug fixes
* Override tag when generating specs
* Update pydocs with formatter
* Add contributing doc
* Add formatters to CONTRIBUTING
* Add working generic logic applied to train
* Update component input and output to inherit
* Downgrade to Python 3.7
* Update add outputValue to arg list
* Updated outputValue to outputPath
* Add empty string default to not-required inputs
* Update path to component relative to root
* Update faulty False-y condition
* Update outputs to write to file
* Update doc formatting
* Update docstrings to match structure
* Add unit tests for component and compiler
* Add unit tests for component
* Add spec unit tests
* Add training unit tests
* Update unit test automation
* Add sample formatting checks
* Remove extra flake8 check in integ tests
* Add unit test black check
* Update black formatting for all files
* Update include black formatting
* Add batch component
* Remove old transform components
* Update region input description
* Add all component specs
* Add deploy component
* Add ground truth component
* Add HPO component
* Add create model component
* Add processing component
* Add workteam component
* Add spec unit tests
* Add deploy unit tests
* Add ground truth unit tests
* Add tuning component unit tests
* Add create model component unit test
* Add process component unit tests
* Add workteam component unit tests
* Remove output_path from required_args
* Remove old component implementations
* Update black formatting
* Add assume role feature
* Compiled all components
* Update doc formatting
* Fix process terminate syntax error
* Update compiler to use kfp structures
* Update nits
* Update unified requirements
* Rebase on debugging commit
* Add debugger unit tests
* Update formatting
* Update component YAML
* Fix unit test Dockerfile relative directory
* Update unit test context to root
* Update Batch to auto-generate name
* Update minor docs and formatting changes
* Update deploy name to common autogenerated
* Add f-strings to logs
* Add update support
* Add Amazon license header
* Update autogen and autoformat
* Rename SpecValidator to SpecInputParser
* Split requirements by dev and prod
* Support for checking generated specs
* Update minor changes
* Update deploy component output description
* Update components to beta repository
* Update fix unit test requirements
* Update unit test build spec for new results path
* Update deploy wait for endpoint complete
* Update component configure AWS clients in new method
* Update boto3 retry method
* Update license version
* Update component YAML versions
* Add new version to Changelog
* Update component spec types
* Update deploy config ignore overwrite
* Update component for debugging
* Update images back to 1.0.0
* Remove coverage from components
* Changes for updating existing endpoint
* Review comments addressed
* Review comments addressed
* Review comments addressed
* Changed awscli and boto3 version. Ran black to format integration tests
* Removing temporarily to debug integration failures
* Adding back integration tests
* Control the number of parallel integration tests to 10
* Third Party License updated
* Version changed to 0.9.0
* Fixed a typo in Changelog
* Implemented debugger for training component with sample pipeline, unit tests, and integration test
* Implemented changes from PR, refactored utils.py, made sample pipeline more succinct, removed hardcoding from integration tests
* Added default parameter for sample pipeline and fixed grammar for sample README, refactored _utils.py for fstrings and fixed offset for errors
* Removed aws secret lines
* Terminate debug rules when terminating training job, Terminate debug rules if terminate is pressed after training job has completed, added integration tests for stop_debug_rules, updated READMEs for train and sample, renamed sample pipeline, removed tensorboard, updated sagemaker version to sagemaker 2.1.0.
* Terminate debug rules when terminating training job, Terminate debug rules if terminate is pressed after training job has completed, added integration tests for stop_debug_rules, updated READMEs for train and sample, renamed sample pipeline, removed tensorboard, updated sagemaker version to sagemaker 2.1.0.
* Removed extra files, cleaned integration test
* Changed integration test to use sample debugger pipeline
* Processing jobs created from debug rules will not terminate, fixing other small issues
* Removed debug from pipeline definition, removed extra line, removed unused function
* Changelog and image tag updates
* Create README.md
* Added README
Updated page to include information on Amazon SageMaker components
* Update README.md
* Integrated feedback
* Added link to SageMaker Components tutorial
* Update README.md
added link to sample pipelines
* Add client assume role functionality
* Add assume_role to component.yaml files
* Update image to personal
* Update input to force NoneType on empty
* Update integration test setup with assumed role
* Add assume role integration test
* Update boto session to use refreshing credentials
* Update assume role relax trust relationship
* Add check for defined assumed role name
* Add processing assume integ test
* Add assume role unit test for main methods
* Add assume_role to all READMEs
* Update session to use AssumeRoleProvider
* Remove region from child calls to session
* Fix extra region_name in test
* Update assume role processing integ test name
* Add processing integ test to list
* Update assumed role to remain if not generated
* Update license version
* Update image tag to new version
* Add new version to Changelog
* Add functionality to stop SM jobs
- Unit and Integration tests for the functionality
* unit test update and customer message update
* Changelog and image tag updates
* update version for deploy component and merge conflicts
* Update version in License file
* fix conflicting paths for download, add test for batch
* Update input arguments
* Remove fileOutputs
* Update outputs to new paths
* Modify integ test artifact path
* Add unit test for new output format
* Add unit test for write_output
* Migrate tests into test_utils
* Add clarifying comment
* Remove output path file extension
* Update license to 0.5.2
* Update component to 0.5.2
* Add 0.5.2 to changelog
* Remove JSON
* add more region ARNs to groundtruth lambda map as per sagemaker documentation
* changes for patch release 0.5.1
* Make label_category_config optional in groundtruth component
* merge fix
Co-authored-by: Suraj Kota <surakota@amazon.com>
* Added preprocessing step to sample pipeline
* Add processing to components README
* Remove unused import
* Add data set prerequisite to training component
* Remove simple HPO pipeline
* Update MNIST header in README
* Remove simple HPO sample integration test
* Empty commit to trigger google bot
* add integ test for sample pipeline train
* change docker build context integ test
* add spot test and use train ccomponent test for sample train pipeline
* small changes and ran flake8 and black
* address comments
* Add working FSx setup and test
* Removed duplicate test function
* Replaced failure return with exit
* Update parallel methods to export
* Update EKS cluster name outside parallel task
* Add SKIP_FSX_TEST in buildspec
* Add revoke security group ingress
* Add default pytest FSx values
* add more tests for deploy and ground_truth components
* add more tests for workteam component
* add unit tests for model component
* add more unit tests for batchTransform component
* add more tests
* add 'request' function tests
* add more unit tests for ground truth
* Integration Test for AWS SageMaker GroundTruth Component
* Unfix already fixed bug
* Fix the README I overwrote by mistake
* Remove use of aws-secret for OIDC
* Rev 2: Fix linting errors
* don't use aws-secret and update readme for sample pipelines
* Addressed comments on PR and few more readme changes
* small changes to readme
* nit change
* Address comments
* # This is a combination of 5 commits.
# This is the 1st commit message:
Add initial scripts
# This is the commit message #2:
Add working pytest script
# This is the commit message #3:
Add initial scripts
# This is the commit message #4:
Add environment variable files
# This is the commit message #5:
Remove old cluster script
* Add initial scripts
Add working pytest script
Add initial scripts
Add environment variable files
Remove old cluster script
Update pipeline credentials to OIDC
Add initial scripts
Add working pytest script
Add initial scripts
Add working pytest script
* Remove debugging mark
* Update example EKS cluster name
* Remove quiet from Docker build
* Manually pass env
* Update env list vars as string
* Update use array directly
* Update variable array to export
* Update to using read for splitting
* Move to helper script
* Update export from CodeBuild
* Add wait for minio
* Update kubectl wait timeout
* Update minor changes for PR
* Update integration test buildspec to quiet build
* Add region to delete EKS
* Add wait for pods
* Updated README
* Add fixed interval wait
* Fix CodeBuild step order
* Add file lock for experiment ID
* Fix missing pytest parameter
* Update run create only once
* Add filelock to conda env
* Update experiment name ensuring creation each time
* Add try/catch with create experiment
* Remove caching from KFP deployment
* Remove disable KFP caching
* Move .gitignore changes to inside component
* Add blank line to default .gitignore
* integration tests for aws sagemaker components with comment
* address comment related to S3 dataset creation
* rev3: bug fix in conda env yaml and resuse sagemaker method to get image URI
* Add createModel test
- reduce code duplication
- add some utility methods
* Add initial unit test buildspec
* Add docker log output
* Add force no pytest color
* Update docker build to be quiet
* Add pass all environment variables
* Update unit test container env file
* Update env to use different syntax
* Remove daemon mode
* Remove TTY from docker run
* Add dryrun and dockercfg setup
* Update dryrun into CodeBuild logic
* Add mkdir for Docker config
* Update app version temporarily
* Revert app version temporarily
* Update unit test log file
* Add tag minor and major versions
* Update version temporarily
* Add print for major and minor tags
* Revert version back down
* Add deploy version override
* Update path to testing directories
* Fix tab formatting
* Fix pytest log directory
* Initial changes
* add one test for each component
* Add readme for unit tests
* add empty string test and dockerfile
* added dockerfile
* use python3 in dockerfile
* add coverage report to unit tests
* update readme for PR
* small changes to resolve git comments
* copy requirements.txt separately in dockerfile
* small changes
* pin pip package versions in unit_tests
Suraj Kota
Leonard O' Sullivan
Kartik Kalamadi
Cesar
Nicholas Thomson
Meghna Baijal
jkuruba
Dustin Luong
Matthew Rose
Bohdan
IvyBazan
Gautam Kumar
Jiaxin Shan