58d22d4ba9
chore(components/pytorch) - Samples fix for PT and PTL Upgrade ( #8148 )
...
* WIP: chore(components/pytorch) - Samples fix for PT and PTL Upgrade
* fix: bert example
- fix minio secret
- remove pth file upload to minio
- add captum to pip packages
* fix: bert-dist training args
* fix: cifar10 example
* fix: cifar10 example notebook
* fix: captum example
* fix: gpu fixes for bert and cifar10 example
* fix: bert dist ptl upgrade
2022-09-22 05:29:38 +00:00
49c3587591
chore(components/pytorch):kserve migration ( #7615 )
...
* chore(components/pytorch):kserve migration
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* fix: pytorch dist training
- enable env vars in config.properties
- upgrade pip in dockerfile
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* Bert - KServe v2 handler changes
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* fix: bert notebook for kserve v2
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* fix: add protocol verion to bert gpu yaml
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* Adding utility to convert image to bytes - Cifar
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Cifar10 - captum update
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* fix: cifar10 example
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* fix: predictor component for kserve v2
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* fix: pytorch dist training for kserve v2
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* fix: cifar10 hpo example
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
* Bumping pytorch-kfp-components version
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
Co-authored-by: Shrinath Suresh <shrinath@ideas2it.com>
2022-07-08 08:55:52 +00:00
131b116fb9
chore(components/pytorch): Upgrade PTL ( #6902 )
...
* chore(components/pytorch): Upgrade PTL
* Bump PTL version
Signed-off-by: Jagadeesh J <jagadeeshj@ideas2it.com>
2022-01-12 07:09:11 +00:00
a54aa33f51
chore(components/pytorch): bert samples - confusion matrix ( #6458 )
...
* Fixing confusion matrix
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Enabling confusion matrix
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Passing attention masks to compute_bert_outputs
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-09-03 12:05:10 -07:00
b4ad3d1488
chore(components/pytorch): Samples Unit tests and lint fixes ( #6288 )
...
* Applying yapf on all the python files in samples
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Test compile yaml fixes
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Remove unused file
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Lint fixes: generate_templates.py
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Lint fixes: gen_image_timestamp.py
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Lint fixes
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Cifar10 handler lint fixes
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Lint fixes for the remaining files
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-08-11 22:59:02 -07:00
f9d47d0ef9
fix(components/pytorch): PyTorch Samples - Generating component.yaml using templates ( #6231 )
...
* Adding code to generate component.yaml files from templates
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding templates for train and pre process
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating the build script to generate component.yaml from templates
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating the jupyter notebook to use templates
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing all component.yaml files
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Revert "Removing all component.yaml files"
This reverts commit db75951949
2021-08-08 23:34:25 -07:00
71317711be
fix(components/pytorch): Custom package installation - torchserve ( #6055 )
...
* Swiching on custom installation
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding sklearn and captum
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-07-19 23:22:15 -07:00
01ec07b467
fix(components/pytorch): Clean up notebook and yaml files ( #6070 )
...
* Fix: Clean up notebook and yaml files
* Fix: Dockerfile
2021-07-16 10:33:38 -07:00
b1d0eb799b
feat(components/pytorch): Pytorch Distributed Training ( #6021 )
...
* Feature: Bert distributed training
* Feat: Adds staging volume for pytorch job
* Feat: Add PVC storage URI for KFserving
- Update copy component
* Fix: gpu explain handler
* fix: notebook cleanup
* Fix: Update Dockerfile, requirement.txt
* Fix: Dockerfile
2021-07-16 05:29:38 -07:00
d88394ba4a
fix(components/pytorch): pytorch kfp components and Sample - GPU updates ( #5939 )
...
* Updating trainer test wiht model parameter to support multi gpu multi node scenario
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Changing get_model to lightning_module in bert example as PTL is used from source
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* removing ptl 1.3.x from dependency as Pytorch operator needs ptl to be installed from source
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating trainer args with gpu parameters
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Installing PTL from source
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating get_model to lightning_module
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-07-11 22:06:26 -07:00
5b78b01410
chore(samples/pytorch) - Build script fix - Updating Dockerfile name ( #5928 )
...
* Fixing build script
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Setting custom package installation to false for bert
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-06-29 12:09:35 -07:00
551756ee39
chore(components/pytorch) Updating pytorch lightning version to 1.3.7 ( #5905 )
...
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-06-27 23:49:14 -07:00
a437b92c24
chore(components/pytorch) [WIP] Updating docker image location - pytorch kfp samples ( #5911 )
...
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-06-24 00:03:19 -07:00
192b8e8756
fix(components/pytorch) Pytorch Lightning Arguments string pass through ( #5870 )
...
* Updating bert script to use input arguments as string
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding utility to parse input arguments
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing tensorboard root and checkpoint dirs
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding string pass through in component.yaml
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing pipeline.py file
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing pipeline keys
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing args in component.yaml
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing extra comma
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing unused code
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating cifar10 example
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Uncommenting confusion matrix
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating jupyter notebooks
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing cifar10 train component.yaml
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Addressing review comments
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing lint issues
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-06-23 23:10:19 -07:00
2d7f3a21f2
feat(components/pytorch): Add requirement for mar generation ( #5850 )
...
* Feat: Add requirement for mar generation
- Notebook clean to skip root access
- Adds package.json
* Fix: Update config.properties for cifar10 and bert examples
* Fix: Update s3 urls with git urls
- readme fixes
- add config.properties
* Fix: Docker files copy command permission
2021-06-17 03:10:28 -07:00
9cfa4dfc0a
fix(components/pytorch) Pytorch - Tensorboard Profiler fix ( #5860 )
...
* Fix: module_file_args overriding the trainer_args variable. Updating module_file_args as the superset
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Setting PTL to 1.3.5 in requirements.txt
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing typo
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding print statements for profiler debugging
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing cpuonly tag
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
2021-06-16 16:06:27 -07:00
703822c420
feat(components): Adding samples for PyTorch - KFP ( #5812 )
...
* Adding Cifar10, Bert examples, component.yamls and Dockerfile
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding cifar10 and bert notebooks
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing component yaml paths
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Changing docker image name
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* updating notebooks
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating image name in component.yaml
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding --no-cache option to pip install
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating readme file with cluster steps
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing links in README.md
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing README.md indendation
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing local pipelines files
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Updating docker image name
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding Docstrings, yapf, pylint
Signed-off-by: ankan94 <ankan@ideas2it.com>
* fix test images paths in the notebook
Signed-off-by: Arvind-Ideas2IT <arvindkumarsingh.gautam@ideas2it.com>
* Adding docker file for gpu
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding pipeline.py file to run the example from local environment
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding build script for local testing
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing name
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding license header and description in component.yaml files
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixin docker file license
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Renaming docker file
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding docker file name to build script
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* updating readme.md with local steps
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Installing pytorch-components from pypi
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Fixing cpu docker file
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Adding --no-cache-dir for pip
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
* Removing headers
Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
Co-authored-by: ankan94 <ankan@ideas2it.com>
Co-authored-by: Arvind-Ideas2IT <arvindkumarsingh.gautam@ideas2it.com>
2021-06-09 06:44:16 -07:00
Jagadeesh J
shrinath-suresh