add nuclio components (to build/deploy, delete, invoke functions) (#1295)
* add support for flexible config (via env var) for the pipline service and UI, fix broken links (pointed to API vs UI service) * add namespace and change address to endpoint
This commit is contained in:
parent
b61bef04a3
commit
442c80449d
|
|
@ -0,0 +1,44 @@
|
|||
# Nuclio (Serverless) Components
|
||||
|
||||
[Nuclio](https://nuclio.io/) is a native and high performance serverless platform over Kubernetes
|
||||
which automate the process of build, deployment, monitoring, and auto-scaling of micro-services.
|
||||
Nuclio support variety of data and data-science related features (e.g. stream processing,
|
||||
GPUs, volume/DB mounts, high concurrency, etc.)
|
||||
|
||||
To install Nuclio over Kubernetes follow the [instruction in Github](https://github.com/nuclio/nuclio),
|
||||
or this [interactive tutorial](https://www.katacoda.com/javajon/courses/kubernetes-serverless/nuclio).
|
||||
|
||||
Nuclio functions can be used in the following ML pipline tasks:
|
||||
* Data collectors, ETL, stream processing
|
||||
* Data preparation and analysis
|
||||
* Hyper parameter model training
|
||||
* Real-time model serving
|
||||
* Feature vector assembly (real-time data preparation)
|
||||
|
||||
Read more on the use of Nuclio in [data-science here](https://towardsdatascience.com/serverless-can-it-simplify-data-science-projects-8821369bf3bd).
|
||||
Nuclio functions can be generated automatically from 8 code languages, from Jupyter Notebooks, Zip, Git, Docker, etc.
|
||||
The [nuclio-jupyter repo](https://github.com/nuclio/nuclio-jupyter) provide guidance and many examples.
|
||||
|
||||
## Components
|
||||
|
||||
There are currently 3 components in this package:
|
||||
* [deploy](deploy/component.yaml) - Automatically build and deploy/re-deploy functions
|
||||
from code/zip/notebooks/git/.. and/or override various deployment configurations such as
|
||||
setting cpu/mem/gpu resources, scaling, environment variables, triggers, etc.
|
||||
* [delete](delete/component.yaml) - Delete a function
|
||||
* [invoker](invoker/component.yaml) - invoke a function and return the results/logs
|
||||
|
||||
Additional components and examples will be added soon for parallel batch/stream processing
|
||||
|
||||
## Examples
|
||||
|
||||
**Deploy a function (from Github)**
|
||||
|
||||
```python
|
||||
nuclio_dep = kfp.components.load_component_from_file('deploy/component.yaml')
|
||||
|
||||
def my_pipeline():
|
||||
new_func = nuclio_dep(url='git://github.com/nuclio/nuclio#master:/hack/examples/python/helloworld', name='myfunc', project='myproj', tag='0.11')
|
||||
|
||||
...
|
||||
```
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
name: nuclio delete
|
||||
description: delete a nuclio function.
|
||||
inputs:
|
||||
- {name: Name, type: String, description: 'function name'}
|
||||
- {name: Namespace, type: String, description: 'Kubernetes namespace', default: ''}
|
||||
- {name: Dashboard, type: String, description: 'nuclio dashboard service url', default: 'http://nuclio-dashboard.nuclio:8070'}
|
||||
implementation:
|
||||
container:
|
||||
image: nuclio/pydeploy
|
||||
command: [
|
||||
python, -m, nuclio, del, {inputValue: Name},
|
||||
--dashboard-url, {inputValue: Dashboard},
|
||||
--namespace, {inputValue: Namespace},
|
||||
]
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
name: nuclio deploy
|
||||
description: auto build and deploy nuclio function.
|
||||
inputs:
|
||||
- {name: Url, type: String, description: 'url/path to source code, zip archive, git path, or notebook'}
|
||||
- {name: Name, type: String, description: 'function name'}
|
||||
- {name: Project, type: String, description: 'project name', default: 'default'}
|
||||
- {name: Tag, type: String, description: 'function version tag', default: ''}
|
||||
- {name: Dashboard, type: String, description: 'nuclio dashboard service url', default: 'http://nuclio-dashboard.nuclio:8070'}
|
||||
- {name: Spec, type: String, description: 'override function spec, Json {key: value, ..}', default: ''}
|
||||
- {name: Env, type: String, description: 'override function env var, Json {key: value, ..}', default: ''}
|
||||
- {name: Mount, type: String, description: 'volume mount, [vol-type:]<vol-url>:<dst-path>', default: ''}
|
||||
outputs:
|
||||
- {name: Endpoint, type: String, description: 'function endpoint url'}
|
||||
implementation:
|
||||
container:
|
||||
image: nuclio/pydeploy
|
||||
command: [
|
||||
python, -m, nuclio, deploy, {inputValue: Url},
|
||||
--dashboard-url, {inputValue: Dashboard},
|
||||
--name, {inputValue: Name},
|
||||
--project, {inputValue: Project},
|
||||
--tag, {inputValue: Tag},
|
||||
--env-json, {inputValue: Env},
|
||||
--spec-json, {inputValue: Spec},
|
||||
--mount, {inputValue: Mount},
|
||||
]
|
||||
fileOutputs:
|
||||
Endpoint: /tmp/output
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
name: nuclio invoker
|
||||
description: invoke nuclio function.
|
||||
inputs:
|
||||
- {name: Url, type: String, description: 'function URL/endpoint'}
|
||||
- {name: Body, type: String, description: 'request body', default: ''}
|
||||
- {name: Log level, type: String, description: 'log level', default: 'info'}
|
||||
- {name: Method, type: String, description: 'HTTP method GET|POST|..', default: ''}
|
||||
outputs:
|
||||
- {name: output, type: String, description: 'function output'}
|
||||
implementation:
|
||||
container:
|
||||
image: nuclio/invoker
|
||||
command: [
|
||||
invoke,
|
||||
-a, {inputValue: Url},
|
||||
-b, {inputValue: Body},
|
||||
-m, {inputValue: Method},
|
||||
-l, {inputValue: Log level},
|
||||
]
|
||||
fileOutputs:
|
||||
output: /tmp/output
|
||||
Loading…
Reference in New Issue