docs/code-samples/community/serving/machinelearning-python-bentoml
kahirokunn 8e917ed012
chore: multi-arch support with buildx in all Dockerfiles (#5273)
* chore: use buildx to build grpc-ping-go

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support grpc-ping-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-clojure multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-dart multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-deno multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-elixir multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-haskell multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-java-micronaut multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-java-quarkus multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-r multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-rserver multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-rust multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-swift multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-vertx multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support iris-classifier multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-python multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support node-heartbeat-source multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support cloudevents-dotnet multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support cloudevents-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support cloudevents-nodejs multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support cloudevents-rust multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support gitwebhook-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-csharp multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-java-spark multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-java-spring multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-kotlin multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-nodejs multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-php multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-python multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-shell multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support knative-routing-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support kong-routing-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support multi-container multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support secrets-go multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support knative-docs multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-r Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support helloworld-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support cloudevents-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support gitwebhook-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support knative-routing-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support kong-routing-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support servingcontainer Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support sidecarcontainer Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support secrets-go Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

* chore: support multi-container Dockerfile multiple architecture

Signed-off-by: kahirokunn <okinakahiro@gmail.com>

---------

Signed-off-by: kahirokunn <okinakahiro@gmail.com>
2023-09-06 13:28:13 +00:00
..
README.md chore: multi-arch support with buildx in all Dockerfiles (#5273) 2023-09-06 13:28:13 +00:00
index.md chore: multi-arch support with buildx in all Dockerfiles (#5273) 2023-09-06 13:28:13 +00:00
iris_classifier.py Move community docs onto MkDocs site (#4544) 2021-12-13 06:57:20 -08:00
main.py Move community docs onto MkDocs site (#4544) 2021-12-13 06:57:20 -08:00
service.yaml Move community docs onto MkDocs site (#4544) 2021-12-13 06:57:20 -08:00

README.md

Hello World - Python BentoML

A simple machine learning model with API serving that is written in python and using BentoML. BentoML is an open source framework for high performance ML model serving, which supports all major machine learning frameworks including Keras, Tensorflow, PyTorch, Fast.ai, XGBoost and etc.

This sample will walk you through the steps of creating and deploying a machine learning model using python. It will use BentoML to package a classifier model trained on the Iris dataset. Afterward, it will create a container image and deploy the image to Knative.

Knative deployment guide with BentoML is also available in the BentoML documentation

Before you begin

  • A Kubernetes cluster with Knative installed. Follow the Knative installation instructions if you need to create one.
  • Docker installed and running on your local machine, and a Docker Hub account configured. Docker Hub will be used for a container registry).
  • Python 3.6 or above installed and running on your local machine.
    • Install scikit-learn and bentoml packages:

      pip install scikit-learn
      pip install bentoml
      

Recreating sample code

Run the following code on your local machine, to train a machine learning model and deploy it as API endpoint with Knative Serving.

  1. BentoML creates a model API server, via prediction service abstraction. In iris_classifier.py, it defines a prediction service that requires a scikit-learn model, asks BentoML to figure out the required pip dependencies, also defines an API, which is the entry point for accessing this machine learning service.

    from bentoml import env, artifacts, api, BentoService
    from bentoml.handlers import DataframeHandler
    from bentoml.artifact import SklearnModelArtifact
    
    @env(auto_pip_dependencies=True)
    @artifacts([SklearnModelArtifact('model')])
    class IrisClassifier(BentoService):
    
        @api(DataframeHandler)
        def predict(self, df):
            return self.artifacts.model.predict(df)
    
  2. In main.py, it uses the classic iris flower data set to train a classification model which can predict the species of an iris flower with given data and then save the model with BentoML to local disk.

    from sklearn import svm
    from sklearn import datasets
    
    from iris_classifier import IrisClassifier
    
    if __name__ == "__main__":
        # Load training data
        iris = datasets.load_iris()
        X, y = iris.data, iris.target
    
        # Model Training
        clf = svm.SVC(gamma='scale')
        clf.fit(X, y)
    
        # Create a iris classifier service instance
        iris_classifier_service = IrisClassifier()
    
        # Pack the newly trained model artifact
        iris_classifier_service.pack('model', clf)
    
        # Save the prediction service to disk for model serving
        saved_path = iris_classifier_service.save()
    
  3. Run the main.py file to train and save the model:

    python main.py
    
  4. Use BentoML CLI to check saved model's information.

    bentoml get IrisClassifier:latest
    

    Example:

    > bentoml get IrisClassifier:latest
    {
      "name": "IrisClassifier",
      "version": "20200305171229_0A1411",
      "uri": {
        "type": "LOCAL",
        "uri": "/Users/bozhaoyu/bentoml/repository/IrisClassifier/20200305171229_0A1411"
      },
      "bentoServiceMetadata": {
        "name": "IrisClassifier",
        "version": "20200305171229_0A1411",
        "createdAt": "2020-03-06T01:12:49.431011Z",
        "env": {
          "condaEnv": "name: bentoml-IrisClassifier\nchannels:\n- defaults\ndependencies:\n- python=3.7.3\n- pip\n",
          "pipDependencies": "bentoml==0.6.2\nscikit-learn",
          "pythonVersion": "3.7.3"
        },
        "artifacts": [
          {
            "name": "model",
            "artifactType": "SklearnModelArtifact"
          }
        ],
        "apis": [
          {
            "name": "predict",
            "handlerType": "DataframeHandler",
            "docs": "BentoService API",
            "handlerConfig": {
              "orient": "records",
              "typ": "frame",
              "input_dtypes": null,
              "output_orient": "records"
            }
          }
        ]
      }
    }
    
  5. Test run API server. BentoML can start an API server from the saved model. Use BentoML CLI command to start an API server locally and test it with the curl command.

    bentoml serve IrisClassifier:latest
    

    In another terminal window, make curl request with sample data to the API server and get prediction result:

    curl -v -i \
    --header "Content-Type: application/json" \
    --request POST \
    --data '[[5.1, 3.5, 1.4, 0.2]]' \
    127.0.0.1:5000/predict
    

Building and deploying the sample

BentoML supports creating an API server docker image from its saved model directory, where a Dockerfile is automatically generated when saving the model.

  1. To build an API model server docker image, replace {username} with your Docker Hub username and run the following commands.

    # jq might not be installed on your local system, please follow jq install
    # instruction at https://stedolan.github.io/jq/download/
    saved_path=$(bentoml get IrisClassifier:latest -q | jq -r ".uri.uri")
    
    # Build and push the container on your local machine.
    docker buildx build --platform linux/arm64,linux/amd64 -t "{username}/iris-classifier" --push $saved_path
    
  2. In service.yaml, replace {username} with your Docker hub username:

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: iris-classifier
      namespace: default
    spec:
      template:
        spec:
          containers:
            - image: docker.io/{username}/iris-classifier
              ports:
              - containerPort: 5000 # Port to route to
              livenessProbe:
                httpGet:
                  path: /healthz
                initialDelaySeconds: 3
                periodSeconds: 5
              readinessProbe:
                httpGet:
                  path: /healthz
                initialDelaySeconds: 3
                periodSeconds: 5
                failureThreshold: 3
                timeoutSeconds: 60
    
  3. Deploy the Service to Knative Serving with kubectl by running the command:

    kubectl apply --filename service.yaml
    
  4. Now that your service is created, Knative performs the following steps:

    • Create a new immutable revision for this version of the app.
    • Network programming to create a route, ingress, service, and load balance for your application.
    • Automatically scale your pods up and down (including to zero active pods).
  5. Run the following command to find the domain URL for your service:

    kubectl get ksvc iris-classifier --output=custom-columns=NAME:.metadata.name,URL:.status.url
    
    NAME              URL
    iris-classifier   http://iris-classifer.default.example.com
    
  6. Replace the request URL with the URL return in the previous command, and execute the command to get prediction result from the deployed model API endpoint.

    curl -v -i \
      --header "Content-Type: application/json" \
      --request POST \
      --data '[[5.1, 3.5, 1.4, 0.2]]' \
      http://iris-classifier.default.example.com/predict
    
    [0]
    

Removing the sample app deployment

To remove the application from your cluster, delete the service record:

kubectl delete --filename service.yaml