dapr-agents/cookbook/llm/nvidia_chat_structured_output.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM: NVIDIA Chat Completion with Structured Output\n",
    "\n",
    "This notebook demonstrates how to use the `NVIDIAChatClient` from `dapr_agents` to generate structured output using `Pydantic` models.\n",
    "\n",
    "We will:\n",
    "\n",
    "* Initialize the `NVIDIAChatClient` with the `meta/llama-3.1-8b-instruct` model.\n",
    "* Define a Pydantic model to structure the response.\n",
    "* Use the `response_model` parameter to get structured output from the LLM."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Required Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install dapr-agents python-dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Environment Variables\n",
    "\n",
    "Load your API keys or other configuration values using `dotenv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dotenv import load_dotenv\n",
    "load_dotenv()  # Load environment variables from a `.env` file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enable Logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dapr_agents import NVIDIAChatClient\n",
    "from dapr_agents.types import UserMessage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize LLM Client\n",
    "\n",
    "Create an instance of the `NVIDIAChatClient`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dapr_agents.llm.nvidia.client:Initializing NVIDIA API client...\n"
     ]
    }
   ],
   "source": [
    "llmClient = NVIDIAChatClient(\n",
    "    model=\"meta/llama-3.1-8b-instruct\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the Pydantic Model\n",
    "\n",
    "Define a Pydantic model to represent the structured response from the LLM."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydantic import BaseModel\n",
    "\n",
    "class Dog(BaseModel):\n",
    "    name: str\n",
    "    breed: str\n",
    "    reason: str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate Structured Output (JSON)\n",
    "\n",
    "Use the generate method of the `NVIDIAChatClient` with the `response_model` parameter to enforce the structure of the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dapr_agents.llm.utils.request:A response model has been passed to structure the response of the LLM.\n",
      "INFO:dapr_agents.llm.utils.structure:Structured response enabled.\n",
      "INFO:dapr_agents.llm.nvidia.chat:Invoking ChatCompletion API.\n",
      "INFO:httpx:HTTP Request: POST https://integrate.api.nvidia.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
      "INFO:dapr_agents.llm.nvidia.chat:Chat completion retrieved successfully.\n",
      "INFO:dapr_agents.llm.utils.response:Structured output was successfully validated.\n",
      "INFO:dapr_agents.llm.utils.response:Returning an instance of <class '__main__.Dog'>.\n"
     ]
    }
   ],
   "source": [
    "response = llmClient.generate(\n",
    "    messages=[UserMessage(\"One famous dog in history.\")],\n",
    "    response_model=Dog\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dog(name='Laika', breed='Soviet space dog (mixed breeds)', reason='First animal in space')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}