dapr-agents/cookbook/llm/hf_chat_basic.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LLM: Hugging Face Chat Endpoint Basic Examples\n",
    "\n",
    "This notebook demonstrates how to use the `HFHubChatClient` in `dapr-agents` for basic tasks with the Hugging Face Chat API. We will explore:\n",
    "\n",
    "* Initializing the Hugging Face Chat client.\n",
    "* Generating responses to simple prompts.\n",
    "* Using a `.prompty` file to provide context/history for enhanced generation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Required Libraries\n",
    "Before starting, ensure the required libraries are installed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install dapr-agents python-dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Environment Variables\n",
    "\n",
    "Load API keys or other configuration values from your `.env` file using `dotenv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dotenv import load_dotenv\n",
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Enable Logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "\n",
    "logging.basicConfig(level=logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import HFHubChatClient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dapr_agents import HFHubChatClient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Chat Completion\n",
    "\n",
    "Initialize the `HFHubChatClient`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "llm = HFHubChatClient(\n",
    "    api_key=os.getenv(\"HUGGINGFACE_API_KEY\"),\n",
    "    model=\"microsoft/Phi-3-mini-4k-instruct\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generate a response to a simple prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
      "INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
     ]
    }
   ],
   "source": [
    "# Generate a response\n",
    "response = llm.generate('Name a famous dog!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ChatCompletion(choices=[Choice(finish_reason='stop', index=0, message=MessageContent(content='One of the most famous dogs in history is Lassie, a fictional character created by Eric Knight. Lassie is an energetic and intelligent collie dog who has appeared in many popular films, television shows, and literature. The original book, \"Lassie Come-Home\" by Eleanor Smith, was published in 1940 and tells the story of a collie pup named Lassie who crosses the English Channel to seek her brother who has been sent to live with new owners in the American West. Today, Lassie\\'s legacy continues through various adaptations and remakes, making her one of the most recognizable and cherished dog characters in entertainment history.', role='assistant'), logprobs=None)], created=1737847799, id='', model='microsoft/Phi-3-mini-4k-instruct', object='chat.completion', usage={'completion_tokens': 150, 'prompt_tokens': 8, 'total_tokens': 158})"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the response\n",
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'content': 'One of the most famous dogs in history is Lassie, a fictional character created by Eric Knight. Lassie is an energetic and intelligent collie dog who has appeared in many popular films, television shows, and literature. The original book, \"Lassie Come-Home\" by Eleanor Smith, was published in 1940 and tells the story of a collie pup named Lassie who crosses the English Channel to seek her brother who has been sent to live with new owners in the American West. Today, Lassie\\'s legacy continues through various adaptations and remakes, making her one of the most recognizable and cherished dog characters in entertainment history.',\n",
       " 'role': 'assistant'}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.get_message()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'One of the most famous dogs in history is Lassie, a fictional character created by Eric Knight. Lassie is an energetic and intelligent collie dog who has appeared in many popular films, television shows, and literature. The original book, \"Lassie Come-Home\" by Eleanor Smith, was published in 1940 and tells the story of a collie pup named Lassie who crosses the English Channel to seek her brother who has been sent to live with new owners in the American West. Today, Lassie\\'s legacy continues through various adaptations and remakes, making her one of the most recognizable and cherished dog characters in entertainment history.'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.get_content()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a Prompty File for Context\n",
    "\n",
    "Use a `.prompty` file to provide context for chat history or additional instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = HFHubChatClient.from_prompty('basic-hf-chat.prompty')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dapr_agents.llm.huggingface.chat:Using prompt template to generate messages.\n",
      "INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
      "INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "ChatCompletion(choices=[Choice(finish_reason='length', index=0, message=MessageContent(content='I\\'m Phi and Microsoft calls me Assistant; it’s a role rather than just having one specific identity like humans do with names such as \"John\" or “Jane”. My purpose revolves around assisting users in finding answers to their queries across various topics including but not limited by science fiction literature analysis! So while there isn‘t any personal \\'name\\', feel free reach out whenever need assistance navigating through complex narratives of interstellar travel tales written between 1950-23rd century Earth timeframe (or even beyond). Let us embark on this intellectual journey together today - let', role='assistant'), logprobs=None)], created=1737847810, id='', model='microsoft/Phi-3-mini-4k-instruct', object='chat.completion', usage={'completion_tokens': 128, 'prompt_tokens': 36, 'total_tokens': 164})"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.generate(input_data={\"question\":\"What is your name?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chat Completion with Messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
      "INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
     ]
    }
   ],
   "source": [
    "from dapr_agents.types import UserMessage\n",
    "\n",
    "# Initialize the client\n",
    "llm = HFHubChatClient()\n",
    "\n",
    "# Generate a response using structured messages\n",
    "response = llm.generate(messages=[UserMessage(\"hello\")])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'content': 'Hello! How can I assist you today? If you have any questions, concerns, or just want to chat, feel free! 😊', 'role': 'assistant'}\n"
     ]
    }
   ],
   "source": [
    "# Display the structured response\n",
    "print(response.get_message())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm.prompt_template"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}