mirror of https://github.com/dapr/dapr-agents.git
343 lines
9.0 KiB
Plaintext
343 lines
9.0 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# LLM: Hugging Face Chat Endpoint Basic Examples\n",
|
||
"\n",
|
||
"This notebook demonstrates how to use the `HFHubChatClient` in `dapr-agents` for basic tasks with the Hugging Face Chat API. We will explore:\n",
|
||
"\n",
|
||
"* Initializing the Hugging Face Chat client.\n",
|
||
"* Generating responses to simple prompts.\n",
|
||
"* Using a `.prompty` file to provide context/history for enhanced generation."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Install Required Libraries\n",
|
||
"Before starting, ensure the required libraries are installed:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"!pip install dapr-agents python-dotenv"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Load Environment Variables\n",
|
||
"\n",
|
||
"Load API keys or other configuration values from your `.env` file using `dotenv`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"True"
|
||
]
|
||
},
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"from dotenv import load_dotenv\n",
|
||
"load_dotenv()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Enable Logging"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import logging\n",
|
||
"\n",
|
||
"logging.basicConfig(level=logging.INFO)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Import HFHubChatClient"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"from dapr_agents import HFHubChatClient"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Basic Chat Completion\n",
|
||
"\n",
|
||
"Initialize the `HFHubChatClient`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"\n",
|
||
"llm = HFHubChatClient(\n",
|
||
" api_key=os.getenv(\"HUGGINGFACE_API_KEY\"),\n",
|
||
" model=\"microsoft/Phi-3-mini-4k-instruct\"\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Generate a response to a simple prompt"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
|
||
"INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Generate a response\n",
|
||
"response = llm.generate('Name a famous dog!')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"ChatCompletion(choices=[Choice(finish_reason='stop', index=0, message=MessageContent(content='A famous dog is Lassie. Lassie was a fictional collie first introduced in the 1943 film \"Lassie Come Home.\" She went on to have her own television series that aired from 1954 to 1973, in which she starred as Rin Tin Tin Jr. Her adventurous and heroic stories captured the hearts of audiences worldwide, and she became an iconic figure in the world of television.', role='assistant'), logprobs=None)], created=1741085108, id='', model='microsoft/Phi-3-mini-4k-instruct', object='chat.completion', usage={'completion_tokens': 105, 'prompt_tokens': 8, 'total_tokens': 113})"
|
||
]
|
||
},
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"# Display the response\n",
|
||
"response"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"{'content': 'A famous dog is Lassie. Lassie was a fictional collie first introduced in the 1943 film \"Lassie Come Home.\" She went on to have her own television series that aired from 1954 to 1973, in which she starred as Rin Tin Tin Jr. Her adventurous and heroic stories captured the hearts of audiences worldwide, and she became an iconic figure in the world of television.',\n",
|
||
" 'role': 'assistant'}"
|
||
]
|
||
},
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"response.get_message()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"'A famous dog is Lassie. Lassie was a fictional collie first introduced in the 1943 film \"Lassie Come Home.\" She went on to have her own television series that aired from 1954 to 1973, in which she starred as Rin Tin Tin Jr. Her adventurous and heroic stories captured the hearts of audiences worldwide, and she became an iconic figure in the world of television.'"
|
||
]
|
||
},
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"response.get_content()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Using a Prompty File for Context\n",
|
||
"\n",
|
||
"Use a `.prompty` file to provide context for chat history or additional instructions."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"llm = HFHubChatClient.from_prompty('basic-hf-chat.prompty')"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"INFO:dapr_agents.llm.huggingface.chat:Using prompt template to generate messages.\n",
|
||
"INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
|
||
"INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"ChatCompletion(choices=[Choice(finish_reason='length', index=0, message=MessageContent(content=\"I'm Phi and my purpose as Microsoft GPT-3 developed by MS Corporation in 2019 serves to assist users with a wide range of queries or tasks they may have at hand! How can i help today ? Let me know if there’s anything specific that comes up for which assistance would be beneficial ! :) 😊✨ #AIAssistant#MicrosoftGptPhilosophyOfHelpfulness@MSCorporationTechnologyInnovationsAndEthicsAtTheCoreofOurDesignProcessesWeStriveToCreateAnExperience\", role='assistant'), logprobs=None)], created=1741085113, id='', model='microsoft/Phi-3-mini-4k-instruct', object='chat.completion', usage={'completion_tokens': 128, 'prompt_tokens': 36, 'total_tokens': 164})"
|
||
]
|
||
},
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"llm.generate(input_data={\"question\":\"What is your name?\"})"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Chat Completion with Messages"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"INFO:dapr_agents.llm.huggingface.chat:Invoking Hugging Face ChatCompletion API.\n",
|
||
"INFO:dapr_agents.llm.huggingface.chat:Chat completion retrieved successfully.\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"from dapr_agents.types import UserMessage\n",
|
||
"\n",
|
||
"# Initialize the client\n",
|
||
"llm = HFHubChatClient()\n",
|
||
"\n",
|
||
"# Generate a response using structured messages\n",
|
||
"response = llm.generate(messages=[UserMessage(\"hello\")])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{'content': \"Hello! How can I assist you today? Whether you have a question, need help with a problem, or just want to chat, I'm here to help. 😊\", 'role': 'assistant'}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Display the structured response\n",
|
||
"print(response.get_message())"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"llm.prompt_template"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": []
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": ".venv",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.12.1"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
}
|