{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# LLM: OpenAI Embeddings Endpoint Basic Examples\n", "\n", "This notebook demonstrates how to use the `OpenAIEmbedder` in `dapr-agents` for generating text embeddings. We will explore:\n", "\n", "* Initializing the `OpenAIEmbedder`.\n", "* Generating embeddings for single and multiple inputs.\n", "* Using the class both as a direct function and via its `embed` method." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Install Required Libraries\n", "Before starting, ensure the required libraries are installed:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "!pip install dapr-agents python-dotenv tiktoken" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load Environment Variables\n", "\n", "Load API keys or other configuration values from your `.env` file using `dotenv`." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dotenv import load_dotenv\n", "load_dotenv()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Import OpenAIEmbedder" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "from dapr_agents.document.embedder import OpenAIEmbedder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Initialize the OpenAIEmbedder\n", "\n", "To start, create an instance of the `OpenAIEmbedder` class. You can customize its parameters if needed, such as the `model` or `chunk_size`." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Initialize the embedder\n", "embedder = OpenAIEmbedder(\n", " model=\"text-embedding-ada-002\", # Default embedding model\n", " chunk_size=1000, # Batch size for processing\n", " max_tokens=8191 # Maximum tokens per input\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Embedding a Single Text\n", "\n", "You can use the embed method to generate an embedding for a single input string." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Embedding (first 5 values): [0.0015723939, 0.005963983, -0.015102495, -0.008559333, -0.011583589]\n" ] } ], "source": [ "# Input text\n", "text = \"The quick brown fox jumps over the lazy dog.\"\n", "\n", "# Generate embedding\n", "embedding = embedder.embed(text)\n", "\n", "# Display the embedding\n", "print(f\"Embedding (first 5 values): {embedding[:5]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Embedding Multiple Texts\n", "\n", "The embed method also supports embedding multiple texts at once." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Text 1 embedding (first 5 values): [0.0015723939, 0.005963983, -0.015102495, -0.008559333, -0.011583589]\n", "Text 2 embedding (first 5 values): [0.03261204, -0.020966679, 0.0026475298, -0.009384127, -0.007305047]\n" ] } ], "source": [ "# Input texts\n", "texts = [\n", " \"The quick brown fox jumps over the lazy dog.\",\n", " \"A journey of a thousand miles begins with a single step.\"\n", "]\n", "\n", "# Generate embeddings\n", "embeddings = embedder.embed(texts)\n", "\n", "# Display the embeddings\n", "for i, emb in enumerate(embeddings):\n", " print(f\"Text {i + 1} embedding (first 5 values): {emb[:5]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the OpenAIEmbedder as a Callable Function\n", "\n", "The OpenAIEmbedder class can also be used directly as a function, thanks to its `__call__` implementation." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Embedding (first 5 values): [-0.0022105372, -0.022207271, 0.017802631, -0.00742872, 0.007270942]\n" ] } ], "source": [ "# Use the class instance as a callable\n", "text_embedding = embedder(\"A stitch in time saves nine.\")\n", "\n", "# Display the embedding\n", "print(f\"Embedding (first 5 values): {text_embedding[:5]}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For multiple inputs:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Text 1 embedding (first 5 values): [0.0038562817, -0.020030975, 0.01792581, -0.014723405, -0.014608578]\n", "Text 2 embedding (first 5 values): [0.011255961, 0.004331666, 0.029073123, -0.01053614, 0.021288864]\n" ] } ], "source": [ "text_list = [\"The early bird catches the worm.\", \"An apple a day keeps the doctor away.\"]\n", "embeddings_list = embedder(text_list)\n", "\n", "# Display the embeddings\n", "for i, emb in enumerate(embeddings_list):\n", " print(f\"Text {i + 1} embedding (first 5 values): {emb[:5]}\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.1" } }, "nbformat": 4, "nbformat_minor": 2 }