From 4e82d4a7f36973939ef942b4dab6f96611e49418 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 00:27:31 -0500 Subject: [PATCH 01/10] Edits to README.md and .env.docker.ui config for deploying UI + simple_calculator Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../simple_calculator/.env.docker.ui | 27 ++++++ .../functions/simple_calculator/README.md | 82 ++++++++++++++++++- .../simple_calculator/compose_calculator.yaml | 33 ++++++++ 3 files changed, 140 insertions(+), 2 deletions(-) create mode 100644 examples/basic/functions/simple_calculator/.env.docker.ui create mode 100644 examples/basic/functions/simple_calculator/compose_calculator.yaml diff --git a/examples/basic/functions/simple_calculator/.env.docker.ui b/examples/basic/functions/simple_calculator/.env.docker.ui new file mode 100644 index 000000000..e20fe2590 --- /dev/null +++ b/examples/basic/functions/simple_calculator/.env.docker.ui @@ -0,0 +1,27 @@ +# Next.js Docker Configuration for Nemo-Agent-Toolkit UI +# +# This file defines Next.js environment variables to build the Nemo-Agent-Toolkit UI. +# Next.js favors build-time environment variables for performance and security. +# Once the UI image is built, a docker compose is run to deploy both the UI with backend aiq server e.g., simple_calculator +# on the same network. + +# Note: Next.js automatically loads environment files in this priority order during the build process: +# .env.production.local (highest priority) +# .env.local +# .env.production ← This is what we'd use (this file gets renamed to .env.production) +# .env (lowest priority) + +# Note: aiq-server below corresponds to the aiq service defined in the docker compose file. +NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL=http://aiq-server:8000/chat/stream +NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL_STREAMING=http://aiq-server:8000/chat/stream +NEXT_PUBLIC_HTTP_FILE_UPLOAD_URL=http://aiq-server:8000/files/upload +NEXT_PUBLIC_HTTP_DELETE_CONVERSATION_URL=http://aiq-server:8000/conversations + +# WebSocket URLs using Docker service name +NEXT_PUBLIC_WEBSOCKET_CHAT_COMPLETION_URL=ws://aiq-server:8000/websocket +NEXT_PUBLIC_WS_CHAT_COMPLETION_URL=ws://aiq-server:8000/chat/stream + +# Default feature flags +NEXT_PUBLIC_CHAT_HISTORY_DEFAULT_ON=true +NEXT_PUBLIC_WEB_SOCKET_DEFAULT_ON=false +NEXT_PUBLIC_ENABLE_INTERMEDIATE_STEPS=true diff --git a/examples/basic/functions/simple_calculator/README.md b/examples/basic/functions/simple_calculator/README.md index 7e26b2155..be93d8a11 100644 --- a/examples/basic/functions/simple_calculator/README.md +++ b/examples/basic/functions/simple_calculator/README.md @@ -33,6 +33,7 @@ This example demonstrates an end-to-end (E2E) agentic workflow using the AIQ too - [Run the Docker Container](#run-the-docker-container) - [Test the API](#test-the-api) - [Expected API Output](#expected-api-output) + - [Deployment with UI](#deployment-with-ui) --- @@ -171,7 +172,7 @@ For a production deployment, use Docker: Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the AIQ toolkit virtual environment. -From the root directory of the Simple Calculator repository, build the Docker image: +From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: ```bash docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/basic/functions/simple_calculator/Dockerfile . @@ -181,7 +182,7 @@ docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calcu Deploy the container: ```bash -docker run -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator +docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator ``` Note, a phoenix telemetry service will be exposed at port 6006. @@ -206,3 +207,80 @@ The API response should be similar to the following: "output": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." } ``` + +To stop and remove the container: + +```bash +docker stop simple_calculator_container +docker rm simple_calculator_container +``` + +### Deployment with UI + +The example can be run with the NeMo-Agent-Toolkit UI. After building the simple_calculator image previously, this will build the UI docker image, and deploy both with network connectivity via a docker compose file. + +**Prerequisites:** +- You must have built the `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) + +#### Build UI Docker Image + +```bash +export WORKPATH=~/Projects # Set this to the parent directory containing NeMo-Agent-Toolkit + +git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit-UI.git $WORKPATH/NeMo-Agent-Toolkit-UI + +cd $WORKPATH/NeMo-Agent-Toolkit-UI + +# Set Next.js build environment variables for UI +cp $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator/.env.docker.ui .env.production + +docker build -t aiqtoolkit-ui . +``` + +#### Deploy Simple LLM Calculator + UI + +After building the UI image above use docker compose to deploy: + +```bash +cd $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator + +docker compose -f compose_calculator.yaml up -d +``` + +This will: +- Build and start the calculator server on port 8000 +- Build and start the web UI on port 3000 +- Set Phoenix telemetry on port 6006 +- Create a secure bridge network for container communication + +#### Verify Deployment + +1. **Check service status:** + ```bash + docker compose ps -a + ``` + +2. **View logs:** + ```bash + docker logs aiq-server + docker logs aiq-ui + ``` + +3. **Access the services:** + - Web UI: http://localhost:3000 + - API Documentation: http://localhost:8000/docs + - Phoenix Telemetry: http://localhost:6006 + +Interact with the chat interface by prompting the agent with the message: +``` +Is 4 + 4 greater than the current hour of the day? +``` + +![AIQ Toolkit Web UI Workflow Result](public/screenshots/ui_generate_example.png) + +#### Stop the Services + +To stop all services: +```bash +docker compose -f compose_calculator.yaml down +``` \ No newline at end of file diff --git a/examples/basic/functions/simple_calculator/compose_calculator.yaml b/examples/basic/functions/simple_calculator/compose_calculator.yaml new file mode 100644 index 000000000..d60d5ed52 --- /dev/null +++ b/examples/basic/functions/simple_calculator/compose_calculator.yaml @@ -0,0 +1,33 @@ +services: + + aiq-server: + image: simple_calculator + container_name: aiq-calculator-server + ports: + - "8000:8000" + - "6006:6006" # Phoenix telemetry service + environment: + - NVIDIA_API_KEY=${NVIDIA_API_KEY} + networks: + - aiq-network + healthcheck: + test: ["CMD-SHELL", "python -c 'import urllib.request; urllib.request.urlopen(\"http://localhost:8000/docs\")' || exit 1"] + interval: 30s + timeout: 10s + retries: 3 + start_period: 40s + + aiq-ui: + image: aiqtoolkit-ui + container_name: aiq-ui + ports: + - "3000:3000" + networks: + - aiq-network + depends_on: + aiq-server: + condition: service_healthy + +networks: + aiq-network: + driver: bridge From 24c5626ec4caf6747e83b3423bef186e165fd708 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 01:04:41 -0500 Subject: [PATCH 02/10] Small comment edits in .env.docker.ui Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../basic/functions/simple_calculator/.env.docker.ui | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/examples/basic/functions/simple_calculator/.env.docker.ui b/examples/basic/functions/simple_calculator/.env.docker.ui index e20fe2590..031fa4ba3 100644 --- a/examples/basic/functions/simple_calculator/.env.docker.ui +++ b/examples/basic/functions/simple_calculator/.env.docker.ui @@ -1,10 +1,11 @@ # Next.js Docker Configuration for Nemo-Agent-Toolkit UI # -# This file defines Next.js environment variables to build the Nemo-Agent-Toolkit UI. -# Next.js favors build-time environment variables for performance and security. -# Once the UI image is built, a docker compose is run to deploy both the UI with backend aiq server e.g., simple_calculator -# on the same network. - +# Please see the README.md "Build UI Docker Image" to see full usage. This file defines Next.js build-time environment variables to +# connect the agent server with the Nemo-Agent-Toolkit UI. Next.js favors build-time environment variables for performance and security. +# +# Once the UI image is built, a docker compose is run to deploy both the UI + agent server e.g., simple_calculator +# on the same docker network. +# # Note: Next.js automatically loads environment files in this priority order during the build process: # .env.production.local (highest priority) # .env.local From 3789d9391b49f0672aa9df867a34946423c72555 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 08:12:52 -0500 Subject: [PATCH 03/10] Updated simple_calculator README.md to provide instructions for NeMo-Agent-Toolkit-UI Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../simple_calculator/README.md | 226 +++++++++++++++--- 1 file changed, 194 insertions(+), 32 deletions(-) diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index 9f9f804f3..be93d8a11 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -17,65 +17,150 @@ limitations under the License. # A Simple LLM Calculator -This example demonstrates an end-to-end (E2E) agentic workflow using the NeMo Agent toolkit library, fully configured through a YAML file. It showcases the NeMo Agent toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. +This example demonstrates an end-to-end (E2E) agentic workflow using the AIQ toolkit library, fully configured through a YAML file. It showcases the AIQ toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. ## Table of Contents -- [Key Features](#key-features) -- [Installation and Setup](#installation-and-setup) - - [Install this Workflow:](#install-this-workflow) - - [Set Up API Keys](#set-up-api-keys) - - [Run the Workflow](#run-the-workflow) -- [Deployment-Oriented Setup](#deployment-oriented-setup) - - [Build the Docker Image](#build-the-docker-image) - - [Run the Docker Container](#run-the-docker-container) - - [Test the API](#test-the-api) - - [Expected API Output](#expected-api-output) +- [A Simple LLM Calculator](#a-simple-llm-calculator) + - [Table of Contents](#table-of-contents) + - [Key Features](#key-features) + - [Installation and Setup](#installation-and-setup) + - [Install this Workflow:](#install-this-workflow) + - [Set Up API Keys](#set-up-api-keys) + - [Run the Workflow](#run-the-workflow) + - [Deployment-Oriented Setup](#deployment-oriented-setup) + - [Build the Docker Image](#build-the-docker-image) + - [Run the Docker Container](#run-the-docker-container) + - [Test the API](#test-the-api) + - [Expected API Output](#expected-api-output) + - [Deployment with UI](#deployment-with-ui) + --- ## Key Features -- **Custom Calculator Tools:** Demonstrates five mathematical tools - `calculator_multiply`, `calculator_inequality`, `calculator_divide`, `calculator_subtract`, and `current_datetime` for mathematical operations and time-based comparisons. -- **ReAct Agent Integration:** Uses a `react_agent` that performs reasoning between tool calls to solve complex mathematical queries requiring multiple steps. -- **Multi-step Problem Solving:** Shows how an agent can break down complex questions like "Is the product of 2 * 4 greater than the current hour?" into sequential tool calls. -- **Custom Function Registration:** Demonstrates the NeMo Agent toolkit plugin system for registering custom mathematical functions with proper validation and error handling. -- **YAML-based Configuration:** Fully configurable workflow that showcases how to orchestrate multiple tools through simple configuration. +- **Pre-built Tools:** Leverages core AIQ toolkit library tools. +- **Custom Plugin System:** Developers can bring in new tools using plugins. +- **High-level API:** Enables defining functions that transform into asynchronous LangChain tools. +- **Agentic Workflows:** Fully configurable via YAML for flexibility and productivity. +- **Ease of Use:** Simplifies developer experience and deployment. --- ## Installation and Setup -If you have not already done so, follow the instructions in the [Install Guide](../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install NeMo Agent toolkit. +If you have not already done so, follow the instructions in the [Install Guide](../../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install AIQ toolkit. ### Install this Workflow: -From the root directory of the NeMo Agent toolkit library, run the following commands: +From the root directory of the AIQ toolkit library, run the following commands: ```bash -uv pip install -e examples/getting_started/simple_calculator +uv pip install -e examples/basic/functions/simple_calculator ``` ### Set Up API Keys -If you have not already done so, follow the [Obtaining API Keys](../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: +If you have not already done so, follow the [Obtaining API Keys](../../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: ```bash export NVIDIA_API_KEY= -export OPENAI_API_KEY= # OPTIONAL ``` ### Run the Workflow -Return to your original terminal, and run the following command from the root of the NeMo Agent toolkit repo to execute this workflow with the specified input: +Return to your original terminal, and run the following command from the root of the AIQ toolkit repo to execute this workflow with the specified input: ```bash -aiq run --config_file examples/getting_started/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" +aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" ``` -**Expected Workflow Output** -Note that the output is subject to the time of day when the workflow was run. For this example output, it was run in the afternoon. -``` -No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). +**Expected Output** +The workflow output can be quite lengthy, the end of the workflow output should contain something similar to the following (the final answer will depend on the time of day the workflow is run): +```console +$ aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" +2025-04-23 15:58:34,877 - aiq.runtime.loader - WARNING - Loading module 'aiq_automated_description_generation.register' from entry point 'aiq_automated_description_generation' took a long time (440.151215 ms). Ensure all imports are inside your registered functions. +2025-04-23 15:58:35,193 - aiq.cli.commands.start - INFO - Starting AIQ toolkit from config file: 'examples/basic/functions/simple_calculator/configs/config.yml' +2025-04-23 15:58:35,199 - aiq.cli.commands.start - WARNING - The front end type in the config file (fastapi) does not match the command name (console). Overwriting the config file front end. + +Configuration Summary: +-------------------- +Workflow Type: react_agent +Number of Functions: 5 +Number of LLMs: 2 +Number of Embedders: 0 +Number of Memory: 0 +Number of Retrievers: 0 + +2025-04-23 15:58:36,674 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Agent input: Is the product of 2 * 4 greater than the current hour of the day? +Agent's thoughts: +Thought: To answer this question, I need to calculate the product of 2 and 4, and then compare it to the current hour of the day. + +Action: calculator_multiply +Action Input: {'text': '2 * 4'} + + +------------------------------ +2025-04-23 15:58:36,682 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Calling tools: calculator_multiply +Tool's input: {"text": "2 * 4"} +Tool's response: +The product of 2 * 4 is 8 +------------------------------ +2025-04-23 15:58:37,704 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Agent input: Is the product of 2 * 4 greater than the current hour of the day? +Agent's thoughts: +Thought: Now that I have the product of 2 and 4, I need to get the current hour of the day to compare it with the product. + +Action: current_datetime +Action Input: None +------------------------------ +2025-04-23 15:58:37,710 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Calling tools: current_datetime +Tool's input: None +Tool's response: +The current time of day is 2025-04-23 15:58:37 +------------------------------ +2025-04-23 15:58:38,865 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Agent input: Is the product of 2 * 4 greater than the current hour of the day? +Agent's thoughts: +Thought: Now that I have the current time of day, I can extract the hour and compare it with the product of 2 and 4. + +Action: calculator_inequality +Action Input: {'text': '8 > 15'} +------------------------------ +2025-04-23 15:58:38,871 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Calling tools: calculator_inequality +Tool's input: {"text": "8 > 15"} +Tool's response: +First number 8 is less than the second number 15 +------------------------------ +2025-04-23 15:58:39,978 - aiq.agent.react_agent.agent - INFO - +------------------------------ +[AGENT] +Agent input: Is the product of 2 * 4 greater than the current hour of the day? +Agent's thoughts: +Thought: I now know the final answer + +Final Answer: No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). +------------------------------ +2025-04-23 15:58:39,981 - aiq.front_ends.console.console_front_end_plugin - INFO - +-------------------------------------------------- +Workflow Result: +['No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15).'] ``` @@ -85,19 +170,19 @@ For a production deployment, use Docker: ### Build the Docker Image -Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the NeMo Agent toolkit virtual environment. +Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the AIQ toolkit virtual environment. -From the root directory of the Simple Calculator repository, build the Docker image: +From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: ```bash -docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/getting_started/simple_calculator/Dockerfile . +docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/basic/functions/simple_calculator/Dockerfile . ``` ### Run the Docker Container Deploy the container: ```bash -docker run -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY -e OPENAI_API_KEY simple_calculator +docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator ``` Note, a phoenix telemetry service will be exposed at port 6006. @@ -119,6 +204,83 @@ The API response should be similar to the following: ```bash { "input": "Is the product of 2 * 4 greater than the current hour of the day?", - "value": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." + "output": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." } ``` + +To stop and remove the container: + +```bash +docker stop simple_calculator_container +docker rm simple_calculator_container +``` + +### Deployment with UI + +The example can be run with the NeMo-Agent-Toolkit UI. After building the simple_calculator image previously, this will build the UI docker image, and deploy both with network connectivity via a docker compose file. + +**Prerequisites:** +- You must have built the `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) + +#### Build UI Docker Image + +```bash +export WORKPATH=~/Projects # Set this to the parent directory containing NeMo-Agent-Toolkit + +git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit-UI.git $WORKPATH/NeMo-Agent-Toolkit-UI + +cd $WORKPATH/NeMo-Agent-Toolkit-UI + +# Set Next.js build environment variables for UI +cp $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator/.env.docker.ui .env.production + +docker build -t aiqtoolkit-ui . +``` + +#### Deploy Simple LLM Calculator + UI + +After building the UI image above use docker compose to deploy: + +```bash +cd $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator + +docker compose -f compose_calculator.yaml up -d +``` + +This will: +- Build and start the calculator server on port 8000 +- Build and start the web UI on port 3000 +- Set Phoenix telemetry on port 6006 +- Create a secure bridge network for container communication + +#### Verify Deployment + +1. **Check service status:** + ```bash + docker compose ps -a + ``` + +2. **View logs:** + ```bash + docker logs aiq-server + docker logs aiq-ui + ``` + +3. **Access the services:** + - Web UI: http://localhost:3000 + - API Documentation: http://localhost:8000/docs + - Phoenix Telemetry: http://localhost:6006 + +Interact with the chat interface by prompting the agent with the message: +``` +Is 4 + 4 greater than the current hour of the day? +``` + +![AIQ Toolkit Web UI Workflow Result](public/screenshots/ui_generate_example.png) + +#### Stop the Services + +To stop all services: +```bash +docker compose -f compose_calculator.yaml down +``` \ No newline at end of file From 1b2901ddcca57b01e371a3da63405cd4360c5ac3 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 08:16:27 -0500 Subject: [PATCH 04/10] Reverting simple_web_query README.md after accidently overwriting Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../simple_web_query/README.md | 262 ++++-------------- 1 file changed, 47 insertions(+), 215 deletions(-) diff --git a/examples/getting_started/simple_web_query/README.md b/examples/getting_started/simple_web_query/README.md index be93d8a11..8f5bf7aa3 100644 --- a/examples/getting_started/simple_web_query/README.md +++ b/examples/getting_started/simple_web_query/README.md @@ -15,272 +15,104 @@ See the License for the specific language governing permissions and limitations under the License. --> -# A Simple LLM Calculator +# A Simple LangSmith-Documentation Agent -This example demonstrates an end-to-end (E2E) agentic workflow using the AIQ toolkit library, fully configured through a YAML file. It showcases the AIQ toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. +A minimal example demonstrating a simple LangSmith-Documentation agent. This agent leverages the NeMo Agent toolkit plugin system and `Builder` to integrate pre-built and custom tools into the workflow to answer questions about LangSmith. Key elements are summarized below: ## Table of Contents -- [A Simple LLM Calculator](#a-simple-llm-calculator) - - [Table of Contents](#table-of-contents) - - [Key Features](#key-features) - - [Installation and Setup](#installation-and-setup) - - [Install this Workflow:](#install-this-workflow) - - [Set Up API Keys](#set-up-api-keys) - - [Run the Workflow](#run-the-workflow) - - [Deployment-Oriented Setup](#deployment-oriented-setup) - - [Build the Docker Image](#build-the-docker-image) - - [Run the Docker Container](#run-the-docker-container) - - [Test the API](#test-the-api) - - [Expected API Output](#expected-api-output) - - [Deployment with UI](#deployment-with-ui) - +* [Key Features](#key-features) +* [Prerequisites](#prerequisites) +* [Installation and Setup](#installation-and-setup) +* [Running the Workflow](#running-the-workflow) +* [Deployment-Oriented Setup](#docker-quickstart) --- ## Key Features -- **Pre-built Tools:** Leverages core AIQ toolkit library tools. -- **Custom Plugin System:** Developers can bring in new tools using plugins. -- **High-level API:** Enables defining functions that transform into asynchronous LangChain tools. -- **Agentic Workflows:** Fully configurable via YAML for flexibility and productivity. -- **Ease of Use:** Simplifies developer experience and deployment. +- **Webpage Query Tool:** Demonstrates a `webpage_query` tool that retrieves and processes documentation from LangSmith's website (https://docs.smith.langchain.com) using web scraping and vector search. +- **ReAct Agent Integration:** Uses a `react_agent` that reasons about user queries and determines when to retrieve relevant documentation from the web. +- **Document Retrieval and Embedding:** Shows how to automatically generate embeddings from web content and perform semantic search to answer questions about LangSmith. +- **End-to-End Web RAG:** Complete example of Retrieval-Augmented Generation (RAG) using web-scraped content as the knowledge source. +- **YAML-based Configuration:** Fully configurable workflow demonstrating integration of web scraping, embeddings, and agent reasoning through simple configuration. ---- +## Prerequisites + +Ensure that Docker is installed and the Docker service is running before proceeding. + +- Install Docker: Follow the official installation guide for your platform: [Docker Installation Guide](https://docs.docker.com/engine/install/) +- Start Docker Service: + - Linux: Run`sudo systemctl start docker` (ensure your user has permission to run Docker). + - Mac & Windows: Docker Desktop should be running in the background. +- Verify Docker Installation: Run the following command to verify that Docker is installed and running correctly: +```bash +docker info +``` ## Installation and Setup -If you have not already done so, follow the instructions in the [Install Guide](../../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install AIQ toolkit. +If you have not already done so, follow the instructions in the [Install Guide](../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install NeMo Agent toolkit. ### Install this Workflow: -From the root directory of the AIQ toolkit library, run the following commands: +From the root directory of the NeMo Agent toolkit library, run the following commands: ```bash -uv pip install -e examples/basic/functions/simple_calculator +uv pip install -e examples/getting_started/simple_web_query ``` ### Set Up API Keys -If you have not already done so, follow the [Obtaining API Keys](../../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: +If you have not already done so, follow the [Obtaining API Keys](../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: ```bash export NVIDIA_API_KEY= ``` -### Run the Workflow +## Running the Workflow -Return to your original terminal, and run the following command from the root of the AIQ toolkit repo to execute this workflow with the specified input: +Run the following command from the root of the NeMo Agent toolkit repo to execute this workflow with the specified input: ```bash -aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" +aiq run --config_file examples/getting_started/simple_web_query/configs/config.yml --input "What is LangSmith?" ``` -**Expected Output** -The workflow output can be quite lengthy, the end of the workflow output should contain something similar to the following (the final answer will depend on the time of day the workflow is run): +**Expected Workflow Output** ```console -$ aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" -2025-04-23 15:58:34,877 - aiq.runtime.loader - WARNING - Loading module 'aiq_automated_description_generation.register' from entry point 'aiq_automated_description_generation' took a long time (440.151215 ms). Ensure all imports are inside your registered functions. -2025-04-23 15:58:35,193 - aiq.cli.commands.start - INFO - Starting AIQ toolkit from config file: 'examples/basic/functions/simple_calculator/configs/config.yml' -2025-04-23 15:58:35,199 - aiq.cli.commands.start - WARNING - The front end type in the config file (fastapi) does not match the command name (console). Overwriting the config file front end. - -Configuration Summary: --------------------- -Workflow Type: react_agent -Number of Functions: 5 -Number of LLMs: 2 -Number of Embedders: 0 -Number of Memory: 0 -Number of Retrievers: 0 - -2025-04-23 15:58:36,674 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: To answer this question, I need to calculate the product of 2 and 4, and then compare it to the current hour of the day. - -Action: calculator_multiply -Action Input: {'text': '2 * 4'} - - ------------------------------- -2025-04-23 15:58:36,682 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_multiply -Tool's input: {"text": "2 * 4"} -Tool's response: -The product of 2 * 4 is 8 ------------------------------- -2025-04-23 15:58:37,704 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the product of 2 and 4, I need to get the current hour of the day to compare it with the product. - -Action: current_datetime -Action Input: None ------------------------------- -2025-04-23 15:58:37,710 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: current_datetime -Tool's input: None -Tool's response: -The current time of day is 2025-04-23 15:58:37 ------------------------------- -2025-04-23 15:58:38,865 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the current time of day, I can extract the hour and compare it with the product of 2 and 4. - -Action: calculator_inequality -Action Input: {'text': '8 > 15'} ------------------------------- -2025-04-23 15:58:38,871 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_inequality -Tool's input: {"text": "8 > 15"} -Tool's response: -First number 8 is less than the second number 15 ------------------------------- -2025-04-23 15:58:39,978 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: I now know the final answer - -Final Answer: No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). ------------------------------- -2025-04-23 15:58:39,981 - aiq.front_ends.console.console_front_end_plugin - INFO - --------------------------------------------------- + + Workflow Result: -['No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15).'] +['LangSmith is a platform for building production-grade LLM (Large Language Model) applications, allowing users to monitor and evaluate their applications, and providing features such as observability, evaluation, and prompt engineering. It is framework-agnostic and can be used with or without LangChain's open source frameworks.'] ``` +## Docker Quickstart -## Deployment-Oriented Setup - -For a production deployment, use Docker: +Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the NeMo Agent toolkit virtual environment. -### Build the Docker Image +Set your NVIDIA API Key in the `NVIDIA_API_KEY` environment variable. -Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the AIQ toolkit virtual environment. +```bash +export NVIDIA_API_KEY="your_nvidia_api_key" +``` -From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: +From the git repository root, run the following command to build NeMo Agent toolkit and the simple agent into a Docker image. ```bash -docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/basic/functions/simple_calculator/Dockerfile . +docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -f examples/getting_started/simple_web_query/Dockerfile -t simple-web-query-agent . ``` -### Run the Docker Container -Deploy the container: +Then, run the following command to run the simple agent. ```bash -docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator +docker run -p 8000:8000 -e NVIDIA_API_KEY simple-web-query-agent ``` -Note, a phoenix telemetry service will be exposed at port 6006. - -### Test the API -Use the following curl command to test the deployed API: +After the container starts, you can access the agent at http://localhost:8000. ```bash curl -X 'POST' \ 'http://localhost:8000/generate' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ - -d '{"input_message": "Is the product of 2 * 4 greater than the current hour of the day?"}' -``` - -### Expected API Output -The API response should be similar to the following: - -```bash -{ - "input": "Is the product of 2 * 4 greater than the current hour of the day?", - "output": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." -} -``` - -To stop and remove the container: - -```bash -docker stop simple_calculator_container -docker rm simple_calculator_container -``` - -### Deployment with UI - -The example can be run with the NeMo-Agent-Toolkit UI. After building the simple_calculator image previously, this will build the UI docker image, and deploy both with network connectivity via a docker compose file. - -**Prerequisites:** -- You must have built the `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) - -#### Build UI Docker Image - -```bash -export WORKPATH=~/Projects # Set this to the parent directory containing NeMo-Agent-Toolkit - -git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit-UI.git $WORKPATH/NeMo-Agent-Toolkit-UI - -cd $WORKPATH/NeMo-Agent-Toolkit-UI - -# Set Next.js build environment variables for UI -cp $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator/.env.docker.ui .env.production - -docker build -t aiqtoolkit-ui . -``` - -#### Deploy Simple LLM Calculator + UI - -After building the UI image above use docker compose to deploy: - -```bash -cd $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator - -docker compose -f compose_calculator.yaml up -d + -d '{"input_message": "What is LangSmith?"}' ``` - -This will: -- Build and start the calculator server on port 8000 -- Build and start the web UI on port 3000 -- Set Phoenix telemetry on port 6006 -- Create a secure bridge network for container communication - -#### Verify Deployment - -1. **Check service status:** - ```bash - docker compose ps -a - ``` - -2. **View logs:** - ```bash - docker logs aiq-server - docker logs aiq-ui - ``` - -3. **Access the services:** - - Web UI: http://localhost:3000 - - API Documentation: http://localhost:8000/docs - - Phoenix Telemetry: http://localhost:6006 - -Interact with the chat interface by prompting the agent with the message: -``` -Is 4 + 4 greater than the current hour of the day? -``` - -![AIQ Toolkit Web UI Workflow Result](public/screenshots/ui_generate_example.png) - -#### Stop the Services - -To stop all services: -```bash -docker compose -f compose_calculator.yaml down -``` \ No newline at end of file From 2a499de5bfc65cff4a9ac3c563942884e2199996 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 09:02:16 -0500 Subject: [PATCH 05/10] Updating simple_calculator README.md to have UI instructions Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../simple_calculator/README.md | 151 ++++-------------- 1 file changed, 33 insertions(+), 118 deletions(-) diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index be93d8a11..6ddba82de 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -17,150 +17,65 @@ limitations under the License. # A Simple LLM Calculator -This example demonstrates an end-to-end (E2E) agentic workflow using the AIQ toolkit library, fully configured through a YAML file. It showcases the AIQ toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. +This example demonstrates an end-to-end (E2E) agentic workflow using the NeMo Agent toolkit library, fully configured through a YAML file. It showcases the NeMo Agent toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. ## Table of Contents -- [A Simple LLM Calculator](#a-simple-llm-calculator) - - [Table of Contents](#table-of-contents) - - [Key Features](#key-features) - - [Installation and Setup](#installation-and-setup) - - [Install this Workflow:](#install-this-workflow) - - [Set Up API Keys](#set-up-api-keys) - - [Run the Workflow](#run-the-workflow) - - [Deployment-Oriented Setup](#deployment-oriented-setup) - - [Build the Docker Image](#build-the-docker-image) - - [Run the Docker Container](#run-the-docker-container) - - [Test the API](#test-the-api) - - [Expected API Output](#expected-api-output) - - [Deployment with UI](#deployment-with-ui) - +- [Key Features](#key-features) +- [Installation and Setup](#installation-and-setup) + - [Install this Workflow:](#install-this-workflow) + - [Set Up API Keys](#set-up-api-keys) + - [Run the Workflow](#run-the-workflow) +- [Deployment-Oriented Setup](#deployment-oriented-setup) + - [Build the Docker Image](#build-the-docker-image) + - [Run the Docker Container](#run-the-docker-container) + - [Test the API](#test-the-api) + - [Expected API Output](#expected-api-output) + - [Deployment with UI](#deployment-with-ui) --- ## Key Features -- **Pre-built Tools:** Leverages core AIQ toolkit library tools. -- **Custom Plugin System:** Developers can bring in new tools using plugins. -- **High-level API:** Enables defining functions that transform into asynchronous LangChain tools. -- **Agentic Workflows:** Fully configurable via YAML for flexibility and productivity. -- **Ease of Use:** Simplifies developer experience and deployment. +- **Custom Calculator Tools:** Demonstrates five mathematical tools - `calculator_multiply`, `calculator_inequality`, `calculator_divide`, `calculator_subtract`, and `current_datetime` for mathematical operations and time-based comparisons. +- **ReAct Agent Integration:** Uses a `react_agent` that performs reasoning between tool calls to solve complex mathematical queries requiring multiple steps. +- **Multi-step Problem Solving:** Shows how an agent can break down complex questions like "Is the product of 2 * 4 greater than the current hour?" into sequential tool calls. +- **Custom Function Registration:** Demonstrates the NeMo Agent toolkit plugin system for registering custom mathematical functions with proper validation and error handling. +- **YAML-based Configuration:** Fully configurable workflow that showcases how to orchestrate multiple tools through simple configuration. --- ## Installation and Setup -If you have not already done so, follow the instructions in the [Install Guide](../../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install AIQ toolkit. +If you have not already done so, follow the instructions in the [Install Guide](../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install NeMo Agent toolkit. ### Install this Workflow: -From the root directory of the AIQ toolkit library, run the following commands: +From the root directory of the NeMo Agent toolkit library, run the following commands: ```bash -uv pip install -e examples/basic/functions/simple_calculator +uv pip install -e examples/getting_started/simple_calculator ``` ### Set Up API Keys -If you have not already done so, follow the [Obtaining API Keys](../../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: +If you have not already done so, follow the [Obtaining API Keys](../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: ```bash -export NVIDIA_API_KEY= +export OPENAI_API_KEY= # OPTIONAL ``` ### Run the Workflow -Return to your original terminal, and run the following command from the root of the AIQ toolkit repo to execute this workflow with the specified input: +Return to your original terminal, and run the following command from the root of the NeMo Agent toolkit repo to execute this workflow with the specified input: ```bash -aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" +aiq run --config_file examples/getting_started/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" ``` -**Expected Output** -The workflow output can be quite lengthy, the end of the workflow output should contain something similar to the following (the final answer will depend on the time of day the workflow is run): -```console -$ aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" -2025-04-23 15:58:34,877 - aiq.runtime.loader - WARNING - Loading module 'aiq_automated_description_generation.register' from entry point 'aiq_automated_description_generation' took a long time (440.151215 ms). Ensure all imports are inside your registered functions. -2025-04-23 15:58:35,193 - aiq.cli.commands.start - INFO - Starting AIQ toolkit from config file: 'examples/basic/functions/simple_calculator/configs/config.yml' -2025-04-23 15:58:35,199 - aiq.cli.commands.start - WARNING - The front end type in the config file (fastapi) does not match the command name (console). Overwriting the config file front end. - -Configuration Summary: --------------------- -Workflow Type: react_agent -Number of Functions: 5 -Number of LLMs: 2 -Number of Embedders: 0 -Number of Memory: 0 -Number of Retrievers: 0 - -2025-04-23 15:58:36,674 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: To answer this question, I need to calculate the product of 2 and 4, and then compare it to the current hour of the day. - -Action: calculator_multiply -Action Input: {'text': '2 * 4'} - - ------------------------------- -2025-04-23 15:58:36,682 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_multiply -Tool's input: {"text": "2 * 4"} -Tool's response: -The product of 2 * 4 is 8 ------------------------------- -2025-04-23 15:58:37,704 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the product of 2 and 4, I need to get the current hour of the day to compare it with the product. - -Action: current_datetime -Action Input: None ------------------------------- -2025-04-23 15:58:37,710 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: current_datetime -Tool's input: None -Tool's response: -The current time of day is 2025-04-23 15:58:37 ------------------------------- -2025-04-23 15:58:38,865 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the current time of day, I can extract the hour and compare it with the product of 2 and 4. - -Action: calculator_inequality -Action Input: {'text': '8 > 15'} ------------------------------- -2025-04-23 15:58:38,871 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_inequality -Tool's input: {"text": "8 > 15"} -Tool's response: -First number 8 is less than the second number 15 ------------------------------- -2025-04-23 15:58:39,978 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: I now know the final answer - -Final Answer: No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). ------------------------------- -2025-04-23 15:58:39,981 - aiq.front_ends.console.console_front_end_plugin - INFO - --------------------------------------------------- -Workflow Result: -['No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15).'] +**Expected Workflow Output** +Note that the output is subject to the time of day when the workflow was run. For this example output, it was run in the afternoon. +``` +No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). ``` @@ -170,19 +85,19 @@ For a production deployment, use Docker: ### Build the Docker Image -Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the AIQ toolkit virtual environment. +Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the NeMo Agent toolkit virtual environment. From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: ```bash -docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/basic/functions/simple_calculator/Dockerfile . +docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/getting_started/simple_calculator Dockerfile . ``` ### Run the Docker Container Deploy the container: ```bash -docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator +docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e OPENAI_API_KEY simple_calculator ``` Note, a phoenix telemetry service will be exposed at port 6006. @@ -204,7 +119,7 @@ The API response should be similar to the following: ```bash { "input": "Is the product of 2 * 4 greater than the current hour of the day?", - "output": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." + "value": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." } ``` @@ -232,7 +147,7 @@ git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit-UI.git $WORKPATH/NeMo-Agent-T cd $WORKPATH/NeMo-Agent-Toolkit-UI # Set Next.js build environment variables for UI -cp $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator/.env.docker.ui .env.production +cp $WORKPATH/NeMo-Agent-Toolkit/examples/getting_started/simple_calculator/.env.docker.ui .env.production docker build -t aiqtoolkit-ui . ``` From b2895f1d3c7b79d26e8c8fe61158dc467c734a09 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 09:41:37 -0500 Subject: [PATCH 06/10] Made adjustment to not use aiq naming for docker compose ui deployment Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../simple_calculator/.env.docker.ui | 14 ++++++------- .../simple_calculator/README.md | 12 +++++------ .../simple_calculator/compose_calculator.yaml | 21 +++++++++---------- 3 files changed, 23 insertions(+), 24 deletions(-) diff --git a/examples/getting_started/simple_calculator/.env.docker.ui b/examples/getting_started/simple_calculator/.env.docker.ui index 031fa4ba3..5acdc2141 100644 --- a/examples/getting_started/simple_calculator/.env.docker.ui +++ b/examples/getting_started/simple_calculator/.env.docker.ui @@ -12,15 +12,15 @@ # .env.production ← This is what we'd use (this file gets renamed to .env.production) # .env (lowest priority) -# Note: aiq-server below corresponds to the aiq service defined in the docker compose file. -NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL=http://aiq-server:8000/chat/stream -NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL_STREAMING=http://aiq-server:8000/chat/stream -NEXT_PUBLIC_HTTP_FILE_UPLOAD_URL=http://aiq-server:8000/files/upload -NEXT_PUBLIC_HTTP_DELETE_CONVERSATION_URL=http://aiq-server:8000/conversations +# Note: nemoagenttoolkit-server below corresponds to the service defined in the docker compose file. +NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL=http://nemoagenttoolkit-server:8000/generate +NEXT_PUBLIC_HTTP_CHAT_COMPLETION_URL_STREAMING=http://nemoagenttoolkit-server:8000/chat/stream +NEXT_PUBLIC_HTTP_FILE_UPLOAD_URL=http://nemoagenttoolkit-server:8000/files/upload +NEXT_PUBLIC_HTTP_DELETE_CONVERSATION_URL=http://nemoagenttoolkit-server:8000/conversations # WebSocket URLs using Docker service name -NEXT_PUBLIC_WEBSOCKET_CHAT_COMPLETION_URL=ws://aiq-server:8000/websocket -NEXT_PUBLIC_WS_CHAT_COMPLETION_URL=ws://aiq-server:8000/chat/stream +NEXT_PUBLIC_WEBSOCKET_CHAT_COMPLETION_URL=ws://nemoagenttoolkit-server:8000/websocket +NEXT_PUBLIC_WS_CHAT_COMPLETION_URL=ws://nemoagenttoolkit-server:8000/chat/stream # Default feature flags NEXT_PUBLIC_CHAT_HISTORY_DEFAULT_ON=true diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index 6ddba82de..2b2859824 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -135,7 +135,7 @@ docker rm simple_calculator_container The example can be run with the NeMo-Agent-Toolkit UI. After building the simple_calculator image previously, this will build the UI docker image, and deploy both with network connectivity via a docker compose file. **Prerequisites:** -- You must have built the `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) +- You must have the built `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) #### Build UI Docker Image @@ -149,7 +149,7 @@ cd $WORKPATH/NeMo-Agent-Toolkit-UI # Set Next.js build environment variables for UI cp $WORKPATH/NeMo-Agent-Toolkit/examples/getting_started/simple_calculator/.env.docker.ui .env.production -docker build -t aiqtoolkit-ui . +docker build -t nemoagenttoolkit-ui . ``` #### Deploy Simple LLM Calculator + UI @@ -157,7 +157,7 @@ docker build -t aiqtoolkit-ui . After building the UI image above use docker compose to deploy: ```bash -cd $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator +cd $WORKPATH/NeMo-Agent-Toolkit/examples/getting_started/simple_calculator docker compose -f compose_calculator.yaml up -d ``` @@ -177,8 +177,8 @@ This will: 2. **View logs:** ```bash - docker logs aiq-server - docker logs aiq-ui + docker logs nemoagenttoolkit-server + docker logs nemoagenttoolkit-ui ``` 3. **Access the services:** @@ -191,7 +191,7 @@ Interact with the chat interface by prompting the agent with the message: Is 4 + 4 greater than the current hour of the day? ``` -![AIQ Toolkit Web UI Workflow Result](public/screenshots/ui_generate_example.png) +![AIQ Toolkit Web UI Workflow Result](https://github.com/NVIDIA/NeMo-Agent-Toolkit-UI/raw/main/public/screenshots/ui_generate_example.png) #### Stop the Services diff --git a/examples/getting_started/simple_calculator/compose_calculator.yaml b/examples/getting_started/simple_calculator/compose_calculator.yaml index d60d5ed52..986961974 100644 --- a/examples/getting_started/simple_calculator/compose_calculator.yaml +++ b/examples/getting_started/simple_calculator/compose_calculator.yaml @@ -1,15 +1,14 @@ services: - aiq-server: + nemoagenttoolkit-server: image: simple_calculator - container_name: aiq-calculator-server + container_name: nemoagenttoolkit-server ports: - "8000:8000" - - "6006:6006" # Phoenix telemetry service environment: - - NVIDIA_API_KEY=${NVIDIA_API_KEY} + - NVIDIA_API_KEY=${OPENAI_API_KEY} networks: - - aiq-network + - nemoagenttoolkit-network healthcheck: test: ["CMD-SHELL", "python -c 'import urllib.request; urllib.request.urlopen(\"http://localhost:8000/docs\")' || exit 1"] interval: 30s @@ -17,17 +16,17 @@ services: retries: 3 start_period: 40s - aiq-ui: - image: aiqtoolkit-ui - container_name: aiq-ui + nemoagenttoolkit-ui: + image: nemoagenttoolkit-ui + container_name: nemoagenttoolkit-ui ports: - "3000:3000" networks: - - aiq-network + - nemoagenttoolkit-network depends_on: - aiq-server: + nemoagenttoolkit-server: condition: service_healthy networks: - aiq-network: + nemoagenttoolkit-network: driver: bridge From 95601d21dafde73ad7894d563877b4c69c0b86ce Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 09:44:59 -0500 Subject: [PATCH 07/10] Minor edit to README.md Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- examples/getting_started/simple_calculator/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index 2b2859824..2c1ea1109 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -184,7 +184,6 @@ This will: 3. **Access the services:** - Web UI: http://localhost:3000 - API Documentation: http://localhost:8000/docs - - Phoenix Telemetry: http://localhost:6006 Interact with the chat interface by prompting the agent with the message: ``` From 921c6f75007e6229a9612e8e4a0516f7efc3c9d0 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 10:08:27 -0500 Subject: [PATCH 08/10] Use NVIDIA_API_KEY in README.md to be consistent with instructions. Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- examples/getting_started/simple_calculator/README.md | 6 +++--- .../simple_calculator/compose_calculator.yaml | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index 2c1ea1109..cbc2d4cee 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -61,7 +61,7 @@ uv pip install -e examples/getting_started/simple_calculator If you have not already done so, follow the [Obtaining API Keys](../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: ```bash -export OPENAI_API_KEY= # OPTIONAL +export NVIDIA_API_KEY= ``` ### Run the Workflow @@ -90,14 +90,14 @@ Prior to building the Docker image ensure that you have followed the steps in th From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: ```bash -docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/getting_started/simple_calculator Dockerfile . +docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/getting_started/simple_calculator/Dockerfile . ``` ### Run the Docker Container Deploy the container: ```bash -docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e OPENAI_API_KEY simple_calculator +docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator ``` Note, a phoenix telemetry service will be exposed at port 6006. diff --git a/examples/getting_started/simple_calculator/compose_calculator.yaml b/examples/getting_started/simple_calculator/compose_calculator.yaml index 986961974..920d224bc 100644 --- a/examples/getting_started/simple_calculator/compose_calculator.yaml +++ b/examples/getting_started/simple_calculator/compose_calculator.yaml @@ -6,7 +6,7 @@ services: ports: - "8000:8000" environment: - - NVIDIA_API_KEY=${OPENAI_API_KEY} + - NVIDIA_API_KEY=${NVIDIA_API_KEY} networks: - nemoagenttoolkit-network healthcheck: From d04d3948e3402136db8fd906dc34fb46fad1e57d Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 10:10:34 -0500 Subject: [PATCH 09/10] Minor edit to README.md Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- examples/getting_started/simple_calculator/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/examples/getting_started/simple_calculator/README.md b/examples/getting_started/simple_calculator/README.md index cbc2d4cee..2297ff4e5 100644 --- a/examples/getting_started/simple_calculator/README.md +++ b/examples/getting_started/simple_calculator/README.md @@ -165,7 +165,6 @@ docker compose -f compose_calculator.yaml up -d This will: - Build and start the calculator server on port 8000 - Build and start the web UI on port 3000 -- Set Phoenix telemetry on port 6006 - Create a secure bridge network for container communication #### Verify Deployment From d1db658fa1570f6770bee4c4d443fb7108173af5 Mon Sep 17 00:00:00 2001 From: Ed Lee <16417837+edlee123@users.noreply.github.com> Date: Mon, 28 Jul 2025 10:13:40 -0500 Subject: [PATCH 10/10] Delete this old README.md Signed-off-by: Ed Lee <16417837+edlee123@users.noreply.github.com> --- .../functions/simple_calculator/README.md | 286 ------------------ 1 file changed, 286 deletions(-) delete mode 100644 examples/basic/functions/simple_calculator/README.md diff --git a/examples/basic/functions/simple_calculator/README.md b/examples/basic/functions/simple_calculator/README.md deleted file mode 100644 index be93d8a11..000000000 --- a/examples/basic/functions/simple_calculator/README.md +++ /dev/null @@ -1,286 +0,0 @@ - - -# A Simple LLM Calculator - -This example demonstrates an end-to-end (E2E) agentic workflow using the AIQ toolkit library, fully configured through a YAML file. It showcases the AIQ toolkit plugin system and `Builder` to seamlessly integrate pre-built and custom tools into workflows. - -## Table of Contents - -- [A Simple LLM Calculator](#a-simple-llm-calculator) - - [Table of Contents](#table-of-contents) - - [Key Features](#key-features) - - [Installation and Setup](#installation-and-setup) - - [Install this Workflow:](#install-this-workflow) - - [Set Up API Keys](#set-up-api-keys) - - [Run the Workflow](#run-the-workflow) - - [Deployment-Oriented Setup](#deployment-oriented-setup) - - [Build the Docker Image](#build-the-docker-image) - - [Run the Docker Container](#run-the-docker-container) - - [Test the API](#test-the-api) - - [Expected API Output](#expected-api-output) - - [Deployment with UI](#deployment-with-ui) - - ---- - -## Key Features - -- **Pre-built Tools:** Leverages core AIQ toolkit library tools. -- **Custom Plugin System:** Developers can bring in new tools using plugins. -- **High-level API:** Enables defining functions that transform into asynchronous LangChain tools. -- **Agentic Workflows:** Fully configurable via YAML for flexibility and productivity. -- **Ease of Use:** Simplifies developer experience and deployment. - ---- - -## Installation and Setup - -If you have not already done so, follow the instructions in the [Install Guide](../../../../docs/source/quick-start/installing.md#install-from-source) to create the development environment and install AIQ toolkit. - -### Install this Workflow: - -From the root directory of the AIQ toolkit library, run the following commands: - -```bash -uv pip install -e examples/basic/functions/simple_calculator -``` - -### Set Up API Keys -If you have not already done so, follow the [Obtaining API Keys](../../../../docs/source/quick-start/installing.md#obtaining-api-keys) instructions to obtain an NVIDIA API key. You need to set your NVIDIA API key as an environment variable to access NVIDIA AI services: - -```bash -export NVIDIA_API_KEY= -``` - -### Run the Workflow - -Return to your original terminal, and run the following command from the root of the AIQ toolkit repo to execute this workflow with the specified input: - -```bash -aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" -``` - -**Expected Output** -The workflow output can be quite lengthy, the end of the workflow output should contain something similar to the following (the final answer will depend on the time of day the workflow is run): -```console -$ aiq run --config_file examples/basic/functions/simple_calculator/configs/config.yml --input "Is the product of 2 * 4 greater than the current hour of the day?" -2025-04-23 15:58:34,877 - aiq.runtime.loader - WARNING - Loading module 'aiq_automated_description_generation.register' from entry point 'aiq_automated_description_generation' took a long time (440.151215 ms). Ensure all imports are inside your registered functions. -2025-04-23 15:58:35,193 - aiq.cli.commands.start - INFO - Starting AIQ toolkit from config file: 'examples/basic/functions/simple_calculator/configs/config.yml' -2025-04-23 15:58:35,199 - aiq.cli.commands.start - WARNING - The front end type in the config file (fastapi) does not match the command name (console). Overwriting the config file front end. - -Configuration Summary: --------------------- -Workflow Type: react_agent -Number of Functions: 5 -Number of LLMs: 2 -Number of Embedders: 0 -Number of Memory: 0 -Number of Retrievers: 0 - -2025-04-23 15:58:36,674 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: To answer this question, I need to calculate the product of 2 and 4, and then compare it to the current hour of the day. - -Action: calculator_multiply -Action Input: {'text': '2 * 4'} - - ------------------------------- -2025-04-23 15:58:36,682 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_multiply -Tool's input: {"text": "2 * 4"} -Tool's response: -The product of 2 * 4 is 8 ------------------------------- -2025-04-23 15:58:37,704 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the product of 2 and 4, I need to get the current hour of the day to compare it with the product. - -Action: current_datetime -Action Input: None ------------------------------- -2025-04-23 15:58:37,710 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: current_datetime -Tool's input: None -Tool's response: -The current time of day is 2025-04-23 15:58:37 ------------------------------- -2025-04-23 15:58:38,865 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: Now that I have the current time of day, I can extract the hour and compare it with the product of 2 and 4. - -Action: calculator_inequality -Action Input: {'text': '8 > 15'} ------------------------------- -2025-04-23 15:58:38,871 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Calling tools: calculator_inequality -Tool's input: {"text": "8 > 15"} -Tool's response: -First number 8 is less than the second number 15 ------------------------------- -2025-04-23 15:58:39,978 - aiq.agent.react_agent.agent - INFO - ------------------------------- -[AGENT] -Agent input: Is the product of 2 * 4 greater than the current hour of the day? -Agent's thoughts: -Thought: I now know the final answer - -Final Answer: No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15). ------------------------------- -2025-04-23 15:58:39,981 - aiq.front_ends.console.console_front_end_plugin - INFO - --------------------------------------------------- -Workflow Result: -['No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 15).'] -``` - - -## Deployment-Oriented Setup - -For a production deployment, use Docker: - -### Build the Docker Image - -Prior to building the Docker image ensure that you have followed the steps in the [Installation and Setup](#installation-and-setup) section, and you are currently in the AIQ toolkit virtual environment. - -From the root directory of the NeMo-Agent-Toolkit repository, build the Docker image: - -```bash -docker build --build-arg AIQ_VERSION=$(python -m setuptools_scm) -t simple_calculator -f examples/basic/functions/simple_calculator/Dockerfile . -``` - -### Run the Docker Container -Deploy the container: - -```bash -docker run --name simple_calculator_container -p 8000:8000 -p 6006:6006 -e NVIDIA_API_KEY simple_calculator -``` - -Note, a phoenix telemetry service will be exposed at port 6006. - -### Test the API -Use the following curl command to test the deployed API: - -```bash -curl -X 'POST' \ - 'http://localhost:8000/generate' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{"input_message": "Is the product of 2 * 4 greater than the current hour of the day?"}' -``` - -### Expected API Output -The API response should be similar to the following: - -```bash -{ - "input": "Is the product of 2 * 4 greater than the current hour of the day?", - "output": "No, the product of 2 * 4 (which is 8) is less than the current hour of the day (which is 16)." -} -``` - -To stop and remove the container: - -```bash -docker stop simple_calculator_container -docker rm simple_calculator_container -``` - -### Deployment with UI - -The example can be run with the NeMo-Agent-Toolkit UI. After building the simple_calculator image previously, this will build the UI docker image, and deploy both with network connectivity via a docker compose file. - -**Prerequisites:** -- You must have built the `simple_calculator` image (see [Build the Docker Image](#build-the-docker-image) above) - -#### Build UI Docker Image - -```bash -export WORKPATH=~/Projects # Set this to the parent directory containing NeMo-Agent-Toolkit - -git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit-UI.git $WORKPATH/NeMo-Agent-Toolkit-UI - -cd $WORKPATH/NeMo-Agent-Toolkit-UI - -# Set Next.js build environment variables for UI -cp $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator/.env.docker.ui .env.production - -docker build -t aiqtoolkit-ui . -``` - -#### Deploy Simple LLM Calculator + UI - -After building the UI image above use docker compose to deploy: - -```bash -cd $WORKPATH/NeMo-Agent-Toolkit/examples/basic/functions/simple_calculator - -docker compose -f compose_calculator.yaml up -d -``` - -This will: -- Build and start the calculator server on port 8000 -- Build and start the web UI on port 3000 -- Set Phoenix telemetry on port 6006 -- Create a secure bridge network for container communication - -#### Verify Deployment - -1. **Check service status:** - ```bash - docker compose ps -a - ``` - -2. **View logs:** - ```bash - docker logs aiq-server - docker logs aiq-ui - ``` - -3. **Access the services:** - - Web UI: http://localhost:3000 - - API Documentation: http://localhost:8000/docs - - Phoenix Telemetry: http://localhost:6006 - -Interact with the chat interface by prompting the agent with the message: -``` -Is 4 + 4 greater than the current hour of the day? -``` - -![AIQ Toolkit Web UI Workflow Result](public/screenshots/ui_generate_example.png) - -#### Stop the Services - -To stop all services: -```bash -docker compose -f compose_calculator.yaml down -``` \ No newline at end of file