Spring AI 2.0 Order Management Agent

This project demonstrates building an intelligent AI agent using Spring Boot 4.1.0 and Spring AI 2.0 (RC2). The agent uses a local LLM (Llama 3.1 via Ollama) and reliably executes function calls to retrieve real-time order status from your system.

Thanks to fine-tuned system prompt and model parameter calibration, the agent communicates directly with the customer, avoiding internal technical dialogue about tool invocations.

Tech Stack

• Java 21 (Temurin SDKMAN)
• Spring Boot 4.1.0
• Spring AI 2.0.0-RC2
• Ollama (with local llama3.1 model)
• Lombok
• Apache Tomcat 11 & Netty (with optimized native resolvers for macOS)

Architecture & Components

The application is built around the new declarative tool concept in Spring AI 2.0.

1. Entry Point (AiApplication.java)

Standard Spring Boot application that initializes the context and triggers the automatic flow via CommandLineRunner.

2. Agent Tools (OrderManagementTools.java)

Instead of classic @Bean functions, Spring AI 2.0 introduces a convenient declarative approach using the @Tool annotation:

@Tool(description = "Returns the current order status by its ID. Use when the user asks 'where is my order' or wants to know the status.")
public String getOrderStatus(String orderId) {
    // Order lookup logic
}

For stable operation with small local models (Llama 3.1 8B), the method returns a plain String instead of complex JSON structures, guaranteeing 100% accurate data integration into the response.

3. Client Configuration (SpringAiAgentRunner.java)

The ChatClient configuration uses the Fluent API to automatically bind the system prompt and default tools:

this.chatClient = chatClientBuilder
        .defaultTools(orderManagementTools)
        .defaultSystem(systemPrompt)
        .build();

Configuration (application.yaml)

The configuration contains two critical aspects for working with local AI models:

• temperature: 0.0 — Minimizes model creativity, making responses predictable and precise.
• Strict system prompt — Prevents the model from "talking to itself" or commenting on tool invocations.

spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        model: llama3.1
        temperature: 0.0

app:
  chat:
    system-prompt: |
      You are a polite support operator. Your only task is to respond to the customer.
      RULES:
      1. Use the data returned by the getOrderStatus tool.
      2. Format the response ONLY for the customer.
      3. DO NOT write phrases like: "The tool response was...", "I received data...", "I passed the information".
      4. Write the response as if you checked the database yourself and are now telling the customer directly.

Running & Request Lifecycle

Ensure Ollama is running locally and the model is loaded:

ollama run llama3.1

Build and run:

./mvnw spring-boot:run

The application is a CommandLineRunner — it starts, makes one request to the LLM, logs the response, and exits.

Example log output:

2026-06-14T20:41:14.259+03:00  INFO --- [main] c.a.ai.SpringAiAgentRunner : --- Starting Spring AI 2.0 Agent ---
2026-06-14T20:41:14.259+03:00  INFO --- [main] c.a.ai.SpringAiAgentRunner : Agent prompt: 
Hello! where is my order 123-abc?
2026-06-14T20:41:16.301+03:00  INFO --- [main] c.a.ai.OrderManagementTools : [Tool Called] Checking status for: 123-abc
2026-06-14T20:41:18.132+03:00  INFO --- [main] c.a.ai.SpringAiAgentRunner : Agent Response: 
"Order #123-abc has already been shipped and will arrive tomorrow. If you have any questions, feel free to contact us!"