How AI Agents Use Tools, APIs and Memory to Work Autonomously

TL;DR

• AI agents use tool calling: the LLM outputs structured JSON and the runtime executes the actual function
• Tools can be any Python function: web search, API call, database query, file write
• Short-term memory is the context window: all prior steps in the current run
• Long-term memory lives in vector databases: retrieved by semantic similarity
• The agent loop: perceive → reason → act → observe → repeat
• Good tool design is the most important factor in agent reliability

The Agent Loop Explained

Every AI agent runs a continuous loop. Understanding this loop is the key to understanding how agents work:

The LLM reads the system prompt (its instructions), the conversation history, tool results from previous steps, and any retrieved memories. This is the full context it reasons from.

The LLM thinks about what to do next. In ReAct agents this is visible as a "Thought:" section. The model decides: do I have enough information? Do I need to call a tool? Am I done?

The LLM outputs a tool call: a JSON object naming the function and its arguments. The agent runtime intercepts this, executes the real function, and captures the result.

The tool result is added to the context window as an observation. The loop repeats, perceive the new state, reason about what to do next, act again, until the goal is achieved.

Tool Calling: How Agents Interact with the World

Tool calling is the mechanism by which an agent moves from language to action. Here's the process:

Step 1: Define the tool. You write a Python function with a docstring describing what it does, its parameters, and what it returns. This description is shown to the LLM.

def search_web(query: str) -> str:
 """
 Search the web for current information.
 Use this when you need up-to-date facts,
 competitor data, or recent news.

 Args:
 query: The search query string
 Returns:
 Top 5 search results with titles and snippets
 """
 results = serper_api.search(query)
 return format_results(results)

Step 2: LLM decides to use it. When the LLM determines it needs web data, it outputs a structured call:

{
 "tool": "search_web",
 "arguments": {
 "query": "HubSpot pricing 2026 UK"
 }
}

Step 3: Runtime executes it. The agent framework intercepts this JSON and calls the actual Python function. It gets the result and appends it to the context as an observation. The LLM never directly touches the internet. It just "requests" actions, and the runtime executes them.

The Standard Tool Library

These are the tools we include in most production agents:

How Agent Memory Works

Memory is what allows an agent to persist knowledge across runs, recall past interactions, and improve over time. There are two types:

Short-Term Memory

The active context window: every message, tool call, and observation in the current run.

Capacity: 128k–1M tokens depending on the model. Automatically managed by the framework.

Cleared when the run ends.

Long-Term Memory

External storage: vector databases (Pinecone, Qdrant, Chroma) or structured SQL. Persists between runs.

Retrieved by semantic similarity: the agent asks "what do I know about X?" and gets the most relevant stored chunks.

Survives run termination: enables learning over time.

How long-term memory retrieval works: When an agent needs to recall something, it calls the memory_search tool with a query. The system converts that query to a vector embedding. Then it searches the vector database for the most semantically similar stored content. Finally it returns the top-k results to the LLM.

This is why agents can "remember" a client's preferences from six months ago. They can also know that a particular supplier always needs a specific format. That information was stored as embeddings after a previous run.

API Integration: How Agents Connect to Business Systems

Connecting an AI agent to your business systems is the same as building an API integration. But it is wrapped as a tool the LLM can call. Here's the pattern:

def update_hubspot_contact(
 contact_id: str,
 properties: dict
) -> dict:
 """
 Update a HubSpot contact record.
 Use this to save information gathered
 about a prospect to the CRM.

 Args:
 contact_id: HubSpot contact ID
 properties: Dict of property names and values
 Returns:
 Updated contact record confirmation
 """
 response = requests.patch(
 f"https://api.hubapi.com/contacts/v1/contact/vid/{contact_id}/profile",
 headers={"Authorization": f"Bearer {HUBSPOT_API_KEY}"},
 json={"properties": [{"property": k, "value": v}
 for k,v in properties.items()]}
 )
 return response.json()

The agent never sees your API keys. It just knows the tool exists and what it does. The runtime handles authentication, rate limiting, error handling, and retries.

What Makes a Well-Designed Agent Tool

Tool design is the biggest predictor of agent reliability. Common mistakes and how to avoid them: