Most AI agent deployments today send every request, from a trivial status check to a complex analysis, to the same large cloud-hosted model. That is simple to build but costly to run, because each query pays for a network round trip and a big model's compute, even when the task is routine. AWS has published a reference architecture called the Adaptive Agentic Intelligence Framework (AAIF), built on its Strands Agents toolkit, that tries to fix this by splitting work between small local models and larger cloud models depending on how hard the task is.

What is new

AAIF is not a new foundation model. It is an orchestration pattern: a way of structuring an AI agent so that a lightweight model decides, for each incoming request, whether it can be handled locally or needs to go to a larger model in the cloud. AWS built the example using Strands Agents (its open framework for building agents), small language models (SLMs) running locally through Ollama, and larger models accessed through Amazon Bedrock, AWS's managed service for foundation models. The design draws on a research position from NVIDIA researchers (Belcak et al.), who argue that small models are often powerful enough, better suited, and cheaper for many of the individual steps inside an agentic system, reserving large models for the genuinely hard cases.

How it works

AAIF organizes processing into three layers. A smart orchestration layer uses a small model, llama3.2:1b (1 billion parameters), as a lightweight dispatcher. It reads each incoming request and decides, based on its own understanding of the text rather than hardcoded keyword rules, which of three tools to call. A local processing layer handles routine jobs: retrieving device telemetry data, or running image analysis using a local vision model (llama3.2-vision) on camera feeds, all without the data leaving the device. A cloud processing layer, backed by Amazon Bedrock, is reserved for requests that need broader context or multi-step reasoning that small models cannot reliably perform.

In AWS's demonstration scenario, an Internet of Things (IoT) monitoring setup, the agent exposes three tools: get_device_telemetry for reading sensor data, analyze_camera_feed for on-device vision processing, and get_expert_insights, which calls Amazon Bedrock for complex questions such as recommending optimization strategies. A simple status request ("what's the current status of our devices?") gets resolved locally with low latency. A request for strategic recommendations gets routed to the cloud model because it needs deeper reasoning.

  • Orchestrator model: llama3.2:1b, run locally via Ollama
  • Local vision model: llama3.2-vision, for on-device image analysis
  • Cloud model: accessed via Amazon Bedrock, for complex multi-step reasoning tasks
  • Framework: Strands Agents (AWS), using the @tool decorator to register capabilities
  • Deployment options shown: edge-only (e.g. a Raspberry Pi with 8 GB memory), full cloud, or hybrid
  • Claimed benefit: in the hybrid setup, AWS says cost savings can come from the roughly 80 percent of traffic that does not need to hit a paid API endpoint

Why it matters

For practitioners building agentic systems, the appeal is straightforward: not every step an agent takes needs a frontier model. Classifying a request, fetching a cached value, or running a basic computer vision check can often be done by a model with a fraction of the parameters, running on local hardware, with no network dependency. That matters for cost control, for latency-sensitive applications, and for situations where network connectivity cannot be guaranteed or where data should not leave a device for privacy or compliance reasons. AWS frames this as broadly applicable beyond IoT, suggesting a healthcare organization could keep patient monitoring local while routing diagnosis support to the cloud, a financial services firm could process transactions on-premises and reserve cloud capacity for risk analysis, or a manufacturing team could run quality-control vision locally while sending optimization requests to the cloud.

The architecture's flexibility comes from separating the routing logic (the small orchestrator model) from the actual processing tools, which can be swapped independently. AWS shows three configurations: replacing the local orchestrator with Amazon Bedrock entirely for teams with reliable connectivity and no data residency constraints; the default hybrid setup described above; and a fully air-gapped configuration that swaps the cloud tool for a larger local model (a 3B-parameter version) for environments with strict data sovereignty requirements, at the cost of reduced reasoning depth on the hardest queries.

Limits and open questions

The post is a technical walkthrough rather than an independent benchmark study, and the cost-saving figure it cites — around 80 percent of traffic avoiding a paid API call — is specific to the example hybrid configuration AWS built, not a general guarantee for any deployment. How well the 1B-parameter orchestrator classifies requests correctly in more varied or ambiguous real-world traffic is not quantified in the post beyond the three example routing cases shown. AWS also notes that production use needs added safety controls: it recommends attaching Amazon Bedrock Guardrails to the cloud-facing model to filter harmful content and block ungrounded responses, and calling a separate ApplyGuardrail API on the local small-model tier so that both halves of the system follow the same safety standard. The post also points to a future extension using the Model Context Protocol (MCP) to let tools run as separate, centrally managed servers across multiple devices, which it describes as a direction rather than something fully demonstrated here.

How to try it

AWS published full sample code for the walkthrough. It requires an AWS account with Amazon Bedrock model access, Python 3.10 or later, Ollama installed locally with the llama3.2:1b and llama3.2-vision models pulled, the strands-agents and strands-agents-tools Python packages, boto3, and AWS CLI credentials configured. The post walks through defining the local tools with Strands' @tool decorator, configuring the Strands agent with the Ollama-hosted orchestrator model, wiring in the Bedrock-backed expert-insights tool, and running example queries that demonstrate local and cloud routing. AWS also includes cleanup steps afterward, such as revoking Bedrock model access and removing the local Ollama models, to avoid ongoing charges or disk usage.

Sources