AWS · PiniShv

CLI Agent Orchestrator: When One AI Agent Isn't Enough

Wed, 05 Nov 2025 00:00:00 +0000

You’ve hit this wall before. You’re working on some complex modernization project with Claude Code or Amazon Q Developer CLI, and the agent starts losing coherence. Too much context. Too many domains. Architecture bleeding into security bleeding into performance optimization. The agent can’t maintain focus.

Your options have been to manually coordinate between separate agent sessions, copying context around like it’s 2010. Or overload one agent with everything and watch quality degrade as the context window fills up.

AWS to the rescue; they just released CLI Agent Orchestrator (CAO): multiple specialized agents working together under a supervisor. Hierarchical orchestration for your AI CLI tools.

It’s early, opinionated, and requires AWS infrastructure. But it shows where developer AI tooling is headed when single agents aren’t enough.

The Problem
#

Single agents work great for focused tasks. Refactoring? Boilerplate? Debugging? Claude Code or Amazon Q handles it.

But try modernizing a legacy mainframe application. Architecture design, security review, performance optimization, testing, data migration. That’s a project spanning multiple disciplines. Load all that context into one agent and watch quality degrade. The agent contradicts itself, forgets earlier decisions, outputs get generic.

The alternative is running separate agents manually. One for architecture. Another for security. Another for performance. Now you’re copying context between them, manually synthesizing outputs, spending more time coordinating than working. You’ve become the orchestration layer.

CAO is AWS’s answer: a supervisor agent manages specialized workers. Each focuses on its domain. The supervisor handles coordination. Configure the team once, let them collaborate.

How It Works
#

A supervisor agent delegates to specialized workers. One for architecture. One for security. One for performance. The supervisor manages sequencing and maintains context. Workers focus on their specialty and report back.

Each agent runs in its own isolated tmux session. No context pollution. The architecture agent’s history doesn’t leak into the security agent’s work. Sessions communicate through Model Context Protocol (MCP) servers, which handle local communication between the isolated sessions, running entirely on your machine.

CAO supports three patterns. Handoff (synchronous): supervisor waits for completion before proceeding. Assign (asynchronous): supervisor delegates and moves on. Send Message: supervisor checks status without blocking. All implemented through Amazon Bedrock action groups.

Example: Mainframe Modernization
#

The supervisor receives “Create a modernization plan for this COBOL banking system.” It hands off sequentially: architecture agent designs the structure, security agent reviews it, then performance and test agents work in parallel. The supervisor synthesizes outputs into a unified plan.

You could apply this to building microservices applications or migrating monoliths. In practice, you’ll iterate on prompts and intervene when agents drift. But the pattern works when configured well.

The Reality Check
#

CAO only works with Amazon Q Developer CLI and Claude Code. Nothing else has shipped despite “future plans” for other tools.

The supervisor runs on Amazon Bedrock, AWS’s managed service for foundation models. You need AWS credentials, Bedrock access, and an AWS account. It’s open source code you can’t run without AWS infrastructure. This is lock-in you should choose consciously.

Everything runs in tmux sessions. Great for transparency, but it’s another dependency with a learning curve. Running this in CI/CD adds complexity.

Multiple agents mean multiple API calls, more token usage, higher latency. For simple tasks, this is wasteful overkill. You need to be selective about when orchestration overhead is worth it.

This is infrastructure for developers comfortable with AWS, tmux, and orchestration concepts. It’s not polished. Limited early reactions on social media praise the privacy focus but flag AWS lock-in and tmux hurdles as barriers to adoption.

Why It Matters
#

The interesting part isn’t CAO specifically. It’s the shift from “AI tool as standalone assistant” to “AI tools as orchestrated teams.”

Single-agent tools hit walls. Context windows don’t solve everything. At some point, more context just means more noise. Multi-agent architectures divide cognitive labor. Each agent has a focused job. The supervisor ensures pieces fit together.

We’re seeing this everywhere. OpenAI’s Swarm. LangGraph. CrewAI. AutoGPT. The underlying idea is the same: complex tasks need coordination, not just more context. Specialization plus orchestration beats generalization with bigger context windows.

The question: does this remain infrastructure developers explicitly configure, or does it become invisible? CAO is clearly “you configure this.” But the long-term direction is probably toward tools that orchestrate automatically, with developers intervening only when defaults fail.

Getting Started
#

If you want to experiment with CAO:

You need an AWS account with Bedrock access and permissions to use Claude models. Install Amazon Q Developer CLI or Claude Code. Install tmux (brew install tmux on macOS). Clone the repo: git clone https://github.com/awslabs/cli-agent-orchestrator. The README has configuration examples and workflows.

Realistically, plan to spend an afternoon getting this working. This isn’t a tool you spin up in 10 minutes.

Should You Use This?
#

Use CAO if you’re handling complex, multi-disciplinary tasks where single agents struggle. If you’re already on AWS and Bedrock, integration is straightforward. You need comfort with tmux and orchestration concepts.

Skip it if your tasks are straightforward. If you’re not on AWS or want to avoid lock-in, skip it. If you need something polished, this isn’t it.

For most developers, single-agent tools remain the right choice. For teams tackling large-scale modernizations or complex migrations, CAO offers a pattern worth exploring.

Check out the GitHub repository and the AWS blog post.

The future of AI tooling is coordination, not just capability. CAO is AWS’s bet on how that works. Whether it becomes standard or just one experiment, the pattern it represents is where things are headed.

Amazon S3 Vectors: When Storage Learns to Think

Fri, 31 Oct 2025 00:00:00 +0000

AWS did something interesting. They turned S3, the storage service that holds something like half the internet’s data, into a vector database. Not a separate service that integrates with S3. Not a new database that happens to live near S3. They built vector search directly into object storage itself.

Amazon S3 Vectors is currently in preview, and it’s exactly what it sounds like: you can now store billions of vector embeddings in S3 and query them with sub-second performance. No servers to provision, no clusters to manage, and according to AWS, up to 90% cheaper than running a traditional vector database.

That last number matters. Because the biggest problem with vector databases right now isn’t performance. It’s cost at scale.

The Problem S3 Vectors Actually Solves
#

Let’s talk about the economics of AI applications for a moment. If you’re building a RAG system for a large enterprise, you might need to embed millions of documents. Each document gets chunked, each chunk becomes a vector, and suddenly you’re storing hundreds of millions or billions of embeddings.

Traditional vector databases are fast, but they’re expensive at scale. They keep everything in memory or on fast SSDs, they run on dedicated compute, and they charge accordingly. For a billion 512-dimension vectors with moderate query loads, you might be paying close to $10,000 per month on a service like Pinecone. That’s not a criticism of Pinecone, by the way. High-performance infrastructure costs money.

But here’s the thing: most use cases don’t need single-digit millisecond latency. A support chatbot can wait 200 milliseconds to retrieve relevant documents. An internal knowledge search can tolerate half a second. A recommendation system that updates nightly doesn’t need real-time indexing at all.

S3 Vectors bets that for many real-world applications, “fast enough” is actually fast enough. And if “fast enough” costs 90% less, that changes what becomes economically feasible.

How It Actually Works
#

The architecture is straightforward. You create a vector bucket in S3 (a special bucket type for vectors), then create one or more vector indexes inside it. Each index has a fixed dimension (up to 4,096) and a distance metric (currently Cosine or Euclidean).

You insert vectors using the PutVectors API, up to 500 at a time. Each vector gets an ID and optional metadata (up to 10 key-value pairs). Under the hood, S3 automatically builds and maintains an index structure for similarity search. The exact algorithms aren’t exposed, but they’re using some form of approximate nearest neighbor search, likely HNSW or similar.

When you query with QueryVectors, you provide a query vector and optionally filter by metadata. S3 returns the top K most similar vectors (up to 30 results) with their IDs, distances, and metadata. The whole operation typically takes 100 to 300 milliseconds for indexes with millions of vectors.

The key insight is that storage and search are now the same thing. Your embeddings live in S3’s durable, elastic object storage, but you can query them semantically without pulling everything into memory first.

Where This Actually Makes Sense
#

If you’re building a RAG system with Amazon Bedrock, S3 Vectors is now the obvious choice for the vector store. Bedrock Knowledge Bases can use it natively. You point Bedrock at your documents, choose an embedding model, and it handles chunking, embedding, and storage in S3 Vectors automatically. When users ask questions, Bedrock queries the index, retrieves relevant chunks, and feeds them to the LLM. The whole pipeline is managed.

For semantic search over large document collections, S3 Vectors shines. Legal firms indexing millions of case documents, media companies searching video archives by content, enterprises making their entire knowledge base searchable by meaning rather than keywords. These are all scenarios where you need massive scale but can tolerate sub-second latency.

For recommendation systems and personalization, it depends. If your embeddings update in batch (nightly retraining, periodic refreshes), S3 Vectors works well. If you need real-time updates per user interaction, it’s less suitable. The write throughput limit is currently 5 requests per second per index (though you can batch 500 vectors per request).

For fraud detection and anomaly detection, S3 Vectors provides a cost-effective way to store historical patterns. You might keep recent data in a faster system like OpenSearch for real-time checks, while using S3 Vectors for deep historical comparisons or retrospective analysis.

The pattern is consistent: S3 Vectors is ideal when you have large, relatively stable datasets with moderate query loads. It’s not for high-frequency trading systems or real-time ad serving. It’s for the long tail of AI applications where scale matters more than the last millisecond of latency.

The Integration Story
#

AWS built this knowing that different use cases need different performance tiers. That’s why S3 Vectors integrates directly with Amazon OpenSearch Service in two ways.

First, you can configure OpenSearch to use S3 Vectors as its storage layer. OpenSearch still handles queries and aggregations, but the actual vector data lives in S3. This dramatically reduces storage costs while keeping OpenSearch’s rich query capabilities and hybrid search features.

Second, you can export an S3 Vector index directly into OpenSearch Serverless when you need faster performance. This lets you start with S3 (cheap, massive scale) and promote hot data to OpenSearch (expensive, 10-50ms latency) when usage patterns justify it.

This tiered approach is honestly more interesting than S3 Vectors alone. It acknowledges that not all data has the same access patterns, and different queries have different latency requirements. You can optimize for cost most of the time and performance when it matters.

What About the Competition
#

The vector database market is crowded. Pinecone, Weaviate, Milvus, Qdrant, ChromaDB, even PostgreSQL with pgvector. They all do vector search, and many do it faster than S3 Vectors.

But S3 Vectors isn’t trying to be the fastest. It’s trying to be the most practical for AWS customers who already have their data in S3 and want to add semantic search without managing new infrastructure.

The real competition isn’t Pinecone or Milvus. It’s the decision to not build vector search at all because it seems too expensive or complex. If S3 Vectors makes vector search a standard feature of your data lake rather than a separate project with separate infrastructure, that changes the adoption calculation.

For specialized vector databases, this probably means focusing on what they do better: multi-cloud portability, advanced query capabilities, extreme performance optimization, or specific verticals. The “store vectors and do similarity search” use case just became commoditized on AWS.

The Limitations You Should Know
#

S3 Vectors has real constraints. Each index caps at 50 million vectors in preview. If you need more in a single semantic space, you’ll need to partition across multiple indexes or request a limit increase. This is probably the biggest operational consideration.

Query results are limited to top 30. You can’t ask for the top 100 candidates for reranking. You get 30, period. For most applications that’s fine, but if your workflow depends on large candidate sets, you’ll need to adapt.

Distance metrics are limited to Cosine and Euclidean. No dot product (though you can normalize vectors to make cosine equivalent), no Manhattan distance, no custom metrics. This covers most use cases but not all.

Metadata is limited to 10 fields and 2KB of filterable data per vector. If you need complex metadata structures or heavy filtering logic, you might need to combine S3 Vectors with another system that handles the metadata side.

Write throughput is throttled to 5 requests per second per index. For bulk ingestion, you batch 500 vectors per request, giving you roughly 2,500 vectors per second per index. That’s fine for batch loads but not for high-frequency streaming ingestion.

And crucially, it’s in preview. No SLA, API might change, and it’s only available in five regions right now (us-east-1, us-east-2, us-west-2, eu-central-1, and ap-sydney-1).

What This Means Strategically
#

AWS is doing what AWS does: taking a capability that startups innovated on and building it into the platform as a basic feature. They did this with databases (RDS, Aurora), with search (OpenSearch), with machine learning (SageMaker), and now with vector search.

This creates pressure on standalone vector database companies to differentiate on something other than basic storage and similarity search. Speed, features, multi-cloud, ease of use. The floor just got raised.

For AWS, this strengthens data gravity. If your embeddings live in S3 alongside your source data, and Bedrock can use them directly, and SageMaker can access them easily, you’re less likely to move workloads to another cloud. It’s not lock-in exactly, but it’s definitely friction.

The broader impact might be democratization. Vector search stops being a specialized project requiring evaluation of multiple vendors and becomes something you just turn on in your existing data lake. That probably expands the market more than it cannibalizes existing solutions.

The Cost Reality
#

Let’s ground this in actual numbers. For 1 billion 512-dimension vectors with about 50 million queries per month:

Storage costs roughly $46 per month (2TB at $0.023/GB). Indexing those billion vectors costs around $205 as a one-time charge ($0.10 per million write operations). Monthly queries cost about $20 ($0.40 per million query operations).

Total: around $271 per month after the initial indexing cost.

Compare that to running dedicated vector database infrastructure, where you’re provisioning compute regardless of actual usage, and the numbers make sense for many use cases.

The catch: this assumes relatively stable data with moderate query loads. If you’re constantly rewriting vectors or running millions of queries per day, the math changes. But for knowledge bases, document search, and periodic recommendations, the economics are compelling.

When Not to Use S3 Vectors
#

Be honest about your requirements. If you need sub-50ms query latency consistently, S3 Vectors isn’t the answer. Use OpenSearch with in-memory indexes, or a dedicated vector database like Pinecone.

If your application requires extremely high query throughput (thousands of queries per second sustained), S3 Vectors will likely hit limits. It’s not designed for that load profile.

If you need advanced query features like vector arithmetic, multi-vector queries, or complex boolean logic beyond basic metadata filtering, you’ll need a more sophisticated system.

If you can’t use AWS for compliance reasons, obviously S3 Vectors isn’t an option. And if you need multi-cloud portability, tying yourself to an AWS-specific service might not align with your architecture.

And if your embeddings change frequently, the write throttling becomes a real constraint. This isn’t for real-time streaming scenarios where vectors update per user interaction.

The Bottom Line
#

S3 Vectors doesn’t replace purpose-built vector databases. It provides a different trade-off: lower cost and zero infrastructure management in exchange for moderate latency and less control.

For many AI applications, especially those built on AWS already, that trade-off makes complete sense. The difference between 50ms and 250ms query time often doesn’t matter to end users. The difference between $10,000 and $500 per month absolutely does matter to businesses.

The most interesting aspect isn’t the technology itself. It’s what becomes possible when the cost barrier drops by 90%. Suddenly it’s economically feasible to embed everything, to make your entire data lake semantically searchable, to keep years of vector history for analysis.

We’re probably entering a phase where vector search becomes table stakes infrastructure, like key-value stores or message queues. Not every application needs it, but it’s available when you do, at a price point that doesn’t require a business case review.

Whether S3 Vectors specifically becomes the standard or whether it just forces the whole market to compete on cost and simplicity, the outcome is probably the same: vector search stops being a specialized tool and becomes basic infrastructure.

And that’s when things get interesting.

AWS · PiniShv

CLI Agent Orchestrator: When One AI Agent Isn't Enough

The Problem #

How It Works #

Example: Mainframe Modernization #

The Reality Check #

Why It Matters #

Getting Started #

Should You Use This? #

Amazon S3 Vectors: When Storage Learns to Think

The Problem S3 Vectors Actually Solves #

How It Actually Works #

Where This Actually Makes Sense #

The Integration Story #

What About the Competition #

The Limitations You Should Know #

What This Means Strategically #

The Cost Reality #

When Not to Use S3 Vectors #

The Bottom Line #