AWS Open-Sources UAEF to Evaluate AI Agents Across Frameworks
Reported by Unite.AI, The Next Web and Tech Times3 sources ↓
AWS released UAEF, an open-source framework that evaluates AI agents across different software frameworks using 36 metrics. Its judge uses safeguards against prompt injection, but AWS says they reduce risk rather than eliminate it.
Key points
- UAEF converts agent records into a shared format for evaluation across frameworks
- Its 36 metrics cover seven dimensions, with scores normalized from 0 to 1
- AWS says organizations in telecom, healthcare and finance have used it to compare agent versions
- AI judging and result storage rely on AWS services
- The judge safeguards do not eliminate prompt-injection risk or provide runtime security
Sources
Details
UAEF uses adapters to convert agent messages, tool calls and handoffs into a shared format called AgentTrace. Its 36 built-in metrics cover seven dimensions, including tool use, response quality, safety, performance and multi-agent coordination. Some scores are calculated from traces and reference data; others use a model on Amazon Bedrock to judge qualities such as relevance and safety.
AWS released UAEF under the Apache 2.0 license. Teams can use it as a Python library in notebooks or CI/CD pipelines, or through a web interface. AWS credentials are needed for Bedrock-based judging and for storing results in DynamoDB and S3. The Next Web reports that AWS said organizations in telecom, healthcare and financial services have used UAEF to compare agent versions before release.
Tech Times reports that UAEF treats agent output as untrusted input, separates the rubric from that content, escapes it, parses judge responses strictly and requires evidence from the agent output before accepting a score. The safeguards lower the risk of injected text corrupting evaluation results, but do not eliminate it. AWS says UAEF should not be used as a security decision system; runtime enforcement belongs in a separate Amazon Bedrock Guardrails path.
Written from coverage as of
Images
- Unite.AI
- The Next Web
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh