PayPal Raised Its AI Assistant’s Accuracy From 43% to 95% Without Changing the LLM
Reported by Gadget Review1 source ↓
PayPal says its assistant’s question-answer accuracy rose from 43% to 95% in an internal evaluation after the company added a business knowledge layer; it did not change the underlying model.
Key points
- The knowledge layer includes business definitions, query specifications and analytical procedures
- PayPal says the assistant handled more than 1,000 questions and saved an estimated 1,600 hours over 10 weeks
- The reported accuracy result came from an internal evaluation
Details
PayPal built a separate knowledge layer to give its assistant business context that was missing even though it could access vetted data. The layer includes definitions, query patterns and analytical procedures employees use to interpret that data. PayPal vice president Ren Zhang said the underlying language model remained the same.
The company’s “golden queries” specify how data should be retrieved for particular purposes, while analytical playbooks describe how to investigate questions across segments, products and behaviors. PayPal says the layer is designed to work across model upgrades, so business logic can be retained if the company changes models.
PayPal also described agent workflows for investigating service alerts and reviewing security issues, with human checks required for some decisions. Zhang cautioned that question volume does not by itself demonstrate productivity gains, and the company uses testing and employee feedback to look for failures that evaluation scores may miss.
Written from coverage as of
Images
- Gadget Review
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh