How Ora Tests Every Major AI Agent on Vercel (and Builds With It Too)

Ora, a 16-person startup co-founded by Assaf Elovic (previously of Tavily, acquired by Nebius this year), runs AI agents against live websites and records every step they take. The goal: show businesses exactly where agents fail and why. According to Ora's own data, 99% of the web cannot handle an agent that shows up to sign up, integrate, and pay for a product. The company benchmarks six major agent frameworks on Vercel, which also hosts the entire platform, and recently chose Vercel's own eve framework to power its own agents after the data made the case.

What happened

Detail	Fact
Company	Ora (journey.ora.ai)
Team size	16 engineers, hundreds of commits per day
Agents benchmarked	Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, eve
Web readiness estimate	99% of websites are not agent-ready
eve vs Claude Code: step reduction	7% fewer steps to complete a task
eve vs Claude Code: native success rate	2x more tasks finished on the target site without falling back to web search
eve vs Claude Code: valid endpoints	9% more usable endpoints discovered
Cost improvement after prompt-cache fix	Roughly 15% lower total cost
Infrastructure	Front end, back end, and agent runtime all on Vercel

Ora’s testing product works by spawning agents against a real customer site, then recording the cost, latency, and number of steps needed to finish a defined task. When an agent stalls mid-signup, the trace shows which step failed and what the agent tried. Without that trace, a benchmark score is just a number with no diagnosis behind it.

Each agent framework, called a “harness,” wraps a model (the reasoning engine) with software that hands the model its tools and drives it forward. No two harnesses share the same expected environment, so Ora runs a separate runtime for each one and traces every step independently. Ido Finder, who leads engineering at Ora, described that side-by-side coverage as one of the most valuable things the platform delivers.

How the eve benchmark played out

When Vercel shipped its eve agent framework, Ora gave it no preferential treatment. eve ran the same benchmark, on the same models (Claude Fable 5 and Haiku 4.5), across hundreds of real journeys on multiple domains, with the same task: integrate with a product. The results above are from that head-to-head with Claude Code.

One run also surfaced a prompt-caching bug inside eve. Ora shared the findings directly with the Vercel engineering team (Ora is a design partner), the fix shipped, and the next round of benchmarks showed roughly 15% lower total cost. That feedback loop is the kind of thing that only happens when you are both testing a tool and using it in production.

After the results came in, Ora adopted eve as the framework for its own internal agents. The deciding factor was the sandbox override feature. Eve ships with its own isolated execution environment, which is a safe default for most teams. But Ora needed agents to run inside its own instrumented environment so every step could be traced. The override let Ora swap in its own sandbox without writing new infrastructure. Journey.ora.ai now has eve on both sides: it tests eve as a harness, and it runs on eve internally.

Why does it matter that 99% of the web fails agent tasks?

Businesses are starting to assume that AI agents will handle tasks on their behalf, from researching vendors to signing up for software to completing purchases. If your website cannot be navigated by an agent, you will lose those transactions. This is not a distant problem: Ora’s data comes from running agents against live sites today, not simulated environments.

The most common failure points are signup flows, authentication steps, and API endpoints that are structured in ways models cannot reliably parse. Ora’s platform tells customers exactly which steps break and what to change. That is a more actionable signal than a general “your site has accessibility issues” audit.

For teams already thinking about AI integration for their products, agent-readiness is worth treating as a technical requirement alongside mobile performance or page speed, not as a future concern.

Why it matters

Ora’s architecture is a practical argument for keeping all parts of a product on a single deployment platform. With front end, back end, and agent runtime sharing the same logs, authentication, and deployment path, the company’s 16 engineers can let their coding agents operate the stack end to end. Finder puts the time saved at “a few hours a week at least,” and Elovic credits a similar saving to how well coding agents build with Vercel’s libraries.

The microservices split Ora is now undertaking is also notable. New services will deploy to the same Vercel infrastructure and communicate with no extra configuration. Internal eve-based agents will slot in as just another service. That approach scales the team’s leverage without scaling headcount.

For context on how AI startups are structuring their data and compute choices, see our earlier coverage of GPU neocloud options for 2026, relevant given that Ora’s acquirer Nebius is in that space.

Our take

Ora is doing something genuinely useful: turning “will agents work on your site?” from a vague anxiety into a traceable, fixable metric. The 99% stat is striking, and even if the real number is more nuanced, the directional point holds. Most websites were not designed with automated agents as a use case.

The eve adoption story is also worth taking seriously. Ora benchmarks frameworks for a living and still chose eve after a rigorous, unbiased test. That is a stronger signal than any vendor case study. The sandbox override feature is a concrete, specific reason that applies to any team that needs to instrument an agent’s execution, not just Ora.

The part we would watch: eve is Vercel’s framework, and Ora is a Vercel partner. The relationship is disclosed, but the benchmark methodology details (how many domains, which task definitions, what constitutes “native success”) are not fully public. Independent replication would strengthen the numbers considerably.

If you are building on Next.js or exploring what it takes to make your product work with AI agents, talking through the architecture early is worthwhile. Our web development team works with exactly these kinds of stack decisions.

What to do about it

1. Run an agent (Claude, ChatGPT, or similar) against your own signup or checkout flow manually and note where it stalls.
2. Check whether your key API endpoints return structured, parseable responses that a model can act on without guessing.
3. Review your authentication flow: CAPTCHA, multi-step verification, and modal interruptions are common agent failure points.
4. If you are building an agent product yourself, evaluate whether your framework’s default sandbox can be overridden so you can instrument execution traces.
5. Monitor tools like Ora’s journey.ora.ai as agent-readiness auditing matures into a standard part of web QA.

Agent-readiness is becoming a real technical requirement. Testing your own site against a live agent today costs almost nothing and will show you more than any audit checklist.

Source: Vercel Blog