Super44 replaced the agent monitoring it built in-house with Glassray and increased agent reliability 2.2x while traffic almost tripled
Claims verified· how these numbers are measured
2.2x agent reliability. The share of production runs that pass every one of Glassray's checks, compared across six weeks (mid-July to mid-August 2026) on the same methodology, while weekly traffic almost tripled.
55 custom failure patterns.Generated automatically from Super44's own flows and traces; patterns the team retired are excluded.
736 deviation investigations. Automated runs where Glassray takes one failure pattern and reads a batch of production traces looking for it.
28k trace evaluations. Total trace reads across those investigations; each trace is checked against many patterns.
About Super44
Super44 is an AI-native business partner for Europe's 1.5 million cafes, restaurants, and bars. Its multi-agent system plugs into a venue's point-of-sale, reviews, and staffing, and works around the clock to grow revenue and cut costs.
Every morning it briefs each merchant on what sold, what to expect today, and one action ranked by profit impact. The team are solving challenges through browser automation, context management, and agentic memory. Co-founder Henry George was previously Head of Engineering (Order & Pay) at SumUp, building products for these same merchants.
The Challenge
Super44's product is only as good as the numbers its agent reports. The daily briefing alone can run up to 20 tools, pulling point-of-sale data, Google reviews, staffing, and the weather. If any tool fails, or the agent skips one, the merchant gets a confident briefing built on missing data.
The team knew its two biggest failure modes and was ironing them out: the agent saying it used a tool when it didn't, and answering for two data sources after calling one. What they couldn't see was everything else. A merchant who catches the agent being wrong about their own sales stops believing everything it says. Super44 has lost at least one merchant to quality, and most don't complain first. They just go quiet.
Obvious failures get seen and fixed the same day. The dangerous ones look fine: a briefing that reads perfectly but contains a made-up number, a reply in the wrong language, a promised capability that doesn't exist. None of them throws an error, and each can cost a merchant's trust. Finding them meant reading every production trace by hand. The team did, every day, until it could not scale.
“The value of Super44 comes down to: do merchants trust that when the agent tells them something about their data, it's true? If it says ‘you've sold more Aperol Spritzes’ and they think, I don't believe you, our value diminishes. Glassray closes that gap: every run gets analysed systematically, without me reading every trace myself.”
Building Their Own Glassray
Reading every trace himself was Henry's first answer, but it didn't scale with their growth. His second was to automate exactly that: his own version of Glassray, a vibe-coded Claude routine connected to BetterStack that pulled every agent message each day and checked every reply against its tool inputs and outputs. It did its job, keeping the known failure modes under watch while the team worked them down. But the harness had problems of its own:
- Too noisy. Reading every trace every day produced an endless stream of findings, and they ended up ignored
- Oversensitive. Every finding came in flagged critical and urgent, and when Henry looked, half of them weren't real
- No memory. Each day it saw only that day's messages, so there was no way to check a fix for regressions or to compare traces across time
- Difficult to track. Reviewing an issue the moment it appeared was easy. Following it up over time wasn't: a finding was a Slack message, and once it scrolled away, so did the follow-up
“My vibe-coded spot checker was oversensitive, constantly saying it's critical and problematic. And it was noisy: going through every trace every day, you end up with this blob of text whether there's anything important or not. You then start ignoring it.”
Choosing Glassray
Glassray replaced the vibe-coded Claude routine. It watched the big fixed failure modes for regressions, flagged the traces that regressed, and discovered new deviations with far more coverage than a once-a-day sweep.
Tracking each pattern over time, it flags a deviation when it becomes important, not the first time it appears. Less noise means more attention on improving the agent and earning merchant trust.
“The statefulness of Glassray is the biggest win: it helps developer experience, reduces the noise, and helps us figure out what's important over time.”
How Super44 Uses Glassray
- 01Glassray watches every trace and intent changes in the codebase
- 02A deviation lands in Slack
- 03Henry triages it in minutes
- 04Verify the diagnosis in the Glassray dashboard
- 05Glassray drafts a fix grounded in the flagged traces
- 06The coding agent takes over via MCP and opens a pull request
- 07The change lands in GitHub or GitLab and is checked against the agent's runs
Alongside the loop, Glassray surfaced failure modes the team had never gone looking for, like the agent replying in a different language depending on what its knowledge retrieval returned.
“Now we can just use the Glassray MCP, so we can just say: hey, there's a new deviation about x. Can you help me fix it? And from that MCP alone, it can gather all of the data, find all the right traces, and so forth.”
An Example of a Silent Failure
Every morning, Super44's agent compiles a briefing for each merchant. It pulls yesterday's sales, today's staffing, recent reviews, and the weather, then writes a short report with an expected revenue figure and one recommended action. The weather matters: a heat wave means more Aperol Spritzes, and the forecast feeds the revenue estimate.
app.super44.aiGood morning! Today's sunny and mild, so expect the terrace to fill up. Projected revenue: €18,400.
Trace viewmorning briefing- okPoint-of-sale lookupsales data returned
- okReviews + staffingreturned normally
- problemWeather toolreturned empty, no data for today
- problemComposerwrote briefing using yesterday's forecast
- okFinal output‘today’s sunny and mild’ · fluent, confident
If you only check the output, this run looks fine. A merchant planning stock and staffing around invented weather would eventually notice, and they wouldn't blame the weather tool. They'd stop trusting the briefing.
“The thing that really helped the most is that I would not have had the assistance to go after all of the tiniest little nicks and paper cuts that are here and there. It's not to say that was a game-changing problem on its own. But when you have 10 or so of these types of little things, they really can come together to build a lot of trust.”
What Today Looks Like
Super44 is scaling its marketing efforts and growing its daily active merchants, as well as hiring for new roles across go-to-market and engineering. More merchants means more briefings and more conversations for the agent to get right, and Glassray keeps that covered without new headcount. Keeping a growing fleet of agents honest is nobody's full-time job: when a deviation matters, it goes through the Glassray MCP to a coding agent and comes back as a pull request.
The merchant-facing agent is the flow with the most moving parts and the one customers actually see, so that's where Glassray runs today. The team is now evaluating rolling it out to its internal agents too, starting with the lead generation flow and the web agent.
“I would probably go to the effort of building and maintaining this. And as I'm sure you know, when you start building and maintaining more and more, you start to hate yourself for it. So I'm glad I don't have to do that.”
Find the failures your evals miss
Glassray monitors your agent's production traces and surfaces the runs where it looked like it worked but didn't.