Niro RCA · private beta

Alert at 01:19.
Verified cause at 01:27

A team of Niro agents investigates in parallel on the code graph and hands you the high-confidence causes, with the evidence chain behind each one. Not a replacement for your on-call. The layer under it.

Request access
Example incident INC-2041
8m
alert to declared cause
26
tool calls, 5 agents
0
files read
EVIDENCE CHAIN 4 HOPS · 3 REPOS checkout-router CheckoutRouter.route:77 hop 1 infra-config enableCheckoutV2Routing = true hop 2 order-events OrderReconciler.batchUpdateOrderEvents hop 3 order-events OrderReconciler.java:214 ← locks in HashMap order cause ✓ ROOT CAUSE VERIFIED Latency: 11ms · Confidence: 96%
Fig. 1: the evidence chain. The flag opened the path; the lock order at :214 is the cause.

The incident workbench

Interrogate any finding. Every answer cites the edge or the commit it stands on, and says so when it cannot.

INC-2041 · checkout-router p95 Mitigation pending opened 01:19 · cause 01:27
Alert fired 01:19 for checkout-router p95 latency: 4.8s against a 240ms baseline, 3.1% of requests erroring.
alert01:19 · checkout-router p95 4.8s (baseline 240ms) · 3.1% errors
blast5 services behind checkout-router degraded; no data loss
triggerautoscale 2 → 27 pods at 01:12 under the morning surge
cleared01:34, six minutes after the flag was reverted
OrderReconciler.java:214 acquires row locks in HashMap iteration order.
causeOrderReconciler.java:214, locks acquired in HashMap iteration order
justificationconcurrent batches acquire the same rows in different sequences; circular waits follow
why nowcheckout-v2 routing opened the path; 27 pods made collision probable
honestystatic signals confirmed against the deploy timeline, not runtime proof
Four pieces carry the verdict. Each one is checkable.
edgeroute() → batchUpdateOrderEvents exists only behind the flag
timingdeadlock rate follows pod count (01:12), not the flag flip (00:58)
codeHashMap iteration at :214, no ordering guarantee in the type
historyno prior bug on this path; the reverted a3f91c retry storm is unrelated
Two moves: revert the flag now, fix the lock order for good.
nowrevert enableCheckoutV2Routing, one line, reversible
durablesort order IDs before the batch update for a deterministic lock order
reproflag on, 8+ concurrent batches over shared rows, deadlock within seconds
Discussion2 · open
RK
rohan · 01:29
Why 01:12? The flag flipped at 00:58.
niro · 01:29 · cites trace_execution_path
Two pods kept lock order stable by accident. The 01:12 scale-out to 27 pods raised concurrent batches past the point where circular waits become probable. Deadlock rate follows pod count, not the flag timestamp.

Anyone on the incident can reply here: ask Niro, or @mention a teammate.

Investigation threads

Anyone can open a thread against the same graph. A refuted thread closes with its evidence attached; a confirmed one merges into the main investigation.

Main investigation root cause declared
statuscause at OrderReconciler.java:214 · confidence 96%
merged inevidence from 2 side threads, one refuted, one folded
opendurable fix PR for deterministic lock ordering

The trail on the report is this thread. Steering it ("check the failover first") reorders the agents’ queue.

Could it be the DB failover at 00:40? refuted · closed
checkedfailover completed 00:41; connections re-established by 00:43
againstno lock metric moves between 00:43 and 01:12
closed byverifier · evidence attached to the main thread

A refuted thread keeps its evidence. The next engineer who suspects the failover reads why it was ruled out.

5xx spike on payments: related? folded into impact
findingpayments 5xx are downstream of checkout-router timeouts
directionsymptom, not cause; callers of route() time out first
foldedinto the blast-radius list on the main report

Not every thread finds a cause. This one corrected the impact map, which is also the job.

The root cause and its evidence

A cause you cannot check is a guess with confidence. Every link in the chain is an edge in the graph or a line in the deploy log.

Causal chain 5 links · all checkable
100:58 · infra-config ships enableCheckoutV2Routing: true
↳flag opens route() → batchUpdateOrderEvents
↳:214 acquires row locks in HashMap iteration order
↳01:12 · autoscale 2 → 27 pods, concurrent batches ×13
✗circular waits · p95 4.8s · deadlock storm
Evidence 7
✓The failing path is unreachable with the flag off; the edge exists only behind route()
✓Deadlock rate first moves at 01:12, with the scale-out, not at the 00:58 flip
✓HashMap key set at :214 has no ordering guarantee; two pods masked it by accident
✓No timeout guards the batch; one stalled lock holds the connection pool
✓5 downstream services degrade in caller order, checkout-router first
✓No prior bug recorded on this path; a3f91c retry storm ruled out
✓Errors stop at 01:34, six minutes after the revert; both patterns at once
The same incident, two ways
On-call, reading files The agent team, on the graph
Where you start grep for the error string the alerting service
Hypotheses one at a time, in your head in parallel, each a thread
Config as evidence not read read as nodes
A ruled-out theory lost in the channel scrollback a closed thread, evidence attached
When you stop when it looks plausible when the chain is complete

FAQ

Can I use RCA today?

It is in private beta: an add-on for Team, as packs or a subscription, and included org-wide on Enterprise. It is not on Free or Solo.

Do the agents touch production?

No. They work on the graph and config read as nodes, never logs or live systems. Findings are structure resolved on the graph, then confirmed against the deploy timeline. Niro writes nothing to your repositories and ships nothing to production. What you do with a cause is entirely yours.

How is Niro’s team of agents different from one agent in a loop?

Multiple hypotheses run in parallel instead of in sequence, and the Verifier argues with the findings before you see them, the same self-check discipline the code reviewer uses. A refuted thread closes with its evidence attached rather than disappearing.

What if the cause is not in the code at all?

Then no cause is declared. You get the candidates with why each is a candidate, the threads stay open, and the trail records what was ruled out, which is itself worth having at 2am.

Run it on your next incident

The one that would have taken an afternoon. Index the repos it touches and let the team take the first eight minutes.

Request access See the graph underneath