For network engineers deciding how much of an incident to hand to a model, and how much to keep on a leash.
Every tool in Network Ghost Agent sorts into one of two categories: deterministic automation, or AI reasoning applied to what the automation produced. Lining up all eight and classifying each one puts a number on something worth stating plainly — five of the eight run with zero AI anywhere in the execution path, and every one of the other three still leans on automation to do the parts that don’t require judgment. Not “mostly AI with some guardrails.” Mostly automation, with AI kept to the parts nothing else can do.
Ghost Agent investigates Azure connectivity failures end to end: NSGs, route tables, OS-layer firewalls, packet captures, VM-to-VM throughput. Building each tool inside it meant answering the same question every time. Does this step reason under uncertainty, or does it need to be exactly right the same way on the millionth run as the first? Doing that audit properly, tool by tool, is the rest of this post.
The Audit
| Tool | Bucket | Why |
|---|---|---|
| Ghost Agent orchestrator | AI | Hypothesis formation, tool selection, escalation reasoning — no fixed step sequence covers every fault |
| Agentic Safety Shell | Automation | Four-tier command classifier. No model anywhere in the decision path |
| PCAP Forensic Engine | Mixed | tshark extraction + percentile stats + pattern counts = automation. Cross-protocol correlation + severity ranking = AI |
Netfilter Inspector (--explain) |
Mixed | Structural parsing = automation. First-match evaluation, chain reachability, negation = still AI — no deterministic equivalent found, commercial or open source |
| Effective Network Inspector | Automation | Snapshot + SHA-256-verified diff of effective routes and NSGs |
| Effective Route Inspector | Automation | Deterministic longest-prefix-match algorithm. Returns TIED_BGP and stops rather than guess a winner |
| Security Rule Inspector | Automation | Dual-gate NSG evaluation engine. States outright: it does not produce AI-generated verdicts |
| Agentic Pipe Meter | Automation | P90 statistics, anomaly detection, baseline delta. No LLM in the measurement pipeline, by design |
Five of eight, fully deterministic — and the other three still hand the parts that don’t need judgment to automation, keeping AI for the parts that do. That’s not what “AI network agent” sounds like from the outside. That’s reason enough to pause on the two that split the work before moving on.
Built to Stop, Not to Guess
Three of the automation-bucket tools don’t just happen to avoid AI. They were built to fail closed, and each one says so explicitly.
The route inspector applies Azure’s actual route-selection algorithm — CIDR containment, longest prefix match, source precedence, BGP tie-break — in plain Python. When two VirtualNetworkGateway routes tie on prefix length, the real tiebreaker is BGP AS Path length, and that field isn’t present in the effective route table JSON. A tool that wanted to look decisive could pick one route arbitrarily and move on. This one returns TIED_BGP and stops. Nothing gets guessed.
The security rule inspector hits the same wall from a different angle. An NSG rule that references an Application Security Group can’t be evaluated without a separate query for ASG membership. Skipping that rule and evaluating the next one down the priority list still produces a verdict — just not necessarily the correct one, since the skipped rule might have matched first. The engine returns INDETERMINATE instead, and the second gate never runs. A wrong verdict delivered with confidence is worse than an honest one that admits it doesn’t know.
The pipe meter states the same principle up front rather than backing into it. Keeping the model out of the measurement pipeline is a deliberate choice, made so correctness stays exact instead of probabilistic. P90 computation, anomaly detection, baseline deltas — same path, same answer, every run.
None of these three tools would gain capability from a model bolted on. They’d lose the one property that makes them worth trusting: the same input produces the same output, checkable by anyone, every time.
Where the Line Actually Moves
The two tools that don’t sort into a single bucket deserve closer attention — they’re where the real work of drawing this line happens.
The PCAP forensic engine splits cleanly in two. A raw .pcap file can run to tens of thousands of frames, and handing that straight to a model produces what you’d expect from asking someone to eyeball a spreadsheet with 50,000 rows. It misses things. Silently.
capture.pcap (134 packets — could be 134,000)
│
▼
┌───────────────────────────────────────────-─┐
│ AUTOMATION — deterministic, every time │
│ │
│ tshark extraction: per-protocol fields │
│ Percentile math: min / median / p95 │
│ Pattern counts: retransmits, │
│ duplicate IP-to-MAC claims │
│ Output: Semantic JSON, ~10–50 KB │
│ Up to 95% size reduction, no loss of │
│ the fields that matter │
└─────────────────────┬───────────────────────┘
│
▼
┌────────────────────────────────────────────┐
│ AI — judgment, applied to a summary │
│ it didn't have to build itself │
│ |
│ Cross-protocol correlation: │
│ "unanswered ARP + ICMP unreachable │
│ for the same host = confirmed gap, │
│ not two unrelated LOW findings" │
│ Severity ranking, RFC-grounded │
│ interpretation, remediation commands │
└────────────────────────────────────────────┘
The counting, the percentiles, the reduction to a summary in the 10–50KB range — none of that touches a model. The actual job left for AI — noticing that a LOW-severity unanswered ARP finding and a HIGH-severity ICMP unreachable finding describe the same dead host — is judgment a fixed rule would have to be told in advance to look for.
The --explain feature doesn’t split this cleanly. It has a parser in front of it too, extracting tables, chains, rules, typed fields like _negated: true, a rule’s jump target. That part is deterministic, same as the PCAP pipeline. But the parser doesn’t resolve what the ruleset actually does. First-match order, tracing a chain jump to its terminal verdict, reading a negation correctly — that evaluation still happens inside the model, even though every one of those questions has exactly one right answer. There’s no judgment call in whether ! -i docker0 means traffic not arriving on docker0. It either does or it doesn’t.
An early version got that wrong in eight distinct ways across 23 test rulesets: negations inverted, near-identical rules merged into a single description with the unreachable duplicate never flagged at all, a chain jump described as a destination instead of traced to what actually happens there, and — the one that should have been impossible — a fully invented call into a chain no rule in the file referenced at all.
The fix wasn’t to move the evaluation into the parser. It stayed in the model. Reliability came from testing instead: twenty-plus explicit principles written into the system prompt, each backed by a unit test, plus a mandatory reachability-mapping step the model has to run before it’s allowed to describe any chain at all.
Deterministic firewall rule analysis isn’t new — Al-Shaer and Hamed published algorithms for detecting shadowed and redundant rules back in 2003, and commercial platforms like AlgoSec and Tufin have shipped rule-base analysis for enterprise firewalls for close to two decades. What that work targets is different, though: generic packet-filter rule models and appliance firewalls, not the specific chain-traversal and negation mechanics of Linux netfilter. I haven’t found a tool, commercial or open source, that takes raw iptables-save or nftables output and narrates evaluation-order-aware English explanations the way --explain does. Testing against 23 rulesets, not a belief the underlying problem was untouched, is what got this one reliable.
What Correct Numbers Don’t Protect You From
A tool can hand Ghost Agent a perfectly correct fact and still leave the hardest part of the job undone. The pipe meter’s own documentation names this directly, and one case lays it out clearly enough to walk through in full.
One of the tested scenarios injects a tc tbf rate 5mbit rule on the source VM alongside an unrelated iptables DROP ICMP rule on the destination. The pipe meter runs its measurement and reports back: is_stable: true, anomaly_type: null, throughput_p90: 0.005 Gbps. Every sample came back identical. No anomaly fired, because the anomaly detector only flags connection drops and inconsistent results across iterations — and this path is neither dropped nor inconsistent. It’s just wrong, consistently, every single time.
5 Mbps is not a healthy number on an Azure accelerated-networking path that should deliver over 2 Gbps. Nothing in the tool’s own output says so. is_stable=True looks like a clean bill of health to anyone who stops reading at the flag. Catching it requires knowing what a healthy number looks like on this specific path — which is exactly the judgment the measurement layer was deliberately built not to embed, because platform throughput expectations vary by VM SKU and nobody wants that assumption baked into a tool that has to work the same way across environments.
Correct measurement, wrong conclusion, is possible even when every deterministic tool in the chain did its job exactly right. That gap — between a fact being correct and a fact being correctly understood — is where the tools that compute facts nobody can dispute hand off to the one layer that has to reason about what those facts mean. That’s the layer where “network AI” earns the AI part of its name.
So, Is It Just Automation?
No, and the audit is the reason, not a hunch. Five of eight tools in Ghost Agent run with no model anywhere in their execution path, several of them built explicitly to stop and admit uncertainty rather than let a model paper over a gap they can’t resolve. The other three still split the work rather than handing it to AI wholesale: the orchestrator reasons from the start, while PCAP correlation and firewall evaluation lean on automation for everything that doesn’t require judgment and reserve AI for the one piece that does.
What AI does in this stack is narrow in surface area and total in importance. It doesn’t compute the route, evaluate the NSG, or measure the throughput — five tools do that without it. What it does do: choose which of those tools to call and in what order, read what each one hands back, and weigh whether that finding confirms or refutes the working hypothesis. Whether a hypothesis has been tested enough to move on, and whether the investigation is finished or needs another layer, are calls only this part of the stack makes. None of that is a computation with one correct answer the way a route lookup is — it’s a sequence of judgment calls under partial information, made one after another, each one changing what the next tool call should be.
There’s no deterministic version of that sequence to fall back on. You can’t write an algorithm for “is this hypothesis confirmed enough to stop investigating” the way you can write one for longest-prefix match. What stands in for an algorithm is a system prompt: the same discipline used to keep --explain reliable without a parser underneath it, extended to the harder problem of running an entire investigation. Five deterministic tools can be exactly right and still leave the actual job undone, because none of them decide anything — they answer the one question they were asked and stop. The layer that strings their answers into an investigation, and knows when to call it finished, is the one job in this stack that’s AI’s alone.
The Questions Behind Every Tool Decision
Three questions did the actual sorting for all eight tools above, and they generalize past this toolkit.
Does the step have exactly one correct answer given the same input? If yes, it’s automation, and testing that it’s always right matters more than testing that it explains itself well. This is what put route selection, dual-gate NSG evaluation, and P90/anomaly detection in the automation column without much debate.
Can it fail by staying silent instead of guessing? If a step is going to sometimes hit a genuine gap — a tie, a missing field, a reference it can’t resolve — build the refusal into the algorithm itself, rather than asking a model to flag its own uncertainty. This is what the route and security rule inspectors do: when the input doesn’t support a confident answer, they return TIED_BGP or INDETERMINATE and stop, instead of picking a route or a verdict just to look decisive. An algorithm can check for that gap directly, before it produces anything. A model can’t check for it the same way — it produces a plausible answer regardless, and there’s no way to trust its stated confidence to reveal when that answer was actually a guess. If the step you’re building will face real ambiguity sometimes, that’s a vote for putting the certainty check in code, not for asking the model to self-report it.
Does answering it require weighing more than one prior finding against each other? No tool in the audit could answer this with a static algorithm, because the right answer depends on everything found so far, not just the input sitting in front of it. That’s the pattern behind every judgment call the orchestrator makes — and the one job in this stack no static algorithm was built to do.
Run whatever you’re building against these three questions to get a better idea of what you will use the AI model for. They held across eight tools in a fairly narrow domain. See if they hold for yours.
I’d like to hear where other people have drawn these same lines — in network engineering, security operations, database administration, anywhere an agent gets shell access to something that’s currently on fire. Where did automation stop and judgment start in what you built? What told you that you’d drawn it in the wrong place the first time?
GitHub: github.com/ranga-sampath/agentic-network-tools
Clone the repo. Pick a tool. Decide for yourself which bucket it belongs in.