Inside the Prism Dashboard: A CM Control Plane in Production

The framework this series has described — Baseline, Change Control, Drift Detection, Audit & Remediation, layered with a Key Risk Indicator tier model — is not theoretical. It is operating right now, on production infrastructure, governing the very system this firm runs on.

This post walks through the implementation. It is honest about what the implementation does today and what it does not. The dashboard described here runs Prism AI Analytics' business operations — clients, projects, financials, the daily-review cadence, the activity log — and the framework governs that system: its code, its configuration, its deployed state. It is not yet wired up against fully autonomous AI agents in production — though, as the later section on extending to agents describes precisely, the framework’s application to Prism’s own internal agents has already begun and sits in the gated, pre-production state the framework itself prescribes. The architectural patterns are the same; the asset class being governed is different.

Two reasons to publish this walkthrough rather than wait until we have a fully agent-governing version:

First, the framework’s claim is that the disciplines that worked for systems work for agents. Demonstrating the disciplines on a working system is honest. Claiming we already run them on agents we have not yet deployed would be dishonest. The post is an evidence walkthrough, not a product demo.

Second, most organizations evaluating CM discipline for AI agents do not have specialized tooling either. They have what we have — a single-server stack, a SQLite or Postgres database, an Express or FastAPI route layer, a vanilla frontend. The framework was deliberately designed to be implementable in that environment. This walkthrough shows it is.

What the dashboard is

The Prism dashboard is a single-file Express.js backend (one server.js), backed by a SQLite database with WAL mode enabled, serving a vanilla HTML and JavaScript frontend from a public/ directory. There is no bundler, no framework, no TypeScript, no microservices. The deploy target is a single Railway service. The total operational footprint is about what a small business accounting package would consume.

It runs Prism’s CRM with a twelve-stage pipeline, integrates Stripe for payments and QuickBooks for accounting, ingests daily review notes from an Obsidian vault via a sync script, captures a Claude Code activity log, scores AI readiness assessments using a documented rubric, syncs canonical state to Notion, and serves the AI-readiness assessment landing page.

It is, in other words, the operational mission-control surface for a small consultancy. That is what makes it instructive. Most organizations deploying AI agents will be running them inside a similarly-sized operational stack. The CM discipline scales down before it scales up.

Where the four disciplines show up today

A note on what is being governed. The configuration items here are the system itself — its deployed code, its runtime configuration, and the definitions that set its behavior. They are not the business data the dashboard manages. A customer record or an invoice is data flowing through the application; it is not a configuration item held to a baseline. The disciplines below run on the system, and the same mechanics are what extend to an agent.

Baseline

A baseline is the approved state of a configuration item. The dashboard has one for itself. Its production state is a specific commit on main — nothing runs in production that did not arrive there through the change-controlled path below. Its behavior is set by declared configuration: the integration settings, the constants that define each workflow’s rules, the database schema with its constrained, enumerated states. The approved shape of each is known and recorded, so a change is measured against it rather than discovered after the fact.

Where the framework runs deepest, the baseline is captured as a content hash rather than prose. Each governed agent configuration carries a hash of its approved definition; the firm’s wider population of governable items is captured in a dated baseline snapshot against which any later state can be compared. A baseline you can hash is a baseline you can check automatically.

Change Control

Change control governs how that approved state is allowed to move. Every change to the dashboard travels one path: work happens on a branch, never on main; a written description and an automated review precede a human merge; merges are squashed so each change lands as a single, revertible commit. The protocol is not aspirational — it exists because production was wiped twice, and the incident-findings document is required reading before any work begins. The audit trail is the git history, the pull-request record, and the deploy events.

Applied to an AI agent, the same path governs prompt revisions, tool-inventory changes, and model-version upgrades — each one a change to the agent’s approved configuration, each one routed through the same gate. The mechanics are unchanged.

Drift Detection

Drift is any divergence of the live state from the approved baseline. Because the baseline is a content hash, detecting drift is a comparison: hash the configuration as it exists now, check it against the approved hash, and a mismatch with no authorizing change-log entry is a configuration-integrity event. At fleet scale this runs as an automated scan across every governable item the firm tracks, with the content hash as the drift signal — the comparison the framework was built to make routine.

What the scan does not yet carry is the full tier model on top of it: the instrumentation point exists, and the threshold-and-tier calibration is the work described later. The next section is the time this fired on a change no one had staged as a test.

Audit & Remediation

Every change leaves a record, and every change can be undone. The audit trail is the git history, the pull-request record, and the deploy log — each deploy named, tagged, and individually rollbackable. Remediation is rehearsed rather than improvised: the deploy runbook documents the rollback procedure, and it has been exercised in production — twice, after the incidents that produced the protocol now in place.

Where the framework runs deepest, the same discipline is pushed down to the configuration item. Each governed agent carries a rollback procedure rehearsed before it is needed and a hash-chained change log, so the restore point — the last approved baseline — is never ambiguous.

The drift the framework was not looking for

The most persuasive evidence that drift detection works did not come from a test we designed. It came from one we did not.

During a routine rollback rehearsal on one of Prism’s own governed configuration files, the file had already been edited earlier in the session — a new section added for an unrelated reason. The edit was legitimate and benign, but it shifted every line below it, leaving the governed content byte-for-byte unchanged while moving where that content sat in the file.

A naive integrity check keyed to line numbers would have raised a drift alarm: the governed text was no longer on the lines the baseline named. The framework’s production method keys the baseline hash to a content boundary — a named heading — rather than a line range. It extracted the same bytes from the moved location, matched the approved hash, and correctly reported no drift.

The lesson is the one the discipline exists to teach. The value of drift detection is not catching the failures you stage on purpose; it is making the right call on the changes you did not see coming — including the benign ones, where a false alarm is its own kind of failure. The control distinguished a harmless structural edit from a substantive one, against a change no one had set up as an exercise.

The KRI tier model, on its own terms

The four disciplines above keep an asset in its approved state. The Key Risk Indicator tier model is what the framework adds specifically for agents: it decides how hard to respond when an agent moves away from that state. It is defined on its own terms — not borrowed from any business-records surface.

A KRI is a measurable signal that an agent is drifting from its approved baseline — a rising rate of unsupported claims, an output distribution that no longer matches the approved profile, a token cost per action climbing past its band, or the agent’s own configuration diverging from the hash that was signed off. Each KRI carries three graded responses:

• Tier 1 — warn. Crossing the threshold logs the event and surfaces it to the operator within a defined window. The agent keeps running.
• Tier 2 — constrain. Crossing the threshold triggers an automatic action — throttling the agent’s budget, pausing a capability, or blocking a write — and pages a human. The agent runs in a reduced state until it is reviewed.
• Tier 3 — halt. Crossing the threshold trips a kill-switch, rolls the agent back to its last approved baseline, and opens a written incident record.

The thresholds are calibrated against an observation window rather than guessed: in the starter set, Tier 1 sits at the 95th percentile of observed behavior, Tier 2 at the 99th, and Tier 3 at the floor below which the agent’s behavior is no longer defensible. Until that window closes for a given agent, the thresholds carry explicit pending-calibration markers rather than false precision.

This is the layer the dashboard’s business surfaces do not need. The CRM manages a sales pipeline; the financials surface reconciles revenue. Neither is a risk-governance instrument, and the framework does not pretend they are. The tier model is the part built for the asset class that can act on its own.

What it would take to extend this to AI agents

The honest answer to "how do you operationalize this framework?" is: the same way Prism operationalized it for its own business records — and, increasingly, the same way Prism has begun operationalizing it for its own AI agents.

That second effort is underway, and it is worth describing precisely, because precision is the point. Prism’s internal Claude-based agents — the ones that run its chief-of-staff and marketing operations — each now carry the framework’s four mandatory configuration artifacts: a documented baseline, a KRI threshold set, a rollback procedure, and a hash-chained change log. Rollback has been rehearsed against a non-production replica and passed.

What has not yet happened is the formal pre-production sign-off the framework itself requires before an agent is treated as governed — and Prism is holding its own agents at that gate rather than waving them through. The sign-off also carries a documented exception, handled the way a maturing program should. Where a one-person firm cannot yet separate a baseline’s author from its approver, the framework requires the gap to be logged as a named, dated deviation with a resolution owner — a designated external reviewer as the firm grows — rather than passed over in silence. It is written into the change log, because a control you quietly relax is not a control.

There is a second number worth stating plainly, because the framework measures it and Prism holds itself to it. A governance program’s performance is its coverage — the share of in-scope configuration items carrying an evidence-backed, current approval, not merely an author’s initials. Measured across Prism’s full population of governable items — the agents, skills, hooks, and connected servers that run the firm — that coverage is about ten percent today, and the highest-stakes items are uncovered — explicitly sequenced for the next baseline cycle, not quietly assumed covered. Configuration drift reads clean, but only because the baseline clock just started; the honest live signal is exposure, not drift. We state it the way the framework requires: a program that cannot report its own coverage has no standing to ask anyone else to adopt one.

So the first asset class — business records — is governed today. The second — Prism’s own agents — sits in the gated, pre-production state the framework prescribes: artifacts authored, rollback rehearsed, sign-off pending. The walkthrough below describes the remaining reach: a fully autonomous AI agent deployed in production under live governance. If Prism deployed one tomorrow — say, an outbound sales agent generating discovery-call follow-up emails — the extension would look like this:

Six steps, each reusing infrastructure the dashboard already runs — a table, a route, a constants block, a rollback path:

1. Add the agent to the asset inventory (Baseline). A new row in a production_agents table, built the same way the dashboard’s other tracked-entity tables are — capturing model identifier, model version, system-prompt content hash, tool-inventory JSON, data-scope JSON, owner, baseline-approved date, and current-status enum.

1. Define the baseline (Baseline). An output-distribution profile over a 14- to 30-day observation window, plus latency band, refusal-rate baseline, and cost-per-action target — written into a baseline record linked to the agent, modeled on how the dashboard already captures daily metric snapshots.

1. Wire change control (Change Control). Any modification to the agent’s row runs through the same PR-and-merge protocol the dashboard already uses. The audit trail is the git history plus the existing change-tracking record.

1. Instrument drift detection (Drift Detection). Per-agent metrics from new instrumentation — signals such as unsupported-claim rate, output-distribution shift, tool-call error rate, and refusal-rate inversion. A new /api/agents/triggers endpoint raises any agent whose behavior has crossed a tier threshold.

1. Define the tier-escalation table (KRI tier overlay). A constants block in server.js defining the Tier 1 / 2 / 3 thresholds and their responses, with per-agent overrides for organizations whose risk appetite differs from the starter-set defaults.

1. Add the rollback path (Audit & Remediation). A /api/agents/:id/rollback endpoint that restores the agent’s most recent approved baseline — tested before the agent enters production, rehearsed quarterly, reusing the existing soft-delete and Railway deploy-rollback patterns.

The effort is an estimate, not a measured result: roughly three weeks of engineering for the first agent, declining to about three days per additional agent once the patterns are factored — and they are already factored, for business records. We will publish the actual figure once the extension ships.

The framework’s architectural claim

The dashboard is the existence proof for an architectural claim the framework makes: configuration management for AI agents does not require new tooling. It requires the discipline of treating every agent in production as a configuration item with a documented baseline, governed by a defined change control process, monitored by a tiered KRI escalation model, and audited against the baseline that authorized any given action.

The infrastructure to do this is the infrastructure most operational teams already have. A database. A route layer. A defined enum of approved states. A change-tracking timestamp. A tier-aware escalation endpoint. A rollback procedure that has been exercised before it was needed.

What is rare is not the tooling. What is rare is the discipline. The dashboard described here is the discipline applied to one class of asset. Extending it to autonomous agents is the next class. The mechanics are the same.

What this post is not

This post is not a vendor pitch for the Prism dashboard as an AI agent governance platform. The dashboard is internal infrastructure. We do not license it, sell it, or position it as a product. The dashboard is offered here as evidence — that the framework is implementable inside a small operational footprint, that the disciplines transfer from business records to agents by the same mechanics, and that the architecture is straightforward enough to describe in a single blog post.

The framework reference document and the KRI starter set are the deliverables. The dashboard is the proof that we use them on ourselves.

The final post in this series crosswalks the framework against the IAPP AIGP Body of Knowledge for governance professionals studying for the certification. It is the closing of the series — the operational layer the AIGP curriculum implies but does not yet fully detail.

This is part 5 of a six-part series on Configuration Management for AI Agents. Part 1 stakes out the latency gap. Part 2 introduces the four-discipline framework. Part 3 details the KRI tier model. Part 4 maps the framework to NIST AI RMF. Part 6 crosswalks the framework against the AIGP Body of Knowledge.