The Agent’s Word Is Not Enough: External Validation in the Agentic Governance Stack
An agent that runs its own tests and reports on its own results is the oldest closed verification loop in software engineering. The final layer…
Read more →Insights & engineering
Blog
Technical analysis and engineering decisions from building enterprise data reliability on Databricks.
Exit Code 2: How Claude Hooks Turn Agentic Rules Into Runtime Barriers
Documented rules don't stop a confident subagent. Hooks do. The runtime-enforcement layer of the agentic governance stack — what a protected-branch guard actually looks like…
Read more →
CLAUDE.md Is Not Enough: The Governance Stack for Agentic Development
The standard advice for governing AI coding agents is to write a good CLAUDE.md. That answers orientation. It does not answer governance, error rates, or…
Read more →
The Entropy Floor: When an Entropy Drop Means Your Data Is Doing Its Job
Not every drop in column entropy is a data quality failure. The harder problem is distinguishing unintended collapse from intended convergence — the signal a…
Read more →
Why Semantic Layers Need Distributional Validation, Not Just Schema Validation
Schema and freshness checks pass while semantic-layer metrics silently degrade. Distributional validation — grounded in Shannon entropy — is the missing trust layer for dbt…
Read more →
From Theory to Evidence: Validating Shannon Entropy for Data Quality at Scale
We ran a structured sequence of preregistered experiments across three real-world datasets totaling nearly 6.6 million rows to demonstrate that Shannon entropy catches data quality…
Read more →
Why Shannon Entropy Catches What Schema Validation Misses
Schema checks validate shape. Entropy measures information content. Here’s why the gap between them is where your pipeline silently fails — and how to close…
Read more →