Apache Polaris
Open REST catalog for Apache Iceberg with role-based access control and audit logging. Lets agents resolve table identity and permissions through a standard interface, rather than each vendor rolling its own.
Specification →§ 03 · Governance
Human analysts bring implicit governance to every query. Autonomous agents lack these safeguards entirely. In agentic systems, access controls, data quality validation, and comprehensive audit logging transition from organizational best practices to load-bearing infrastructure.
§ Role in the stack
Human analysts understand access boundaries, apply contextual judgment to anomalous results, and recognize when data looks wrong. They ask a colleague before exporting a customer list. They re-run a query when a number seems implausible. They pause before writing to production.
An agent does none of this by default. Whatever it is instructed to do, it does, at machine speed, across every table it has a credential for. Governance is where the organizational rules an analyst carries implicitly get encoded so the agent is bound by them explicitly.
The four primitives at this layer: access control (who, or what agent identity, can read or write which data), data quality (does the data meet contracted expectations before the agent reasons over it), lineage (which queries the agent ran and what they touched), and audit (an immutable record sufficient for debugging and compliance). Absent any one of these, agents amplify errors at machine speed with no human checkpoint.
§ Open standards
Governance is the layer with the most consequential open-standards shift in 2024–2026: catalog interfaces moved from vendor-proprietary to open specs. This is what makes multi-engine, multi-agent governance tractable.
Open REST catalog for Apache Iceberg with role-based access control and audit logging. Lets agents resolve table identity and permissions through a standard interface, rather than each vendor rolling its own.
Specification →Databricks open-sourced Unity Catalog in 2024. Governs tables, models, functions, and files under one permission model. The open version is now the governance substrate for non-Databricks agent deployments too.
Specification →Standard event model for data lineage. When an agent queries a table, OpenLineage events capture what ran, what it read, and what it wrote. This is what makes "which agent touched this table?" a structured question instead of a log search.
Specification →A YAML format for data contracts: schema, quality expectations, freshness, and access terms between producers and consumers. Agents reading from contracted tables get machine-readable guarantees about what they can assume.
Specification →§ Failure modes
Agent identity collapsed into a service account. Every agent invocation runs under the same warehouse role, with the same permissions. Audit logs show the service account, not the agent, not the user. This makes "who did what" unanswerable. Fix: propagate user identity through the agent, or mint per-invocation scoped credentials.
Quality checks as an offline job. Data quality runs nightly. Agents query in real time. The agent reasons over a stale row, emits a confident answer, and the quality check finds the problem six hours later. Fix: treat quality as a query-time gate for agent reads, not just a monitoring signal.
Lineage that stops at the warehouse. The warehouse knows which SQL ran. It does not know which agent plan step dispatched the SQL, which prompt produced the plan, or which user asked the question. Without end-to-end lineage, debugging an agent mistake means reconstructing state from multiple logs. OpenLineage with GenAI context fills this.
No circuit breakers for destructive operations. Most analytics agents are read-only today. A growing number call write-capable tools (CRM updates, notification dispatch, workflow triggers). Without per-action policies and dry-run modes, a plan error becomes a production incident. Fix: tool-level policy gates, distinct from credential-level permissions.