§ 01 · Data
The warehouse, lakehouse, and file formats underneath everything else. Open formats at this layer are what make agents portable across engines, tools, and vendors. Closed formats are where lock-in compounds silently.
§ Role in the stack
The data layer looks like plumbing. For human analysts, it often is: once the warehouse works, you move on. For agents, format choices at this layer shape which tools are available above it. An agent that can call DuckDB, Polars, Trino, and Spark natively, because the underlying data is Iceberg or Delta on Parquet, has more reachable context than an agent confined to a single engine.
Three practical consequences. First, tool coverage tracks format openness: open table formats have agent tool libraries; proprietary ones have vendor-hosted access paths only. Second, cross-engine queries become tractable: an analytics agent that joins CRM data in one engine with warehouse data in another needs a common table format to avoid per-engine ETL. Third, evaluation is cheaper: benchmarks that can run against the same data on different engines isolate engine performance from data quality.
The SQL dialect choice matters too. Agents trained predominantly on vendor dialects learn idioms that do not transfer. When you evaluate an agent's text-to-SQL capability, test against ANSI-conformant patterns and document where the agent leans on non-standard extensions.
§ Open standards
Five specifications cover the substrate. Iceberg and Delta are the dominant open table formats; Parquet and Arrow underlie both; ANSI SQL is the query baseline.
Open table format for analytic datasets at warehouse scale. Engine-agnostic: the same table reads from Spark, Trino, Snowflake, BigQuery, DuckDB, and Polars. For agents, this means a single semantic address resolves regardless of which engine the tool uses underneath.
Specification →Columnar file format that underlies most open table formats. Matters because agent tools (DuckDB, Polars, Arrow-based readers) assume Parquet as the baseline. If your data is locked in a proprietary columnar format, agent tool surface collapses.
Specification →The other widely adopted open table format. Originated at Databricks, now governed by the Linux Foundation. Delta and Iceberg are converging on interoperability; agents operating across both should use a catalog abstraction rather than format-specific tools.
Specification →In-memory columnar format (Arrow) and a network transport for it (Flight). This is how agent tools move data between steps without serializing to JSON. For long-running agent plans with intermediate results, Arrow is what keeps per-step latency manageable.
Specification →The oldest open standard in the stack. Agents trained on standard SQL port across engines. Agents that learn vendor-specific dialects (Snowflake, BigQuery, T-SQL extensions) carry portability debt. Portable dialects stay cheaper to evaluate.
Specification →§ Failure modes
Non-standard SQL dialects. The agent generates SQL that runs on the original engine and fails on everything else. This is an evaluation failure first: if your benchmarks only run against one dialect, you will not notice when your text-to-SQL quality collapses on the next engine you add.
Closed catalog formats. The warehouse exposes tables through a proprietary catalog protocol. Every agent tool needs custom integration. Every identity mapping is bespoke. Apache Polaris and Unity Catalog OSS exist so this stops being true; pick an engine that speaks them.
Schemas that obscure semantics. Tables named t_1, columns named col_a. The agent, reasoning over schema alone, cannot produce grounded SQL. This is technically a semantic-layer problem, but it originates here: the data team chose names before the semantic layer was introduced, and the agent inherits the opacity.
Mixing open and closed formats silently. Some tables live in Iceberg, others in a proprietary format the query engine handles transparently. The agent cannot tell the difference until a tool breaks on one table and not another. Document which tables are format-portable and which are engine-locked, and surface this to the agent's tool registry.