12 articles tagged
Databricks changed how dbt tasks refresh materialized views and streaming tables on 20 August 2026. Jobs that already had Performance optimized turned off for cost now pick up 4 to 6 minute startup latency.
AWS Glue 6.0 went generally available on 21 August 2026 with a 30% price reduction. The cut is version-gated, several meters are excluded, and Iceberg v3 tables cannot be read by Athena SQL.
AWS Clean Rooms can now export Spark logs for SQL queries. The logs are redacted: record counts round down to an order of magnitude, values under 100 report as zero, and error text and stack traces are removed.
MWAA Serverless can finally run your own Python and shell code, but the 250 MB package cap, 60-minute task timeout, fixed 1 vCPU worker and no-internet default decide whether your DAGs can actually move.
AWS shipped long-term Redshift system table retention on 20 August 2026. The launch post says AWS manages retention. The management guide says Redshift defines none, and unconfigured data is kept forever.
Most data platform budgets miss because the build is quoted and the run is not. Here are the August 2026 list prices for Snowflake, BigQuery, Redshift, Databricks, dbt and Fivetran, what drives the bill, and how eCorpIT
India's DPDP Rules were notified on 13 November 2025 with a three-phase timetable: consent managers in November 2026, substantive obligations in May 2027. How to build a first-party CDP that survives both.
The open table format is the most consequential lakehouse decision, and in 2026 the answer is no longer obvious. Iceberg v3, Delta UniForm and catalog federation are pulling the two formats together. Here is where each
Databricks LTAP unifies transactions and analytics on one copy of data in the lake; Snowflake answers with Snowflake Postgres and Unistore. How each kills the ETL pipeline, and which fits your stack in 2026.
Databricks Lakehouse//RT runs real-time queries directly on Delta and Iceberg at sub-100ms and 12,000 QPS. We compare it with ClickHouse, Apache Druid and Apache Pinot on latency, concurrency, cost and ops burden.
The RBI's draft Guidance on Regulatory Expectations for Data Governance runs to 63 paragraphs across 6 chapters and covers 11 categories of regulated entity. Here is the engineering work it implies.
Agents run heavy queries across your whole organisation. If the data is fragmented, the agent is guessing. Here is what the data layer needs before an agent is worth deploying.
