Apache Spark
Snowflake prepares more autonomous data pipelines and AI agents
Snowflake has unveiled several data management updates designed to support AI agents and applications. The announcements cover everything from automated compute…
Databricks makes Real-Time Mode generally available for Spark Declarative Pipelines
Databricks has made Real-Time Mode (RTM) generally available on Spark Declarative Pipelines (SDP), bringing end-to-end streaming latencies in the millisecond range to…
Databricks Benchmark: Declarative Spark Pipelines Cut Agent Compute Time About 34x
Databricks says Spark Declarative Pipelines (SDP) are the better fit for AI agents that build production data pipelines. In its own benchmark, declarative builds used…
NVIDIA Shows How AI Agents Can Rewrite Spark UDFs for GPU Speedups
NVIDIA has published a guide to using coding agents to rewrite Apache Spark user-defined functions (UDFs) into GPU-accelerated equivalents, which it says can deliver…