Learn extra at:
Lakeflow Designer is a part of Lakeflow, which can now be typically out there. Lakeflow has three modules — Lakeflow Join, Lakeflow Declarative Pipelines, and Lakeflow Jobs. Designer is built-in inside Declarative Pipelines.
United in goal, divided in strategy
Lakeflow Designer, regardless of being similar to rival Snowflake’s Openflow, differs in its strategy, analysts say.
“Lakeflow and OpenFlow mirror two philosophies: Databricks integrates information engineering right into a Spark-native, open orchestration material, whereas Snowflake’s OpenFlow affords declarative workflow management with deep Snowflake-native semantics. One favors flexibility and openness; the opposite favors consolidation and ease,” Ni stated.
Each the choices additionally differ in maturity, ISG’s Aslett stated. Whereas Snowflake’s OpenFlow is comparatively new, Lakeflow has matured in performance over time, with Designer being its newest instrument. “The Join capabilities had been acquired together with Arcion in 2023. Declarative pipelines performance is the evolution of DLT (Delta Stay Tables), and Jobs is the evolution of Databricks Workflows,” Aslett added.
Individually, Databricks additionally launched one other pro-code built-in improvement surroundings (IDE) for information engineers, which unifies the info engineer’s full pipeline lifecycle — code, DAGs, pattern information, and debugging — multi function built-in workspace.
Releasing Lakeflow Designer and the brand new IDE collectively is a strategic play, based on Ni, because the lakehouse supplier is concentrating on each ends of the pipeline maturity curve — low-code to maneuver quick and the complete IDE to scale and preserve pipelines.
