p/dagster-cloud The data orchestrator that puts developer experience first.

Start new thread

Dagster+ - Ship data pipelines with extraordinary velocity

flo merian

Bucket

•

11mo ago

Dagster+ is a category-redefining data orchestrator that brings together unique capabilities into one tool in order to speed up your development work, reduce your costs, and simplify your data platform. Try Dagster+ for free, no credit card required.

Replies

Best

Pete Hunt

Dagster

Maker

📌

Today we're launching Dagster+, the next evolution of Dagster Cloud. Dagster+ is targeted at data engineers, ML engineers, and analytics engineers, and is a major rethinking of what a data orchestrator is, where it sits in the stack, and how data platforms are constructed. Since our launch in August 2022, we have seen enormous growth in both the Dagster Core open-source project and our commercial product. We now have hundreds of commercial customers and the fastest-growing open-source community in the category. With this growth comes broader and deeper relationships with a diverse set of customers. As we engaged with these customers throughout 2022 and 2023, we heard a few key challenges: • They were spending too much money on infrastructure, especially on the data warehouse, ETL tools, and AI workloads, and struggled to understand, optimize, and control their spend. • Their stacks were sprawling and complex, with hundreds of different integrations and point solutions with varying levels of maintenance, and no single pane of glass for understanding the data platform as a whole. • They wanted to empower individual teams to work autonomously end-to-end, but struggled to balance this with the centralized needs of the data organization. Dagster+ is a suite of tools that sits on top of Dagster's data orchestration engine, and replaces the existing Dagster Cloud product. Dagster+ brings four new capabilities above and beyond our existing commercial product: 1️⃣ We have built a brand new data catalog and data discovery capability in Dagster+, which enables all users of the data platform to discover and leverage data autonomously. 2️⃣ Dagster+ builds on Dagster Core’s asset checks feature, introducing additional capabilities around anomaly detection, freshness, and alerting. 3️⃣ We’ve made massive improvements to our Branch Deployments feature from Dagster Cloud, and are launching a killer new feature, Change Tracking, with Dagster+. 4️⃣ Dagster Insights enables data teams to manage their spend on tools like Snowflake, Fivetran and OpenAI without doing any additional integration work. With Dagster+, Insights is now generally available and comes with many new features and integrations. All of these capabilities are enabled by Dagster's unique asset-oriented approach to data orchestration. I'm really proud of the product we've built, and I'm proud to be a part of the team that delivered it. We hope that Dagster+ helps you reduce your spend, eliminates the need for many point solutions, and empowers your teams to work more autonomously and less chaotically.