Latest news with #dbt
Yahoo
28-05-2025
- Business
- Yahoo
dbt Labs Launches AI-Powered Features to Onboard Data Analysts into dbt
Analysts can now build, explore, and validate models leveraging the power of dbt PHILADELPHIA, May 28, 2025 /PRNewswire/ -- dbt Labs, the leader in standards for AI-ready structured data, today announced a powerful new suite of AI-enhanced features that give data analysts a fast and governed way to explore data and deliver insights within dbt's workflows. These new capabilities empower analysts across a range of technical backgrounds to lean on natural language or visual interfaces to build, explore and validate data in the same version-controlled environment trusted by data teams. This release includes dbt Canvas (a visual, drag-and-drop interface for model development), dbt Insights (an AI-powered query tool for quick analysis and sharing), and an enhanced dbt Catalog (for global asset discovery). Additionally, organizations can now use the new cost management dashboard to optimize their data warehouse spend. Bridging the gap between self-service and governance Gartner® predicts that, "by 2027, 60% of organizations will fail to realize the full value of their AI use cases due to fragmented data governance frameworks."* One contributing factor is the rise of ungoverned data workflows, often driven by analysts working around limited engineering support. To get the insights they need, data analysts rely on unsupported, disconnected tools and un-tested, bespoke logic to build, query, and explore data, leading to compliance risks, increased costs, and poor data quality that undermine organizational decision making. dbt's new AI-powered capabilities are purpose-built to solve this issue by giving analysts greater autonomy while ensuring every action remains governed, version-controlled and aligned with organizational data standards. "Data teams today face a fundamental tension – analysts need speed and independence, while organizations require strong governance and security," said Tristan Handy, founder and CEO of dbt Labs. "Our new AI-powered solutions break down these traditional barriers for data analysts across any skill level and collaborate with developers in the same platform, which will have a significant, positive impact throughout the business." Unlocking trusted self-service for analysts with dbt The Analytics Development Lifecycle (ADLC) is a vendor-agnostic framework that helps organizations mature how they build, maintain, and scale trusted data products. As the data control plane for the modern enterprise, dbt brings the ADLC to life, enabling version-controlled, governed workflows that power analytics across teams. dbt Labs is now making it easy for downstream analysts to participate in the ADLC with the following new capabilities: dbt Canvas, a new visual editing environment in dbt, enables users more comfortable with drag-and-drop tooling to build and edit data models. Analysts can describe what they want to build in natural language using dbt Copilot, allowing teams with limited SQL knowledge to build effective data models using context-rich AI. It automatically maintains governance and quality standards, while reducing reliance on data engineers, boosting collaboration and improving productivity. dbt Canvas is now GA. dbt Insights, a new AI-powered query interface that helps analysts ask questions and get answers faster, all within dbt. With full awareness of an organization's models, lineage and governance rules, it enables users to query, validate, visualize, and share findings using SQL or natural language in one seamless, governed workspace. This eliminates the need to wait on central data teams to process requests or switch tabs to get answers. dbt Insights is available in preview. An expanded dbt Catalog (formerly dbt Explorer) includes a unified discovery experience that enables global search and exploration for overall Snowflake assets not managed by dbt, offering analysts a comprehensive view of their data landscape. Analysts can easily discover, understand and trust the assets they use, without switching tools. dbt Catalog is now generally available, with the ability to explore Snowflake data assets currently in preview. Integrations for additional data platforms are coming soon. "Lowering the technical barrier to entry for data analysts has been important to Tableau from the beginning of the company," said Dan Jewett, Senior Vice President, Product Management at Tableau. "dbt's expanded offering is a game changer for customers that are looking to reduce the sizable burden on their data engineering teams, while simultaneously enabling analysts across the business in a meaningful way. It's a massive step forward for the future of data teams and one we're thrilled to continue to partner on." dbt Labs customer WHOOP is eager to boost self-service for its analysts, while leaning on easy workflows. "As our data needs evolve, empowering analysts with seamless self-exploration becomes increasingly critical," said William Tsu, Senior Analytics Engineer at WHOOP. "By keeping them within the familiar dbt Catalog they already use daily, dbt's new analyst offerings enhance discoverability and enable faster, more intuitive, and governed self-service." For dbt systems integrator InterWorks, dbt Canvas is poised to remove bottlenecks and power trusted self-service analytics across the organization. "dbt Canvas is unlocking a future where analysts can build confidently alongside engineers within the same trusted and governed workflows," said James Wright, Chief Strategy Officer at InterWorks. "We're excited about how this new development environment will help our customers unlock true self-service while maintaining the standards, security, and collaboration required to scale analytics responsibly." Empowering Organizations to Manage Data Warehouse Spend dbt Labs is also providing new features that allow organizations to optimize data platform costs and ensure the long-term flexibility of their data investments. This includes a cost management dashboard that helps organizations understand data platform costs from their dbt workloads, and also view consumption and realized savings from standardizing on dbt. Powered by the dbt Fusion engine, the cost management dashboard offers visibility into costs at the project, environment, model, and test level, helping users identify and resolve cost inefficiencies. No other vendor owns the transformation workflow from development to production, allowing dbt to embed cost optimization natively rather than as an add-on. The cost management dashboard is in preview for Snowflake customers ahead of the 2025 Snowflake Summit, June 2-5 in San Francisco. A Better-than-ever Developer Experience Announced earlier today, dbt Labs launched the new dbt Fusion engine, incorporating the technology from its acquisition of SDF Labs this year. Fusion delivers massive performance improvements and introduces features that significantly enhance the developer experience. These include next-gen data transformation capabilities that improve code quality by providing real-time feedback, lower costs by avoiding unnecessary warehouse compute, and make dbt 30x faster than dbt Core. For more information on the future of the dbt platform, visit For more information on the new dbt features for analysts, visit *Gartner Insights, Adopt a Data Governance Approach That Enables Business Outcomes, GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved. About dbt LabsSince 2016, dbt Labs has been on a mission to help data practitioners create and disseminate organizational knowledge. dbt is the standard for AI-ready structured data. Powered by the dbt Fusion engine, it unlocks the performance, context, and trust that organizations need to scale analytics in the era of AI. Globally, more than 60,000 data teams use dbt, including those at Siemens, Roche and Condé Nast. Learn more at and follow dbt Labs on LinkedIn, X, Instagram, and YouTube. View original content to download multimedia: SOURCE dbt Labs Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
Forbes
01-04-2025
- Business
- Forbes
Data Engineering: Transforming The Backbone Of Modern Data Solutions
Mukul Garg is the Head of Support Engineering at PubNub , which powers apps for virtual work, play, learning and health. getty In my journey through data engineering, one of the most remarkable shifts I've witnessed occurred during the integration of real-time data pipelines for a fast-growing SaaS platform. Initially, our data team was bogged down with batch processing and delayed analytics, which severely hindered decision-making speed. However, when we implemented a real-time data architecture using technologies like Apache Kafka and cloud-native solutions, we were able to process and analyze data on the fly, dramatically increasing our business agility. This experience solidified my belief in the transformative power of modern data engineering. This year, data engineering will have become the backbone of every digital transformation strategy. With the growing complexity of data sources and increasing demand for real-time analytics, companies are adopting cutting-edge technologies to build robust, scalable and efficient data infrastructures. Data engineering today plays a pivotal role in unlocking the true value of data across industries, allowing businesses to harness their data in real time, improve decision-making and create personalized experiences for customers. Several companies are at the forefront of implementing advanced data engineering solutions that set benchmarks for the industry: • Netflix's Real-Time Data Pipelines: Netflix utilizes a data architecture that combines both batch and stream processing methods to handle massive quantities of data. This approach balances latency, throughput and fault tolerance by using batch processing for comprehensive views and real-time stream processing for immediate data insights. • Uber's Predictive Analytics Engine: Uber has developed a sophisticated predictive analytics engine to optimize route planning and demand forecasting. By using real-time data processing, Uber can anticipate surge pricing and provide drivers with the most efficient routes in real time. • Shopify's Automated Data Warehouse: Shopify recently moved to an automated data warehouse powered by cloud-native solutions like dbt (data build tool). This has allowed them to integrate sales, inventory and customer data more efficiently, resulting in quicker data-driven insights and better decision-making. • Airbnb's Data Mesh Architecture: Airbnb has embraced a data mesh approach to scale its data infrastructure, decoupling data storage and processing across multiple teams. This approach enables each team to take ownership of its own data domain while using shared infrastructure, improving data discoverability and reducing bottlenecks. Benefits Of Modern Data Engineering Modern data engineering offers several key benefits that have become essential for businesses today: • Real-Time Analytics: With the advent of real-time data pipelines, companies can process and analyze data as it comes in. This has allowed businesses like Uber and Netflix to offer more timely, relevant insights and optimize decision-making in real time. • Scalability: Data engineering solutions today can scale horizontally, handling increasing volumes of data without a corresponding increase in cost. Cloud data platforms like Snowflake and Google BigQuery are prime examples of scalable solutions that allow organizations to scale operations as they grow. • Data Democratization: The rise of self-service data tools such as dbt and Looker has democratized data access, enabling teams across organizations to leverage data without needing deep technical expertise. This leads to faster decision-making across departments. • Cost Efficiency: Cloud-native data solutions enable companies to optimize storage and compute costs by only paying for what they use, making it easier for small and medium-sized businesses to manage their data infrastructure without heavy upfront investments. Challenges And Considerations In Data Engineering While the benefits are clear, there are also challenges in integrating modern data engineering solutions. • Data Quality: Ensuring that the data being ingested into the system is clean, consistent and accurate is one of the most challenging aspects of data engineering. Poor data quality can lead to incorrect insights and missed opportunities. • Data Privacy And Compliance: As data privacy regulations like GDPR continue to evolve, organizations need to ensure that their data pipelines comply with these regulations. This requires robust data governance and regular audits to maintain compliance. • Integration Complexity: Integrating multiple data sources, especially from legacy systems, can be complex. Data engineering teams must ensure seamless integration while maintaining the integrity of the data and ensuring minimal latency. • Maintaining Real-Time Performance: As real-time data processing becomes more prevalent, maintaining low-latency pipelines becomes increasingly difficult. Ensuring high throughput and minimal delays, especially with large datasets, requires careful infrastructure management. Maintaining Data Integrity And Security In an age where data is the new currency, ensuring data integrity and security has never been more important. Implementing secure access controls, encrypted data pipelines and comprehensive monitoring systems can help safeguard data from potential breaches. Companies like Shopify and Airbnb are taking proactive steps to ensure their data infrastructures are both secure and resilient, using advanced data masking and encryption techniques to protect sensitive information. Future Outlook Based on patterns observed in the industry, here are my predictions for data engineering in the next two to three years. • Data Fabric And Data Mesh Expansion: The concept of a data fabric, which integrates disparate data sources into a unified layer, will continue to gain traction. Combined with data mesh architecture, organizations will see greater flexibility and scalability in their data operations, enabling more efficient collaboration across departments. • Serverless Data Platforms: Serverless computing will take on an even greater role in data engineering. Companies will increasingly shift to serverless data architectures, reducing the overhead of managing infrastructure while focusing more on the logic of data processing. • Data Privacy By Design: As privacy concerns grow, companies will build privacy-enhancing technologies into their data pipelines from the outset, ensuring compliance with global regulations without sacrificing performance. Conclusion In 2025, data engineering is not just about building infrastructure—it's about creating agile, scalable and secure systems that can process vast amounts of data in real time. The future of data engineering looks bright as organizations continue to innovate, leveraging modern technologies to unlock new insights, improve decision-making and drive business growth. Companies that invest in the latest data engineering solutions should be well-positioned to gain a competitive edge in an increasingly data-driven world. Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
