Latest news with #ApacheIceberg™
Cision Canada
13 hours ago
- Business
- Cision Canada
Databricks Eliminates Table Format Lock-in and Adds Capabilities for Business Users with Unity Catalog Advancements
Unity Catalog is now the most complete catalog for Apache Iceberg™ and Delta Lake, enabling open interoperability with governance across compute engines, and adds unified semantics and a rich discovery experience for business users SAN FRANCISCO, June 11, 2025 /CNW/ -- Data + AI Summit -- Databricks, the Data and AI company, today extends its leadership in the unified governance category with powerful new capabilities. Unity Catalog adds full support for Apache Iceberg™ tables, including native support for the Apache Iceberg REST Catalog APIs. Now, Unity Catalog is the only catalog that enables external engines to read and write, with fine-grained governance, to performance-optimized, Iceberg managed tables, eliminating lock-in and enabling seamless interoperability. Databricks is also introducing two new enhancements that extend Unity Catalog to business users. Business metrics and KPIs are the foundation of how companies manage their business, and can now be defined as first-class data assets with Unity Catalog Metrics. In addition, data + AI discovery is enhanced for business users with a new, curated internal marketplace that surfaces the highest-value data, AI and AI/BI assets, organized by business domain. All these assets are augmented with automated data intelligence, so every team can find, trust and act on the right data. Unity Catalog Now Eliminates the Need to Choose Between Formats Built on open standards, Unity Catalog is designed to work across every table format and engine. Databricks is now taking that vision further with the Public Preview of full Apache Iceberg support, uniting the Apache Iceberg and Delta Lake ecosystems with a single approach to governance. The preview adds three new capabilities. First, organizations can create Apache Iceberg managed tables that any Iceberg‑compatible engine can read and write through Unity Catalog's Iceberg REST Catalog API. These Iceberg managed tables benefit from the full power of Unity Catalog: best price performance with AI-powered Predictive Optimization; and unified governance and policy enforcement both within Databricks and across external engines, including Trino, Snowflake, Amazon EMR, etc. Second, Unity Catalog's pioneering Lakehouse Federation capabilities enable seamless access to Iceberg tables managed in external catalogs, so those tables can be discovered and governed alongside native tables. Third, Iceberg tables get all the benefits of the Delta Sharing ecosystem, including seamless cross-organizational sharing of Iceberg tables. These capabilities eliminate format-driven data silos — no other catalog in the industry provides these capabilities. A Growing Disconnect Between Data Platforms and Business Users While data platforms have advanced rapidly for technical users, teams across the business remain disconnected from the systems that power their decisions. Technical teams center their world on tables, files, compute and code, while business users operate in BI tools, AI chatbots and focus on KPIs and business metrics in their business domains. These fundamentally different languages result in business users unsure of what data to trust or reliant on engineers for basic questions. Without a unified foundation for business context, organizations face duplicated work, decision paralysis and a persistent gap between data and action. A Single Source of Truth for Metrics Across the Business To address this need, Unity Catalog Metrics brings business metric definitions traditionally embedded within BI tools to the data platform. This creates consistency and accuracy for how everyone in the organization understands business performance. Unlike proprietary Bl semantic layers, Unity Catalog Metrics are fully addressable via SQL to ensure that everyone in the organization can have the same view of metrics, irrespective of what tool they choose. Unity Catalog Metrics is available to all customers today as a Public Preview and will be Generally Available later this summer. A Unified Foundation for Context: From Guided Discovery to Intelligent Insights To make trusted data truly usable for business users, Databricks is introducing new Unity Catalog capabilities that blend intuitive discovery with built-in intelligence. A new Discover experience offers a curated internal marketplace of certified data products — organized by business domains like Sales, Marketing, or Finance and enriched with documentation, ownership, tagging and usage insights. Automated, intelligent recommendations coupled with data steward curation tools ensure the highest value assets - metrics, dashboards, tables, AI agents, Genie spaces, etc. - can easily be explored, understood, trusted, and accessed through a self-serve workflow, without manual approvals or engineering support. Unity Catalog Discover is now in Private Preview. Unity Catalog also now adds intelligence across the experience, surfacing data quality signals, usage patterns, relationships across assets, and certification and deprecation status to help users quickly assess trust and relevance. With Databricks Assistant built into Unity Catalog, they can ask natural language questions and get grounded, contextual answers based on governed metrics — turning discovery into a guided journey where data is accessible, explainable, trustworthy, and ready for use. "We created the Unified Governance category with Unity Catalog four years ago," said Matei Zaharia, Co-founder and CTO of Databricks. "With these updates to Unity Catalog, we are now offering the best catalog in the industry for Apache Iceberg and all open table formats, and the only one that allows reads and writes to managed tables from external engines, for a truly open enterprise catalog. No matter what table format our customers choose, we ensure it's accessible, optimized, and governed. And with our expanded focus on business users, we're ensuring we deliver on the promise of democratizing data + AI to every user in the enterprise." Customer + Partner Quotes "At Riskified, we want to store all our data in an open format and want a single catalog that can connect to all the tools we use," said Hen Ben-Hemo, Data Platform Architect at Riskified. "Unity Catalog allows us to write Iceberg tables that are fully open to any Iceberg client, unlocking the entire lakehouse ecosystem and future proofing our architecture." "Unity Catalog Metrics gives us a central place to define business KPIs and standardize semantics across teams, ensuring everyone works from the same trusted definitions across dashboards, SQL, and AI applications." — Richard Masters, Vice President, Data & AI, Virgin Atlantic "Unity Catalog Metrics presents an exciting opportunity to establish consistency, trust, and control in how business metrics are defined and consumed across Zalando. It is a promising contribution to aligned, data-driven decisions across our BI dashboards, notebooks, and other tools." — Timur Yuere, Engineering Manager, Zalando "Unity Catalog Metrics represents an exciting opportunity for Tableau customers to leverage the value of centralized governance with Databricks Unity Catalog. Through our deep integration and expanding roadmap with Databricks, we're thrilled to help remove the friction for our customers in leveraging Databricks to define their core business metrics" - Nicolas Brisoux, Sr. Director Product Management Tableau "We're excited to partner with Databricks to integrate Unity Catalog Metrics into Sigma. This gives business teams direct access to trusted, standardized business metrics within their dashboards, so everyone can make decisions based on consistent definitions, without relying on data teams for every question." — Dillion Morrison, VP of Product, Sigma Computing Availability Databricks is introducing Public Preview of full Apache Iceberg support in Unity Catalog. Unity Catalog Metrics is available to all customers today as a Public Preview and will be Generally Available later this summer. Unity Catalog Discover is now in Private Preview. About Databricks Databricks is the Data and AI company. More than 15,000 organizations worldwide — including Block, Comcast, Condé Nast, Rivian, Shell and over 60% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to take control of their data and put it to work with AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake, MLflow, and Unity Catalog. To learn more, follow Databricks on X, LinkedIn and Facebook.
Business Wire
2 days ago
- Business
- Business Wire
Qlik Expands Integration with the Databricks Data Intelligence Platform
SAN FRANCISCO--(BUSINESS WIRE)-- Qlik ®, a global leader in data integration, data quality, analytics, and artificial intelligence, today announced a series of new capabilities for customers of Databricks, the Data and AI company, built on the Databricks Data Intelligence Platform. These enhancements include streaming real-time data into Unity Catalog's Uniform tables via change data capture (CDC), automated Apache Iceberg™ optimization through Qlik Open Lakehouse, and the creation of high-quality data products. These enhancements give data teams greater flexibility across open formats, improve operational performance with Delta and Iceberg data, and accelerate the path to trusted, AI-ready architectures, without compromising Databricks-native governance or performance. New capabilities include: Real-Time Data Streaming to Databricks Uniform Tables via CDC: Qlik Replicate ® now streams continuous CDC from enterprise data sources directly into Unity Catalog's managed Iceberg tables, enabling low-latency ingestion that supports strict business SLAs for both Delta and Iceberg formats. Adaptive Iceberg Optimization: As data is ingested into Apache Iceberg tables by Qlik Talend Cloud ®, Qlik Open Lakehouse's fully automated optimizer will intelligently handle compactions, partitioning, and pruning, reducing storage footprint and delivering faster queries. Optimized Iceberg tables will be queryable via Databricks Photon or any Iceberg-compatible engine with consistently low-latency performance. High-Quality, AI-Ready Data Products: Enables data teams to build governed Data Products and push down data quality computation for Databricks assets—including Delta Live Tables—ensuring products remain trusted, accurate, and ready for AI use cases. Spark-Aware Studio Roadmap Enhancements: Qlik will soon introduce new developer-focused capabilities including schema inference, Databricks notebook import, and native Spark debugging—empowering teams to manage governed, self-service data pipelines within their existing Databricks workflows. 'Databricks customers continue to push the boundaries of what's possible with open data formats and AI,' said Ariel Amster, Director, Strategic Technology Partners at Databricks. 'By delivering real-time change data capture into UniForm tables and its native integration with Mosaic AI, Qlik is helping our joint customers simplify and accelerate innovation on the Databricks Data Intelligence Platform.' 'From ingestion to insight, Databricks customers are demanding more speed, flexibility, and trust across their data estate,' said David Zember, Senior Vice President of Worldwide Channels and Alliances at Qlik. 'These new capabilities allow teams to do more with their Databricks investment—especially around governance, interoperability, and AI readiness.' The new capabilities—including Uniform table CDC and Iceberg optimization—are now available in private preview. Qlik's planned Open Lakehouse integration for Databricks is under development with timing to be announced. To learn more, request early access, or see a demo, visit Qlik at booth #521 during Databricks Data + AI Summit or visit our website. About Qlik Qlik converts complex data landscapes into actionable insights, driving strategic business outcomes. Serving over 40,000 global customers, our portfolio provides advanced, enterprise-grade AI/ML, data integration, and analytics. Our AI/ML tools, both practical and scalable, lead to better decisions, faster. We excel in data integration and governance, offering comprehensive solutions that work with diverse data sources. Intuitive analytics from Qlik uncover hidden patterns, empowering teams to address complex challenges and seize new opportunities. As strategic partners, our platform-agnostic technology and expertise make our customers more competitive. © 2025 QlikTech International AB. All rights reserved. All company and/or product names may be trade names, trademarks and/or registered trademarks of the respective owners with which they are associated.
Business Wire
02-06-2025
- Business
- Business Wire
Qlik Adds Native Support for Snowflake Managed Iceberg Tables, Expands Open Lakehouse Options for Snowflake Customers
SAN FRANCISCO--(BUSINESS WIRE)-- Qlik®, a global leader in data integration, data quality, analytics, and artificial intelligence, today announced at Snowflake's annual user conference, Snowflake Summit 2025, the launch of native support for Snowflake-managed Apache Iceberg™ tables, enabling fast, open-format data pipelines directly into Snowflake's highly performant, governed environment. Alongside this, Qlik is introducing additional capabilities that allow customers to leverage Qlik Open Lakehouse, powered by Apache Iceberg, in conjunction with Snowflake for greater architectural flexibility and AI scalability. These advancements are designed to help Snowflake customers reduce latency, optimize storage and compute efficiency, and accelerate the development of AI-powered applications, including retrieval-augmented generation (RAG) via Snowflake Cortex AI. Newly announced capabilities include: Native Streaming to Snowflake-managed Iceberg Tables: Qlik Talend Cloud® now supports continuous change data capture (CDC) from enterprise systems directly into Snowflake-managed Iceberg tables, enabling low-latency ingestion supporting strict business SLAs for analytics and AI use cases. Qlik Open Lakehouse Optimization & Mirroring: Qlik Open Lakehouse combines low-latency ingestion into Apache Iceberg tables with an automated optimizer that manages compactions, partitioning, and pruning in S3—delivering faster queries and a reduced storage footprint without manual tuning. It also mirrors Iceberg data back into Snowflake for downstream transformations without duplicating data. One-Click Data Products with In-Snowflake Quality Execution: Qlik data products can be generated directly within customers' Snowflake ecosystems, leveraging the Qlik Talend Trust Score™ to push down data quality computation in Snowflake—enabling teams to produce governed, high-quality outputs that elevate the value of curated assets. Knowledge Mart for RAG on Snowflake Cortex: Qlik's Knowledge Mart transforms structured and unstructured content—including PDFs, call transcripts, and relational records—into AI-ready vectorized assets in Snowflake, powering retrieval-augmented generation pipelines through Cortex with full explainability and governance. 'Open standards like Apache Iceberg are foundational to an interoperable data stack, including both Qlik and Snowflake,' said Saurin Shah, Senior Product Manager, Data Engineering at Snowflake. 'By combining real-time ingestion, automated optimization, and Cortex-ready AI pipelines, Qlik, together with Snowflake, helps customers accelerate time to insight while maximizing the value of their data investments.' 'The integration between Qlik and Snowflake has transformed how we manage and operationalize data,' said Michael Benassi, Vice President of Enterprise Analytics at United Federal Credit Union. 'By operationalizing near real-time data ingestion and streamlined engineering pipelines, we're able to scale insights across the business and support faster, more trusted AI initiatives.' 'This launch gives our joint customers the power to do more with their Snowflake investment,' said David Zember, Senior Vice President of Worldwide Channels and Alliances at Qlik. 'By combining Qlik's real-time ingestion and Iceberg optimization with native Snowflake governance, we're unlocking a smarter path to analytics and AI that's as open as it is scalable.' The new Qlik capabilities are now available in private preview, with general availability targeted in July 2025. To request early access, see a live demo, or speak with Qlik product experts, visit booth #1219 at Snowflake Summit 2025 or visit us online. About Snowflake Snowflake makes enterprise AI easy, efficient and trusted. More than 11,000 companies around the globe, including hundreds of the world's largest, use Snowflake's AI Data Cloud to share data, build applications, and power their business with AI. The era of enterprise AI is here. Learn more at (NYSE: SNOW). About Qlik Qlik converts complex data landscapes into actionable insights, driving strategic business outcomes. Serving over 40,000 global customers, our portfolio provides advanced, enterprise-grade AI/ML, data integration, and analytics. Our AI/ML tools, both practical and scalable, lead to better decisions, faster. We excel in data integration and governance, offering comprehensive solutions that work with diverse data sources. Intuitive analytics from Qlik uncover hidden patterns, empowering teams to address complex challenges and seize new opportunities. As strategic partners, our platform-agnostic technology and expertise make our customers more competitive. © 2025 QlikTech International AB. All rights reserved. All company and/or product names may be trade names, trademarks and/or registered trademarks of the respective owners with which they are associated.
TECHx
19-03-2025
- Business
- TECHx
Confluent Enhances Tableflow with Apache Iceberg and Delta Lake Support News Desk - 19/03/2025 ShareConfluent, Inc., the data streaming company, has announced key advancements in its Tableflow platform, providing enhanced access to operational data from data lakes and warehouses. With these updates, including full support for Apache Iceberg™ and the launch of an Early Access Program for Delta Lake in partnership with Databricks, Tableflow enables businesses to unlock new possibilities for real-time analytics, artificial intelligence (AI), and next-generation applications.The new updates to Tableflow allow data engineers and data scientists to access streaming data in popular open table formats, empowering AI-driven decision-making and simplifying the integration of operational data into analytical systems. With the general availability of Apache Iceberg support, teams can now seamlessly represent Apache Kafka® topics as Iceberg tables for real-time and batch processing use cases. This development significantly reduces the maintenance burden of tasks like table compaction, giving data engineers more time to focus on driving business value.'At Confluent, we're all about making your data work for you, whenever you need it and in whatever format is required,' said Shaun Clowes, Chief Product Officer at Confluent. 'With Tableflow, we're bringing our expertise of connecting operational data to the analytical world. Now, data scientists and data engineers have access to a single, real-time source of truth across the enterprise, making it possible to build and scale the next generation of AI-driven applications.'Tableflow also introduces the Early Access Program for Delta Lake, a widely used open-format storage layer pioneered by Databricks. Delta Lake processes over 10 exabytes of data daily, making it a key enabler for AI-driven applications. Through this integration, customers can now access a unified view of real-time data across operational and analytic applications, speeding up AI-driven decision-making and allowing for smarter, more agile business processes. Interested users can apply for the Early Access Program to explore these capabilities.To offer more flexibility, Tableflow now supports the Bring Your Own Storage feature, enabling customers to store Iceberg or Delta tables once and reuse them multiple times with their preferred storage solutions. This added flexibility allows businesses to have full control over their data storage and compliance requirements, ensuring that data governance needs are met without sacrificing performance.Confluent has further enhanced Tableflow's capabilities with seamless integrations with AWS Glue Data Catalog and Snowflake's Open Catalog, ensuring easy management of Iceberg tables and providing access to popular analytical engines like Amazon Athena, AWS EMR, and RedShift. This integration streamlines data accessibility for a range of data lake and warehouse solutions, including Snowflake, Dremio, Imply, and others.With support from global and regional system integrators, including Tata Consultancy Services (TCS), Onibex, GoodLabs Studio, and Psyncopate, Confluent is positioning Tableflow as a critical tool for enterprises seeking to drive AI innovation and scale real-time analytics. The continued development of Tableflow underscores Confluent's commitment to providing cutting-edge tools that bridge the gap between operational data and analytical systems, enabling businesses to accelerate their AI-driven digital transformation.
Confluent Enhances Tableflow with Apache Iceberg and Delta Lake Support Confluent, Inc., the data streaming company, has announced key advancements in its Tableflow platform, providing enhanced access to operational data from data lakes and warehouses. With these updates, including full support for Apache Iceberg™ and the launch of an Early Access Program for Delta Lake in partnership with Databricks, Tableflow enables businesses to unlock new possibilities for real-time analytics, artificial intelligence (AI), and next-generation applications. The new updates to Tableflow allow data engineers and data scientists to access streaming data in popular open table formats, empowering AI-driven decision-making and simplifying the integration of operational data into analytical systems. With the general availability of Apache Iceberg support, teams can now seamlessly represent Apache Kafka® topics as Iceberg tables for real-time and batch processing use cases. This development significantly reduces the maintenance burden of tasks like table compaction, giving data engineers more time to focus on driving business value. 'At Confluent, we're all about making your data work for you, whenever you need it and in whatever format is required,' said Shaun Clowes, Chief Product Officer at Confluent. 'With Tableflow, we're bringing our expertise of connecting operational data to the analytical world. Now, data scientists and data engineers have access to a single, real-time source of truth across the enterprise, making it possible to build and scale the next generation of AI-driven applications.' Tableflow also introduces the Early Access Program for Delta Lake, a widely used open-format storage layer pioneered by Databricks. Delta Lake processes over 10 exabytes of data daily, making it a key enabler for AI-driven applications. Through this integration, customers can now access a unified view of real-time data across operational and analytic applications, speeding up AI-driven decision-making and allowing for smarter, more agile business processes. Interested users can apply for the Early Access Program to explore these capabilities. To offer more flexibility, Tableflow now supports the Bring Your Own Storage feature, enabling customers to store Iceberg or Delta tables once and reuse them multiple times with their preferred storage solutions. This added flexibility allows businesses to have full control over their data storage and compliance requirements, ensuring that data governance needs are met without sacrificing performance. Confluent has further enhanced Tableflow's capabilities with seamless integrations with AWS Glue Data Catalog and Snowflake's Open Catalog, ensuring easy management of Iceberg tables and providing access to popular analytical engines like Amazon Athena, AWS EMR, and RedShift. This integration streamlines data accessibility for a range of data lake and warehouse solutions, including Snowflake, Dremio, Imply, and others. With support from global and regional system integrators, including Tata Consultancy Services (TCS), Onibex, GoodLabs Studio, and Psyncopate, Confluent is positioning Tableflow as a critical tool for enterprises seeking to drive AI innovation and scale real-time analytics. The continued development of Tableflow underscores Confluent's commitment to providing cutting-edge tools that bridge the gap between operational data and analytical systems, enabling businesses to accelerate their AI-driven digital transformation.
Tahawul Tech
19-03-2025
- Business
- Tahawul Tech
'Tableflow makes it possible to build and scale the next generation of AI-driven applications.' – Shaun Clowes, Confluent
Confluent have formally announced the general availability of Tableflow, which brings real-time business context to analytical systems to make AI and next-generation applications enterprise-ready. With Tableflow, all streaming data in Confluent Cloud can be accessed in popular open table formats, unlocking limitless possibilities for advanced analytics, real-time artificial intelligence (AI), and next-generation applications. Support for Apache Iceberg™ is now generally available (GA). And as a result of an expanded partnership with Databricks, a new early access program for Delta Lake is now open. Additionally, Tableflow now offers enhanced data storage flexibility and seamless integrations with leading catalog providers, including AWS Glue Data Catalog and Snowflake's managed service for Apache Polaris™, Snowflake Open Catalog. 'At Confluent, we're all about making your data work for you, whenever you need it and in whatever format is required,' said Shaun Clowes, Chief Product Officer at Confluent. 'With Tableflow, we're bringing our expertise of connecting operational data to the analytical world. Now, data scientists and data engineers have access to a single, real-time source of truth across the enterprise, making it possible to build and scale the next generation of AI-driven applications.' Bridging the Data Gap for Enterprise-Ready AI Tableflow simplifies the integration between operational data and analytical systems. It continuously updates tables used for analytics and AI with the exact same data from business applications connected to Confluent Cloud. Within Confluent, processing and governance happen as data is generated, shifting these tasks upstream to ensure that only high-quality, consistent data is used to feed data lakes and warehouses. This is a breakthrough for AI, as it's only as powerful as the data that shapes it. Today, Confluent announces significant updates to Tableflow: ● Support for Apache Iceberg is ready for production workloads. Teams can now instantly represent Apache Kafka® topics as Iceberg tables to feed any data warehouse, data lake, or analytics engine for real-time or batch processing use cases. Expensive and error-prone table maintenance tasks, such as compaction, are automatically handled by Tableflow, giving time back to data engineers to deliver more business value. It also provides a single source of truth for one of the most widely adopted open-format storage options, enabling data scientists and data engineers to scale AI innovation and next-generation applications. ● New Early Access Program for Delta Lake is now open. This open-format storage layer, pioneered by Databricks, processes more than 10 exabytes of data daily and is used alongside many popular AI engines and tools. With this integration, customers will have a consistent view of real-time data across operational and analytic applications, enabling faster, smarter AI-driven decision-making. Apply for the Tableflow Early Access Program here. ● Increase flexibility through Bring Your Own Storage. Store fresh, up-to-date Iceberg or Delta tables once and reuse them many times with the freedom to choose a storage bucket. Customers now have full control over storage and compliance to meet their unique data ownership needs. ● Enhance data accessibility and governance with partners. Direct integrations with Amazon SageMaker Lakehouse via AWS Glue Data Catalog (GA) and Snowflake Open Catalog (GA) enable seamless catalog management for Tableflow's Iceberg tables. They also streamline access for analytical engines such as Amazon Athena, AWS EMR, and Amazon RedShift, and leading data lake and warehouse solutions including Snowflake, Dremio, Imply, Onehouse, and Starburst. Additionally, Confluent has strengthened enterprise adoption for Tableflow with support from global and regional system integrators, including GoodLabs Studio, Onibex, Psyncopate, and Tata Consultancy Services (TCS).
