Latest news with #ApacheKafka
Techday NZ
24-04-2025
- Business
- Techday NZ
OVHcloud launches all-in-one data platform for analytics & AI
OVHcloud has announced the general availability of its Data Platform, an all-in-one platform-as-a-service designed for organisations seeking to manage and analyse data while maintaining compliance and cost predictability. The Data Platform, according to OVHcloud, offers end-to-end capabilities for data collection, storage, processing, analysis, and visualisation in a cloud environment that aims to address the challenges of increasing data volume, complexity, and the growing adoption of artificial intelligence tools. Built on open-source technologies, the solution is aimed at helping organisations avoid dependency on major cloud hyperscalers and is pitched as particularly relevant for highly regulated industries and any business concerned with data sovereignty and vendor lock-in. "With the launch of OVHcloud Data Platform we are providing customers with a complete and integrated solution for their data journey. Businesses can leverage their data to find new insights through AI and analytics projects. We are proud to deliver this important milestone having implemented customer feedback throughout the Beta phase," said Alexis Gendronneau, Chief Data Officer OVHcloud. The platform consists of a set of managed services, including data streaming, storage, pipeline orchestration, and advanced visualisation and exploration tools. Users are able to focus on utilising data for value creation rather than managing infrastructure, according to the company. Target use cases span several sectors. For retail and e-commerce, the platform is designed to assist in identifying customer groups and predicting inventory needs. In financial services, it provides tools for portfolio risk assessment, fraud detection, and credit scoring. Healthcare use cases include analysis of clinical trial data to speed up drug development. The platform can also measure key performance indicators for media and entertainment advertising campaigns and perform audience sentiment analysis. In Industry 4.0 settings, it may be used for supply chain optimisation, predictive maintenance, and quality control. The platform supports a broad spectrum of data sources, including Object Storage, Apache Kafka, ClickHouse, MongoDB, MySQL, Oracle, HTTP/FTP, Google Analytics, Google BigQuery, Snowflake, X, the OVHcloud API, and others. Data can be processed, stored, and made available for analytics and sharing via built-in applications or APIs. Listed technical features include compatibility with languages and frameworks such as ANSI SQL, Python, Apache Iceberg, Spark, Pandas, Jupyter notebooks, Trino, SuperSet, Prometheus, and Kubernetes. The solution has been optimised using technology developed by ForePaaS, which OVHcloud acquired previously, and is now integrated within the wider OVHcloud portfolio. OVHcloud presents the Data Platform as suitable for use by data engineers, analytics engineers, data analysts, data scientists, and dataops teams, providing a single user interface aimed at facilitating cross-department data collaboration. The service is available for businesses of varying sizes, including small and mid-size companies requiring advanced analytics services. The workflow of the Data Platform can be further enhanced with OVHcloud's AI Endpoints for tasks such as document data extraction, multi-modal transcription, automated data cleansing, or anomaly detection. The service integrates with serverless GPU-powered OVHcloud AI Training and AI Deploy services, supporting the acceleration of data-to-model lifecycle processes. On data sovereignty and security, the company states that all data is hosted in OVHcloud's European cloud infrastructure, designed to meet high security and compliance standards. "Data is hosted in Europe, providing protection against non-European regulations and giving organisations technical and strategic autonomy. The platform is also based on open-source technologies, providing users with superior data portability, control and freedom of choice," the company stated.
Forbes
01-04-2025
- Business
- Forbes
Data Engineering: Transforming The Backbone Of Modern Data Solutions
Mukul Garg is the Head of Support Engineering at PubNub , which powers apps for virtual work, play, learning and health. getty In my journey through data engineering, one of the most remarkable shifts I've witnessed occurred during the integration of real-time data pipelines for a fast-growing SaaS platform. Initially, our data team was bogged down with batch processing and delayed analytics, which severely hindered decision-making speed. However, when we implemented a real-time data architecture using technologies like Apache Kafka and cloud-native solutions, we were able to process and analyze data on the fly, dramatically increasing our business agility. This experience solidified my belief in the transformative power of modern data engineering. This year, data engineering will have become the backbone of every digital transformation strategy. With the growing complexity of data sources and increasing demand for real-time analytics, companies are adopting cutting-edge technologies to build robust, scalable and efficient data infrastructures. Data engineering today plays a pivotal role in unlocking the true value of data across industries, allowing businesses to harness their data in real time, improve decision-making and create personalized experiences for customers. Several companies are at the forefront of implementing advanced data engineering solutions that set benchmarks for the industry: • Netflix's Real-Time Data Pipelines: Netflix utilizes a data architecture that combines both batch and stream processing methods to handle massive quantities of data. This approach balances latency, throughput and fault tolerance by using batch processing for comprehensive views and real-time stream processing for immediate data insights. • Uber's Predictive Analytics Engine: Uber has developed a sophisticated predictive analytics engine to optimize route planning and demand forecasting. By using real-time data processing, Uber can anticipate surge pricing and provide drivers with the most efficient routes in real time. • Shopify's Automated Data Warehouse: Shopify recently moved to an automated data warehouse powered by cloud-native solutions like dbt (data build tool). This has allowed them to integrate sales, inventory and customer data more efficiently, resulting in quicker data-driven insights and better decision-making. • Airbnb's Data Mesh Architecture: Airbnb has embraced a data mesh approach to scale its data infrastructure, decoupling data storage and processing across multiple teams. This approach enables each team to take ownership of its own data domain while using shared infrastructure, improving data discoverability and reducing bottlenecks. Benefits Of Modern Data Engineering Modern data engineering offers several key benefits that have become essential for businesses today: • Real-Time Analytics: With the advent of real-time data pipelines, companies can process and analyze data as it comes in. This has allowed businesses like Uber and Netflix to offer more timely, relevant insights and optimize decision-making in real time. • Scalability: Data engineering solutions today can scale horizontally, handling increasing volumes of data without a corresponding increase in cost. Cloud data platforms like Snowflake and Google BigQuery are prime examples of scalable solutions that allow organizations to scale operations as they grow. • Data Democratization: The rise of self-service data tools such as dbt and Looker has democratized data access, enabling teams across organizations to leverage data without needing deep technical expertise. This leads to faster decision-making across departments. • Cost Efficiency: Cloud-native data solutions enable companies to optimize storage and compute costs by only paying for what they use, making it easier for small and medium-sized businesses to manage their data infrastructure without heavy upfront investments. Challenges And Considerations In Data Engineering While the benefits are clear, there are also challenges in integrating modern data engineering solutions. • Data Quality: Ensuring that the data being ingested into the system is clean, consistent and accurate is one of the most challenging aspects of data engineering. Poor data quality can lead to incorrect insights and missed opportunities. • Data Privacy And Compliance: As data privacy regulations like GDPR continue to evolve, organizations need to ensure that their data pipelines comply with these regulations. This requires robust data governance and regular audits to maintain compliance. • Integration Complexity: Integrating multiple data sources, especially from legacy systems, can be complex. Data engineering teams must ensure seamless integration while maintaining the integrity of the data and ensuring minimal latency. • Maintaining Real-Time Performance: As real-time data processing becomes more prevalent, maintaining low-latency pipelines becomes increasingly difficult. Ensuring high throughput and minimal delays, especially with large datasets, requires careful infrastructure management. Maintaining Data Integrity And Security In an age where data is the new currency, ensuring data integrity and security has never been more important. Implementing secure access controls, encrypted data pipelines and comprehensive monitoring systems can help safeguard data from potential breaches. Companies like Shopify and Airbnb are taking proactive steps to ensure their data infrastructures are both secure and resilient, using advanced data masking and encryption techniques to protect sensitive information. Future Outlook Based on patterns observed in the industry, here are my predictions for data engineering in the next two to three years. • Data Fabric And Data Mesh Expansion: The concept of a data fabric, which integrates disparate data sources into a unified layer, will continue to gain traction. Combined with data mesh architecture, organizations will see greater flexibility and scalability in their data operations, enabling more efficient collaboration across departments. • Serverless Data Platforms: Serverless computing will take on an even greater role in data engineering. Companies will increasingly shift to serverless data architectures, reducing the overhead of managing infrastructure while focusing more on the logic of data processing. • Data Privacy By Design: As privacy concerns grow, companies will build privacy-enhancing technologies into their data pipelines from the outset, ensuring compliance with global regulations without sacrificing performance. Conclusion In 2025, data engineering is not just about building infrastructure—it's about creating agile, scalable and secure systems that can process vast amounts of data in real time. The future of data engineering looks bright as organizations continue to innovate, leveraging modern technologies to unlock new insights, improve decision-making and drive business growth. Companies that invest in the latest data engineering solutions should be well-positioned to gain a competitive edge in an increasingly data-driven world. Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
Yahoo
25-03-2025
- Business
- Yahoo
2 Hypergrowth Tech Stocks to Buy in 2025
2025 is proving to be a volatile year for the stock market, as economic uncertainty and unpredictable policies from the Trump administration throw investors for a loop. While anything can happen in the near term, growth investors should focus on investing in fast-growing companies with no shortage of long-term potential. Reddit (NYSE: RDDT) and Confluent (NASDAQ: CFLT) undoubtedly fit the bill. Reddit is a social media network unlike any other. The company's platform, split into subreddits covering specific topics, has emerged as one of the best ways for people to find useful information on the internet. While Alphabet's Google Search is still the king, a focus on serving advertisements and artificial intelligence (AI) overviews of dubious quality can lead to poor results for users. As Reddit works to better monetize its platform, revenue and usage are soaring. Daily active unique users shot up 39% year over year in the fourth quarter of 2024 to 101.7 million, and revenue surged 71% to $427.7 million. Reddit currently makes money through advertising sales and deals with AI companies for data access. Later this year, Reddit is reportedly planning to launch a feature that could lock some content behind paywalls. While this will be tricky to get right without upsetting users, it represents another potential revenue stream. Reddit is already profitable, generating a generally accepted accounting principles (GAAP) operating margin of about 12% in the fourth quarter of 2024. Gross margin was an impressive 93%, allowing the company to spend heavily on research and development while still churning out profits. Reddit runs its platform on third-party cloud computing platforms rather than its own infrastructure, so capital spending is minimal. This asset-light business model helped produce $215.8 million in free cash flow last year. Valued at nearly $21 billion, Reddit is an expensive stock relative to earnings. The stock has also taken a beating lately, tumbling more than 40% from its all-time high. While these two factors may scare some investors away, Reddit is a unique platform that can leverage its status as a source of trusted information to grow revenue and profit rapidly in the coming years. The stock will likely be volatile, but it's a great choice for long-term investors. Large enterprises with complex IT infrastructures need a robust way to connect applications together. Connecting applications directly to each other is fragile, resulting in a web of complexity that could break mission-critical data flows when anything goes wrong. Apache Kafka, an open-source event streaming platform, has become extremely popular among large companies to solve this problem. Major players in manufacturing, banking, insurance, telecom, and other industries rely on Kafka. One problem is that Kafka is a complex piece of software that requires proper configuration and management. Confluent, which was founded by the creators of Kafka, solves this problem by building propriety features on top of Kafka and other open-source software. The company's data streaming platform now has around 5,800 customers, including nearly 1,400 that spend more than $100,000 annually. Confluent's growth has been impressive, particularly for its Confluent Cloud platform. Cloud revenue rose 38% year over year in the fourth quarter, which helped push up total revenue by 23% to $261.2 million. Confluent is just scratching the surface of its total addressable market, which the company pegs at $60 billion. Confluent is still in growth mode, so it's not yet profitable on a GAAP basis. However, the company is producing positive free cash flow, an important step toward profitability. Shares of Confluent have taken an absolute beating since peaking in late 2021, down around 70%. While the stock is tough to value given the lack of profits, strong revenue growth and an attractive value proposition for its enterprise customers makes Confluent a long-term buy for growth investors. Before you buy stock in Reddit, consider this: The Motley Fool Stock Advisor analyst team just identified what they believe are the for investors to buy now… and Reddit wasn't one of them. The 10 stocks that made the cut could produce monster returns in the coming years. Consider when Nvidia made this list on April 15, 2005... if you invested $1,000 at the time of our recommendation, you'd have $721,394!* Now, it's worth noting Stock Advisor's total average return is 839% — a market-crushing outperformance compared to 164% for the S&P 500. Don't miss out on the latest top 10 list, available when you join . See the 10 stocks » *Stock Advisor returns as of March 24, 2025 Timothy Green has no position in any of the stocks mentioned. The Motley Fool recommends Confluent. The Motley Fool has a disclosure policy. 2 Hypergrowth Tech Stocks to Buy in 2025 was originally published by The Motley Fool Sign in to access your portfolio
Zawya
03-03-2025
- Business
- Zawya
VAST Data delivers the first fully unified AI data platform, adding native block storage
RELATED TOPICS TECHNOLOGY RELATED COMPANIES Apache Kafka LinkedIn Gartner VMware AI Labs. Vast Data Simplified Manag Dubai, United Arab Emirates – VAST Data, the AI data platform company, recently announced new capabilities that deliver on VAST's mission to provide a truly unified data platform for the AI era. Available next month, VAST has added Block storage functionality, completing the initial vision for the VAST DataStore as a universal, multiprotocol storage platform. The VAST Data Platform, which seamlessly combines storage, databases, and virtualized compute engine services into a unified AI operating system, is now the only exascale solution on the market capable of linearly scaling parallel data access performance for every type of data – file, object, block, table, and streaming data – for all data workloads. By having all dataaccessible in a single system, organizations can now address all workloads within one unified architecture without trade-offs in performance, scalability, or economics – allowing them to accelerate their journey to real-time insights and seamless AI adoption. According to Gartner®, 'Multiprotocol storage platforms. These are designed to support multiple storage access protocols and address the growing needs of businesses. These platforms are versatile, allowing data to be stored and accessed using different protocols, such as Network File System (NFS), Server Message Block (SMB), block and object. This flexibility enables seamless integration with diverse IT environments and ensures that the storage system can meet the varied requirements of applications and users with different protocol preferences or compatibility needs.' [1] By incorporating Block storage, VAST has transformed data management for large-scale enterprises, consolidating siloed infrastructure into one platform with a full suite of enterprise data services such as snapshots, multi-tenancy, quality of service (QoS), encryption, and granular Role-Based Access Control (RBAC). These new features allow VAST to meet the demands of modern enterprise IT infrastructure: Seamless Integration with Virtualization Platforms: The VAST Data Platform now provides robust support for environments such as VMware, Hyper-V, and other hypervisors. Features such as multi-tenancy and QoS allow IT teams to isolate workloads, guarantee performance, and simplify resource management across hundreds or thousands of virtual machines. Optimized for Kubernetes and Containerized Applications: For organizations leveraging Kubernetes, Openshift or other container orchestration platforms, the addition of Block storage enables persistent storage for containerized workloads. From transactional databases in containers to stateful microservices, the platform delivers the scalability and performance required for DevOps workflows, test environments, and production-grade applications. Unified Infrastructure for Hybrid Workloads: In modern enterprise environments, virtualized and containerized workloads often coexist, creating challenges in managing disparate storage systems. The VAST Data Platform eliminates this complexity by consolidating both workload types onto a single, unified storage architecture. This approach reduces operational overhead, simplifies provisioning, and ensures optimal performance across diverse application environments. Boot from SAN for Simplified Management and High Availability: With support for Boot from SAN, enterprises can streamline server deployment and management by eliminating reliance on local disks. This approach enhances disaster recovery, improves redundancy, and enables rapid provisioning of new virtual or bare-metal servers while ensuring consistent performance across IT environments. 'Our vision for the VAST Data Platform was always to enable organizations to process, analyze, and act on data in real-time, empowering them to scale effortlessly, reduce their infrastructure costs, and innovate faster by unifying AI data infrastructure within a single, powerful platform," said Aaron Chaisson, Vice President, Product & Solutions Marketing at VAST Data. 'With today's announcement, we're eliminating the data silos that once hindered AI and analytics initiatives, affording customers faster, more accurate decisions and unlocking data-driven growth.' Additional Resources: BLOG: The Latest VAST Release is All About the Protocols, by Howard Marks DEMO: VAST Data Block Support PRESS RELEASE: VAST Data Announces the VAST Event Broker for Apache Kafka, A Breakthrough for Real-Time Event Processing and Analytics About VAST Data VAST Data is the data platform company built for the AI era. As the new standard for enterprise AI infrastructure, organizations trust the VAST Data Platform to serve their most data-intensive computing needs. VAST Data empowers enterprises to unlock the full potential of all of their data by providing AI infrastructure that is simple, scalable, and architected from the ground up to power deep learning and GPU-accelerated data centers and clouds. Launched in 2019, VAST Data is the fastest growing data infrastructure company in history. For more information, please visit and follow VAST Data on X (formerly Twitter) and LinkedIn. Media Contact vastdata@ [1] Gartner, Stop Buying Storage, Embrace Platforms Instead, Julia Palmer, Jeff Vogel, Chandra Mukhyala, January 15, 2025. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
