In this week’s real-time analytics news: The Databricks Data + AI Summit produced dozens of company and partner announcements.
Keeping pace with news and developments in the real-time analytics and AI market can be a daunting task. Fortunately, we have you covered with a summary of the items our staff comes across each week. And if you prefer it in your inbox, sign up here!
Databricks made a number of announcements at this week’s Data + AI Summit. Some of the top news items from the summit include:
The open sourcing of Unity Catalog, the unified solution for data and artificial intelligence (AI) governance across clouds, data formats, and data platforms. Unity Catalog OSS offers a universal interface that supports any data format and compute engine, including the ability to read tables with Delta Lake, Apache Iceberg, and Apache Hudi clients via Delta Lake UniForm. It also supports the Iceberg REST Catalog and Hive Metastore (HMS) interface standards. The move has support from Amazon Web Services (AWS), Google Cloud, Microsoft, NVIDIA, Salesforce, and others.
The launch of Databricks LakeFlow, a new solution that unifies all aspects of data engineering, from data ingestion to transformation and orchestration. With LakeFlow, data teams can simply ingest data at scale from databases such as MySQL, Postgres, and Oracle, as well as enterprise applications such as Salesforce, Dynamics, Sharepoint, Workday, NetSuite, and Google Analytics. Databricks is also introducing Real Time Mode for Apache Spark, which allows stream processing at ultra-low latency.
The launch of Databricks AI/BI, a new type of business intelligence (BI) product that aims to democratize analytics and insights for anyone in an organization. Databricks AI/BI features a pair of complementary experiences: Dashboards, an AI-powered, low-code interface for creating and distributing fast, interactive dashboards; and Genie, a conversational interface for addressing ad-hoc and follow-up questions through natural language. Both are powered by a compound AI system that continuously learns from usage across an organization’s entire data stack, including ETL pipelines, lineage, and other queries.
The announcement of new Mosaic AI capabilities to help customers build production-quality Generative AI applications. Databricks is investing in Mosaic AI in three key areas: support for building compound AI systems, capabilities to improve model quality, and new AI governance tools.
An expanded collaboration with NVIDIA to optimize data and AI workloads by bringing NVIDIA CUDA accelerated computing to the core of Databricks’ Data Intelligence Platform. Through this broadened alliance, Databricks is adding native support for NVIDIA GPU acceleration on the Databricks Data Intelligence Platform.
Partner announcements at the Data + AI Summit
Alation announced a deepened integration with Databricks to deliver trusted data for AI initiatives to the enterprise. This strengthened partnership through the Alation Open Data Quality Initiative improves data reliability, streamlines compliance, and facilitates more effective decision-making.
Informatica announced an expanded partnership with Databricks with new product capabilities and offerings, bringing together the full range of Informatica’s AI-powered Intelligent Data Management Cloud (IDMC) platform capabilities within the Databricks Data Intelligence Platform. New capabilities through the partnership include a GenAI solution blueprint for Databricks DBRX, native Databricks SQL ELT, Cloud Data Integration-Free Service (CDI-Free) on Databricks partner connect, and full IDMC support via Unity Catalog.
Infoworks.io announced that it has added Databricks Unity Catalog integration to Infoworks Replicator, the company’s solution to automate the migration of Hadoop data and metadata to the cloud. Replicator with Unity Catalog integration enables automated data and metadata migration and cataloging in Databricks Unity Catalog in a single step, which establishes and maintains a robust unified data governance framework at every stage of migration.
KPMG announced that its audit practice is embedding the Databricks Data Intelligence Platform within KPMG Clara, its global smart audit platform. By leveraging Databricks technology, KPMG auditors can analyze billions of financial transactions across thousands of audits, driving enhanced audit quality and deeper insights into audit-related risks in order to maintain public trust in the capital markets.
Matillion announced the launch of Retrieval Augmented Generation (RAG) and pushdown AI components for Databricks, bringing AI and LLMs directly into data pipelines to transform and enrich any data type, structured or unstructured. Matillion’s Data Productivity Cloud GenAI components seamlessly integrate with Databricks SQL, the serverless data warehouse, enabling more than 50 use cases.
Precisely announced it has expanded its partnership with Databricks, making its Precisely Data Integrity Suite available on Databricks Partner Connect. The suite brings enhanced data management capabilities to Databricks environments that address all aspects of data integrity – starting with data quality and observability and geo addressing – so users can leverage trustworthy data in their AI or advanced analytics initiatives.
Qlik announced a new strategic integration with Databricks AI Functions, enhancing enterprise AI capabilities through advanced data solutions. The fusion between Qlik’s data integration platform and Databricks AI model APIs empowers users to tap into advanced AI functionalities on the Databricks Data Intelligence Platform directly within their analytics workflows, making it easier to deploy AI-driven transformations and analyses.
Other real-time analytics news in brief
Amazon Web Services (AWS) announced a $230 million commitment for startups around the world to accelerate the creation of generative AI applications. This will provide startups, especially early-stage companies, with AWS credits, mentorship, and education. Part of the new commitment will fund the second cohort of the AWS Generative AI Accelerator, a program that provides hands-on expertise and up to $1 million in credits to each of the top 80 early-stage startups that are using generative AI to solve complex challenges.
In other AWS news, the company announced the launch of two AWS Certifications focused on artificial intelligence (AI), machine learning (ML), and generative AI. They include the AWS Certified AI Practitioner, a foundational-level certification for people to showcase their understanding of AI and generative AI concepts; and the AWS Certified Machine Learning Engineer – Associate, which is designed for individuals with at least one year of experience building, deploying, and maintaining AI and ML solutions on AWS.
Oracle and Google Cloud announced a partnership that gives customers the choice to combine Oracle Cloud Infrastructure (OCI) and Google Cloud technologies to help accelerate their application migrations and modernization. Google Cloud’s Cross-Cloud Interconnect will be initially available for customer onboarding in 11 global regions, allowing customers to deploy general-purpose workloads with no cross-cloud data transfer charges. Later this year, a new offering, Oracle Database@Google Cloud, will be available with the highest level of Oracle database and network performance, along with feature and pricing parity with OCI.
Confluent announced Build with Confluent, a new partner program that helps system integrators speed up the development of data streaming use cases and quickly get them in front of the right audiences. Build with Confluent makes it faster and easier than ever to monetize real-time solutions built on Confluent’s data streaming platform. It includes specialized software bundles for developing joint solutions faster and support from data streaming experts to certify offerings.
Credo AI announced a comprehensive AI Risk and Controls Library. The library of AI-specific Risk Scenarios and Controls builds on academic research, Credo AI’s domain expertise, and industry frameworks from leaders like MITRE and NIST. The library is designed to anticipate and mitigate negative incidents, enabling the development and deployment of safe and controlled AI systems with unparalleled speed to governance.
The Digital Twin Consortium (DTC) announced that MongoDB has joined the consortium. Adding MongoDB to the consortium simplifies access to its developer data platform for DTC’s network, creating digital replicas of physical assets, processes, and systems through bi-directional data and information synchronization.
DiffusionData announced a new partnership with Technaura to provide markets such as Financial Services, IoT, Telco, and Healthcare with reliable and secure real-time data distribution at Internet scale. Contributing to the effort, DiffusionData’s flagship product, Diffusion, offers a framework that is purpose-built to simplify and speed data-driven, real-time application development, reduce operational costs, and economically deliver hyper-personalized data. Technaura provides Big Data processing, high volume data streams, consolidating scattered data flows, reliable data queues to record incoming data, scalable microservices and analytics.
Elastic announced a partner integration package with LangChain that will simplify the import of vector database and retrieval capabilities of Elasticsearch into LangChain applications. This deeper collaboration and native support will empower developers to leverage common retrieval strategies to bring greater context, relevancy, and accuracy to their application builds.
Frame AI announced that it is offering its Enterprise API through Amazon’s AWS Marketplace. Complementing existing Retrieval-Augmented Generation (RAG) systems, Frame AI’s Enterprise API provides companies access to a composable AI infrastructure featuring Stream-Trigger Augmented Generation (STAG) components for proactively acting on streaming, unstructured data.
Gretel has released a new high-quality synthetic dataset designed to help AI developers effectively detect and remove sensitive personal information in financial documents. The dataset, created using the company’s state-of-the-art AI system Gretel Navigator, addresses critical challenges in scaling highly performant and privacy-preserving generative AI applications. The dataset supports multiple languages and regionalization out of the box.
Hammerspace announced the addition of the S3 interface to its Global Data Platform, advancing the orchestration of existing data sets to available compute resources. By adding S3 to all other standard protocols for data ingestion, the Hammerspace platform ensures a seamless and efficient transfer of existing data into a global file system fueled by its automated orchestration system.
Insight Enterprises has launched a fast-start service that accelerates the adoption of data analytics across the enterprise through the use of Microsoft Fabric. The Insight Lens architecture solution reduces the implementation time of Microsoft Fabric. Organizations benefit from the creation of a Microsoft Fabric proof of concept, tailored support from Insight Data and AI experts, and access to highly templated deployment architecture.
Pegasystems announced significant product enhancements and new tools to complement its lineup of enterprise solutions. The announcements include Pega GenAI Socrates, a new tool aimed at transforming the traditional corporate learning process, and new capabilities coming to its Pega GenAI Blueprint solution. The Pega GenAI Socrates tool engages in interactive dialogue that constantly adapts to each person’s industry, skill level, and learning needs. Pega GenAI Blueprint, launched earlier this year, crafts tailored workflows for any business use case and now sports new capabilities and UI features that make innovating even easier.
In other Pega news, the company announced an expansion of its Pega GenAITM capabilities to connect to Amazon Web Services (AWS) and Google Cloud’s Large Language Models (LLMs). With this expansion, Pega’s enterprise platform will provide clients with the ability to connect with a vast range of generative AI services and models with Pega GenAI architecture to support generative AI models within Pega solutions.
Rivery announced Rivery Blueprint, a new way to quickly and easily build data pipelines for GenAI-powered and data applications. By using a simple AI prompt, data engineers can automatically generate the required configuration to connect to any REST API without sifting through API docs and detailing formats. They can build and see all data pipelines in one place without having to use a combination of tools to make custom pipeline connections. Non-technical data professionals can also use Rivery Blueprint to quickly input data for analytics and AI applications. Rivery Blueprint is currently available in Private Preview with select customers and will be generally available later in 2024.
SymphonyAI announced IRIS Workspace, a no-code application for industrial knowledge workers to engage in a live, collaborative environment with all relevant information for rapid problem-solving. IRIS Workspace empowers cross-functional teams to simultaneously engage with time-series, P&IDs (piping and instrument diagrams), documents, drawings, and images in a shared workspace for real-time problem solving using unified and contextualized data from IRIS Foundry, an AI-powered, composable industrial dataops platform.
Unbabel announced the launch of TowerLLM, a Large Language Model (LLM) designed, trained, and optimized for translation. Unbabel customers using TowerLLM will significantly improve machine translation accuracy, reducing errors and cost, and will benefit from a more cost-effective price than popular LLMs.
Unravel Data announced the release of three new AI agents: the Unravel DataOps Agent, the Unravel FinOps Agent, and the Unravel Data Engineering Agent. While generative AI capabilities such as Large Language Models allow users to ask general questions and receive broad answers, AI agents are designed to perform specific, actionable tasks within their domains. These AI agents leverage domain-specific knowledge graphs and advanced automation to tackle precise problems faced by data teams, significantly enhancing efficiency and accuracy. All three new AI agents are included in the latest version of Unravel Platform.
If your company has real-time analytics news, send your announcements to [email protected].
In case you missed it, here are our most recent previous weekly real-time analytics news roundups:
- Real-time Analytics News for the Week Ending June 8
- Real-time Analytics News for the Week Ending June 1
- Real-time Analytics News for the Week Ending May 25
- Real-time Analytics News for the Week Ending May 18
- Real-time Analytics News for the Week Ending May 11
- Real-time Analytics News for the Week Ending May 4
- Real-time Analytics News for the Week Ending April 27
- Real-time Analytics News for the Week Ending April 20
- Real-time Analytics News for the Week Ending April 13
