IBM will acquire data streaming platform provider Confluent for $ US 11 billion. Confluent operates a commercial distribution of Apache Kafka, an open-source platform that enables realtime data streaming between applications and systems.

IBM claims the transaction addresses growing enterprise requirements for real-time data integration as organisations deploy generative AI and manage increasingly complex information architectures.

Data streaming refers to the continuous flow of information between systems as events occur, rather than processing data in batches. Traditional enterprise architectures move data in scheduled intervals - nightly batch jobs transfer records between databases, for example. Data streaming creates persistent connections that transmit information the moment it is generated.

Apache Kafka functions as a distributed message broker that captures, stores and routes data streams. Applications publish events to Kafka topics - categorised feeds of information. Other applications subscribe to these topics and receive events in real time. Kafka retains event logs, enabling systems to replay historical data or recover from failures.

It addresses a fundamental challenge: maintaining data consistency across siloed systems. When customer records update in a CRM system, inventory changes in an ERP platform, or transactions process through payment systems, streaming platforms propagate those changes immediately to downstream applications, data warehouses and analytics tools.

The technology has direct implications for records management and compliance. Data streaming platforms create immutable event logs - permanent records of every state change across systems. This provides audit trails showing exactly when information was created, modified or accessed across the enterprise.

Organisations managing complex compliance requirements use streaming platforms to enforce data governance policies in realtime. When personal information moves between systems, streaming infrastructure can apply masking, encryption or access controls at the point of transfer rather than relying on downstream systems to enforce policies.

"IBM and Confluent together will enable enterprises to deploy generative and agentic AI better and faster by providing trusted communication and data flow between environments, applications and APIs," said Arvind Krishna, IBM chairman, president and chief executive officer.

Technical Architecture

Confluent's platform extends Apache Kafka with enterprise features including pre-built connectors for popular enterprise systems, stream processing capabilities, and governance tools. The platform integrates with major cloud platforms including AWS, Google Cloud Platform and Microsoft Azure, as well as data warehouses like Snowflake.

Deployment options include fully managed cloud services, self-managed on-premises installations, and hybrid bring-your-own-cloud configurations.

The platform serves more than 6,500 clients. The company states that more than 40% of Fortune 500 organisations use its services. Jay Kreps, Confluent's CEO and co-founder, co-created Apache Kafka while working at LinkedIn before founding Confluent in 2014.

https://www.ibm.com