Billions of Calls, Stored and Searchable? No Worries Says EMC

October 2nd, 2006: EMC has teamed up with Intech and SenSage to forge a Internet, messaging and telephony transaction solution that satisfies the considerable demands of the recent EU Data Retention Directive.

With new regulations looming, the three companies put their thinking caps on and come up with a solution that they say not only enable telecommunication and Internet providers a cheaper way to compliantly manage communication records, but also one that allows law enforcement agencies to quickly access phone and Internet records to pinpoint serious criminal activities.

Announced in March 2006, the EU Data Retention Directive requires Telco’s and ISPs to securely retain traffic and location data and enable it to be analysed. The mandate covers fixed lines, mobile phones and Internet data (e-mail, VoIP and access) for up to two years.

No small task considering millions of calls and connections occur each day, equating to billions of records and terabytes of transaction data that must be securely stored and rapidly analysed. The Directive also requires service providers to produce answers to law enforcement inquiries without undue delay.

The three have demonstrated the ability to manage and obtain results in minutes from over 100 billion Call Detail Records (CDR). Intech collected and processed the sample data, randomizing and replicating it until it hit the necessary 100 billion records, or 13 terabytes of raw, uncompressed data. From here, SenSage provided the scalable event data management platform that powers the collection, compression, management and high-speed analysis of the CDR data. Finally, an EMC Centera storage cluster handled retention and online availability of the stored data.

The system executed un-indexed queries at more than 27.9 million records per second, obtaining answers in less than seven minutes within a three month search range.

“The EU Data Retention Directive puts tremendous pressure and a clear timeline on Telcos and ISPs to meet security and investigation obligations,” says Fernando Elizalde, senior telecommunications analyst at Frost & Sullivan. “The newly developed technology offered by EMC, Intec and SenSage supports the Directive mandates by providing a progressive, scalable platform, to cost-effectively manage the broad event data integration, storage and analysis requisites.”

The Directive states that all service providers must be compliant by August, 2007.

Comment on this story