Elastic has announced the launch of Search AI Lake, to optimize realtime, low-latency applications such as search, retrieval augmented generation (RAG), observability, and security. Search AI Lake also drives the new Elastic Cloud Serverless offering, which simplifies operations by automatically scaling and managing workloads.

Combining the vast storage capacity of a data lake with the advanced search and AI relevance capabilities of Elasticsearch, Search AI Lake provides low-latency query performance while maintaining scalability, relevance, and cost-effectiveness.

By fully separating storage and compute, the platform promises scalability and reliability using object storage. Dynamic caching supports high throughput, frequent updates, and interactive querying of large data volumes, eliminating the need for replicating indexing operations across multiple servers, thus reducing costs and data duplication.

Enhancements like smart caching and segment-level query parallelization maintain excellent query performance, even with data stored on object stores. These improvements reduce latency by enabling faster data retrieval and processing more requests quickly.

By separating indexing and search processes, the platform can independently and automatically scale to meet diverse workload needs.

Users can utilize a suite of powerful AI relevance, retrieval, and reranking capabilities, including a native vector database integrated into Lucene, open inference APIs, semantic search, and transformer models. These features work seamlessly with a range of search functionalities.

Elasticsearch’s query language, ES|QL, transforms, enriches, and simplifies investigations with fast concurrent processing, regardless of data source and structure. It supports precise and efficient full-text search and time series analytics for pattern identification in geospatial analysis.

Users can build, deploy, and optimize machine learning directly on all data for superior predictions. For security analysts, prebuilt threat detection rules can easily run across historical information, and unsupervised models perform near-real-time anomaly detection on data spanning long periods, surpassing other SIEM platforms.

Users can query data from the region or data centre where it was generated through one interface. Cross-cluster search (CCS) avoids the need to centralize or synchronize data, allowing for rapid querying and analytics while reducing data transfer and storage costs.

Search AI Lake powers the new Elastic Cloud Serverless offering, leveraging the architecture's speed and scale to eliminate operational overhead. Users can quickly start and scale workloads with Elastic managing all operations, including monitoring, backup, configuration, and sizing. Users only need to provide their data and select Elasticsearch, Elastic Observability, or Elastic Security on Serverless.

Ken Exner, Chief Product Officer at Elastic, highlighted the need for a new architecture capable of handling compute and storage at enterprise speed and scale for AI and real-time workloads.

"Search AI Lake addresses the limitations of traditional data lakes, providing the necessary architecture for the search, observability, and security workloads of tomorrow," Exner said.

Currently available in tech preview, Search AI Lake and Elastic Cloud Serverless represent the next step in advancing real-time, low-latency search capabilities. For more information on how to get started, visit the Elastic blog.