Komprise has released Transparent File Tables, which lets analytics and AI platforms such as Snowflake and Databricks query enterprise file data without copying or moving it.
The feature presents unstructured data as an Apache Iceberg table, so data engineers and scientists can work with files in familiar tools. Komprise says this avoids the heavy cost of large-scale data movement.
The company points to a gap it aims to fill. Although unstructured data is over 80% of an enterprise’s data footprint, less than 1% of this data is being used in AI, according to IDC
That data has stayed dark because it lacks consistent schema, is often poor quality, and is slow and costly to move, the company said. Ingesting petabytes across NAS and cloud can take weeks or months.
Transparent File Tables uses Komprise metadata enrichment to build a schema and a pointer to the files. Its Transparent Move Technology loads data only when needed, and Komprise applies governance based on user access permissions.
“The reason 99 per cent of enterprise unstructured data has been dark to AI and analytics is because discovering and generating its schema and moving it is inherently complex and costly,” said Kumar K. Goswami, chief executive and co-founder of Komprise.
“Komprise brings to light the huge petabytes of enterprise unstructured data in a form that data teams can access easily and transparently for analytics. Komprise Transparent File Tables opens a whole new world to AI.”
When full files are needed, Komprise says its Intelligent AI Ingest moves them at twice the speed of standard transfer tools.