What is a data lake? Definition and principles
A data lake is a vast storage repository able to hold very large volumes of data in a wide variety of formats, with no fixed size limit and no schema imposed on entry. It takes in structured data from relational databases, semi-structured data such as JSON or application logs, and unstructured content alike: images, videos, audio files, documents, sensor telemetry.
Faced with exploding volumes, it has established itself as the foundation of modern data architectures, built on cloud object storage, Amazon S3, Azure Data Lake Storage, Google Cloud Storage, whose cost per terabyte bears no comparison with that of a classic warehouse. Its promise: keep raw data as it arrives, then prepare it the moment a use demands it, whether analytics, business intelligence or training machine-learning models.
The concept emerged in the early 2010s with the Hadoop ecosystem, deployed on clusters of on-premise servers. That first generation showed the way, but at the cost of heavy operations and rare skills. The shift to the clouds' managed object storage lifted those constraints: near-unlimited capacity, guaranteed durability, pay-as-you-go, and above all a clean separation between storage, pooled and cheap, and compute engines, chosen and sized per use. It is this decoupled architecture that dominates today.
The Vs of big data: volume, velocity, variety and veracity
Volume reflects the continuous acceleration in data production, driven by application telemetry, industrial IoT and generated content, now including content generated by AI itself. Velocity refers to the speed at which data is produced and must be processed: event streams, streaming, navigation data, real-time transactions.
These dimensions reinforce one another: an e-commerce site produces at once a massive volume of navigation events, a continuous stream of transactions to process in near real time and a wide variety of content, customer reviews, product images, technical logs. An architecture that can only handle one dimension out of three ends up bypassed by teams, who recreate uncontrolled local copies.
Variety denotes the multiplicity of formats to absorb: logs, JSON, multimedia content, sensor data, binary formats, office documents. It is precisely this diversity that the data lake is designed to accommodate with no prior constraint, where traditional systems impose a single structure.
Veracity, the V that became critical in the AI era
A fourth V has imposed itself in practice: veracity, meaning the reliability and traceability of data. An AI model trained or fed with dubious data produces dubious results, with the confidence of an automated system. Data contracts, quality tests and lineage have become first-order requirements, not optional refinements.
From data lake to lakehouse: open table formats
The historical limit of the data lake was the absence of transactional guarantees: overwritten files, inconsistent reads, schemas drifting silently. Open table formats lifted that limit by bringing database properties to object storage: ACID transactions, controlled schema evolution, time travel, reliable updates and deletes, indispensable in particular to honour GDPR erasure requests. Without them, it is impossible to fix a partial write, to track how a table evolves over time or to guarantee that a dashboard and a model read the same version of the data.
Apache Iceberg and Delta Lake, foundations of interoperability
Apache Iceberg and Delta Lake have established themselves as de facto standards, adopted by every major platform, Databricks, Snowflake, BigQuery, Microsoft Fabric and the clouds' managed services. The strategic consequence is major: data is no longer captive to one engine. The same table can be written by a Spark pipeline, queried by a SQL warehouse and read by a machine-learning framework, without duplication.
The choice of catalogue then becomes structuring: it is the catalogue that references tables, arbitrates concurrent access and carries permissions. Technical catalogues exposed through standardised REST APIs let several engines write and read the same tables safely, and let teams change compute tools without migrating a single piece of data.
This convergence gave birth to the lakehouse: an architecture that combines the flexibility and cost of the data lake with the performance, transactions and governance of a warehouse, on a single copy of the data.
The benefits of the data lake for analytics and AI
The first advantage remains simplicity of storage: no need to model data upfront, which drastically shortens the delay between collection and first exploration. Add to that cost-effective scalability, object storage decouples storage cost from compute cost, and versatility in the face of heterogeneous sources: each team consumes the same data with the tool suited to its use, from SQL to Python notebooks.
The storage-compute decoupling also changes project economics: a one-off exploration only mobilises compute for the duration of a query, a model training run can scale up for a few hours then release everything. Teams stop arbitrating between keeping data and controlling costs: they keep everything, compress intelligently and pay for compute based on actual use.
The data foundation of generative AI projects
The data lake has become the natural feeding point of AI projects: training and fine-tuning corpora, source documents for RAG architectures, embeddings and vector indexes, evaluation sets. Platforms now support these uses natively, vector search over lake tables, document preparation pipelines, which makes a governed lake a concrete prerequisite of any enterprise AI strategy.
Add to that continuous analytics: lake tables are now fed as streams, through change data capture (CDC) from operational databases or event ingestion, which brings data freshness down from the previous day to a few minutes. Operational dashboards and anomaly detection thus draw directly on the lake, with no parallel chain to maintain.
Data lake vs data warehouse: schema on read or on write
The founding distinction lies in when preparation happens. The data lake applies schema on read: it ingests raw data quickly and structures it at access time. The data warehouse applies schema on write: data is modelled, cleaned and structured before entering, which guarantees consistent indicators but slows the integration of new sources.
Concretely, the same revenue indicator can exist in three states: raw events in the ingestion zone, a cleaned and deduplicated table in the refined zone, a certified aggregate exposed to BI tools. Each state has its owner, its quality level and its audience, it is this gradation, more than the tooling, that makes the architecture legible.
The data lake also keeps immutable ingestion layers: source data remains intact, which provides audit trails, eases data discovery and makes it possible to replay a failing pipeline from the origin, without loss.
The choice is reasoned by use: regulatory reporting and financial indicators demand the rigour of schema on write; exploration, data science and AI benefit from the latitude of schema on read. The right question is no longer choosing between lake and warehouse, but deciding which zone serves which consumer, with what contractualised level of quality.
An opposition that is fading
In practice, the opposition is fading: warehouses read the lake's open formats, lakehouses deliver warehouse-grade SQL performance, and most organisations have raw zones, refined zones and business views coexist within a single platform, organised in successive layers of increasing quality.
Avoiding the data swamp: governance, quality and cataloguing
The most documented risk of the data lake is its degeneration into a data swamp: a marsh of files with no owner, no documentation and no measured quality, where nobody finds anything and which nobody trusts any more. The remedy is organisational as much as technical: clear ownership of datasets, ingestion rules, sensitivity classification, fine-grained access control.
Security follows the same logic of granularity: encryption at rest and in transit, permissions at table, column and row level, masking of personal data, full access logging. On a lake that concentrates most of the company's information assets, these controls do not only serve compliance: they condition the business lines' trust, and therefore their willingness to share their data.
Catalogues and lineage: making data findable and reliable
Modern data catalogues expose every table with its description, owner, freshness and lineage, where the data comes from, what transformations it has undergone, who consumes it. Combined with automated quality tests in pipelines, they turn the lake into an asset that can be used with confidence, including by AI agents that must be able to cite their sources. Recent platforms add a semantic layer on top: shared business definitions, certified metrics and automatically generated documentation, which bring the data closer to those who consume it.
Designing a governed, scalable data lake with Adservio
At Adservio, we view the data lake not as an end in itself but as one brick in a data architecture designed around uses. The choice of open formats, the organisation of zones, governance, security and preparation for access are weighed against real analytical and AI needs, never in the abstract.
We support your teams in designing a scalable, secure and governed data repository, and in evolving it towards the lakehouse when uses justify it, so that it lastingly feeds your analytics, machine-learning and generative AI projects while remaining under their control. In the field, this support runs from architecture framing and format choices to putting the first use cases into production, with a transfer of skills that leaves your teams fully autonomous on the platform.
STAY POSTED
Get our next analyses and field notes straight to your inbox.
