# Rihan Hai et al. - Data Lakes: A Survey of Functions and Systems (Highlights)  ## Metadata **Review**:: [readwise.io](https://readwise.io/bookreview/61259460) **Source**:: #from/readwise #from/zotero **Zettel**:: #zettel/fleeting **Status**:: #x **Authors**:: [[Rihan Hai]], [[Christos Koutras]], [[Christoph Quix]], [[Matthias Jarke]] **Full Title**:: Data Lakes: A Survey of Functions and Systems **Category**:: #articles #readwise/articles **Category Icon**:: 📰 **Highlighted**:: [[2026-06-12]] **Created**:: [[2026-06-13]] ## Highlights -  (Page 6) ^1024613988 - In contrast to traditional ‘schema-on-write’ approaches such as data warehouses, data lakes are repositories storing raw data in its original formats and providing a common access interface. ([Page 12571](zotero://open-pdf/library/items/ABH6J6HW?page=1&annotation=J47WG2IU)) ^1024613989 ### II. A BRIEF HISTORY OF DATA LAKES - In 2013, Pivot proposed an architecture for a business data lake [19], which ingests multiple data sources in three abstract tiers: (1) an ingestion tier takes data in real-time/microbatch/batch, (2) an insight tier analyzes data in real-time or interactive time and derives insights, and (3) an action tier that links insights with the existing applications; additional tiers monitor and manage the data. ([Page 12573](zotero://open-pdf/library/items/ABH6J6HW?page=3&annotation=TSQYD7T9)) ^1024613991 - Dixon revisited the general concept [38] and emphasized that a data lake should also be equipped with metadata and governance, so that even with data in its raw form, a data lake could enable ad-hoc data analytics. ([Page 12574](zotero://open-pdf/library/items/ABH6J6HW?page=4&annotation=UKLRJSE5)) ^1024613992 ### III. DATA LAKE ARCHITECTURE AND PROPOSED CATEGORIZATION CRITERIA ### IV. STORAGE ### V. INGESTION #### A. Metadata Extraction - The process of DATAMARAN does not require human supervision and provides a high extraction accuracy compared to existing works. ([Page 12578](zotero://open-pdf/library/items/ABH6J6HW?page=8&annotation=XGW935UE)) ^1024613997 #### B. Metadata Modeling - For structured or semi-structured data in typical business scenarios, a promising conceptual modeling environment is data vault [57], [107]. It has three main types of elements: hubs representing business concepts, links indicating the many-to-many relationships among hubs, and satellites with descriptive properties of hubs and links [86], [87]. ([Page 12578](zotero://open-pdf/library/items/ABH6J6HW?page=8&annotation=QW9HB5BH)) ^1024613999 - Nogueira et al. [107] show how their conceptual model based on data vault can be transformed into relational and document-oriented logical models, and further to physical models (PostgreSQL and MongoDB, respectively). ([Page 12578](zotero://open-pdf/library/items/ABH6J6HW?page=8&annotation=33XTFLIM)) ^1024614000 - To efficiently discover relevant datasets from massive data sources, Aurum [48] devises an enterprise knowledge graph (EKG) to capture and query relationships among datasets. ([Page 12578](zotero://open-pdf/library/items/ABH6J6HW?page=8&annotation=AJELH2WQ)) ^1024614001 - he metadata model might be easily extensible, but the management of the metadata (i.e., its storage, user interfaces for creation and manipulation, etc.) is much more challenging. ([Page 12578](zotero://open-pdf/library/items/ABH6J6HW?page=8&annotation=UWXMWR9H)) ^1024614002 ### VI. MAINTENANCE #### A. Dataset Organization #### B. Related Dataset Discovery - To construct it, Aurum first profiles each table column by adding signatures, i.e., information extracted from column values such as cardinality, data distribution, and a representation of data values (i.e., MinHash). Then, it indexes these signatures using locality-sensitive hashing (LSH). ([Page 12580](zotero://open-pdf/library/items/ABH6J6HW?page=10&annotation=PJMFHCTX)) ^1024614006 - Given a table T in the data lake, and one specific column C from T , JOSIE can return tables in the data lake that could be joined with T on C. ([Page 12581](zotero://open-pdf/library/items/ABH6J6HW?page=11&annotation=3LNFS59B)) ^1024614007 - The core solution of DLN is building random-forest classification models. ([Page 12582](zotero://open-pdf/library/items/ABH6J6HW?page=12&annotation=N7S993LY)) ^1024614008 #### C. Data Integration - Specifically, ALITE embeds columns by using state-of-the-art techniques such as TURL [32], and then applies hierarchical clustering in order to obtain sets of columns that are related. ([Page 12583](zotero://open-pdf/library/items/ABH6J6HW?page=13&annotation=NGFVXKZW)) ^1024614010 #### D. Metadata Enrichment - That is, given a set of input tables, D4 discovers their semantic domains and represents each domain with a set of terms. ([Page 12583](zotero://open-pdf/library/items/ABH6J6HW?page=13&annotation=EVYBI6MA)) ^1024614012 - In order to obtain metadata that describes dataset origin, ownership, and its possible usage, it is often beneficial to keep human experts in the loop. ([Page 12583](zotero://open-pdf/library/items/ABH6J6HW?page=13&annotation=G6T4P6M7)) ^1024614013 #### E. Data Cleaning #### F. Schema Evolution #### G. Data Provenance ### VII. EXPLORATION ### VIII. CHALLENGES AND FUTURE DIRECTIONS #### B. Data Lakes Meet Machine Learning - Earlier, a common industrial practice was to apply data lakes (e.g., Amazon S3, GCP) as a cheap storage of large-scale raw data, before the datasets are selected and transformed for data warehouses (e.g., Snowflake, BigQuery). The overhead and complexity of maintaining two systems, a data lake and a data warehouse, have led to Lakehouses ([Page 12587](zotero://open-pdf/library/items/ABH6J6HW?page=17&annotation=TMWJMEEQ)) ^1024614020