From data lake to flexible data lakehouse
Many companies and organizations have invested large sums of money in data warehousing (DWH) over the past few decades. In recent years, however, DWHs have fallen somewhat into disrepute for being cumbersome and complex obstacles on the road to data-driven organizations, rather than accelerating the journey there. Data Lakes have long been hoped to take the pressure off DWHs and make them more agile. Here and there, the end of data warehousing has already been proclaimed, but a data lake alone does not replace a structured data warehouse. Only since the emergence of so-called “lakehouses", which combine the advantages of DWHs and data lakes, has there been a real alternative to the DWH. According to the principle "from data lake to flexible data lakehouse", or is this just a DWH in a new guise? That's what we want to find out in this article.
The data and the data structures of a DWH can be specifically queried and changed with SQL - the files in a data lake cannot be so far. This possibility of targeted querying and modification is central for robust ETL or ELT processes for data integration and for efficient historization, i.e., for core tasks of data collection.
New technologies, cloud standards and open interfaces are fueling the competition between technologies in the data lakehouse. There are already prefabricated lakehouse architectures that can be set up in the cloud almost at the push of a button, such as the Eraneos Data Hub.
A data lakehouse can replace a data warehouse, but the historization of data, professional refinement and visualization remain tasks that data engineers have to deal with. No one is spared professional data management.
Managing many different data silos is not only inefficient, but also leads to redundancies and high costs. This paper is currently only available in German.
The e-paper series "Digital Platforms - Enabler of an Agile Enterprise Architecture" sheds light on the design of business applications based on digital platforms.The e-papers are currently only available in German.
Data Analytics & AI | data from cost factor to asset value
Every company has large amounts of data in which valuable business-relevant knowledge lies dormant. Our data science specialists can help you extract and utilize this knowledge.
We have compiled reports on our projects, interesting facts from the various competence and customer areas as well as information about our company for you here.