![]() All users need to do in this new ecosystem is shop around and choose any open project or vendor that implements the open standard and your migration cost will be practically non-existent. This new freedom gives the user the features of a data warehouse, with the scalability of the cloud, and a free market of a la carte services to handle your needs at whatever price point you need at any given time. This is not just open in the sense of the freedom to use and contribute to a project, but the open standard that enables you the freedom to simply move between different projects. With the open S3 storage standard, open file standards like Parquet, open table standards like Iceberg, and the ANSI SQL spec closely followed by Trino, the entire analytics warehouse has become a modular stack of truly open components. However, that’s a fallacy I hope vendors will slowly begin to understand is not true. It pushes the incentives for vendors to support these open standards which puts them in a seemingly vulnerable position compared to locking users in. This approach to data virtualization is so interesting as each system offers full leverage over vendors trying to lock you in their particular query language or storage format. Preaching the combination of these two became somewhat of a mission of mine after that. Just as I had the epiphany with Trino ( Presto at the time ) of how big of a productivity booster SQL queries over multiple systems were, I had a similar experience with Iceberg that night. The whole package was just such an elegant solution to problems that had caused me and many in the Trino community failed deployments and late-night calls. While researching I learned about hidden partitioning, schema evolution, and most importantly, the open specification. I mean really, what could be better than Hive? □ I had heard the Trino users and maintainers talk about Iceberg replacing Hive but it just didn’t sink in for the first couple of months. While setting up that demo, I really started to understand what a game-changer Iceberg was. In true ADHD form, I crammed learning about Apache Iceberg well into the night before the broadcast with the creator of Iceberg, Ryan Blue. Let’s time-travel ( pun intended ) to the first Iceberg episode of the Trino Community Broadcast.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |