Dbt Labs will soon add a semantic layer in the modern data stack

Pennsylvania headquartered dbt Labs (formerly Fishtown Analytics), which offers an open-source data transformation tool called dbt and is known to be a significant part of the modern data stack, is gearing up to take things to the next level with a new semantic layer.

“On our move-the-ecosystem-forward initiatives, there are a bunch of irons in the fire. The biggest one is something we’re calling the semantic layer, which is a brand new way for Business Intelligence (BI) and analytics tools to access a single set of business concepts (metrics, entities, and more),” Tristan Handy, the CEO and founder of the dbt Labs, told VentureBeat.

Current architecture

The current architecture of the modern data stack sees information flow from warehouses and lakehouses to AI and BI tools for analytics projects. However, the problem is, organizations (especially big ones with complex structures) tend to have different tools for different analysis needs. This could lead to tools accessing different copies of data from warehouses and lakehouses.

To solve this challenge, a single open semantic layer of dbt code, where business metrics and concepts can be defined and made universally accessible, will sit in between. It will utilize any existing programming construct that dbt authors could express – refs, macros, sources – to offer the same version of the truth to all the BI and analytic tools, simplifying the whole process.

“This will solve, once and for all, one of the biggest problems in our space: the “single source of truth” problem. The reason we’ve been struggling to solve this as an industry for 30+ years is that it keeps being commercial/proprietary vendors that attempt to solve it,” Tristan added.

Possibilities

Though the CEO did not share the specifics of the semantic layer planned by dbt Labs, a blog post shared by him did indicate some directions the company may take.

“Imagine being able to ref a model instead of selecting from a physical table name inside your BI tool! With this one change you could get native environment support everywhere you work,” the post read. In addition to this, dbt community members and partners have also suggested leveraging the layer to define semantic entities and to build dynamic governance and privacy tooling, among other things. 

Ultimately, the company believes that this move will create a lot of whitespace to help enterprises drive innovation and build better products faster. A large portion of its $222 million series D round will also go toward this effort. The investment was led by Altimeter with multiple participants including Databricks, Snowflake, and Salesforce Ventures. 

Dbt Labs’ growth and competition

Dbt Labs claims that its transformation tool is currently used by over 9,000 companies. In the last year alone, the company’s customer base has tripled while revenue has grown by six times. 

“Dbt takes something that was once arcane, once required a tremendous amount of technical expertise, and turns it into something that anyone with an analytical background can participate in. It removes the barriers between data analysts, who are close to the business and understand its data needs, and data engineers, who are experts in data technology. Enabling both of these user groups to work together in the same tool to build production-grade data infrastructure is dbt’s superpower,” Tristan said.

While there are some companies that offer tools for transforming and preparing the data for analysis, including Datameer and Mozart Data, the company doesn’t see them as “meaningful challengers” at present.

“dbt has become such a standard in the industry — growing its install base by nine times over the past two years to 9000 companies!—that it truly doesn’t have meaningful challengers. This is actually not unusual in the open-source world: the most successful open source technologies end up becoming standards. Linux, Docker, Kubernetes, etc. However, as with many open-source companies, that (also) means that our biggest commercial competition is our own open-source product,” he said.