the.com/unity catalog
one metadata layer to rule every table, model, and file so your data stops living in silos.
means a unified governance system for data and ai assets across an organization, tracking permissions, lineage, and metadata in one place instead of scattered per-tool catalogs.
from built by databricks, announced 2021 and generally available 2023, to solve the mess of every workspace and engine keeping its own separate permissions and metadata store.
open sourceddatabricks open sourced core in 2024
scopegoverns tables, files, models, and notebooks alike
lineagetracks column level lineage automatically, no extra tagging
three level namespacecatalog, schema, table replaces old two level model
for instance
databricks lakehouse — native governance layer across all workspaces since 2023 ga
delta sharing — lets unity catalog share live data across organizations without copying
open source uc — apache 2.0 release, june 2024, for non-databricks engines like spark and trino