OpenLineage dbt integration
Wrapper script for automatic metadata collection from dbt
Features
Metadata
- Model run lifecycle
- Model inputs / outputs
Requirements
Right now, openlineage-dbt only supports bigquery, snowflake, spark and redshift dbt adapters.
Installation
$ pip3 install openlineage-dbt
To install from source, run:
$ pip install .
Configuration
HTTP Backend Environment Variables
openlineage-dbt uses the OpenLineage client to push data to the OpenLineage backend.
The OpenLineage client depends on environment variables:
OPENLINEAGE_URL- point to service which will consume OpenLineage eventsOPENLINEAGE_API_KEY- set if consumer of OpenLineage events requiresBearerauthentication keyOPENLINEAGE_NAMESPACE- set if you are using something other than thedefaultnamespace for job namespace.
Usage
To begin collecting dbt metadata with OpenLineage, replace dbt run with dbt-ol run.
Additional table and column level metadata will be available if catalog.json, a result of running dbt docs generate, will be found in the target directory.
SPDX-License-Identifier: Apache-2.0
Copyright 2018-2023 contributors to the OpenLineage project
1 maintainer
Metadata
1 maintainer
Release files for openlineage-dbt 0.20.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| openlineage-dbt-0.20.6.tar.gz | 4.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| openlineage_dbt-0.20.6-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 8.2 kB
Release files / openlineage-dbt-0.20.6.tar.gz
| Download URL | openlineage-dbt-0.20.6.tar.gz |
|---|---|
| Size | 4.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
159f0b226c7c4f64ccb6173e2efd687ae3eae7f963da3399beae765e4359170c
|
|
BLAKE2b-256 checksum How to use checksums |
01cbbcad71d9d61e40a2b7dd9d7d9467c78b3559ecfb673426b662e8d54fe6bd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.16
|
Release files / openlineage_dbt-0.20.6-py3-none-any.whl
| Download URL | openlineage_dbt-0.20.6-py3-none-any.whl |
|---|---|
| Size | 4.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
07dcfa0b19a8c03d50a3e1d875c29826ff2f4d880f551e0443af43549c6e8a3f
|
|
BLAKE2b-256 checksum How to use checksums |
d0d5f076ecd9f0111871fea6d93403a924c1b7699ad73d4bf012eb29b1d9187c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.16
|
