OpenLineage dbt integration

Wrapper script for automatic metadata collection from dbt

Features

Metadata

  • Model run lifecycle
  • Model inputs / outputs

Requirements

Right now, openlineage-dbt only supports bigquery, snowflake, spark and redshift dbt adapters.

Installation

$ pip3 install openlineage-dbt

To install from source, run:

$ pip install .

Configuration

HTTP Backend Environment Variables

openlineage-dbt uses the OpenLineage client to push data to the OpenLineage backend.

The OpenLineage client depends on environment variables:

  • OPENLINEAGE_URL - point to service which will consume OpenLineage events
  • OPENLINEAGE_API_KEY - set if consumer of OpenLineage events requires Bearer authentication key
  • OPENLINEAGE_NAMESPACE - set if you are using something other than the default namespace for job namespace.

Usage

To begin collecting dbt metadata with OpenLineage, replace dbt run with dbt-ol run.

Additional table and column level metadata will be available if catalog.json, a result of running dbt docs generate, will be found in the target directory.

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2022 contributors to the OpenLineage project

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.53.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openlineage from gravatar.com openlineage

Credits

Author: OpenLineage

Requires

Python >=3.7

Tags

openlineage

Report project as malware

Metadata

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.53.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openlineage from gravatar.com openlineage

Credits

Author: OpenLineage

Requires

Python >=3.7

Tags

openlineage

Report project as malware

Release files for openlineage-dbt 0.18.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for openlineage-dbt 0.18.0
File Size Uploaded
openlineage-dbt-0.18.0.tar.gz 3.9 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for openlineage-dbt 0.18.0
File Interpreter ABI Platform
openlineage_dbt-0.18.0-py3-none-any.whl 3.9 kB Python 3 none any Details

Total release size: 7.8 kB

Release files / openlineage-dbt-0.18.0.tar.gz

Download URL openlineage-dbt-0.18.0.tar.gz
Size 3.9 kB
Tags Source
SHA-256 checksum
How to use checksums 		de57c1d96244bd176dd8d7a7e88f32a33265297b8074959ae880d3be4d4dd31a
BLAKE2b-256 checksum
How to use checksums 		5652cb8c366b0974ea824f724fcd81fb9dcfd63e3910f320ecb131578b8d480e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/4.0.2 CPython/3.7.15

Release files / openlineage_dbt-0.18.0-py3-none-any.whl

Download URL openlineage_dbt-0.18.0-py3-none-any.whl
Size 3.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums 		c17da398aefd24445e38453baac9e5b7a1ff39b4f9909f4c71b8b86a8eb65980
BLAKE2b-256 checksum
How to use checksums 		06805a4909e314839db6b70850fe87622638799db6c9c62808bd794ce5bf2443
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/4.0.2 CPython/3.7.15

Release history Release notifications | RSS feed

1.53.0

1 release file

1.52.0

1 release file

1.51.0

1 release file

1.50.0

1 release file

1.49.0

1 release file

1.48.0

1 release file

1.47.1

1 release file

1.47.0

1 release file

1.46.0

1 release file

1.45.0

1 release file

1.44.1

1 release file

1.44.0

1 release file

1.43.0

1 release file

1.42.1

1 release file

1.41.0

1 release file

1.40.1

1 release file

1.40.0

1 release file

1.39.0

2 release files

1.38.0

2 release files

1.37.0

2 release files

1.36.0

2 release files

1.35.0

2 release files

1.34.0

2 release files

1.33.0

2 release files

1.32.1

2 release files

1.32.0

2 release files

1.31.0

2 release files

1.30.1

2 release files

1.30.0

2 release files

1.29.0

2 release files

1.28.0

2 release files

1.27.0

2 release files

1.26.0

2 release files

1.25.0

2 release files

1.24.2

2 release files

1.24.1

2 release files

1.24.0

2 release files

1.23.0

2 release files

1.22.0

2 release files

1.21.1

2 release files

1.21.0

2 release files

1.20.6

2 release files

1.20.5

2 release files

1.20.4

2 release files

Pre-release

1.20.3.dev12141

2 release files

1.19.0

2 release files

1.18.0

2 release files

1.17.1

2 release files

1.17.0

2 release files

1.16.0

2 release files

1.15.0

2 release files

1.14.0

2 release files

1.13.1

2 release files

1.12.0

2 release files

1.11.3

2 release files

1.11.2

2 release files

1.11.1

2 release files

1.10.2

2 release files

1.10.1

2 release files

1.10.0

2 release files

1.9.1

2 release files

1.9.0

2 release files

1.8.0

2 release files

1.7.0

2 release files

1.6.2

2 release files

1.6.1

2 release files

1.6.0

2 release files

1.5.0

2 release files

1.4.1

2 release files

1.3.1

2 release files

1.3.0

2 release files

1.2.2

2 release files

1.2.1

2 release files

1.2.0

2 release files

1.1.0

2 release files

1.0.0

2 release files

0.30.1

2 release files

0.30.0

2 release files

0.29.2

2 release files

0.28.0

2 release files

0.27.2

2 release files

0.27.1

2 release files

0.26.0

2 release files

0.25.0

2 release files

0.24.0

2 release files

0.23.0

2 release files

0.22.0

2 release files

0.21.1

2 release files

0.21.0

2 release files

0.20.6

2 release files

0.20.4

2 release files

0.19.2

2 release files

0.19.1

2 release files

This release

0.18.0 This release

2 release files

0.17.0

2 release files

0.16.1

2 release files

0.15.1

2 release files

0.14.1

2 release files

0.14.0

2 release files

0.13.1

2 release files

0.13.0

2 release files

0.12.0

2 release files

0.11.0

2 release files

0.10.0

2 release files

0.9.0

2 release files

0.8.2

2 release files

0.8.1

2 release files

0.7.1

2 release files

0.6.2

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.2

2 release files

0.5.1

2 release files

0.4.0

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

Pre-release

0.1.0rc5

2 release files

Pre-release

0.0.1rc8

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page