OpenLineage dbt integration

Wrapper script for automatic metadata collection from dbt

Features

Metadata

  • Model run lifecycle
  • Model inputs / outputs

Requirements

Right now, openlineage-dbt only supports bigquery, snowflake, spark and redshift dbt adapters.

Installation

$ pip3 install openlineage-dbt

To install from source, run:

$ pip install .

Configuration

HTTP Backend Environment Variables

openlineage-dbt uses the OpenLineage client to push data to the OpenLineage backend.

The OpenLineage client depends on environment variables:

  • OPENLINEAGE_URL - point to service which will consume OpenLineage events
  • OPENLINEAGE_API_KEY - set if consumer of OpenLineage events requires Bearer authentication key
  • OPENLINEAGE_NAMESPACE - set if you are using something other than the default namespace for job namespace.

Usage

To begin collecting dbt metadata with OpenLineage, replace dbt run with dbt-ol run.

Additional table and column level metadata will be available if catalog.json, a result of running dbt docs generate, will be found in the target directory.

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2023 contributors to the OpenLineage project

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.53.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openlineage from gravatar.com openlineage

Credits

Author: OpenLineage

Requires

Python >=3.7

Tags

openlineage

Report project as malware

Metadata

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.53.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for openlineage from gravatar.com openlineage

Credits

Author: OpenLineage

Requires

Python >=3.7

Tags

openlineage

Report project as malware

Release files for openlineage-dbt 0.20.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for openlineage-dbt 0.20.4
File Size Uploaded
openlineage-dbt-0.20.4.tar.gz 4.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for openlineage-dbt 0.20.4
File Interpreter ABI Platform
openlineage_dbt-0.20.4-py3-none-any.whl 4.1 kB Python 3 none any Details

Total release size: 8.2 kB

Release files / openlineage-dbt-0.20.4.tar.gz

Download URL openlineage-dbt-0.20.4.tar.gz
Size 4.1 kB
Tags Source
SHA-256 checksum
How to use checksums 		578a455817b7dff79758a455e5b0aae05a4a1d78060cbf0cde67abf8fb9dba0b
BLAKE2b-256 checksum
How to use checksums 		7ff79c7f0a0a81b5a3eb7b63a4ef6e388e9598ae4f0fd51b3c8f4f197d661265
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/4.0.2 CPython/3.7.16

Release files / openlineage_dbt-0.20.4-py3-none-any.whl

Download URL openlineage_dbt-0.20.4-py3-none-any.whl
Size 4.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums 		c4191a95bb736d23739e8bf40eaf229061e1de910c11abcb981c1bbbad865c23
BLAKE2b-256 checksum
How to use checksums 		b14588d494943e5a32f32b1973bb866cf8f26ad24db40196892b3e6a6ed0c1c9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing? 		No
Uploaded via twine/4.0.2 CPython/3.7.16

Release history Release notifications | RSS feed

1.53.0

1 release file

1.52.0

1 release file

1.51.0

1 release file

1.50.0

1 release file

1.49.0

1 release file

1.48.0

1 release file

1.47.1

1 release file

1.47.0

1 release file

1.46.0

1 release file

1.45.0

1 release file

1.44.1

1 release file

1.44.0

1 release file

1.43.0

1 release file

1.42.1

1 release file

1.41.0

1 release file

1.40.1

1 release file

1.40.0

1 release file

1.39.0

2 release files

1.38.0

2 release files

1.37.0

2 release files

1.36.0

2 release files

1.35.0

2 release files

1.34.0

2 release files

1.33.0

2 release files

1.32.1

2 release files

1.32.0

2 release files

1.31.0

2 release files

1.30.1

2 release files

1.30.0

2 release files

1.29.0

2 release files

1.28.0

2 release files

1.27.0

2 release files

1.26.0

2 release files

1.25.0

2 release files

1.24.2

2 release files

1.24.1

2 release files

1.24.0

2 release files

1.23.0

2 release files

1.22.0

2 release files

1.21.1

2 release files

1.21.0

2 release files

1.20.6

2 release files

1.20.5

2 release files

1.20.4

2 release files

Pre-release

1.20.3.dev12141

2 release files

1.19.0

2 release files

1.18.0

2 release files

1.17.1

2 release files

1.17.0

2 release files

1.16.0

2 release files

1.15.0

2 release files

1.14.0

2 release files

1.13.1

2 release files

1.12.0

2 release files

1.11.3

2 release files

1.11.2

2 release files

1.11.1

2 release files

1.10.2

2 release files

1.10.1

2 release files

1.10.0

2 release files

1.9.1

2 release files

1.9.0

2 release files

1.8.0

2 release files

1.7.0

2 release files

1.6.2

2 release files

1.6.1

2 release files

1.6.0

2 release files

1.5.0

2 release files

1.4.1

2 release files

1.3.1

2 release files

1.3.0

2 release files

1.2.2

2 release files

1.2.1

2 release files

1.2.0

2 release files

1.1.0

2 release files

1.0.0

2 release files

0.30.1

2 release files

0.30.0

2 release files

0.29.2

2 release files

0.28.0

2 release files

0.27.2

2 release files

0.27.1

2 release files

0.26.0

2 release files

0.25.0

2 release files

0.24.0

2 release files

0.23.0

2 release files

0.22.0

2 release files

0.21.1

2 release files

0.21.0

2 release files

0.20.6

2 release files

This release

0.20.4 This release

2 release files

0.19.2

2 release files

0.19.1

2 release files

0.18.0

2 release files

0.17.0

2 release files

0.16.1

2 release files

0.15.1

2 release files

0.14.1

2 release files

0.14.0

2 release files

0.13.1

2 release files

0.13.0

2 release files

0.12.0

2 release files

0.11.0

2 release files

0.10.0

2 release files

0.9.0

2 release files

0.8.2

2 release files

0.8.1

2 release files

0.7.1

2 release files

0.6.2

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.2

2 release files

0.5.1

2 release files

0.4.0

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.0

2 release files

Pre-release

0.1.0rc5

2 release files

Pre-release

0.0.1rc8

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page