dbt-dremio 1.7.0

dbt-dremio v1.7.0 April 29, 2024

What's Changed

• Ensure api call to create folders does not get called when creating a table by @ravjotbrar in #219
• Optimize REST API workload by using Session and slowing down polling loop by @chulucninh09 in #220
• fix for DX-87286, added retries by @mxmarg in #221
• Upgrade to dbt-core 1.7 by @ravjotbrar in #223

1.7 dbt-core upgrades

• #8307 Allow source freshness to be evaluated from table metadata
• #8307 Catalog fetch performance improvements
• #8307 Migrate date_spine macros

New Contributors

• @chulucninh09 made their first contribution in #220

Full Changelog: v1.5.1...v1.7.0

dbt-dremio 1.5.1

dbt-dremio v1.5.1 Jan 31, 2024

Changes

• #199 Populate PyPI's long_description with contents of README.md
• #167 Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
• #211 Make fetching model data false by default. This improves performance where job results do not need to be populated.
• #203 Allow for dots in schema name, by surrounding in single and double quotes.
• #193 Fixes Reflection bug: The name argument to ref() must be a string, got <class 'jinja2.runtime.Undefined'>
• Versioning Added optional parameter v to the ref macro

dbt-dremio 1.5.0

dbt-dremio 1.5.0 - release June 22, 2023

Under the Hood

• #179 Upgrade to support dbt-core v1.5.0.
  • Add support for Python 3.11.
  • Add support for relevant Tests:
    • caching
    • hooks
    • simple_copy
• Add support for model contracts (Stub the feature to let users know the feature is not supported).

dbt-dremio 1.4.5

dbt-dremio 1.4.5 - release March 23, 2023

Features

Fixes

• #142 Ensure ssl verification is enabled in all api calls. Also added an option called verify_ssl so it can be disabled in necessary circumstances.

Under the Hood

• #64 Add BaseArrayTests and throw exceptions for unsupported Array Macros.
• #117 Add support for Query Comment Tests and Python 3.11
• #134 Add dremio:exact_search_enabled variable that if set to true, replaces usage of ilike with a basic equality in dremio__list_relations_without_caching when reflections are not enabled.
• #117 Add Base Current Timestamps Tests
• #117 Replace deprecated dbt-core exceptions
• #117 Add support for changing relation type test

Dependency

• Upgrade dbt-core to 1.4.5.
• Upgrade dbt-tests-adapter to 1.4.5.

dbt-dremio 1.3.2

dbt-dremio 1.3.2 - February 8, 2023

Features

Fixes

• Override dbt-core default__type_string() macro to use Dremio Supported VARCHAR instead of the default string. (#80)

• Change _populate_job_results() to have an optional row_limit argument with default set to 100 (Dremio's default). (#61)

• Implement pagination in _populate_job_results() (#61)

• Fix error handling so the error reported when a job fails is the actual error from Dremio. (#69)

• Override dbt-core default__rename_relation() macro to use Dremio Supported CTAS and DROP instead of ALTER TABLE and RENAME to. (#44)

Under the Hood

• #32 Add pre-commit hooks (most significant being black, flake8, and bandit)

• Implement new Incremental materialization logic from dbt 1.3 as part of the upgrade to support dbt-core v1.3.0. (#44, #16)

Dependency

• Upgrade dbt-core to 1.3.2.

• Upgrade dbt-tests-adapter to 1.3.2.

dbt-dremio 1.3.0

dbt-dremio - November 9, 2022

Features

• New profile configuration aliases to better align with Dremio concepts (53):
  • dremio_space = database
  • dremio_space_folder = schema
  • object_storage_space = datalake
  • object_storage_path = root_folder
• dbt-core 1.2 updates:
  • grants (42)
• dbt-core 1.3 updates:
  • confirmed no breaking changes for new incremental materializations append default (44)

Under the Hood

• dbt-core 1.2 updates:
  • connection retries (40)
  • cross-database macros (41)
    • confirmed no work needed
• dbt-core 1.3 updates:
  • BaseDocsGenerate and BaseDocsGenReferences updates (43)
  • new functional adapter tests (45)
• Miscellaneous bug fixes

Final ODBC Release

Final release using the Dremio Arrow Flight ODBC driver.

Requirements for using ODBC will be dropped in subsequent releases.

Release is designated as beta due to lack of full test coverage. However, functionality and performance is same as v1.0.6.5.

Dependencies:

• dbt-core == 1.0.6
• pyodbc >= 4.0.27
• Dremio Software >= 21.0.0
• dremio.iceberg.enabled, dremio.iceberg.ctas.enabled and dremio.execution.support_unlimited_splits enabled
• Dremio's Arrow Flight SQL ODBC driver

Adapter works with both Dremio Software and Dremio Cloud.

When creating projects using Dremio Cloud, add the following to the bottom of the dbt_proctted.yml file:

vars:
  dremio:reflections_enabled: false

Note that the above may be required in projects connecting to Dremio Software.