Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cosmos/propagate_logs Airflow config support for disabling log pr… #648

Merged
merged 5 commits into from
Nov 9, 2023

Conversation

agreenburg
Copy link
Contributor

@agreenburg agreenburg commented Nov 3, 2023

…opagation if desired

Description

Add Airflow config check for cosmos/propagate_logs to allow override of default propagation behavior.
Expose entry-point so that Airflow can theoretically detect configuration default.

Related Issue(s)

closes #639

Breaking Change?

This is backward-compatible as it falls back to default behavior if the cosmos section or propagate_logs option don't exist.

Checklist

  • I have made corresponding changes to the documentation (if required)
  • I have added tests that prove my fix is effective or that my feature works

@agreenburg agreenburg requested a review from a team as a code owner November 3, 2023 21:33
@agreenburg agreenburg requested a review from a team November 3, 2023 21:33
Copy link

netlify bot commented Nov 3, 2023

👷 Deploy Preview for amazing-pothos-a3bca0 processing.

Name Link
🔨 Latest commit eb5e64d
🔍 Latest deploy log https://app.netlify.com/sites/amazing-pothos-a3bca0/deploys/654ceac79f05e60008e86e25

Copy link

codecov bot commented Nov 6, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9cd46d2) 92.67% compared to head (eb5e64d) 92.70%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #648      +/-   ##
==========================================
+ Coverage   92.67%   92.70%   +0.02%     
==========================================
  Files          54       54              
  Lines        2212     2220       +8     
==========================================
+ Hits         2050     2058       +8     
  Misses        162      162              
Files Coverage Δ
cosmos/__init__.py 100.00% <100.00%> (ø)
cosmos/log.py 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tatiana tatiana added this to the 1.3.0 milestone Nov 7, 2023
Copy link
Collaborator

@tatiana tatiana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @agreenburg for making this improvement to Cosmos logging.

Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/

Could you give more details on the following?
"Expose entry-point so that Airflow can theoretically detect configuration default."

I'm happy to merge this change once I understand this a little bit more :)

@agreenburg
Copy link
Contributor Author

Thank you very much @agreenburg for making this improvement to Cosmos logging.

Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/

Could you give more details on the following? "Expose entry-point so that Airflow can theoretically detect configuration default."

I'm happy to merge this change once I understand this a little bit more :)

HI @tatiana ! The entry-point stuff is based on the guidance here: https://airflow.apache.org/docs/apache-airflow-providers/howto/create-custom-providers.html#how-to-create-a-provider

I have not thoroughly tested that airflow is detecting it properly yet though.

@tatiana
Copy link
Collaborator

tatiana commented Nov 8, 2023

Thank you very much @agreenburg for making this improvement to Cosmos logging.
Initially I hadn't thought about having Cosmos as a provider, but it can make sense. It fits Airflow's definition of a provider: https://airflow.apache.org/docs/apache-airflow-providers/
Could you give more details on the following? "Expose entry-point so that Airflow can theoretically detect configuration default."
I'm happy to merge this change once I understand this a little bit more :)

HI @tatiana ! The entry-point stuff is based on the guidance here: https://airflow.apache.org/docs/apache-airflow-providers/howto/create-custom-providers.html#how-to-create-a-provider

I have not thoroughly tested that airflow is detecting it properly yet though.

Ah, I see - so the motivation is to display the config in the Airflow UI (e.g. http://localhost:8080/configuration). If we didn't add this provider logic (both entry point and get_provider_info), you'd still be able to set this value via an environment variable or Airflow config file, but it would not display in the UI. Makes sense!

@tatiana tatiana added the area:logging Related to logging, like log levels, log formats, error logging, etc label Nov 9, 2023
@tatiana tatiana merged commit 6a2c866 into astronomer:main Nov 9, 2023
42 checks passed
tatiana pushed a commit that referenced this pull request Nov 9, 2023
…opagation if desired (#648)

Add Airflow config check for cosmos/propagate_logs to allow override of
default propagation behavior.
Expose entry-point so that Airflow can theoretically detect
configuration default.

Closes #639 

## Breaking Change?

This is backward-compatible as it falls back to default behavior if the
`cosmos` section or `propagate_logs` option don't exist.

## Checklist

- [X] I have made corresponding changes to the documentation (if
required)
- [X] I have added tests that prove my fix is effective or that my
feature works

---------

Co-authored-by: Andrew Greenburg <[email protected]>
tatiana added a commit that referenced this pull request Dec 7, 2023
Features

* Add ProfileMapping for Vertica by @perttus in #540 and #688
* Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649
* Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728
* Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648
* Add operator_args full_refresh as a templated field by @joppevos in #623
* Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735

Enhancements

* Make Pydantic an optional dependency by @pixie79 in #736
* Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730
* Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731
* Add aws_session_token for Athena mapping by @benjamin-awd in #663

Others

* Replace flake8 for Ruff by @joppevos in #743
* Reduce code complexity to 8 by @joppevos in #738
* Update conflict matrix between Airflow and dbt versions by @tatiana in #731
* Speed up integration tests by @jbandoro in #732
@tatiana tatiana mentioned this pull request Dec 7, 2023
tatiana added a commit that referenced this pull request Dec 7, 2023
Features

* Add ProfileMapping for Vertica by @perttus in #540 and #688
* Add support for Snowflake encrypted private key environment variable by @DanMawdsleyBA in #649
* Add support to select using (some) graph operators when using LoadMode.CUSTOM and LoadMode.DBT_MANIFEST by @tatiana in #728
* Add cosmos/propagate_logs Airflow config support for disabling log pr… by @agreenburg in #648
* Add operator_args full_refresh as a templated field by @joppevos in #623
* Expose environment variables and dbt variables in ProjectConfig by @jbandoro in #735

Enhancements

* Make Pydantic an optional dependency by @pixie79 in #736
* Create a symbolic link to dbt_packages when dbt_deps is False when using LoadMode.DBT_LS by @DanMawdsleyBA in #730
* Support no profile_config for ExecutionMode.KUBERNETES and ExecutionMode.DOCKER by @MrBones757 and @tatiana in #681 and #731
* Add aws_session_token for Athena mapping by @benjamin-awd in #663

Others

* Replace flake8 for Ruff by @joppevos in #743
* Reduce code complexity to 8 by @joppevos in #738
* Update conflict matrix between Airflow and dbt versions by @tatiana in #731
* Speed up integration tests by @jbandoro in #732
jbandoro pushed a commit that referenced this pull request Dec 7, 2023
**Features**

* Add `ProfileMapping` for Snowflake encrypted private key path by
@ivanstillfront in #608
* Add support for Snowflake encrypted private key environment variable
by @DanMawdsleyBA in #649
* Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in
#616
* Add support to select using (some) graph operators when using
`LoadMode.CUSTOM` and `LoadMode.DBT_MANIFEST` by @tatiana in #728
* Add cosmos/propagate_logs Airflow config support for disabling log
propagation by @agreenburg in #648
* Add operator_args ``full_refresh`` as a templated field by @joppevos
in #623
* Expose environment variables and dbt variables in ``ProjectConfig`` by
@jbandoro in #735

**Enhancements**

* Make Pydantic an optional dependency by @pixie79 in #736
* Create a symbolic link to `dbt_packages` when `dbt_deps` is False when
using `LoadMode.DBT_LS` by @DanMawdsleyBA in #730
* Support no `profile_config` for `ExecutionMode.KUBERNETES` and
`ExecutionMode.DOCKER` by @MrBones757 and @tatiana in #681 and #731
* Add `aws_session_token` for Athena mapping by @benjamin-awd in #663

**Others**

* Replace flake8 for Ruff by @joppevos in #743
* Reduce code complexity to 8 by @joppevos in #738
* Update conflict matrix between Airflow and dbt versions by @tatiana in
#731
* Speed up integration tests by @jbandoro in #732
@tatiana tatiana mentioned this pull request Jan 4, 2024
tatiana added a commit that referenced this pull request Jan 4, 2024
**Features**

* Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in #733
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)).
* Add support to select using (some) graph operators when using
``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in #728
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude))
* Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in
#755,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)).
* Add ``ProfileMapping`` for Vertica by @perttus in #540, #688 and #741,
as
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)).
* Add ``ProfileMapping`` for Snowflake encrypted private key path by
@ivanstillfront in #608, as ([documentation](
https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)).
* Add support for Snowflake encrypted private key environment variable
by @DanMawdsleyBA in #649
* Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro
in #616,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)).
* Add cosmos/propagate_logs Airflow config support for disabling log
propagation by @agreenburg in #648,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)).
* Add operator_args ``full_refresh`` as a templated field by @joppevos
in #623
* Expose environment variables and dbt variables in ``ProjectConfig`` by
@jbandoro in #735
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)).
* Support disabling event tracking when using Cosmos profile mapping by
@jbandoro in #768,
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)).

**Enhancements**

* Make Pydantic an optional dependency by @pixie79 in #736
* Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False
when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in #730
* Add ``aws_session_token`` for Athena mapping by @benjamin-awd in #663
* Retrieve temporary credentials from ``conn_id`` for Athena by @octiva
in #758
* Extend ``DbtDocsLocalOperator`` with static flag by @joppevos  in #759

**Bug fixes**

* Remove Pydantic upper version restriction so Cosmos can be used with
Airflow 2.8 by @jlaneve in #772

**Others**

* Replace flake8 for Ruff by @joppevos in #743
* Reduce code complexity to 8 by @joppevos in #738
* Speed up integration tests by @jbandoro in #732
* Fix README quickstart link in by @RNHTTR in #776
* Add package location to work with hatchling 1.19.0 by @jbandoro in
#761
* Fix type check error in ``DbtKubernetesBaseOperator.build_env_args``
by @jbandoro in #766
* Improve ``DBT_MANIFEST`` documentation by @dwreeves in #757
* Update conflict matrix between Airflow and dbt versions by @tatiana in
#731 and #779
* pre-commit updates in #775, #770, #762
ykuc pushed a commit to ykuc/astronomer-cosmos that referenced this pull request Jan 11, 2024
**Features**

* Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)).
* Add support to select using (some) graph operators when using
``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude))
* Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in
astronomer#755,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)).
* Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741,
as
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)).
* Add ``ProfileMapping`` for Snowflake encrypted private key path by
@ivanstillfront in astronomer#608, as ([documentation](
https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)).
* Add support for Snowflake encrypted private key environment variable
by @DanMawdsleyBA in astronomer#649
* Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro
in astronomer#616,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)).
* Add cosmos/propagate_logs Airflow config support for disabling log
propagation by @agreenburg in astronomer#648,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)).
* Add operator_args ``full_refresh`` as a templated field by @joppevos
in astronomer#623
* Expose environment variables and dbt variables in ``ProjectConfig`` by
@jbandoro in astronomer#735
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)).
* Support disabling event tracking when using Cosmos profile mapping by
@jbandoro in astronomer#768,
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)).

**Enhancements**

* Make Pydantic an optional dependency by @pixie79 in astronomer#736
* Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False
when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730
* Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663
* Retrieve temporary credentials from ``conn_id`` for Athena by @octiva
in astronomer#758
* Extend ``DbtDocsLocalOperator`` with static flag by @joppevos  in astronomer#759

**Bug fixes**

* Remove Pydantic upper version restriction so Cosmos can be used with
Airflow 2.8 by @jlaneve in astronomer#772

**Others**

* Replace flake8 for Ruff by @joppevos in astronomer#743
* Reduce code complexity to 8 by @joppevos in astronomer#738
* Speed up integration tests by @jbandoro in astronomer#732
* Fix README quickstart link in by @RNHTTR in astronomer#776
* Add package location to work with hatchling 1.19.0 by @jbandoro in
astronomer#761
* Fix type check error in ``DbtKubernetesBaseOperator.build_env_args``
by @jbandoro in astronomer#766
* Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757
* Update conflict matrix between Airflow and dbt versions by @tatiana in
astronomer#731 and astronomer#779
* pre-commit updates in astronomer#775, astronomer#770, astronomer#762
@zy-wiser
Copy link

zy-wiser commented Feb 5, 2024

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

@agreenburg
Copy link
Contributor Author

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

@zy-wiser
Copy link

zy-wiser commented Feb 5, 2024

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

Got it thanks.

I used the following command to check airflow config and it looks good to me (can get False as return)

airflow config get-value cosmos propagate_logs
False

But actual log still shows doubled

[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.522+0000�[0m] {�[34mlocal.py:�[0m253} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.523+0000�[0m] {�[34mlocal.py:�[0m254} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Outlets: []�[0m

@agreenburg
Copy link
Contributor Author

@agreenburg Have you tested if the config resolved the double logging issue? I found airflow is not taking this as provider so the config doesn't really works for me

I was able to test it with both an environment variable and by adding to the airflow.cfg and it was suppressing the propagation as expected. Not sure what's missing to get it to work as a provider.

Got it thanks.

I used the following command to check airflow config and it looks good to me (can get False as return)

airflow config get-value cosmos propagate_logs
False

But actual log still shows doubled

[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.522+0000�[0m] {�[34mlocal.py:�[0m253} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2024-02-05, 20:39:07 UTC] {log.py:232} WARNING - [�[34m2024-02-05T20:39:07.523+0000�[0m] {�[34mlocal.py:�[0m254} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Outlets: []�[0m

Those are two separate log messages. One with the Inlets[] and the other with the Outlets[]. Before the change I was seeing logs like this:

[2023-12-04, 19:08:25 UTC] {log.py:232} WARNING - [�[34m2023-12-04T19:08:25.566+0000�[0m] {�[34mlocal.py:�[0m244} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m
[2023-12-04, 19:08:25 UTC] {local.py:244} INFO - Inlets: []

@zy-wiser
Copy link

zy-wiser commented Feb 5, 2024

@agreenburg Thanks for the explanation. I guess I misunderstood the outcome with the setting.
Was thinking I can limit it to just showing

[2023-12-04, 19:08:25 UTC] {local.py:244} INFO - Inlets: []

and completely get rid of the cosmos log

[2023-12-04, 19:08:25 UTC] {log.py:232} WARNING - [�[34m2023-12-04T19:08:25.566+0000�[0m] {�[34mlocal.py:�[0m244} INFO�[0m - �[35m(astronomer-cosmos)�[0m - Inlets: []�[0m

as it's not that readable and format well...

@agreenburg
Copy link
Contributor Author

@zy-wiser yes, unfortunately that would require a bigger change. I guess the custom logging wasn't designed with the celery executor in mind.

arojasb3 pushed a commit to arojasb3/astronomer-cosmos that referenced this pull request Jul 14, 2024
…opagation if desired (astronomer#648)

Add Airflow config check for cosmos/propagate_logs to allow override of
default propagation behavior.
Expose entry-point so that Airflow can theoretically detect
configuration default.

Closes astronomer#639 

## Breaking Change?

This is backward-compatible as it falls back to default behavior if the
`cosmos` section or `propagate_logs` option don't exist.

## Checklist

- [X] I have made corresponding changes to the documentation (if
required)
- [X] I have added tests that prove my fix is effective or that my
feature works

---------

Co-authored-by: Andrew Greenburg <[email protected]>
arojasb3 pushed a commit to arojasb3/astronomer-cosmos that referenced this pull request Jul 14, 2024
**Features**

* Add `ProfileMapping` for Snowflake encrypted private key path by
@ivanstillfront in astronomer#608
* Add support for Snowflake encrypted private key environment variable
by @DanMawdsleyBA in astronomer#649
* Add `DbtDocsGCSOperator` for uploading dbt docs to GCS by @jbandoro in
astronomer#616
* Add support to select using (some) graph operators when using
`LoadMode.CUSTOM` and `LoadMode.DBT_MANIFEST` by @tatiana in astronomer#728
* Add cosmos/propagate_logs Airflow config support for disabling log
propagation by @agreenburg in astronomer#648
* Add operator_args ``full_refresh`` as a templated field by @joppevos
in astronomer#623
* Expose environment variables and dbt variables in ``ProjectConfig`` by
@jbandoro in astronomer#735

**Enhancements**

* Make Pydantic an optional dependency by @pixie79 in astronomer#736
* Create a symbolic link to `dbt_packages` when `dbt_deps` is False when
using `LoadMode.DBT_LS` by @DanMawdsleyBA in astronomer#730
* Support no `profile_config` for `ExecutionMode.KUBERNETES` and
`ExecutionMode.DOCKER` by @MrBones757 and @tatiana in astronomer#681 and astronomer#731
* Add `aws_session_token` for Athena mapping by @benjamin-awd in astronomer#663

**Others**

* Replace flake8 for Ruff by @joppevos in astronomer#743
* Reduce code complexity to 8 by @joppevos in astronomer#738
* Update conflict matrix between Airflow and dbt versions by @tatiana in
astronomer#731
* Speed up integration tests by @jbandoro in astronomer#732
arojasb3 pushed a commit to arojasb3/astronomer-cosmos that referenced this pull request Jul 14, 2024
**Features**

* Add new parsing method ``LoadMode.DBT_LS_FILE`` by @woogakoki in astronomer#733
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/parsing-methods.html#dbt-ls-file)).
* Add support to select using (some) graph operators when using
``LoadMode.CUSTOM`` and ``LoadMode.DBT_MANIFEST`` by @tatiana in astronomer#728
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/selecting-excluding.html#using-select-and-exclude))
* Add support for dbt ``selector`` arg for DAG parsing by @jbandoro in
astronomer#755,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/render-config.html#render-config)).
* Add ``ProfileMapping`` for Vertica by @perttus in astronomer#540, astronomer#688 and astronomer#741,
as
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/VerticaUserPassword.html)).
* Add ``ProfileMapping`` for Snowflake encrypted private key path by
@ivanstillfront in astronomer#608, as ([documentation](
https://astronomer.github.io/astronomer-cosmos/profiles/SnowflakeEncryptedPrivateKeyFilePem.html)).
* Add support for Snowflake encrypted private key environment variable
by @DanMawdsleyBA in astronomer#649
* Add ``DbtDocsGCSOperator`` for uploading dbt docs to GCS by @jbandoro
in astronomer#616,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/generating-docs.html#upload-to-gcs)).
* Add cosmos/propagate_logs Airflow config support for disabling log
propagation by @agreenburg in astronomer#648,
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/logging.html)).
* Add operator_args ``full_refresh`` as a templated field by @joppevos
in astronomer#623
* Expose environment variables and dbt variables in ``ProjectConfig`` by
@jbandoro in astronomer#735
([documentation](https://astronomer.github.io/astronomer-cosmos/configuration/project-config.html#project-config-example)).
* Support disabling event tracking when using Cosmos profile mapping by
@jbandoro in astronomer#768,
([documentation](https://astronomer.github.io/astronomer-cosmos/profiles/index.html#disabling-dbt-event-tracking)).

**Enhancements**

* Make Pydantic an optional dependency by @pixie79 in astronomer#736
* Create a symbolic link to ``dbt_packages`` when ``dbt_deps`` is False
when using ``LoadMode.DBT_LS`` by @DanMawdsleyBA in astronomer#730
* Add ``aws_session_token`` for Athena mapping by @benjamin-awd in astronomer#663
* Retrieve temporary credentials from ``conn_id`` for Athena by @octiva
in astronomer#758
* Extend ``DbtDocsLocalOperator`` with static flag by @joppevos  in astronomer#759

**Bug fixes**

* Remove Pydantic upper version restriction so Cosmos can be used with
Airflow 2.8 by @jlaneve in astronomer#772

**Others**

* Replace flake8 for Ruff by @joppevos in astronomer#743
* Reduce code complexity to 8 by @joppevos in astronomer#738
* Speed up integration tests by @jbandoro in astronomer#732
* Fix README quickstart link in by @RNHTTR in astronomer#776
* Add package location to work with hatchling 1.19.0 by @jbandoro in
astronomer#761
* Fix type check error in ``DbtKubernetesBaseOperator.build_env_args``
by @jbandoro in astronomer#766
* Improve ``DBT_MANIFEST`` documentation by @dwreeves in astronomer#757
* Update conflict matrix between Airflow and dbt versions by @tatiana in
astronomer#731 and astronomer#779
* pre-commit updates in astronomer#775, astronomer#770, astronomer#762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:logging Related to logging, like log levels, log formats, error logging, etc
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Annoying duplicate celery logging in Airflow task logs when running in celery worker
3 participants