Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging #1208
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for sunny-pastelito-5ecb04 canceled.
|
pankajkoti
changed the title
pankajkoti
commented
Sep 18, 2024
Co-authored-by: Pankaj Koti <[email protected]>
|
Thanks a lot for working on this, @pankajkoti , and solving the CI issue proactively while we were at the Airflow Summit! |
tatiana
approved these changes
Sep 18, 2024
There was a problem hiding this comment.
Thanks a lot, @pankajkoti , given our discussion with TP on #1201 (comment), this seems the best way to move forward.
dosubot
bot
added
size:S
2 tasks
ags-de
pushed a commit
to ags-de/astronomer-cosmos
that referenced
this pull request
Sep 24, 2024
…tifact to v4 and set min version for packaging (astronomer#1208) I am observing that our CI [jobs](https:/astronomer/astronomer-cosmos/actions/runs/10920015707/job/30308959677?pr=1207) are failing complaining that we're using deprecated version (v2) for `actions/upload-artifact` & `actions/download-artifact`. It is weird that the jobs directly are failed for using deprecating versions, but so is the case that's happening. Hence, to get the CI working I'm upgrading the versions to use v4. Additionally, we are observing failures for packaging version as discussed in astronomer#1201 (comment). Hence, also set the min packaging version to `22.0` as per the discussion in that thread closes: astronomer#1209 --------- Co-authored-by: Tatiana Al-Chueyr <[email protected]>
Merged
tatiana
added a commit
that referenced
this pull request
Oct 4, 2024
New Features * Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by @pankajkoti @pankajastro @tatiana in #1224 #1230. This is a first step in this journey and we would really appreciate feedback from the community. For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by @jonathanrainer @ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/. * Support using ``DatasetAlias`` and fix orphaning unreferenced dataset by @tatiana in #1217 #1240 Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling * Add GCP_CLOUD_RUN_JOB execution mode by @ags-de #1153 Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html Enhancements * Create single virtualenv when ``DbtVirtualenvBaseOperator`` has ``virtualenv_dir=None`` and ``is_virtualenv_dir_temporary=True`` by @kesompochy in #1200 * Consistently handle build and imports in ``cosmos/__init__.py`` by @tatiana in #1215 * Add enum constants to init for direct import by @fabiomx in #1184 Bug fixes * URL encode dataset names to support multibyte characters by @t0momi219 in #1198 * Fix invalid argument (``full_refresh``) passed to DbtTestAwsEksOperator (and others) by @johnhoran in #1175 * Fix ``printer_width`` arg type in ``DbtProfileConfigVars`` by @jessicaschueler in #1191 * Fix task owner fallback by @jmaicher in #1195 Docs * Add scarf to readme and docs for website analytics by @cmarteepants in #1221 * Add ``virtualenv_dir`` param to ``ExecutionConfig`` docs by @pankajkoti in #1173 * Give credits to @LennartKloppenburg in CHANGELOG.rst by @tatiana #1174 * Refactor docs for async mode execution by @pankajkoti in #1241 Others * Remove PR branch added for testing a change in CI in #1224 by @pankajkoti in #1233 * Fix CI wrt broken coverage upload artifact @pankajkoti in #1210 * Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging by @pankajkoti in #1208 * Resolve CI failures for Apache Airflow 2.7 jobs by @pankajkoti in #1182 * CI: Update GCP manifest file path based on new secret update by @pankajkoti in #1237 * Pre-commit hook updates in #1176 #1186, #1186, #1201, #1219, #1231
tatiana
added a commit
that referenced
this pull request
Oct 4, 2024
New Features * Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by @pankajkoti @pankajastro @tatiana in #1224 #1230. This is a first step in this journey and we would really appreciate feedback from the community. For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by @jonathanrainer @ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/. * Support using ``DatasetAlias`` and fix orphaning unreferenced dataset by @tatiana in #1217 #1240 Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling * Add GCP_CLOUD_RUN_JOB execution mode by @ags-de #1153 Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html Enhancements * Create single virtualenv when ``DbtVirtualenvBaseOperator`` has ``virtualenv_dir=None`` and ``is_virtualenv_dir_temporary=True`` by @kesompochy in #1200 * Consistently handle build and imports in ``cosmos/__init__.py`` by @tatiana in #1215 * Add enum constants to init for direct import by @fabiomx in #1184 Bug fixes * URL encode dataset names to support multibyte characters by @t0momi219 in #1198 * Fix invalid argument (``full_refresh``) passed to DbtTestAwsEksOperator (and others) by @johnhoran in #1175 * Fix ``printer_width`` arg type in ``DbtProfileConfigVars`` by @jessicaschueler in #1191 * Fix task owner fallback by @jmaicher in #1195 Docs * Add scarf to readme and docs for website analytics by @cmarteepants in #1221 * Add ``virtualenv_dir`` param to ``ExecutionConfig`` docs by @pankajkoti in #1173 * Give credits to @LennartKloppenburg in CHANGELOG.rst by @tatiana #1174 * Refactor docs for async mode execution by @pankajkoti in #1241 Others * Remove PR branch added for testing a change in CI in #1224 by @pankajkoti in #1233 * Fix CI wrt broken coverage upload artifact @pankajkoti in #1210 * Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging by @pankajkoti in #1208 * Resolve CI failures for Apache Airflow 2.7 jobs by @pankajkoti in #1182 * CI: Update GCP manifest file path based on new secret update by @pankajkoti in #1237 * Pre-commit hook updates in #1176 #1186, #1186, #1201, #1219, #1231
tatiana
added a commit
that referenced
this pull request
Oct 4, 2024
**New Features** * Support using ``DatasetAlias`` and fix orphaning unreferenced dataset by @tatiana in #1217 #1240 Documentation: https://astronomer.github.io/astronomer-cosmos/configuration/scheduling.html#data-aware-scheduling * Add GCP_CLOUD_RUN_JOB execution mode by @ags-de #1153 Learn more about it: https://astronomer.github.io/astronomer-cosmos/getting_started/gcp-cloud-run-job.html * Introduction of experimental support to run dbt BQ models using Airflow deferrable operators by @pankajkoti @pankajastro @tatiana in #1224 #1230. This is the first step in the journey of running dbt resources with native Airflow, and we would appreciate feedback from the community. For more information, check the documentation: https://astronomer.github.io/astronomer-cosmos/getting_started/execution-modes.html#airflow-async-experimental This work has been inspired by the talk "Airflow at Monzo: Evolving our data platform as the bank scales" by @jonathanrainer @ed-sparkes given at Airflow Summit 2023: https://airflowsummit.org/sessions/2023/airflow-at-monzo-evolving-our-data-platform-as-the-bank-scales/. **Enhancements** * Create single virtualenv when ``DbtVirtualenvBaseOperator`` has ``virtualenv_dir=None`` and ``is_virtualenv_dir_temporary=True`` by @kesompochy in #1200 * Consistently handle build and imports in ``cosmos/__init__.py`` by @tatiana in #1215 * Add enum constants to init for direct import by @fabiomx in #1184 **Bug fixes** * URL encode dataset names to support multibyte characters by @t0momi219 in #1198 * Fix invalid argument (``full_refresh``) passed to DbtTestAwsEksOperator (and others) by @johnhoran in #1175 * Fix ``printer_width`` arg type in ``DbtProfileConfigVars`` by @jessicaschueler in #1191 * Fix task owner fallback by @jmaicher in #1195 **Docs** * Add scarf to readme and docs for website analytics by @cmarteepants in #1221 * Add ``virtualenv_dir`` param to ``ExecutionConfig`` docs by @pankajkoti in #1173 * Give credits to @LennartKloppenburg in CHANGELOG.rst by @tatiana #1174 * Refactor docs for async mode execution by @pankajkoti in #1241 Others * Remove PR branch added for testing a change in CI in #1224 by @pankajkoti in #1233 * Fix CI wrt broken coverage upload artifact @pankajkoti in #1210 * Fix CI issues - Upgrade actions/upload-artifact & actions/download-artifact to v4 and set min version for packaging by @pankajkoti in #1208 * Resolve CI failures for Apache Airflow 2.7 jobs by @pankajkoti in #1182 * CI: Update GCP manifest file path based on new secret update by @pankajkoti in #1237 * Pre-commit hook updates in #1176 #1186, #1186, #1201, #1219, #1231 --------- Co-authored-by: Pankaj Koti <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I am observing that our CI jobs are failing complaining that we're using deprecated version (v2) for
actions/upload-artifact&actions/download-artifact. It is weird that the jobs directly are failed for using deprecating versions, but so is the case that's happening. Hence, to get the CI working I'm upgrading the versions to use v4.Additionally, we are observing failures for packaging version as discussed in #1201 (comment). Hence, also set the min packaging version to
22.0as per the discussion in that threadcloses: #1209