Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop python3.8 support core and providers #42742

Conversation

jscheffl
Copy link
Contributor

@jscheffl jscheffl commented Oct 4, 2024

As Python 3.8 is getting out-of support, this PR removes support from Airflow providers and code.

See Python release schedule: https://peps.python.org/pep-0569/

Note: As I got a head-ache thinking about how to un-bundle the removal w/o 4-5PRs this is a combination with to commits from PR #42738 and #42739 - Maybe both is needed to make CI in Airflow v2-10 branch happy anyway.

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:CLI area:core-operators Operators, Sensors and hooks within Core Airflow area:dev-tools area:logging area:production-image Production image improvements and fixes area:providers area:serialization area:webserver Webserver related Issues provider:amazon-aws AWS/Amazon - related issues provider:cloudant provider:cncf-kubernetes Kubernetes provider related issues provider:common-io provider:openlineage AIP-53 labels Oct 4, 2024
@jscheffl jscheffl changed the title Feature/drop python3.8 support core and providers Drop python3.8 support core and providers Oct 4, 2024
Add newsfragment

(cherry picked from commit f0c14e7)
@jscheffl jscheffl force-pushed the feature/drop-python3.8-support-core-and-providers branch from 3aa9534 to 0578f53 Compare October 4, 2024 19:15
@potiuk
Copy link
Member

potiuk commented Oct 5, 2024

Note: As I got a head-ache thinking about how to un-bundle the removal w/o 4-5PRs this is a combination with to commits from PR #42738 and #42739 - Maybe both is needed to make CI in Airflow v2-10 branch happy anyway.

Ah yeah. This is indeed needed to be done in a single PR

@potiuk
Copy link
Member

potiuk commented Oct 5, 2024

Additionally ... This one should be done as a PR from apache repository - not from your fork. The reason is that "Build images" workflow uses "pull_request_target" workflow type and for that one - we need to make sure all the build scripts are taken from "target" branch (i.e. "main" branch in this case) - because otherwise anyone who modifies the build scripts in the PR will be able to run their code in "pull_request_target" workflow - with access to our secrets and write permissions.

So you will need to close that PR and re-open it by pushing your branch to the "apache" repository and open PR from there. When you will do it, the "pull_request_target" workflow will not be used at all - instead images will be built in the "Tests" workflow - and pushed to registry. This is because only PRS from the "main" repository can have write permissions (in this case "packages:write"). By default "pull_request" workflows have no "write" permissions and they would not be able to push the images to our registry - that's why for the "Fork" pull requests we need to build and push images in "pull_request_target" workflow - because it can have write permissions - but we have to make sure that this workflow uses only "approved" workflows and scripts from the "main" branch - rather than those that the PR is modifying.

What happens in "pull_request_target" is that we check-out the PR from the fork, and before we run any scripts in our workflows, we override workflows, actions and scripts/ci folder with the version from main - this way the "PR modified" code cannot modify workflows and scripts used during the "pull_request_target".

@jscheffl
Copy link
Contributor Author

jscheffl commented Oct 5, 2024

So you will need to close that PR and re-open it by pushing your branch to the "apache" repository

Oh, wow. New learning. Did not know that with my committer power I actually could do...

Continued in #42766

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:API Airflow's REST/HTTP API area:CLI area:core-operators Operators, Sensors and hooks within Core Airflow area:dev-tools area:logging area:production-image Production image improvements and fixes area:providers area:serialization area:webserver Webserver related Issues provider:amazon-aws AWS/Amazon - related issues provider:cloudant provider:cncf-kubernetes Kubernetes provider related issues provider:common-io provider:openlineage AIP-53
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants