Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove "from __future__" from airflow/providers __init__.py #43173

Merged
merged 2 commits into from
Oct 18, 2024

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Oct 18, 2024

Cleans-up airflow and providers __init__.py files in order to get providers import work again.

This is done by excluding the two __init__.py files from automated ruff isort rules adding from __future__ import annotations.

Also removed the __init__.py file from "providers" directory, it is not needed there, because "providers" is just a folder where we keep provider files, it's not a Python package.

That should finally get rid of the Intellij teething import problem that has been introduced in #42505.

There were earlier - unsuccesful - attempts to fix it in the #43116 and #43081 that followed #42951 - but the key is that Pycharm requires the namespace's extend_path to be first "real" line of code in the __init__.py to understand that the package is an "explicit" namespace package - and it conflicts with the requirement of "from future import annotations" to be the first line of Python code.

Also this PR fixes a few other teething problems with setup of tests that were introcuded in #42505 and #43802 "masked" by having __init__.py added in providers package:

  • common.sql interface pre-commit used wrong path to generated files
  • openlineage extractor test that should not expect "providers.tests."
    but "tests.
    " package
  • common_sql_api_stubs wrongly calculating generated path for
    stub-generated files
  • pytest_plugin expecting .asf.yml in "airflow" sources - even during
    compatibility tests with older version of airflow (where the
    .asf.yml is not present)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk
Copy link
Member Author

potiuk commented Oct 18, 2024

It nicely works for my IntelliJ:

Screenshot 2024-10-18 at 19 53 11

@potiuk
Copy link
Member Author

potiuk commented Oct 18, 2024

Ha.. Still some errors to fix but we get closer :) . Let me work on those :)

@potiuk potiuk force-pushed the fix-intellij-pycharm-imports-for-providers branch from 0cea19f to a732655 Compare October 18, 2024 20:10
@potiuk
Copy link
Member Author

potiuk commented Oct 18, 2024

I had to fix (I hope 🤞) a few other problems that wer "masked" by having "init.py" in providers folder.

@potiuk potiuk force-pushed the fix-intellij-pycharm-imports-for-providers branch from a732655 to 6ce161e Compare October 18, 2024 20:19
providers/__init__.py Outdated Show resolved Hide resolved
Cleans-up airflow and providers `__init__.py" files in order to
get providers import work again.

This is done by excluding the two `__init__.py` files from
automated ruff isort rules adding `from __future__ import annotations`.

Also removed the `__init__.py` file from "providers" directory,
it is not needed there, because "providers" is just a folder where
we keep provider files, it's not a Python package.

That should finally get rid of the Intellij teething import
problem that has been introduced in apache#42505.

There were earlier - unsuccesful - attempts to fix it in
the apache#43116 and apache#43081 that followed apache#42951 - but the key is that Pycharm
requires the namespace's extend_path to be first "real" line
of code in the `__init__.py` to understand that the package
is an "explicit" namespace package - and it conflicts with
the requirement of "from __future__ import annotations" to be
the first line of Python code.

Also this PR fixes a few other teething problems with setup of
tests that were introcuded in apache#42505 and #43802 "masked" by having
`__init__.py` added in providers package:

* common.sql interface pre-commit used wrong path to generated files
* openlineage extractor test that should not expect "providers.tests.*"
  but "tests.*" package
* common_sql_api_stubs wrongly calculating generated path for
  stub-generated files
* pytest_plugin expecting .asf.yml in "airflow" sources - even during
  compatibility tests with older version of airflow (where the
  .asf.yml is not present)
@kaxil kaxil force-pushed the fix-intellij-pycharm-imports-for-providers branch from f9c4020 to 6cd6564 Compare October 18, 2024 21:27
@kaxil kaxil force-pushed the fix-intellij-pycharm-imports-for-providers branch from 6cd6564 to 78e990c Compare October 18, 2024 21:53
@kaxil kaxil merged commit 6e9c536 into apache:main Oct 18, 2024
80 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants