Move tests_common from "dev" to top-level.

[potiuk]

Follow-up after #42505 fixing teething problem with tests_common.

Originally in #42505 common test code was moved to "dev" folder, but the "dev" folder is really dedicated to "build" scripts and the problem with moving "tests_common" to the folder was that the whole "dev" folder is replaced (for non-committer PRs) with the content from the target branch.

This is done for security reasons, because we can accidentally use any of the scripts from dev in the CI build scripts and we might not notice, which will open us to a security issue where a file in "dev" coming from PR could be accidentally executed during the "pull_request_target" workflow - which would expose our secrets and GitHub Package write permissions to a contributor coming from a fork.

This change moves the files, fixes pre-commit specification and docs, also fixes a number of "doc" issues detected by "ruff" in the tests_common folder as they were detected after the move. The tests_common folder is added to folders mounted when breeze is executed with local folders mounted (in order to avoid accidental mounting of randomly generated files to inside the breeze container).

All imports for the common tests were updated to reflect this move.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

This fixes the problem attempted to be fixed in #42971 by @gopidesupavan who experienced this issue in #42081

[potiuk]

Unfortunately it's huge and we can't do much about it :(

[jscheffl]

Some sanity question - a LOT of copy & paste... smells some decorators or generaotrs could remove a lot of duplication?

Crossing fingers you get it green.

For easier review... might be better to make some smaller incremental moves w/o any content changes, then git diff is faster to scroll through :-D

[potiuk]

For easier review... might be better to make some smaller incremental moves w/o any content changes, then git diff is faster to scroll through :-D

Good point. And really reasonable.

I tried to make it smaller, but all the necessary changes result from the move (i.e. in pretty much all files import will have to change. And then our pre-commit/formatting also intervenes. The comment from @ashb will likely limit the number of content changes. So in general we won't avoid content changes.

But - yes - I could split it to several commits though if you think it help to review it - I will attempt to do it, that will be rather easy. It's also a good exercise with some git commands to produce such commits :D so let me exercise my fingers.

I could also separate docstring fixes - dev has different rules than the "actual" code - including tests, but I figured that it's docstring only and there are not many of those - mostly empty lines and dots and changing to imperative statements in the first line. But I will see if that is easy - if not I would leave them together with the import changes.

[jscheffl]

I could also separate docstring fixes - dev has different rules than the "actual" code - including tests, but I figured that it's docstring only and there are not many of those - mostly empty lines and dots and changing to imperative statements in the first line. But I will see if that is easy - if not I would leave them together with the import changes.

That was not meant to be a blocker for the current PR but to have more friends for the next one :-D

[potiuk]

That was not meant to be a blocker for the current PR but to have more friends for the next one :-D

I split it REGARDLESS -> There are 3 commits now - move, docs, make it works. You can re-review now, I 🤞 it's going to pass tests now.

[potiuk]

It's actually pretty easy to split such PR if you know a few git tricks

git reset HEAD^ --hard
mv dev/tests_common . 
git add .
git commit --no-verify -m "'Move tests common without changes"
git cherry-pick <previous branch tip from git reflog>
git add tests_common
git commit --no-verify -m "Fix docstrings in tests_common"
git add . 
git commit # copy-paste previous commit message

[potiuk]

BTW. When you look at it now - it does not necessarily make it easier to review in GH UI, but git diff HEAD^ is actually pretty reviewable - even if long. (and the first two commits are easily reviewable).

[potiuk]

OK. Looks like we are getting there. There were few other places where there get_test_run had one more empty line.

BTW. That's the first time I started to like sphinx - this time it actually prevented us from accidental error that would have potentially caused silently slower-running system tests.

(well @jscheffl to notice it with review but sphinx also guarding it).

[potiuk]

Green, reviewed, merged.

[gopidesupavan]

cool, Thank you @potiuk 😄