refactor: replace distutils' copy_tree with shutil copytree

[gruzewski]

Fixes issue #963 (partially)

Proposed Changes

distutils.dir_util module is deprecated and will be removed from 3.12 (see this). Unfortunately, shutil.copytree is not a drop-in replacement as it returns destination directory and not the list of copied files, like in distutils.dir_util.copy_tree.

This PR adds a simple wrapper around copy_tree that computes copied files.

Testing

• Added basic unit tests
• Manually ran kapitan init --directory and got the same results. Cases covered: empty dir, existing dir without the structure, existing dir with the structure in place

NOTE: This change only works with Python >= 3.8 as dirs_exist_ok was added there. Otherwise, it will throw FileExistsError.

[MatteoVoges]

Hey @gruzewski ,
Thanks for noticing that the package is depricated or will be depricated soon. But instead of using a wrapper, wouldn't it be better to adapt our functionality and use the new function from the shutil library? There should be a way that we replace the list with the copied files which some os function like listdirs()

[gruzewski]

@MatteoVoges I did some research and couldn't develop a better solution. The challenge with os.listdir() is that it doesn't support recursive listing. And if someone initialises an existing directory (like an infrastructure repo with a .git folder and maybe README.md) with Kapitan inventory, we would need a different way to only list new files (like using a timestamp, but that feels very fragile). Otherwise, we would go back to this issue #306.

[MatteoVoges]

@gruzewski What if we traverse the template dir? Then we would get only the files, which get populated/ copied...
I think with your pr #966 and the tree output we don't even need information about which files existed already and which don't. We only need the current state of the directory

[sapslaj]

Now that Python 3.12 is out, Kapitan can't be run and throws a ModuleNotFoundError: No module named 'distutils'. PEP 632's migration advice even suggests reimplementing distutils.dir_util.create_tree.

@MatteoVoges what do you think about merging this as-is to enable Python 3.12 support? I realize it's been quite a while so if @gruzewski isn't interested in fixing up the branch I'd be willing to take that on.

[gruzewski]

I can pick it up next week :)

[gruzewski]

I have rebased the PR, and the tests are passing, so this is good to go.

@MatteoVoges, coming back to your point - we could technically do that, but there is a risk of reporting files that failed to copy. It wouldn't be a good first impression for users.

[MatteoVoges]

I have rebased the PR, and the tests are passing, so this is good to go.

Thanks, I will have a look at it soon!

@MatteoVoges, coming back to your point - we could technically do that, but there is a risk of reporting files that failed to copy. It wouldn't be a good first impression for users.

I don't see it that critical if we just dump the tree output. Can you explain once again why we need the different handling for new vs. old files?

[MatteoVoges]

Well done!

[ademariag]

Thank you @MatteoVoges and @gruzewski

[MatteoVoges]

@gruzewski Could you remove distutils from pyproject.toml and update the lockfile?