----

Note that this commit also streamlines obtaininig a relative
path for a directory, which previously could panic.

That way it's easier to assure that forbidden names are never used
as part of path components.

…ding paths.

This way, everyone using the stack with the purpose of
altering the working tree will run additional checks to prevent callers
from sneaking in forbidden paths.

Note that these checks don't run otherwise, so one has to be careful
to not forget to run these checks whenever needed.

That way, detailed information about the path-to-be is available not
only for evaluating attributes or excludes, but also for validating
path components (in this case, relevant for `.gitmodules`).

…ctNTFS`.

This also adds `gitoxide.core.protectWindows` as a way to enforce
additional restrictions that are usually only available on Windows.

Note that `core.protectNFS` is always enabled by default, just like
[it is in Git](git/git@9102f95).

It's probably best not to try to protect against violations of constraints
in this free-to-mutate data-structure and instead suggest to validate entry
paths before using them on disk (or use the `gix_worktree::Stack`).

…otectWindows` is enabled.

Note that trailing `.` are forbidden for some reason, but trailing ` ` (space) is forbidden
as it's just ignored when creating directories or files, allowing them to be clobbered
and merged silently.

This should not be incorporated into automated tests in its current
form. It is a proof of concept to generate repositories that attempt
to install real executables in directories where they may be run,
whereas test fixtures should completely limit all effects to testing
directories, even in the event of regressions or unexpected failures.

+ Refactor for brevity.

Because some sed implementations, at least the one on macOS, detect
invalid text in the current locale's encoding and error out. See:

https://stackoverflow.com/questions/19242275/re-error-illegal-byte-sequence-on-mac-os-x

This makes the script work on macOS.

Because committing the staged paths creates the necessary Git tree
objects irrespective of what directories exist or are otherwise
represented.

In addition to simplifying the proof-of-concept repository, this
also makes it so its entries are properly ordered in its Git object
database, so `git fsck` does not report errors about that, and
exits reporting success (though of course still warns about the
presence of `..` components).

Because the output of `git commit` should show that information.

The repo this script makes attempts to check out entries traversing
the default `$INDEX_ALLOCATION` directory stream of the `.git`
directory, whose stream name is documented to be `$I30`.

However, although I am able to access directories under this naming
scheme through other applications, the repositories this script
currently creates do not appear to trigger the bug in gitoxide. The
next step is to try specifying the stream type explicitly.

This seems more effective at revealing such a vulnerability.

I don't know why, since both should in principle work fine.

The repo the script makes contains a filename with slash characters
in it that, if not rejected, will install a pre-commit hook.

This requires xxd now, but it honors its /bin/sh hashbang line, no
longer assuming printf understands \xNN in a format string.

This is needed on some Git versions. It seems it was not needed
on older versions, even though their git-fsck detected the unusual
filenames when run. It is supported even on those older versions,
so the script should still run on them.

The repo this script makes contains a filename with a slash character
in it that, if not rejected, will create a file above the working tree.

This is a modification of make_traverse_dotgit_slashes.sh. Both
require some further revision, and since most of their content is
duplicated, it may be worthwhile to combine them to avoid that.

Co-authored-by: Eliah Kagan <[email protected]>

- assure `con` is checked for, and that it's not overzealous.
- reduce code duplication
- improve documentation about more obscure parts of the code,
  based on the description in [this commit](git/git@e7cb0b4)
- upper-case device names in comparisons as this is their canonical form, which also is more recognizable for people who
  are looking for them.
- make clear why there is asymmetry between COM and LPT numbers.
- Don't make a partial control-character check, but a complete one (i.e. *b < 32|0x20)
- Add more variants for stream type tests (as regression protection, the code doesn't really care)
- various clarifications in path-related tests on Windows

Co-authored-by: Eliah Kagan <[email protected]>

The [Naming Files, Paths, and Namespaces](https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file) article does not state that control characters or non-printable characters are in general forbidden in filenames. Instead, it says that it is okay to

> Use any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:

and then lists various things that are not allowed, where the one that is relevant to control characters is:

> Characters whose integer representations are in the range from 1 through 31, except for alternate data streams where these characters are allowed. *[...]*

No mention is made of 127 (0x7F).

On Windows 10, I used PowerShell 7 for this experiment, which I believe would also work in PowerShell 6, but not Windows PowerShell, which doesn't support `` `u ``. First, as a baseline, I checked what happened if I tried to create a file whose name contained a low-numbered control character:

```text
C:\Users\ek\source\repos\unusual-filenames [main]> echo hello > a`u{8}b
Out-File: The filename, directory name, or volume label syntax is incorrect. : 'C:\Users\ek\source\repos\unusual-filenames\b'
C:\Users\ek\source\repos\unusual-filenames [main]> echo hello > a`u{08}b
Out-File: The filename, directory name, or volume label syntax is incorrect. : 'C:\Users\ek\source\repos\unusual-filenames\b'
```

I created a file whose name contained the `DEL` character, and even a file whose entire name is that character:

```text
C:\Users\ek\source\repos\unusual-filenames [main]> echo hello > a`u{7F}b
C:\Users\ek\source\repos\unusual-filenames [main +1 ~0 -0 !]> echo goodbye > `u{7F}
C:\Users\ek\source\repos\unusual-filenames [main +2 ~0 -0 !]> ls

    Directory: C:\Users\ek\source\repos\unusual-filenames

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           5/20/2024  5:59 PM              9
-a---           5/20/2024  5:59 PM              7 ab
```

Thus this appears to work fine on Windows, and it seems fine that Git permits it:

```text
C:\Users\ek\source\repos\unusual-filenames [main +2 ~0 -0 !]> git status
On branch main

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        "a\177b"
        "\177"

nothing added to commit but untracked files present (use "git add" to track)
C:\Users\ek\source\repos\unusual-filenames [main +2 ~0 -0 !]> git add .
C:\Users\ek\source\repos\unusual-filenames [main +2 ~0 -0 ~]> git commit -m 'Initial commit'
[main (root-commit) 543ccd5] Initial commit
 2 files changed, 2 insertions(+)
 create mode 100644 "a\177b"
 create mode 100644 "\177"
```

Thus, gitoxide should probably permit it too.

To be sure, I also tried creating such a file in Python 3.12 on the same system, by calling the `touch` method on a `Path` object. That worked, too.

Co-authored-by: Eliah Kagan <[email protected]>

These are changes that do not significantly affect behavior but use
the set of tools that should be availalble in testing environments,
as well as refactorings that are useful to do not before really
making this usable as a fixture.

- Use bash shebang, enable pipefail.
- Don't require xxd.
- Don't create an extra temporary file.
- Shorten, simplify, and clarify some logic.

Keeping the changes to make_traverse_dotdot_slashes.sh, this folds
the make_traverse_dotgit_slsahes.sh logic into it, extracting the
twice-used parts (which are most of the script) into a function
that both call. Other changes:

- No longer use command-line arguments. There are two repositories
  that are currently useful to make in this way, and this calls the
  function for each of them.

- Change the style to mostly match that of other fixture scripts,
  including decreasing the indent from 4 to 2 and using the
  function keyword when defining functions.

- Shorten variable names in cases where doing so is unambiguous
  (but not otherwise).

- Eliminate the emit_payload function, since the new make_repo
  function now receives the content on standard input, which can
  be provided by whatever means is convenient (the current calls
  use a here string for the one-line file and a heredoc otherwise).

At least for now, this does not test the creation of multiple files
at a time outside of a repository, nor multi-step upward traversal
with many `../../..` components, since tests using such fixtures
would be complicated, and may or may not be warranted in the test
suite.

However, this combines substantial elements of the scripts that
create repositories with unexpected tree objects (e.g., `..` trees)
to make a make_traverse_trees.sh script that, when run, produces
repositories for testing that traverse:

- Upward with a `..` tree: `traverse_dotdot_tree`
- Downward with `.git` and `hooks` trees: `traverse_dotgit_trees`
- Similar but with an NTFS stream alias: `traverse_dotgit_stream`

This replaces the `make_traverse_dotdot_trees.sh` and
`make_traverse_ntfs_streams.sh` scripts with one script that takes
no command-line arguments and creates multiple repos by calling
a function. This is thus architecturally similar, broadly speaking,
to `make_traverse_literal_slashes.sh`, but that produces repos with
very strangely named blobs, rather than with strangly named trees.

The approach in make_traverse_literal_slases.sh works about equally
well for any top level file with strange characters. Before, it was
only generating such repositores where the filename has slashes,
causing traversal on all platforms. This has is generate two more
repositories, with backslashes instead of slashes. That script's
name is accordingly updated to make_traverse_literal_separators.sh.

Note that while such names with backslashes may be blocked on
multiple systems under various circumstances, they will only
perform traversal on Windows.

This is to see if anything should be done to more effectively
prevent paths containing `.git` (icase).

In conclusion, I think it's fine to keep allowing it as none
of the component-validations really kicks in on Linux if backslashes
are used as path separator. Thus, `.git` shouldn't be more special than
`..` for example.

The only way to fix this on Linux would be to either enable Windows protections,
or to disallow `\` as path seprator by default which seems too limitting.

Windows Users will naturally be protected as path-splitting will turn
these into components, with each of them checked as normal.

If we are dependent on symlinks, we should be sure that the probe
actually detects symlinks.

This is achieved by making filenames unique so they won't clash.

…917)

Previously, malicious trees could be used to create a index with
invalid names, which is one step closer to actually abusing it.

That way it's easy to determine if a component contains a windows device name

Otherwise it's possible to read or write to devices when interacting
with references of the 'right' name.

This behaviour can be controlled with the new `prohibit_windows_device_names` flag,
which is adjustable on the `Store` instance as field, and which now has to be
passed during instantiation as part of the new `store::init::Options` struct.

Co-authored-by: Eliah Kagan <[email protected]>

fix for CVE-2024-35186 and CVE-2024-35197

This started working with the upgradde of the `zip` crate.