EDITOR=sed related error when running make_rebase_i_repo.sh

[calam1]

These seem to be different errors though, maybe the shell on MacOS has trouble?

MacOS

2022-04-14T09:48:59.9196440Z stderr: /Users/runner/work/gitoxide/gitoxide/git-repository/tests/fixtures/make_rebase_i_repo.sh: line 33: [: 2 35 1: integer expression expected
2022-04-14T09:48:59.9196950Z fatal: --preserve-merges was replaced by --rebase-merges
2022-04-14T09:48:59.9197450Z ', tests/tools/src/lib.rs:115:9
2022-04-14T09:48:59.9197760Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2022-04-14T09:48:59.9197950Z 
2022-04-14T09:48:59.9197960Z 
2022-04-14T09:48:59.9198020Z failures:

Windows

2022-04-14T09:40:43.0699609Z stderr: which: no gsed in (/mingw64/bin:/usr/bin:/c/Users/runneradmin/bin:/d/a/gitoxide/gitoxide/target/debug/build/curl-sys-7e85c957548a17f0/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libgit2-sys-a405fc67d8aee456/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libssh2-sys-0acf82cde4d4f0d1/out/build:/d/a/gitoxide/gitoxide/target/debug/build/libz-sys-a02af73321b6f02b/out/lib:/d/a/gitoxide/gitoxide/target/debug/build/wepoll-ffi-169df33785c3c0f8/out:/d/a/gitoxide/gitoxide/target/debug/deps:/d/a/gitoxide/gitoxide/target/debug:/c/Users/runneradmin/.rustup/toolchains/stable-x86_64-pc-windows-msvc/lib/rustlib/x86_64-pc-windows-msvc/lib:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/.rustup/toolchains/stable-x86_64-pc-windows-msvc/bin:/c/Program Files/MongoDB/Server/5.0/bin:/c/aliyun-cli:/c/vcpkg:/c/cf-cli:/c/Program Files (x86)/NSIS:/c/tools/zstd:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/stack/2.7.5/x64:/c/cabal/bin:/c/ghcup/bin:/c/tools/ghc-9.2.2/bin:/c/Program Files/dotnet:/c/mysql/bin:/c/Program Files/R/R-4.1.3/bin/x64:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/GitHub CLI:/bin:/c/Program Files (x86)/pipx_bin:/c/hostedtoolcache/windows/go/1.17.8/x64/bin:/c/hostedtoolcache/windows/Python/3.7.9/x64/Scripts:/c/hostedtoolcache/windows/Python/3.7.9/x64:/c/hostedtoolcache/windows/Ruby/2.5.9/x64/bin:/c/tools/kotlinc/bin:/c/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.322-6/x64/bin:/c/npm/prefix:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/ProgramData/kind:/c/Program Files/Eclipse Foundation/jdk-8.0.302.8-hotspot/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/ProgramData/Chocolatey/bin:/c/Program Files/Docker:/c/Program Files/PowerShell/7:/c/Program Files/Microsoft/Web Platform Installer:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Program Files (x86)/Microsoft SQL Server/110/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/120/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/130/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/140/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/c/Program Files/nodejs:/c/Program Files/OpenSSL/bin:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/ProgramData/chocolatey/lib/pulumi/tools/Pulumi/bin:/c/Program Files/TortoiseSVN/bin:/c/Program Files/CMake/bin:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.8.5/bin:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/cmd:/mingw64/bin:/usr/bin:/c/Program Files/GitHub CLI:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/Program Files/Amazon/AWSCLIV2:/c/Program Files/Amazon/SessionManagerPlugin/bin:/c/Program Files/Amazon/AWSSAMCLI/bin:/c/Program Files (x86)/Google/Cloud SDK/google-cloud-sdk/bin:/c/Program Files (x86)/Microsoft BizTalk Server:/c/Program Files/LLVM/bin:/c/Users/runneradmin/.dotnet/tools:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/AppData/Local/Microsoft/WindowsApps)
2022-04-14T09:40:43.0705473Z D:\a\gitoxide\gitoxide\git-repository\tests\fixtures\make_rebase_i_repo.sh: line 33: bc: command not found
2022-04-14T09:40:43.0706030Z fatal: --preserve-merges was replaced by --rebase-merges
2022-04-14T09:40:43.0706440Z ', tests\tools\src\lib.rs:115:9
2022-04-14T09:40:43.0706836Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Originally posted by @Byron in #382 (comment)

[calam1]

I am on a Mac with an intel chip, running rust 1.60.0, using Nvim v0.7.0

So I am having issues with running the make tests command. I fail on the same place as this above error, but with a different error. I think it has something to do with this line in make_rebase_i_repo.sh EDITOR="sed -i.bak 's/pick/edit/g'" git rebase --rebase-merges --interactive HEAD~2

When i comment out that above line, I get an assertion error, which makes sense. I am wondering if this has something to do with nvim as the error message alludes to that.

  Finished test [unoptimized + debuginfo] target(s) in 0.87s
     Running tests/git.rs (target/debug/deps/git-c1ac60e3f9ad2185)

running 1 test
Archive at 'tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz' not found, creating fixture using script 'make_rebase_i_repo.sh'
thread 'repository::state::rebase_interactive' panicked at 'repo script failed: stdout: [main (root-commit) 98e8fd1] 1
 Author: author <[email protected]>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 1
[main 3b79d4c] 2
 Author: author <[email protected]>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 2
[main 4637843] 3
 Author: author <[email protected]>
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 3

stderr: Vim: Error reading input, exiting...

Vim: Finished.
error: There was a problem with the editor '/usr/local/bin/nvim'.
', tests/tools/src/lib.rs:178:17
stack backtrace:
^[]11;rgb:2828/2828/2828^[\^[[O   0: rust_begin_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
   2: git_testtools::scripted_fixture_repo_read_only_with_args
             at /Users/xcxl200/git/development/gitoxide/tests/tools/src/lib.rs:178:17
   3: git_testtools::scripted_fixture_repo_read_only
             at /Users/xcxl200/git/development/gitoxide/tests/tools/src/lib.rs:51:5
   4: git::named_repo
             at ./tests/git.rs:11:21
   5: git::repository::state::rebase_interactive
             at ./tests/repository/state.rs:55:16
   6: git::repository::state::rebase_interactive::{{closure}}
             at ./tests/repository/state.rs:54:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
test repository::state::rebase_interactive ... FAILED

failures:

failures:
    repository::state::rebase_interactive

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 40 filtered out; finished in 0.73s

error: test failed, to rerun pass '-p git-repository --test git'
The terminal process "cargo 'test', '--package', 'git-repository', '--test', 'git', '--', 'repository::state::rebase_interactive', '--exact', '--nocapture'" terminated with exit code: 101.

[Byron]

So I am having issues with running the make tests command. I fail on the same place as this above error, but with a different error.

Is there a way to see this different error?

[calam1]

Unless I am mistaken the error I get is in the code block in my second comment. I created the issue off of the commit of this unit test, and in the comments of that PR. So that was probably confusing. My bad

[Byron]

Unless I am mistaken the error I get is in the code block in my second comment.

The reason I ignored it is that it appears it was related to removing the EDITOR=sed part of the invocation, which would invoke the default editor of the system which is unrelated.
Please also be sure that you run the test on the latest main. It's likely something with the sed installed on the system, but since it works on CI my current stance is that it should be fixable on the host. The binary it should pick up is /usr/bin/sed, at least that's what's happening on MacOS for me.

[calam1]

Oh, the error that shows, does NOT have that EDITOR=sed removed. I removed it and ran the tests again, and got an assertion error, which makes sense, since the EDITOR part was not there, thus not ran.

Hmm, my sed is also in /usr/bin/sed but if it is working for you on your Mac, then it must be something weird on my terminal setup.

I'll close this and look further. Thanks!

[Byron]

Alright, please keep me posted if you discover the root cause of the issue. Maybe there is something that can be done to make this test more resilient (and it already came a long way).

[calam1]

Thus far, I have traced it down to this line in the tests/tools/src/lib.rs file the

       std::io::copy(&mut decoder, &mut buf)?;

this copy is crapping out and getting an failed to extract 'tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz': lzma data error

For kicks I ran the cargo tests with GITOXIDE_TEST_IGNORE_ARCHIVES=1 although it bypassed the data error, it didn't work overall there were just other failures, and I think this setting was for linux or what-not, and I am running Mac OSX. Still looking though. I ran this with various versions of rust also, 1.59, 1.60, and 1.60-nightly same error

[calam1]

reopening for now, for visibility, feel free to close once reading and either clarifying, etc

so maybe this is the issue this file https:/Byron/gitoxide/blob/main/git-repository/tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz is a binary file. Makes sense it's a compressed file.

When I clone this repo and look at this file on my laptop it looks like this when i cat it.

 cat  gitoxide/git-repository/tests/fixtures/generated-archives/make_rebase_i_repo.tar.xz

version https://git-lfs.github.com/spec/v1
oid sha256:116d62faab198c39a3d11248a17f8efe1d3f78d879905fe6b5570587edc01b2c
size 11576

so the above lzma data error makes sense because it isn't a binary file since you are using xz2 to Decode.

       # test/tools/src/lib.rs
       ...
        let buffered_file = std::io::BufReader::new(input_archive);
        let mut decoder = xz2::bufread::XzDecoder::new(buffered_file);
        std::io::copy(&mut decoder, &mut buf)?;

[Byron]

I am afraid you were led down the wrong path: Archives are purely optional and errors loading them - for whichever reason - are non-fatal and lead to the original shell script to be executed.

Once it fails, errors are printed. It's possible to execute it yourself in any empty directory to reproduce the issue in the terminal which should help narrowing it down. The script is the only place that matters for determining the root cause of the issue, even though fixing it might involve other changes either to the script itself to handle some cases better, or to the host system.

Additional information will be required in order to resolve this - thanks for your persistence.

Edit: It's clear that it uses nvim instead of sed, so that's the issue. It's the question why that happens.

[calam1]

Thanks for the response. I’ll try your suggestions and keep digging. I will see if I can get further and will update.

[calam1]

So the issue for me seems to be in the make_rebase_i_repo.sh file. The following code causes issues for me,

EDITOR="sed -i.bak 's/pick/edit/g'" git rebase --rebase-merges --interactive HEAD~2

The EDITOR setting and the sed command were problematic on my setup.

I edited the file to read the following and commented out the above

# EDITOR="sed -i.bak 's/pick/edit/g'" git rebase --rebase-merges --interactive HEAD~2
GIT_EDITOR="sed -i -e 's/pick/edit/g'" git rebase --rebase-merges --interactive HEAD~2

With this change it works for me. The nvim issue was from my .gitconfig editor setting or my .zshrc file EDITOR setting. So I finally got past this but only to come to another issue, I will close this issue and look into this new error I encountered, and if necessary open up another issue. Thanks for the guidance in pinpointing the issue. sed commands are always tricky on different OS and versions of the same OS, and also the reliance of configuration or shell settings.

-----------------------------------------------------
smart-release
-----------------------------------------------------
[with] 'c'
  [with] -d minor to bump minor dependencies
     [it] succeeds
[when] releasing 'a'
  [with] dry-run only
     [with] conditional version bumping
        [with] explicit bump specification
           [it] succeeds11,12c11,12
< [TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
< [TRACE] WOULD run "gh" "release" "create" "a-v0.8.0" "--title" "a v0.8.0" "--notes" "### Commit Statistics\n…" [note truncated]
\ No newline at end of file
---
> [WARN ] To create github releases, please install the 'gh' program and try again
> [TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
\ No newline at end of file
 - FAIL
$ /Users/xcxl200/git/junk/gitoxide/cargo-smart-release/tests/../../target/debug/cargo-smart-release smart-release a --no-push --no-publish -v --allow-dirty -b minor
Output snapshot did not match snapshot at '/Users/xcxl200/git/junk/gitoxide/cargo-smart-release/tests/snapshots/triple-depth-workspace/a-dry-run-success-multi-crate'
[WARN ] Consider running with --update-crates-index to assure bumping on demand uses the latest information
[WARN ] Crates.io index doesn't exist. Consider using --update-crates-index to help determining if release versions are published already
[INFO ] Manifest version of provided package 'a' at 0.8.0 is sufficient, creating a new release 🎉, ignoring computed version 0.9.0
[INFO ] WOULD adjust version constraints in manifest of 1 package as direct dependencies are changing: b
[INFO ] WOULD modify existing changelog for 'a'.
[TRACE] WOULD persist changes to 2 manifests and 1 changelogs with: "Adjusting changelogs prior to release of a v0.8.0"
[INFO ] Up to 1 changelog would be previewed if the --execute is set and --no-changelog-preview is unset.
[WARN ] WOULD ask for review after commit as the changelog entry is empty for crate: a
[WARN ] To fix the changelog manually, run: cargo changelog --write a
[TRACE] WOULD run "git" "commit" "-am" "Adjusting changelogs prior to release of a v0.8.0"
[WARN ] To create github releases, please install the 'gh' program and try again
[TRACE] WOULD create tag object a-v0.8.0 with changelog message, first line is: '### Commit Statistics'
make: *** [journey-tests-smart-release] Error 1

[Byron]

It's too strange as according to this code git will definitely pick up EDITOR as well and it will use it. The line in question in the .sh file will override any previously set EDITOR variable with the given value so it's absolutely mind boggling how it can manage to be unset.

Regarding the follow-up failure: it appears the journey test for smart release relies on having the gh (github via homebrew) program installed, and not going so yields an additional message that the test fixture doesn't expect.

[calam1]

thanks for all your help! The whole EDITOR problem maybe is on my side? I have a MacBook and an Ubuntu laptop and they both experienced the same issue regarding this. I am running zsh with customizations (similar customizations on both machines), so maybe that is interfering somehow is my guess. i think the test in question is relatively new. so hopefully this issue helps others who may run into this in the future.

[Byron]

thanks for all your help! The whole EDITOR problem maybe is on my side?

Since it's happening on two platforms with similar setups, it's certainly possible. It's just that these customizations shouldn't have any bearing on the tests. They launch /bin/bash and that comes up without customizations when running in interpreter mode like it does when executing scripts.

This is why I'd love to learn what can be causing this to hopefully protect against this if at all possible.
What does /bin/bash --version say? For me it's this:

GNU bash, version 3.2.57(1)-release (arm64-apple-darwin21)
Copyright (C) 2007 Free Software Foundation, Inc

[calam1]

my bash version is

❯❯❯ /bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21)
Copyright (C) 2007 Free Software Foundation, Inc.

[Byron]

I am absolutely out of questions to ask, and see no reason why this wouldn't be working for you :/.

[Byron]

Of course, if sed would be symlinked to nvim which most certainly isn't the case it would all make sense.

[calam1]

yeah, it's odd. I have been a little busy but am just trying to learn this tool you wrote, and hopefully will be able to contribute in the future (after watching your videos) thanks for the involvement!