write: connection reset by peer - during use of build-push-action to harbor

[dashap]

We have a github action that runs something similar to the following:

- name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to internal Docker Registry
        uses: docker/login-action@v3
        with:
          registry: ourharbor.com
          username: ${{ secrets.registry_username }}
          password: ${{ secrets.registry_password }}

      # Download artifact, build and push api image
      - name: Download artifacts from rest build workflow
        uses: actions/download-artifact@v4
        with:
          name: ${{ env.REST_TAG }}
          path: REST-EAR/target/.

      # Docker image build/push
      - name: Build and Push REST
        uses: docker/build-push-action@v6
        with:
          file: REST-EAR/Dockerfile
          context: ./REST-EAR
          push: true
          tags: ourharbor.com/ourapp/api:${{ env.TAG_NAME }}

We see every run connection resets similar to following:

ERROR: failed to solve: failed to push ourharbor.com/ourapp/api:develop-6898: failed to copy: failed to do request: Put "https://ourharbor.com/v2/ourapp/api/blobs/uploads/blahblahblah: write tcp 172.17.0.2:48946->10.1.2.3:443: write: connection reset by peer

The logs do not show much of anything. All we see are these and not sure if they are related or not to issue:

404 errors:
proxy.log:Oct 4 09:06:15 172.21.0.1 proxy[10408]: 10.1.2.3 - "HEAD /v2/ourapp/staging/manifests/develop-1234 HTTP/1.1" 404 0 "-" "buildkit/v0.16" 0.005 0.005 .

and 405 errors:
proxy.log:Oct 4 09:04:04 172.21.0.1 proxy[10408]: 10.1.2.3 - "POST /service/token HTTP/1.1" 405 19 "-" "containerd/1.7.21+unknown" 0.025 0.005 .

Note that this one does show 200 generally for the containerd/1.7.21+unknown:
proxy.log:Oct 4 09:04:05 172.21.0.1 proxy[10408]: 10.1.2.3 - "GET /service/token?scope=repository%3Aourapp%2Fapi%3Apull&scope=repository%3Aourapp%2Fstaging%3Apull%2Cpush&service=harbor-registry HTTP/1.1" 200 1006 "-" "containerd/1.7.21+unknown" 0.064 0.065 .

It generally tries a few times and eventually works, but we have occasions where it fails. We are looking for what we can do to stop the connection resets from happening in the first place. Is there any nginx.conf settings we should be using to assist with this? Any other ideas?How can we help you?

Note that we are using harbor v2.1.1.

[wy65701436]

is is the harbor v2.11.1? how big of the failure image?