Clean up ci-runner dockerfiles & upgrade to nodejs 14.19.1

[peterzhuamazon]

Signed-off-by: Peter Zhu [email protected]

Description

Clean up ci-runner dockerfiles & upgrade to nodejs 14.19.1.

• Remove AL2 runner, as it is not used and deprecated.
• Change 14.18.2 to 14.19.1 nodejs version per this: Bumps Node.js from v14.18.2 to v14.19.1 OpenSearch-Dashboards#1487
• Removed nodejs and cypress from the opensearch build runner as it is not needed anymore.

Issues Resolved

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #2030 (7a2caab) into main (b9f83df) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               main    #2030   +/-   ##
=========================================
  Coverage     94.23%   94.23%           
  Complexity       25       25           
=========================================
  Files           198      198           
  Lines          3902     3902           
  Branches         29       29           
=========================================
  Hits           3677     3677           
  Misses          219      219           
  Partials          6        6           

Impacted Files	Coverage Δ
src/manifests_workflow/input_manifests.py	98.97% <ø> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9f83df...7a2caab. Read the comment docs.

[peterzhuamazon]

New images:

(Will show up once the build is complete)

• https://hub.docker.com/r/opensearchstaging/ci-runner/tags?page=1&ordering=last_updated&name=ci-runner-centos7-opensearch-build-v2
• https://hub.docker.com/r/opensearchstaging/ci-runner/tags?page=1&ordering=last_updated&name=ci-runner-centos7-opensearch-dashboards-build-v2
• https://hub.docker.com/r/opensearchstaging/ci-runner/tags?page=1&ordering=last_updated&name=ci-runner-rockylinux8-opensearch-dashboards-integtest-v2

All images now use the number v2.

[tianleh]

I can perform some sanity check on the new images when they are ready. @peterzhuamazon

[peterzhuamazon]

Everything is built right now all the images above updated to v2.
Cypress on arm64 can now automatically compile during docker image build time.
Thanks.

[peterzhuamazon]

Image too big change to multi-stage now.

opensearchstaging/ci-runner               ci-runner-rockylinux8-opensearch-dashboards-integtest-v2   daf9ba3a8c03   43 minutes ago   14.5GB

[tianleh]

Please optimize the image sizes before merging.

[peterzhuamazon]

Size fixed now:

[ci-runner-rockylinux8-opensearch-dashboards-integtest-v2](https://hub.docker.com/layers/ci-runner/opensearchstaging/ci-runner/ci-runner-rockylinux8-opensearch-dashboards-integtest-v2/images/sha256-1054f5782bbf282d9373dba556c943fb7aa9dd2b559d0d69efaee0a30c89b5f3?context=explore)
Last pushed 7 minutes ago by [opensearchstaging](https://hub.docker.com/u/opensearchstaging)
Digest
OS/ARCH
Compressed Size
1054f5782bbf
linux/amd64
1.05 GB
4d3830db21d0
linux/arm64
1.04 GB

[peterzhuamazon]

[opensearch@c28e1e31fe3e ~]$ cypress run
It looks like this is your first time using Cypress: 9.5.4

✔  Verified Cypress! /usr/share/opensearch/.cache/Cypress/9.5.4/Cypress

Opening Cypress...
[1773:0423/103729.554624:ERROR:gpu_init.cc(453)] Passthrough is not supported, GL is swiftshader, ANGLE is
Could not find a Cypress configuration file in this folder: /usr/share/opensearch
[opensearch@c28e1e31fe3e ~]$ uname -a
Linux c28e1e31fe3e 4.14.273-207.502.amzn2.aarch64 #1 SMP Wed Mar 30 01:13:12 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

Ignore failure as there is no cypress test present.
This proves that on aarch64 the compiled cypress runs correctly.

Thanks.

We have reduced the size now.

[peterzhuamazon]

Rebuilt with cmake installed so no module compilation failure, still the sandbox test seems error out due to we are building a docker image inside a docker, but binary built.

[peterzhuamazon]

@tianleh please review.
Thanks.

[prudhvigodithi]

Hey @peterzhuamazon I see ci-runner is mentioned twice in the image opensearchstaging/ci-runner:ci-runner-centos7-opensearch-build-v2, can we have the image like opensearchstaging/centos7-ci-runner:2.0, would this make simpler ?
Thank you

[peterzhuamazon]

Hey @peterzhuamazon I see ci-runner is mentioned twice in the image opensearchstaging/ci-runner:ci-runner-centos7-opensearch-build-v2, can we have the image like opensearchstaging/centos7-ci-runner:2.0, would this make simpler ? Thank you

It is always like that and I understand where you come with that question.
It is not ideal but we have been putting all the ci runner images in the same repo without creating multiple other repos.
In the next iteration we can try something else, at least standardize the image names, which is still not there yet.

Let's not change now as release is coming up and a lot of coding is depending on these (yes there are actual code dependencies on these names).

Plus even with the help of Jenkins these images build in 3 hours due to multiple compilation on the fly through binary translation (which is by design to do so). So I would not start that now.

There is an issue in this repo already pointing that out.

Thanks.

[gaiksaya]

Hey @peterzhuamazon if you have issue for enhancement for ci images can you add this example as well? opensearch-project/opensearch-ci#107
so that we don't have to wait for someone to build the image, whenever a change is pushed to this repo, the ci image can be built automatically.
Thanks!

[peterzhuamazon]

Hey @peterzhuamazon if you have issue for enhancement for ci images can you add this example as well? opensearch-project/opensearch-ci#107 so that we don't have to wait for someone to build the image, whenever a change is pushed to this repo, the ci image can be built automatically. Thanks!

Not necessarily the same as we have other script to setup env before dockerfile being built, thus cannot directly run docker build from github actions. Please the compiling is intense and it would crash if we do it on the 2 core machine of github runners. It would also take more than 6 hours even if it runs.