Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide dockerx setup for docker builds #1194

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

sikehish
Copy link

@sikehish sikehish commented Oct 18, 2024

This address #1064.
A shell script was created to handle building and pushing multi-architecture images using docker buildx.

Changes

This PR introduces a shell script (buildx_and_push.sh) to automate the process of building and pushing Docker images for multiple architectures using docker buildx. It ensures that the Docker images are always built with the latest version of the sf-hamilton-ui library from PyPI.

Key Changes:

  1. Multi-architecture support:

    • Builds images for multiple platforms (linux/amd64, linux/arm64).
  2. Automated version management:

    • The script dynamically fetches the latest sf-hamilton-ui version from PyPI.
    • The fetched version is used to tag Docker images, ensuring that both the versioned tag and latest tag are always in sync with the current release.
  3. Builder instance setup:

    • Ensures that Docker Buildx is enabled and uses a named builder instance (hamilton-builder).
  4. Image tagging and pushing:

    • Tags both backend and frontend images with:
      • The latest sf-hamilton-ui version.
      • The latest tag.

How I Tested This

  1. Environment:

    • Tested on a local machine with Docker Buildx enabled.
  2. Test Cases:

    • Checked the output to confirm that the fetched version from PyPI was used for tagging the images.
    • Built and pushed the backend and frontend images successfully using:
      ./buildx_and_push.sh
    • Verified that both versioned and latest tags were pushed to the repository.
  3. Validation:

    • Confirmed that the following tags were pushed:
      • dagworks/ui-backend:<fetched_version>
      • dagworks/ui-backend:latest
      • dagworks/ui-frontend:<fetched_version>
      • dagworks/ui-frontend:latest

Notes

  • Ensure Docker Buildx is installed before running the script. The setup logic in the script will handle creating the builder instance if it is not already configured.

  • Why this change is needed: This automation ensures consistency between the versioned tag and the latest tag. It also simplifies multi-architecture builds and makes them easier to manage across multiple platforms.

  • PR has an informative and human-readable title (this will be pulled into the release notes)

  • Changes are limited to a single goal (no scope creep)

  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.

  • Any change in functionality is tested

  • New functions are documented (with a description, list of inputs, and expected output)

  • Placeholder code is flagged / future TODOs are captured in comments

  • Project documentation has been updated if adding/changing functionality.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 19d2d98 in 21 seconds

More details
  • Looked at 34 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. ui/buildx_and_push.sh:7
  • Draft comment:
    Ensure that jq is installed before using it to parse JSON. You can add a check at the beginning of the script to verify its presence and provide a helpful error message if it's missing. Additionally, consider adding error handling for the curl command to ensure the script behaves correctly if the request to PyPI fails. For example, check if the response is empty or invalid and handle it appropriately.
  • Reason this comment was not posted:
    Comment did not seem useful.
2. ui/buildx_and_push.sh:5
  • Draft comment:
    Consider adding this script to the Sphinx documentation under docs/ to guide users on building and pushing multi-architecture Docker images.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The script is straightforward and doesn't have repeated code, except for the docker buildx command which is slightly different for frontend and backend. The script is not overly complicated, so comments are not necessary beyond what is already provided. Variable names are descriptive. There are no secrets or credentials in the code. The script follows a single responsibility of building and pushing docker images. The function naming is consistent. Sensitive data is not logged. The script could be added to the documentation for users who want to build and push docker images.

Workflow ID: wflow_SV8PcasBZOtbB0DH


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

ui/buildx_and_push.sh Show resolved Hide resolved
Copy link
Collaborator

@skrawcz skrawcz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

what would a github action look like to get this to run once a new sf-hamilton-ui version was published?

@sikehish
Copy link
Author

Looks great!

what would a github action look like to get this to run once a new sf-hamilton-ui version was published?
I'll be working on the action soon. But I'm currently building the images without the env variables/args. What do I do? Do I add multi arch builds to the docker compose file, or is there any other approach that I need to take?

@skrawcz
Copy link
Collaborator

skrawcz commented Oct 19, 2024

I'll be working on the action soon. But I'm currently building the images without the env variables/args. What do I do? Do I add multi arch builds to the docker compose file, or is there any other approach that I need to take?

Sorry I'm not following. The multi-arch builds would just use your script here right?

@sikehish
Copy link
Author

I'll be working on the action soon. But I'm currently building the images without the env variables/args. What do I do? Do I add multi arch builds to the docker compose file, or is there any other approach that I need to take?

Sorry I'm not following. The multi-arch builds would just use your script here right?

Yes. Just wanted to ask if you if the script is satisfying the requirements.

@skrawcz
Copy link
Collaborator

skrawcz commented Oct 19, 2024

Yes. Just wanted to ask if you if the script is satisfying the requirements.

Script looks great thanks. I haven't run it yet to verify though; so I assume it works right now.

@sikehish
Copy link
Author

sikehish commented Oct 19, 2024

Looks great!

what would a github action look like to get this to run once a new sf-hamilton-ui version was published?

Looks great!

what would a github action look like to get this to run once a new sf-hamilton-ui version was published?

Hi @skrawcz . Ive created a new workflow, tested it on my fork, and it's building and pushing those images. So it's working fine. Let me know if I need to make any changes :)

@sikehish
Copy link
Author

Hi @skrawcz
I've made the reuqested changes. Let me know if anything else needs to be changed. I've tested it on my local fork, and its working fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants