ESP Code + Repo Formatter 👩‍💻

[marianabuhazi]

ESP Code + Repo Formatter 👩‍💻

Overview

This pull request includes two scripts used to automatically format source files in the ESP repository, and format files that have been modified in Git (format_repo.sh and format.sh, accordingly).

The format_repo.sh script applies standardized formatting rules to various file types, such as C, C++, VHDL, Verilog, and Python, using open-source formatting tools. It recursively traverses directories to format files, excluding submodules, with a few exceptions.

The format.sh script applies the same formatting tools to determine if any files have been modified in the ESP git repo (checks the git diff), and help users find formatting issues in their code, and format files in-place.

The format_repo.sh script should only be run once to format the entire repository, and create a "clean" starting point for the repo. Then format.sh can be used in the future to maintain the organization as contributions are made.

Requirements

Ensure the following tools are installed and available in the system's PATH:

• clang-format-10
• verible-verilog-format
• vsg
• autopep8 (Python)

I have provided a script install.sh (script location: utils/scripts/format/tools/install.sh). Run it as ./install.sh to install the necessary tools before proceeding. If an error occurs, and the tools are not installed correctly, the formatting scripts will display an error message.

Sample error message

ERROR: The following required tools are not installed or not defined in PATH:
  - clang-format-10
  - verible-verilog-format
  - autopep8

Functionality: format_repo.sh

The script:

1. Recursively descends into all directories in the repository.
2. Skips submodules, except for three specified directories.
3. For each file, checks its extension and applies the appropriate formatter.
4. Outputs success or error messages for each formatted file.
5. User must specify the file type they want to format.

Script Location

• utils/scripts/format/format_repo.sh

Usage

Run the script with --help or -h to learn more about its usage, ie. format_repo.sh --help.

Supported File Types

The following file types are supported for automatic formatting:

• C/C++ (.c, .h, .cpp, .hpp)
• Python (.py)
• VHDL (.vhd)
• Verilog/SystemVerilog (.v, .sv)

Note on submodules (format_repo.sh)

The format_repo.sh script skips formatting for most Git submodules (.gitmodules), with the exception of the following directories:

• rtl/caches/esp-caches/*
• rtl/caches/spandex-caches/*
• accelerators/stratus_hls/common/inc/*

Any other submodules will be ignored during the formatting process to prevent unintended modifications.

Functionality: format.sh

The format.sh script is more targeted than format_repo.sh. It checks whether any files have been modified in git and either reports formatting issues or applies formatting in place. The script is typically run in two contexts:

1. As part of a GitHub Actions workflow to ensure code pushed to the repository is properly formatted.
2. As a pre-push git hook to prevent unformatted code from being pushed.

The script:

1. Formats files in place or checks their formatting without making changes.
2. Supports formatting of C, Python, Verilog, and VHDL files based on the file extension.
3. Outputs success or error messages for each processed file.
4. Provides options to format individual files or all modified files in the repository.
5. Can be used in both interactive and automated workflows (e.g., CI pipelines or Git Hooks).

Script Location

• utils/scripts/format/format.sh

Usage

Run the script with --help or -h to learn more about its usage, ie. format.sh --help.

Supported file types

Each file type is formatted using the following tools:

• C/C++: clang-format-10
• Python: autopep8
• VHDL: vsg (VHDL Style Guide)
• Verilog/SystemVerilog: verible-verilog-format

GitHub Actions Integration

The GitHub Actions pipeline is already set up as defined in .github/workflows/format_check.yaml. This pipeline will create an instance of a Docker container with the ubuntu18-small image and run the install.sh script to install all the necessary tools. Then, it will run the formatting check script for all files in "GitHub Actions" mode (-g) format.sh -g -ca to check that all pushed files to dev or master are formatted. The pipeline fails if there are errors.

Pre-push Hook

Users can set up a Git Pre-Push hook locally to ensure that all code pushed to a remote branch follows the formatting expectations. Simply, navigate to .githooks directory, and run the ./setup.sh script. This will create a pre-push hook. It can also be done automatically as part of Dockerfile in the future.