Start your EC2 self-hosted runner right before you need it. Run the job on it. Finally, stop it when you finish. And all this automatically as a part of your GitHub Actions workflow.
Table of Contents
The action can start the EC2 runner in any subnet of your VPC that you need - public or private. In this way, you can easily access any private resources in your VPC from your GitHub Actions workflow.
For example, you can access your database in the private subnet to run the database migration.
Some of your CI workloads may require more powerful hardware that GitHub-hosted runners provide. In the action, you can configure any EC2 instance type for your runner that AWS provides.
For example, you may run c5.4xlarge EC2 runner for some of your compute-intensive workloads. Or r5.xlarge EC2 runner for workloads that process large data sets in memory.
If your CI workloads don't need the power of the GitHub-hosted runners and the execution takes more than a couple of minutes, you can consider running it on a cheaper and less powerful instance from AWS.
According to GitHub's documentation, you don't need to pay for the jobs handled by the self-hosted runners:
Self-hosted runners are free to use with GitHub Actions, but you are responsible for the cost of maintaining your runner machines.
So you will be charged by GitHub only for the time the self-hosted runner start and stop. EC2 self-hosted runner will handle everything else so that you will pay for it to AWS, which can be less expensive than the price for the GitHub-hosted runner.
Use the following steps to prepare your workflow for running on your EC2 self-hosted runner:
1. Prepare AWS access keys
- Create new AWS access keys with the following least-privilege permissions. The action will use the keys for EC2 instance management in the AWS account.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:RunInstances",
"ec2:TerminateInstances",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus"
],
"Resource": "*"
}
]
}
- Add the keys to GitHub secrets.
- Use the aws-actions/configure-aws-credentials action to put the keys into environment variables.
2. Prepare GitHub personal access token
- Create a new GitHub personal access token with the
repo
scope. The action will use the token for self-hosted runners management in the GitHub account on the repository level. - Add the token to GitHub secrets.
3. Prepare EC2 image
- Create a new EC2 image (AMI) from the Linux distribution you need. You don't need to install anything special beforehand into the AMI. The action will install all the necessary tools during the EC2 instance creation.
4. Prepare VPC with subnet and security group
- Create a new VPC and a new subnet in it. Or use the existing VPC and subnet.
- Create a new security group for the runners in the VPC. The runner doesn't require any inbound traffic. However, outbound traffic for port 443 should be allowed so the runner can pull GitHub Actions' tasks.
Note: Make sure your EC2 instance has access to the internet in order to connect to GitHub Actions and pull the tasks.
5. Configure the GitHub workflow
- Create a new GitHub Actions workflow or edit the existing one.
- Use the documentation and example below to configure your workflow.
- Please don't forget to set up a job for removing the EC2 instance at the end of the workflow execution. Otherwise, the EC2 instance won't be removed and continue to run even after the workflow execution is finished.
Now you're ready to go!
Name | Required | Description |
---|---|---|
mode |
Always required. | Specify here which mode you want to use: - start - to start a new runner;- stop - to stop the previously created runner. |
github-token |
Always required. | GitHub Personal Access Token with the repo scope assigned. |
ec2-image-id |
Required if you use the start mode. |
EC2 Image Id (AMI). The new runner will be launched from this image. The action is compatible only with Linux images. |
ec2-instance-type |
Required if you use the start mode. |
EC2 Instance Type. |
subnet-id |
Required if you use the start mode. |
VPC Subnet Id. The subnet should belong to the same VPC as the specified security group. |
security-group-id |
Required if you use the start mode. |
EC2 Security Group Id. The security group should belong to the same VPC as the specified subnet. |
label |
Required if you use the stop mode. |
Name of the unique label assigned to the runner. The label is provided by the output of the action in the start mode. The label is used to remove the runner from GitHub when the runner is not needed anymore. |
ec2-instance-id |
Required if you use the stop mode. |
EC2 Instance Id of the created runner. The id is provided by the output of the action in the start mode. The id is used to terminate the EC2 instance when the runner is not needed anymore. |
In addition to the inputs described above, the action also requires the following environment variables to access your AWS account:
AWS_DEFAULT_REGION
AWS_REGION
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
We recommend using aws-actions/configure-aws-credentials action right before running the step for creating a self-hosted runner. This action perfectly does the job of setting the required environment variables.
Name | Description |
---|---|
label |
Name of the unique label assigned to the runner. The label is used in two cases: - to use as the input of runs-on property for the following jobs; - to remove the runner from GitHub when it is not needed anymore. |
ec2-instance-id |
EC2 Instance Id of the created runner. The id is used to terminate the EC2 instance when the runner is not needed anymore. |
The workflow showed in the picture above and declared in do-the-job.yml
looks like this:
name: do-the-job
on: pull_request
jobs:
start-runner:
name: Start self-hosted EC2 runner
runs-on: ubuntu-latest
outputs:
label: ${{ steps.start-ec2-runner.outputs.label }}
ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Start EC2 runner
id: start-ec2-runner
uses: machulav/[email protected]
with:
mode: start
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
ec2-image-id: ami-123
ec2-instance-type: t3.nano
subnet-id: subnet-123
security-group-id: sg-123
do-the-job:
name: Do the job on the runner
runs-on: ${{ needs.start-runner.outputs.label }} # run the job on the newly created runner
needs: start-runner # required to start the main job when the runner is ready
steps:
- name: Hello World
run: echo 'Hello World!'
stop-runner:
name: Stop self-hosted EC2 runner
runs-on: ubuntu-latest
needs:
- start-runner # required to get output from the start-runner job
- do-the-job # required to wait when the main job is done
if: ${{ always() }} # required to stop the runner even if the error happened in the previous jobs
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Stop EC2 runner
uses: machulav/[email protected]
with:
mode: stop
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
label: ${{ needs.start-runner.outputs.label }}
ec2-instance-id: ${{ needs.start-runner.outputs.ec2-instance-id }}
We recommend that you do not use self-hosted runners with public repositories.
Forks of your public repository can potentially run dangerous code on your self-hosted runner machine by creating a pull request that executes the code in a workflow.
Please find more details about this security note on GitHub documentation.
This code is made available under the MIT license.