Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup staging deployment #6

Merged
merged 6 commits into from
Mar 24, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .delta/ui
Submodule ui updated from 2e8548 to 67c68c
12 changes: 12 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Title of the application shown on the header and wherever the name is needed.
APP_TITLE=Dashboard Delta

# Short description used in meta tags. It shows up when the app url is shared.
APP_DESCRIPTION=Earth changing dashboard

# Email to reach out when an error occurs or the users have questions.
[email protected]

# If the app is being served in from a subfolder, the domain url must be set.
# For example, if the app is served from /mysite:
# PUBLIC_URL=http://example.com/mysite
95 changes: 95 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# This workflow performs basic checks:
#
# 1. run a preparation step to install and cache node modules
# 2. once prep succeeds, run lint and test in parallel
#
# The checks are skipped on the 'main' branch. The project relies on branch
# protection to avoid pushes straight to 'main'.

name: Checks

on:
push:
branches-ignore:
- 'main'

env:
NODE: 16

jobs:
prep:
if: github.event.pull_request.draft == false
runs-on: ubuntu-latest

steps:
- name: Cancel Previous Runs
uses: styfle/[email protected]
with:
access_token: ${{ github.token }}

# Both delta-ui and delta-config are private repos. Since the token issued
# to GH does not have access to the delta-ui submodule, we have to
# manually check it out using a ssh deploy key.
- name: Checkout
uses: actions/checkout@v3
- name: Get submodules
env:
SSH_KEY_SUBMODULE: ${{secrets.DELTA_UI_FETCH_KEY}}
run: |
eval `ssh-agent -s`
ssh-add - <<< "${SSH_KEY_SUBMODULE}"; git submodule update --init --recursive

- name: Use Node.js ${{ env.NODE }}
uses: actions/setup-node@v1
with:
node-version: ${{ env.NODE }}

- name: Cache node_modules
uses: actions/cache@v2
id: cache-node-modules
with:
path: |
node_modules
.delta/ui/node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package.json') }}

- name: Install
run: ./.delta/setup

test:
needs: prep
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3

# See comment on checks-yml - prep step
- name: Checkout
uses: actions/checkout@v3
- name: Get submodules
env:
SSH_KEY_SUBMODULE: ${{secrets.DELTA_UI_FETCH_KEY}}
run: |
eval `ssh-agent -s`
ssh-add - <<< "${SSH_KEY_SUBMODULE}"; git submodule update --init --recursive

- name: Use Node.js ${{ env.NODE }}
uses: actions/setup-node@v1
with:
node-version: ${{ env.NODE }}

- name: Cache node_modules
uses: actions/cache@v2
id: cache-node-modules
with:
path: |
node_modules
.delta/ui/node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package.json') }}

- name: Install
run: ./.delta/setup

- name: Test
run: yarn test
97 changes: 97 additions & 0 deletions .github/workflows/deploy-staging.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Deploy Staging - Surge.sh

on:
push:
branches:
- 'develop'

env:
NODE: 16
SURGE_STAGING_DOMAIN: https://vizex.surge.sh

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Cancel Previous Runs
uses: styfle/[email protected]
with:
access_token: ${{ github.token }}

# See comment on checks-yml - prep step
- name: Checkout
uses: actions/checkout@v3
- name: Get submodules
env:
SSH_KEY_SUBMODULE: ${{secrets.DELTA_UI_FETCH_KEY}}
run: |
eval `ssh-agent -s`
ssh-add - <<< "${SSH_KEY_SUBMODULE}"; git submodule update --init --recursive

- name: Use Node.js ${{ env.NODE }}
uses: actions/setup-node@v1
with:
node-version: ${{ env.NODE }}

- name: Cache node_modules
uses: actions/cache@v2
id: cache-node-modules
with:
path: |
node_modules
.delta/ui/node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package.json') }}

- name: Cache dist
uses: actions/cache@v2
id: cache-dist
with:
path: dist
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ github.workflow }}-${{ github.sha }}

- name: Install
run: ./.delta/setup

- name: Build staging
run: yarn stage

deploy:
runs-on: ubuntu-latest
needs: build

steps:
# See comment on checks-yml - prep step
- name: Checkout
uses: actions/checkout@v3
- name: Get submodules
env:
SSH_KEY_SUBMODULE: ${{secrets.DELTA_UI_FETCH_KEY}}
run: |
eval `ssh-agent -s`
ssh-add - <<< "${SSH_KEY_SUBMODULE}"; git submodule update --init --recursive
- name: Restore node_modules
uses: actions/cache@v2
id: cache-node-modules
with:
path: |
node_modules
.delta/ui/node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package.json') }}

- name: Restore dist cache
uses: actions/cache@v2
id: cache-dist
with:
path: dist
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ github.workflow }}-${{ github.sha }}

- name: Use Node.js ${{ env.NODE }}
uses: actions/setup-node@v1
with:
node-version: ${{ env.NODE }}

- name: Deploy Staging to Surge
run: |
cp ./dist/index.html ./dist/200.html
yarn run surge ./dist ${{ env.SURGE_STAGING_DOMAIN }} --token ${{ secrets.SURGE_TOKEN }}
45 changes: 41 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,11 @@ Setup the project
./.delta/setup
```

## Content & Configuration
## Configuration

The base properties used by Delta are set through the `.env` file. This includes values like the application title, and contact email. These values are then used throughout the app.

## Content

Delta is divided into Thematic Areas, Datasets, and Discoveries.
Each piece of content is written in [MDX](https://mdxjs.com/docs/what-is-mdx/#what-is-mdx) with configuration frontmatter between a set of `---`.
Expand All @@ -47,7 +51,7 @@ thematics: './thematic/*.thematic.mdx'
datasets: './datasets/*.data.mdx'
discoveries: './discoveries/*.discoveries.mdx'
```
But the content can be stores wherever you'd like.
But the content can be stored wherever you'd like.

### Thematic Areas
TODO: info
Expand All @@ -61,8 +65,41 @@ TODO: info
# Development
There's not much development to be done in the config repo. The UI part of Delta is stored at [delta-ui](https:/NASA-IMPACT/delta-ui), and connected to the config via a git submodule.

**To avoid breaking changes you'll have to manually update the version of delta-ui your config uses.**
**To avoid breaking changes, the version of `delta-ui` is pinned to a specific commit making use of a [git-submodule](https://www.atlassian.com/git/tutorials/git-submodule).** This allows development to continue without breaking existent instances.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add documentation about how this pinned commit can be updated for when ui update needs to be deployed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating the commit involves knowing how git submodules work and which commit you pin it to, depends on the use case. During development we'll most likely pin this to random branches, but I added some docs explaining what is going on and an example to update from main. Should be easy to understand how to apply it to other situations.
Let me know what you think @hanbyul-here .

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I see. I was not sure where this is done (was thinking if this is saved somewhere in github action config?) , and documentation make it clear where this is done. Thanks!

You can see the version running:
```
./.delta/delta --info
```
```

The commit you may want to pin the `delta-ui` to will depend, but to get the latest changes from the `main` branch you could do:
```
# Go into the submodule
cd .delta/ui

# Update the submodule info
git fetch

# Switch to the branch you want.
# You can also choose a specific tag.
git switch main

# Get the latest changes
git pull

# Get back to the root directory
cd -
```

At this point you will have changed the commit the submodule is pointed to.
If you run a diff on the submodule `git diff .delta/ui` you'll see that the commit changed.
```diff
diff --git a/.delta/ui b/.delta/ui
index 67c68c4..57d18a1 160000
--- a/.delta/ui
+++ b/.delta/ui
@@ -1 +1 @@
-Subproject commit 67c68c4115283a76fe8f3e0921fcd43ad281077a
+Subproject commit 57d18a12a4c17eaf436e128bd06904cba75f3e4a
```

The only thing left to do is to commit this change like you'd normally do.
Binary file added datasets/img-placeholder-3.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added datasets/img-placeholder-4.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
46 changes: 41 additions & 5 deletions datasets/no2.data.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,46 @@
---
id: no2
name: 'Carbon dioxide or not'
name: 'Carbon Dioxide or Some Other Cool Dataset Name'
description: "Sed sed lectus vitae odio vestibulum mattis. Integer iaculis nisl lectus, vel aliquet nulla imperdiet in."
media:
src: ::file ./img-placeholder-3.jpg
alt: Generic placeholder by Unsplash
author:
name: Unsplash
url: https://unsplash.com/
thematics:
- water
- fire
- agriculture
- air-quality

layers:
- id: no2-monthly
name: No2
type: raster
description: no2 dataset
zoomExtent:
- 0
- 20
sourceParams:
resampling_method: bilinear
bidx: 1
color_formula: gamma r 1.05
colormap_name: coolwarm
rescale:
- 0
- 1.5e16
compare:
datasetId: sandbox
layerId: no2-monthly
datetime: |
::js ({ datetime, dateFns }) => {
return dateFns.sub(datetime, { years: 1 });
}
mapLabel: |
::js ({ datetime, dateFns }) => {
const comparingDate = dateFns.sub(datetime, { years: 1 });
return `${dateFns.format(datetime, 'LLL yyyy')} - ${dateFns.format(comparingDate, 'LLL yyyy')}`;
}
---
# NO2
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut vitae ornare lectus, ac accumsan erat. Cras eget eleifend ligula. Curabitur ac nisi tempor, molestie lorem vitae, consectetur turpis. Etiam hendrerit et sapien ac interdum.

There goes NO2.
Phasellus nec magna tristique, imperdiet mauris hendrerit, posuere sem. Vestibulum in lobortis tortor.
61 changes: 61 additions & 0 deletions datasets/sandbox.data.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
featured: true
id: sandbox
name: Sandbox
description: Travel restrictions and lockdown measures have disrupted the shipping industry and the global economy broadly. NASA researchers are using artificial intelligence to track shipping activities across major ports in the U.S.
media:
src: ::file ./img-placeholder-4.jpg
alt: Generic placeholder by Unsplash
author:
name: Unsplash
url: https://unsplash.com/
thematics:
- agriculture
- air-quality

layers:
- id: no2-monthly
name: No2
type: raster
description: no2 dataset
zoomExtent:
- 0
- 20
sourceParams:
resampling_method: bilinear
bidx: 1
color_formula: gamma r 1.05
colormap_name: coolwarm
rescale:
- 0
- 1.5e16
compare:
datasetId: sandbox
layerId: no2-monthly
datetime: |
::js ({ datetime, dateFns }) => {
return dateFns.sub(datetime, { years: 1 });
}
mapLabel: |
::js ({ datetime, dateFns }) => {
const comparingDate = dateFns.sub(datetime, { years: 1 });
return `${dateFns.format(datetime, 'LLL yyyy')} - ${dateFns.format(comparingDate, 'LLL yyyy')}`;
}
- id: nightlights-hd-monthly
name: Nightlights Monthly
type: raster
description: The nightlights dataset
zoomExtent:
- 4
- 16
sourceParams:
bidx: 1
colormap_name: inferno
rescale:
- 0
- 255
- id: dev-fail
name: Failing layer
type: raster
---
# Nightlights
Binary file added discoveries/history-cover.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading