Content: Water quality and covid 19

[danielfdsilva]

Adding content

Migrate Water Quality and Covid19 from Covid Dashboard.

Several datasets are not yet available in the VEDA dashboard, so images were used instead, trying to maintain the same information structure.

Missing datasets (for reference):

• water turbidity SF and Venice
• Chlorophyll-a anomaly for Chesapeake and Great Lakes
• ls8-iowa and ls8-belize

NOTE: The whole Venice chapter on water turbidity relies on the visualization. Since the dataset doesn't even load on the covid dashboard it was not possible to get images and it was left out.

---

👀 https://deploy-preview-231--visex.netlify.app/covid-19/discoveries/water-quality-and-covid-19

@slesaad @aboydnw Could you have a look at this to see if it makes sense? Or bring in the correct people.

[netlify]

✅ Deploy Preview for visex ready!

Name	Link
🔨 Latest commit	3a3d8d3
🔍 Latest deploy log	https://app.netlify.com/sites/visex/deploys/642ae0476c8f41000796c275
😎 Deploy Preview	https://deploy-preview-231--visex.netlify.app/
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[slesaad]

I was looking at why the datasets in the covid19 dashboard weren't loading. It looks like there are requests to two apis that are failing:

(Counting crops during a lockdown)
https://c50qa6bhpe.execute-api.us-west-2.amazonaws.com/scenes/landsat/tiles/9/121/191.png?sceneid=LC08_L1TP_026031_20200304_20200314_01_T1&bands=B6,B5,B2&color_formula=gamma%20RGB%203.5,%20saturation%201.7,%20sigmoidal%20RGB%2015%200.35

• couldn't find this api in either the covid-response aws account or the veda-mcp account 🤷‍♀️ maybe it's in the devseed side?

(Understanding impacts on the venetian lagoon)
https://8ib71h0627.execute-api.us-east-1.amazonaws.com/v1/9/272/184@1x?url=s3://covid-eo-data/tsm/nas-2020_03_21.tif&resampling_method=bilinear&bidx=1&rescale=-100%2C100&color_map=rdbu_r

• this is failing because the file it's looking for s3://covid-eo-data/tsm/nas-2020_03_21.tif doesn't exist, the whole 'tsm' folder doesn't exist in the bucket

@abarciauskas-bgse @leothomas any ideas?

[abarciauskas-bgse]

@slesaad wrt the second error you are seeing, where the file doesn't exist, I found it in s3://covid-eo-data-archive/tsm/nas-2020_03_21.tif

Is there some process which is moving the files to an archive bucket?

[abarciauskas-bgse]

wrt to the first error, I did a bit of digging around (searching slack and covid-api and covid-dashboard for terms like "landsat", "sceneid") and the best clue I have found so far is NASA-IMPACT/covid-dashboard#466. The spreadsheet linked in that ticket includes the same sceneid as in the URL sceneid=LC08_L1TP_026031_20200304_20200314_01_T1 so perhaps @olafveerman can help us understand how these products were incorporated and where that mystery API gateway was deployed. I'm wondering if it was the API gateway behind remotepixel which is now defunct. cc @vincentsarago

[vincentsarago]

@abarciauskas-bgse it might be an instance of https:/developmentseed/titiler-pds (https:/developmentseed/titiler-pds/blob/master/titiler_pds/routes/landsat.py#L21-L26)

also note that the sceneid refers to Landsat collection1 which is no more available on AWS

[nerik]

Just one cpation to fix, other wise looks good 👍

[olafveerman]

@abarciauskas-bgse @vincentsarago Did a quick search on Slack, which confirmed Vincent's assumption:

https://c50qa6bhpe.execute-api.us-west-2.amazonaws.com/docs
https:/developmentseed/titiler-pds/blob/master/demo/landsat.ipynb
this is deployed on devseed account
besically you can access the tiles with this url
https://c50qa6bhpe.execute-api.us-west-2.amazonaws.com/scenes/landsat/tiles/{z}/{x}/{y}.{format}?sceneid=LC08....&bands=B4,B3,B2&color_formula=gamma RGB 3.5, saturation 1.7, sigmoidal RGB 15 0.35 (edited)

[abarciauskas-bgse]

Thanks so much @vincentsarago and @olafveerman - it looks like this was a titiler built specifically for specific datasets on AWS, with the path /scenes/landsat. So if we need that still @slesaad we will have to look into redeploying https:/developmentseed/titiler-pds or some other way of integrating those datasets into our existing veda-backend deployment.

I will check with the AWS team in case those datasets are intended for publication to AWS's STAC catalog anytime soon.

[abarciauskas-bgse]

@wildintellect suggested using the landsat collections available from USGS https://stacindex.org/catalogs/usgs-landsat-collection-2-api#/?t=3 which can be explored here: https://landsatlook.usgs.gov/explore.

I don't see the exact product that we were using before in that STAC catalog yet but another product available in that catalog may be appropriate. I think the product being used before was the Level 1 "Terrain Precision Correction (L1TP)" product (https://www.usgs.gov/landsat-missions/landsat-level-1-processing-details) and the only Level 1 product in the USGS catalog appears to be the L1GT product (https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l1/items) - I'm guessing this or many of the other higher level products could also be used for the purposes of this story, but not 100% sure.

@slesaad does that help? Sounds like this story probably requires more work than expected?

[wildintellect]

Level 1 Systematic Terrain (Corrected) (L1GT) product — Includes radiometric and
geometric corrections, and uses a Digital Elevation Model (DEM) to correct parallax

• 2 - LSDS-1822
  Version 6.0
  error due to local topographic relief; the accuracy of the terrain-corrected product
  depends on the resolution of the best available DEM.
  Level 1 Precision Terrain (Corrected) (L1TP) product — Includes radiometric,
  geometric, and precision correction, and uses a DEM to correct parallax errors due to
  local topographic relief; the accuracy of the precision/terrain-corrected product depends
  on the availability of Ground Control Points (GCPs), as well as the resolution of the best
  available DEM

https://www.usgs.gov/media/files/landsat-8-9-olitirs-collection-2-level-1-data-format-control-book
@abarciauskas-bgse is probably right that L1GT should be sufficient for the Dashboard, however L1TP should exist in the catalog too (but may be fewer granules)

Also - what's the goal/story of the layer? L2 data might also work.

[wildintellect]

So I've confirmed L1TP are in the catalog, here's a query body to
https://landsatlook.usgs.gov/stac-server/search

{
	"bbox": [-132.1875, 31.95216223802496, -112.5, 40.97989806962013],
	"collections": ["landsat-c2l1"],
	"query": {
		"landsat:correction":{"in":["L1TP"]},
		"platform": {
			"in": ["LANDSAT_8"]
		},
		"datetime": "2021-10-31T00:00:00.000Z/2023-03-27T23:59:59.999Z",
		"limit": 10,
		"fields": {
			"geometry": false,
			"include": [
				["properties.landsat:wrs_path", "properties.landsat:wrs_row"]
			],
			"exclude": [
				["bbox", "assets", "geometry", "links", "collection", "properties.datetime"]
			]
		}
	}
}

[slesaad]

@abarciauskas-bgse @wildintellect thanks for going into so much detail there! we should be able to publish the necessary stac records from the usgs stac catalog to ours. i'll create a ticket for data services team to do that.

[danielfdsilva]

Given that this discovery is somewhat affected by the missing datasets, perhaps we should block it until we have the datasets in. @slesaad Would you prefer we add it as is and update later, or wait?

[slesaad]

@danielfdsilva let's wait.