Upgrade to edal-wms (ncwms 2.0)

[lesserwhirls]

Upgrade TDS 5.0 to use the new wms server based on Reading-eScience-Centre/edal-java

[lesserwhirls]

Working on this issue on my ncwms2 branch if anyone would like to follow along.

[lesserwhirls]

Ran into a potential bug - submitted issue: Reading-eScience-Centre/edal-java#30

[lesserwhirls]

We are using saxon-he transitively (via waterml), but we should make it an explicit dep. @cwardgar - where should we define this?

[lesserwhirls]

Ok, here is an update:

I added an explicit dependency on saxon-he, and the resource files are now being processed correctly by edal-java.

Here is where things stand at the moment:

In the edal-cdm artifact, CdmUtils uses FeatureDatasetFactoryManager to try to make a FeatureType.GRID from a NetcdfDataset.

A GridDataset object is expect to be created, but a Coverage is created instead. This causes edal-cdm to fail as it is trying to cast the FeatureDataset into a GridDataset object on the return of CdmUtils.getGridDataset.

I am cloning the repository now to try to get a workaround to see if this solves all of the issues. I expect to do a workaround be creating a GridDataset directly instead of using FeatureDatasetFactoryManager.

In the long term, I know it would be best to get edal-java to start using Coverage over GridDataset, but I am just trying to get something working at this point.

[ethanrd]

Just took a quick look at the edal-java repository.

Looks like they are now using Velocity templates instead of JSP for the WMS
XML responses. And it is all under a resource directory. Which I'm guessing
means they are loaded as resources, so there may be nothing on the WMS
side, other than the .jar file, that needs to go in the webapps directory.

It also looks like they are no longer using Spring. Not sure what that
means for our having Spring automatically scan the jars and auto-wire a WMS
controller. Unless we have a TDS branch in our forked repo with a TDS
specific WMS Spring Controller?

I was just too curious not to take a peak.

Cheers,

Ethan

On Thu, Nov 19, 2015 at 10:09 AM, Sean Arms [email protected]
wrote:

Ok, here is an update:

I added an explicit dependency on saxon-he, and the resource files are now
being processed correctly by edal-java.

Here is where things stand at the moment:

In the edal-cdm artifact, CdmUtils uses FeatureDatasetFactoryManager to
try to make a FeatureType.GRID from a NetcdfDataset.

A GridDataset object is expect to be created, but a Coverage is created
instead. This causes edal-cdm to fail as it is trying to cast the
FeatureDataset into a GridDataset object on the return of
CdmUtils.getGridDataset.

I am cloning the repository now to try to get a workaround to see if this
solves all of the issues. I expect to do a workaround be creating a
GridDataset directly instead of using FeatureDatasetFactoryManager.

In the long term, I know it would be best to get edal-java to start using
Coverage over GridDataset, but I am just trying to get something working at
this point.

—
Reply to this email directly or view it on GitHub
#234 (comment).

[lesserwhirls]

Hi Ethan,

Here is where the repo for ncwms2/tds lives:

https:/lesserwhirls/thredds/tree/ncwms2

Once I get something working, I will move the branch to the unidata reop and stop force pushing.

There are two java files (so far) that have been added to the tds (thanks to @guygriffiths!):

ThreddsWmsServlet.java
ThreddsWmsCatalogue.java

We map to the ThreddsWmsServlet via web.xml, although I think this could be done programatically.

For a list of all the changes to THREDDS so far, check out this

I aim to test out the changes I've made to CdmUtils (outlined in #234 (comment)) today and make the appropriate pull request to edal-java. I have high hopes that it will be our last thing to adjust.

[lesserwhirls]

Submitted a PR to edal-java for CdmUtils:

Reading-eScience-Centre/edal-java#39

With this fix, I can now generate a GetCapabilities document from WMS, as well as request an image of the legend using the TDS code in

https:/lesserwhirls/thredds/tree/ncwms2

To be clear, using the default catalogs and data from TDS (while enabling the WMS service in the testEnhanced catalog and threddsConfig.xml), the following works:

http://localhost:8080/thredds/wms/testEnhanced/2004050412_eta_211.nc?service=WMS&version=1.3.0&request=GetCapabilities

http://localhost:8080/thredds/wms/testEnhanced/2004050412_eta_211.nc?VERSION=1.3.0&REQUEST=GetLegendGraphic&COLORBARONLY=true

I have tried to generate an image of a layer, but so far only a blank image is returned (no error on the client or server side). As an example, try

http://localhost:8080/thredds/wms/testEnhanced/2004050412_eta_211.nc?VERSION=1.3.0&REQUEST=GetMap&LAYERS=Z_sfc&WIDTH=640&HEIGHT=320&FORMAT=image/png&STYLES=default-scalar/default&CRS=EPSG:3857&BBOX=-130,20,-32,59

@guygriffiths - any thoughts as to why the image would come back empty? The server does return a 1.3 kB png (I think the background is white...will double check in photoshop) image.

[guygriffiths]

The bounding box needs to be in the specified CRS, which I'm fairly sure is measured in metres for EPSG:3857. The image is probably correct but only of a very small area. Try one of:

http://localhost:8080/thredds/wms/testEnhanced/2004050412_eta_211.nc?VERSION=1.3.0&REQUEST=GetMap&LAYERS=Z_sfc&WIDTH=640&HEIGHT=320&FORMAT=image/png&STYLES=default-scalar/default&CRS=EPSG:3857&BBOX=-14500000,2270000,-3560000,8200000

http://localhost:8080/thredds/wms/testEnhanced/2004050412_eta_211.nc?VERSION=1.3.0&REQUEST=GetMap&LAYERS=Z_sfc&WIDTH=640&HEIGHT=320&FORMAT=image/png&STYLES=default-scalar/default&CRS=CRS:84&BBOX=-130,20,-32,59

[lesserwhirls]

@guygriffiths - awesome, that was my mistake. Here are the images:

1st request:

and here is from the 2nd:

We'll still need to implement caching and a few other things, but the basic edal-wms seems to be working in 5.0 now. Thank you so much @guygriffiths for all of your help!

[lesserwhirls]

Ok, I have a working wms that can use Godiva3 - see #343

Note that to build godiva3, I had to use the following from the base level of the thredds repository:

java -cp /Users/lesserwhirls/.m2/repository/uk/ac/rdg/resc/edal-godiva/1.0.5-SNAPSHOT/edal-godiva-1.0.5-SNAPSHOT.jar:/usr/local/Cellar/gwt/2.7.0/share/*:/Users/lesserwhirls/dev/unidata/thredds/50/tds/src/main/resources/uk/ac/rdg/resc/edal/ncwms/:/Users/lesserwhirls/.m2/repository/org/gwtopenmaps/openlayers/gwt-openlayers-client/1.0/gwt-openlayers-client-1.0.jar com.google.gwt.dev.Compiler Godiva3 -war ./tds/src/main/webapp

Basically, make sure the following are on the class path:

• edal-godiva.jar
• all of the GWT jars
• the location of Godiva3.gwt.xml (tds/src/main/resources/uk/ac/rdg/resc/edal/ncwms/)
• the gwt-openlayers-client.jar

Then, to compile, we use the class com.google.wt.dev.Compiler, tell it to compile Godiva3, and then use the -war flag and tell it where the war files live (this is where the compiler will output the compiled files).

Apparently, this can all be handled in gradle such that the javascript files get compiled during the build (thus eliminating a ton of binary files current under git control), but I could not figure that one out.

At any rate, wms and godiva3 are now working in 5.0. w00t. There are still a few things that we can do, such as:

• set defaults
• setup a cache?

[lesserwhirls]

The basics of this are fixed in /pull/610

The next step, besides bug fixes, will be to setup a cache, set defaults, and make sure users can still make changes to the default properties of layers in a way they used to be able to using wmsConfig.xml.

[rsignell-usgs]

@lesserwhirls, if possible, can we try to get ncwms-2.2.2 into the TDS 5.0 release?

Just released today, v2.2.2 has support for datasets written with SGRID conventions (in addition to UGRID conventions), which means that TDS would be finally be able to deliver WMS layers of velocity arrows/barbs from staggered grid models like ROMS and WRF!

https:/Reading-eScience-Centre/ncwms/releases/tag/ncwms-2.2.2
https:/Reading-eScience-Centre/edal-java/releases/tag/edal-1.2.2

[lesserwhirls]

Sure thing! Let me see what I can do.

[rsignell-usgs]

🎉

[lesserwhirls]

Ok, the new war artifact can be found here. Don't forget to rename it to something like thredds##5.0.0-SNAPSHOT.war.

[rsignell-usgs]

@lesserwhirls are you now using the latest ncWMS2 2.2.5 version, with covjson support?
https:/Reading-eScience-Centre/ncwms/releases

[lesserwhirls]

We're using a snapshot of 2.2.5, but it was the second to last commit before that release. I'll update to the official release here shortly.