-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Area definition html representation for Jupyter notebooks #450
Changes from 63 commits
924627c
af2e9f5
e314e70
c875136
1844e66
1f8bc79
9e59637
e73913b
fb0b407
af779f3
225e86e
f2b0509
7baad8d
45fb3b3
fd1c358
b596a08
fab8e37
9d2fb5d
131afd0
9cdaf7e
d0d8230
affeda9
42eabaa
f88ab85
e0da759
1c03707
6e6fd05
710482d
2d39013
98f83bd
8f3858a
cceb48d
abc2b57
f7acc6c
41d3930
212e14d
b8bcc8b
8112895
810eb86
7b4af63
d44a65a
bf47bdc
67f9df2
fc7ebb4
88acf1e
f95a2a4
858d25d
32c3d38
35877f7
1e49766
79c8ff3
55669d6
89b6663
99dc7ad
2019450
f9a3e5b
e99dabe
05dc307
809057d
22b483a
7d6a9b8
d70f0ea
bf32af3
5191248
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,363 @@ | ||
# Copyright (C) 2023 Pyresample developers | ||
# | ||
# This program is free software: you can redistribute it and/or modify it under | ||
# the terms of the GNU Lesser General Public License as published by the Free | ||
# Software Foundation, either version 3 of the License, or (at your option) any | ||
# later version. | ||
# | ||
# This program is distributed in the hope that it will be useful, but WITHOUT | ||
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS | ||
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more | ||
# details. | ||
# | ||
# You should have received a copy of the GNU Lesser General Public License | ||
# along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
"""Functions for html representation of area definition.""" | ||
|
||
from __future__ import annotations | ||
|
||
import uuid | ||
from functools import lru_cache | ||
from html import escape | ||
from importlib.resources import read_binary | ||
from typing import Literal, Optional, Union | ||
|
||
import numpy as np | ||
|
||
import pyresample.geometry as geom | ||
|
||
try: | ||
import cartopy | ||
except ModuleNotFoundError: | ||
cartopy = None | ||
|
||
try: | ||
import xarray as xr | ||
from xarray.core.formatting_html import _obj_repr, datavar_section | ||
xarray = True | ||
except ModuleNotFoundError: | ||
xarray = False | ||
|
||
|
||
STATIC_FILES = ( | ||
("pyresample.static.html", "icons_svg_inline.html"), | ||
("pyresample.static.css", "style.css"), | ||
) | ||
|
||
|
||
@lru_cache(None) | ||
def _load_static_files(): | ||
Comment on lines
+48
to
+49
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there any reason for this not to be 1? |
||
"""Lazily load the resource files into memory the first time they are needed.""" | ||
return [ | ||
read_binary(package, resource).decode("utf-8") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any idea if There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good question. I don't really know. This is probably something that I would move inline in a refactoring step similar to pytroll/satpy#2171. That would also get rid of the directories which I think are not very nice. |
||
for package, resource in STATIC_FILES | ||
] | ||
|
||
|
||
def _icon(icon_name): | ||
# icon_name should be defined in pyresample/static/html/icon-svg-inline.html | ||
return ( | ||
"<svg class='icon pyresample-{0}'>" | ||
"<use xlink:href='#{0}'>" | ||
"</use>" | ||
"</svg>".format(icon_name) | ||
) | ||
|
||
|
||
def plot_area_def(area: Union['geom.AreaDefinition', 'geom.SwathDefinition'], # noqa F821 | ||
feature_res: Optional[str] = "110m", | ||
fmt: Optional[Literal["svg", "png", None]] = None) -> Union[str, None]: | ||
"""Plot area. | ||
|
||
Args: | ||
area : Area/Swath to plot. | ||
feature_res : | ||
Resolution of the features added to the map. Argument is handed over | ||
to `scale` parameter in cartopy.feature. | ||
fmt : Output format of the plot. The output is the string representation of | ||
the respective format xml for svg and base64 for png. Either svg or png. | ||
If None (default) plot is just shown. | ||
|
||
Returns: | ||
svg or png image as string. | ||
""" | ||
import base64 | ||
from io import BytesIO, StringIO | ||
|
||
import matplotlib.pyplot as plt | ||
|
||
if isinstance(area, geom.AreaDefinition): | ||
crs = area.to_cartopy_crs() | ||
fig, ax = plt.subplots(subplot_kw=dict(projection=crs)) | ||
elif isinstance(area, geom.SwathDefinition): | ||
from shapely.geometry.polygon import Polygon | ||
|
||
lx, ly = area.get_edge_lonlats() | ||
|
||
crs = cartopy.crs.Mercator() | ||
fig, ax = plt.subplots(subplot_kw=dict(projection=crs)) | ||
|
||
poly = Polygon(list(zip(lx[::-1], ly[::-1]))) # make lat/lon counterclockwise for shapely | ||
ax.add_geometries([poly], crs=cartopy.crs.CRS(area.crs), facecolor="none", edgecolor="red") | ||
bounds = poly.buffer(5).bounds | ||
ax.set_extent([bounds[0], bounds[2], bounds[1], bounds[3]], crs=cartopy.crs.CRS(area.crs)) | ||
|
||
coastlines = cartopy.feature.NaturalEarthFeature(category="physical", | ||
name="coastline", | ||
scale=feature_res, | ||
linewidth=1, | ||
facecolor="never") | ||
borders = cartopy.feature.NaturalEarthFeature(category="cultural", | ||
name="admin_0_boundary_lines_land", # noqa E114 | ||
scale=feature_res, | ||
edgecolor="black", | ||
facecolor="never") # noqa E1> | ||
ocean = cartopy.feature.OCEAN | ||
|
||
ax.add_feature(borders) | ||
ax.add_feature(coastlines) | ||
ax.add_feature(ocean, color="lightgrey") | ||
|
||
plt.tight_layout(pad=0) | ||
|
||
if fmt == "svg": | ||
svg_str = StringIO() | ||
plt.savefig(svg_str, format="svg", bbox_inches="tight") | ||
plt.close() | ||
return svg_str.getvalue() | ||
|
||
elif fmt == "png": | ||
png_str = BytesIO() | ||
plt.savefig(png_str, format="png", bbox_inches="tight") | ||
img_str = f"<img src='data:image/png;base64, {base64.encodebytes(png_str.getvalue()).decode('utf-8')}'/>" | ||
plt.close() | ||
return img_str | ||
|
||
else: | ||
plt.show() | ||
return None | ||
|
||
|
||
def collapsible_section(name: str, inline_details: Optional[str] = "", details: Optional[str] = "", | ||
enabled: Optional[bool] = True, collapsed: Optional[bool] = False, | ||
icon: Optional[str] = None) -> str: | ||
"""Create a collapsible section. | ||
|
||
Args: | ||
name : Name of the section | ||
inline_details : Information to show when section is collapsed. Default nothing. | ||
details : Details to show when section is expanded. | ||
enabled : Is collapsing enabled. Default True. | ||
collapsed: Is the section collapsed on first show. Default False. | ||
icon : Icon to use for collapsible section. | ||
|
||
Returns: | ||
Html div structure for collapsible section. | ||
|
||
""" | ||
# "unique" id to expand/collapse the section | ||
data_id = "section-" + str(uuid.uuid4()) | ||
|
||
collapse_enabled = "" if enabled else "disabled" | ||
is_collapsed = "" if collapsed else "checked" | ||
tip = " title='Expand/collapse section'" if enabled else "" | ||
|
||
if icon is None: | ||
icon = _icon("icon-database") | ||
|
||
return ("<div class='pyresample-area-section-item'>" | ||
f"<input id='{data_id}' class='pyresample-area-section-in' " | ||
f"type='checkbox' {collapse_enabled} {is_collapsed}>" | ||
f"<label for='{data_id}' {tip}>{icon} {name}</label>" | ||
f"<div class='pyresample-area-section-preview'>{inline_details}</div>" | ||
f"<div class='pyresample-area-section-details'>{details}</div>" | ||
"</div>" | ||
) | ||
|
||
|
||
def map_section(area: Union['geom.AreaDefinition', 'geom.SwathDefinition']) -> str: # noqa F821 | ||
"""Create html for map section. | ||
|
||
Args: | ||
area : AreaDefinition or SwathDefinition. | ||
|
||
Returns: | ||
Html with collapsible section with a cartopy plot. | ||
|
||
""" | ||
map_icon = _icon("icon-globe") | ||
|
||
if cartopy: | ||
coll = collapsible_section("Map", details=plot_area_def(area, fmt="svg"), collapsed=True, icon=map_icon) | ||
else: | ||
coll = collapsible_section("Map", | ||
details="Note: If cartopy is installed a display of the area can be seen here", | ||
collapsed=True, icon=map_icon) | ||
|
||
return f"{coll}" | ||
|
||
|
||
def proj_area_attrs_section(area: 'geom.AreaDefinition') -> str: # noqa F821 | ||
"""Create html for attribute section based on an area Area. | ||
|
||
Args: | ||
area : Area definition. | ||
|
||
Returns: | ||
Html with collapsible section of attributes of Area. | ||
|
||
""" | ||
resolution_str = "/".join([str(round(x, 1)) for x in area.resolution]) | ||
proj_dict = area.proj_dict | ||
proj_str = "{{{}}}".format(", ".join(["'%s': '%s'" % (str(k), str(proj_dict[k])) for k in | ||
sorted(proj_dict.keys())])) | ||
area_units = proj_dict.get("units", "") | ||
|
||
attrs_icon = _icon("icon-file-text2") | ||
|
||
area_attrs = ("<dl>" | ||
f"<dt>Area name</dt><dd>{area.area_id}</dd>" | ||
f"<dt>Description</dt><dd>{area.description}</dd>" | ||
f"<dt>Projection</dt><dd>{proj_str}</dd>" | ||
f"<dt>Width/Height</dt><dd>{area.width}/{area.height} Pixel</dd>" | ||
f"<dt>Resolution x/y (SSP)</dt><dd>{resolution_str} {area_units}</dd>" | ||
f"<dt>Extent (ll_x, ll_y, ur_x, ur_y)</dt>" | ||
f"<dd>{tuple(round(x, 4) for x in area.area_extent)}</dd>" | ||
"</dl>" | ||
) | ||
|
||
coll = collapsible_section("Properties", details=area_attrs, icon=attrs_icon) | ||
|
||
return f"{coll}" | ||
|
||
|
||
def swath_area_attrs_section(area: 'geom.SwathDefinition') -> str: # noqa F821 | ||
"""Create html for attribute section based on SwathDefinition. | ||
|
||
Args: | ||
area : Swath definition. | ||
|
||
Returns: | ||
Html with collapsible section of swath attributes. | ||
|
||
Todo: | ||
- Improve resolution estimation from lat/lon arrays. Maybe use CoordinateDefinition.geocentric_resolution? | ||
djhoese marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
""" | ||
if isinstance(area.lons, np.ndarray) & isinstance(area.lats, np.ndarray): | ||
# Calculate and estimated resolution from lats/lons in meter | ||
area_name = "Arbitrary Swath" | ||
resolution_y = np.mean(area.lats[0:-1, :] - area.lats[1::, :]) | ||
resolution_x = np.mean(area.lons[:, 1::] - area.lons[:, 0:-1]) | ||
resolution = np.mean(np.array([resolution_x, resolution_y])) | ||
resolution = np.round(40075000 * resolution / 360, 1) | ||
resolution_str = f"{resolution}/{resolution}" | ||
area_units = "m" | ||
else: | ||
lon_attrs = area.lons.attrs | ||
lat_attrs = area.lats.attrs | ||
|
||
# use resolution from lat/lons dataarray attributes -> are these always set? -> Maybe try/except? | ||
area_name = f"{lon_attrs.get('sensor')} swath" | ||
resolution_str = "/".join([str(round(x.get("resolution"), 1)) for x in [lat_attrs, lon_attrs]]) | ||
area_units = "m" | ||
|
||
height, width = area.lons.shape | ||
|
||
attrs_icon = _icon("icon-file-text2") | ||
|
||
area_attrs = ("<dl>" | ||
# f"<dt>Area name</dt><dd>{area_name}</dd>" | ||
f"<dt>Description</dt><dd>{area_name}</dd>" | ||
f"<dt>Width/Height</dt><dd>{width}/{height} Pixel</dd>" | ||
f"<dt>Resolution x/y (SSP)</dt><dd>{resolution_str} {area_units}</dd>" | ||
"</dl>" | ||
) | ||
|
||
if xarray and not isinstance(area.lons, np.ndarray): | ||
ds_dict = {i.attrs['name']: i.rename(i.attrs['name']) for i in [area.lons, area.lats]} | ||
dss = xr.merge(ds_dict.values()) | ||
|
||
area_attrs += _obj_repr(dss, header_components=[""], sections=[datavar_section(dss.data_vars)]) | ||
else: | ||
with np.printoptions(threshold=50): | ||
lons = f"{area.lons}".replace("\n", "<br>") | ||
lats = f"{area.lats}".replace("\n", "<br>") | ||
area_attrs += ("<div class='xr-wrap', style='display:none'>" | ||
"<div class='xr-header'></div>" | ||
"<ul class='xr-sections'>" | ||
"<li class='xr-section-item'>" | ||
"<div class='xr-section-details', style='display:contents'>" # noqa E127 | ||
"<ul class='xr-var-list'>" # noqa E127 | ||
"<li class='xr-var-item'>" # noqa E127 | ||
"<div class='xr-var-name'>" # noqa E127 | ||
"<span>Longitude</span>" | ||
"</div>" | ||
f"<div class=xr-var-preview xr-preview>{lons}</div>" | ||
"</li>" | ||
"<li class='xr-var-item'>" | ||
"<div class='xr-var-name'>" | ||
"<span>Latitude</span>" | ||
"</div>" | ||
f"<div class=xr-var-preview xr-preview>{lats}</div>" | ||
"</li>" | ||
"</ul>" | ||
"</div>" | ||
"</li>" | ||
"</ul>" | ||
"</div>" | ||
"</div>" | ||
) | ||
|
||
coll = collapsible_section("Properties", details=area_attrs, icon=attrs_icon) | ||
|
||
return f"{coll}" | ||
|
||
|
||
def area_repr(area: Union['geom.AreaDefinition', 'geom.SwathDefinition'], include_header: Optional[bool] = True, # noqa F821 | ||
include_static_files: Optional[bool] = True): | ||
"""Return html repr of an AreaDefinition. | ||
|
||
Args: | ||
area : Area definition. | ||
include_header : If true a header with object type will be included in | ||
the html. This is mainly intented for display in Jupyter Notebooks. For the | ||
display in the overview of area definitions for the Satpy documentation this | ||
should be set to false. | ||
include_static_files : Load and include css and html needed for representation. | ||
|
||
Returns: | ||
Html. | ||
|
||
""" | ||
if include_static_files: | ||
icons_svg, css_style = _load_static_files() | ||
html = f"{icons_svg}<style>{css_style}</style>" | ||
else: | ||
html = "" | ||
|
||
obj_type = f"pyresample.{type(area).__name__}" | ||
header = ("<div class='pyresample-header'>" | ||
"<div class='pyresample-obj-type'>" | ||
f"{escape(obj_type)}" | ||
"</div>" | ||
"</div>" | ||
) | ||
|
||
html += (f"<pre class='pyresample-text-repr-fallback'>{escape(repr(area))}</pre>" | ||
"<div class='pyresample-wrap' style='display:none'>" | ||
) | ||
|
||
if include_header: | ||
html += f"{header}" | ||
|
||
html += "<div class='pyresample-area-sections'>" | ||
if isinstance(area, geom.AreaDefinition): | ||
html += proj_area_attrs_section(area) | ||
html += map_section(area) | ||
elif isinstance(area, geom.SwathDefinition): | ||
html += swath_area_attrs_section(area) | ||
|
||
html += "</div>" | ||
|
||
return html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have to use a private function (
_obj_repr
)? How does it differ fromrepr(xarr_obj)
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately yes.
repr(xarr_obj)
produces the string representation which is also used as a fallback/ in the non notebook case. I could just "copy" the function into pyresample and make it non private?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And your usage differs from the public
array_repr
anddataset_repr
in that sameformatting_html
module in xarray (these functions use_obj_repr
)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes because I want to be able to customize the header and the displayed sections (I only want to display the data variables from the whole
xarray.Dataset
representation). Obviously that is a (personal) design choice which I am happy to talk about. I just tried to reuse as much as was already available but we could also implement something of our own.Just to give you an idea of the differences (top:
_obj_repr
, bottom:dataset_repr
):There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very interesting. Yeah I see what you mean. I was going to say maybe we copy that functionality here, but then noticed it is loading the static icon SVGs so that gets awkward too. I guess that function is small too but it feels weird to copy them.
@mraspaud @pnuu any opinions on this for what "feels" right from the above screenshots? The bottom has so much extra "cruft" for things that aren't used, but maybe that is OK since we are just completely depending on upstream xarray's "private" functions. Or... @BENR0 what about a PR to xarray (in the long term) that let's you exclude empty sections? That would still include the Attributes but maybe that's a good thing? I could see it being beneficial to users to see the extra metadata hanging around the swath definition.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using a private function/method from another libraries feels wrong I have to say. I agree that the bottom is maybe too verbose, but having the dimension size explicitly here I think is nice actually.