Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make_xarray_grid fails after passing DataArrays #330

Closed
santisoler opened this issue Aug 30, 2021 · 3 comments
Closed

make_xarray_grid fails after passing DataArrays #330

santisoler opened this issue Aug 30, 2021 · 3 comments
Labels
bug Report a problem that needs to be fixed enhancement Idea or request for a new feature

Comments

@santisoler
Copy link
Member

santisoler commented Aug 30, 2021

Description of the problem

When passing xarray objects as arguments to vd.make_xarray_grid we end up on some xarray errors down the line.

Full code that generated the error

import numpy as np
import xarray as xr
import verde as vd


x = np.linspace(0, 10, 11)
y = np.linspace(-4, 4, 8)
xx, yy = np.meshgrid(x, y)
data = xx + yy

dims = ("y", "x")
coords = {"x": x, "y": y}
da = xr.DataArray(data, coords=coords, dims=dims)

vd.make_xarray_grid(coordinates=(da.x, da.y), data_names="data", data=da)

Full error message

Traceback (most recent call last):
  File "/home/santi/verde_bug.py", line 15, in <module>
    vd.make_xarray_grid(coordinates=(da.x, da.y), data_names="data", data=da)
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/verde/utils.py", line 370, in make_xarray_grid
    return xr.Dataset(data_vars, coords)
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/xarray/core/dataset.py", line 754, in __init__
    variables, coord_names, dims, indexes, _ = merge_data_and_coords(
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/xarray/core/merge.py", line 484, in merge_data_and_coords
    return merge_core(
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/xarray/core/merge.py", line 632, in merge_core
    collected = collect_variables_and_indexes(aligned)
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/xarray/core/merge.py", line 294, in collect_variables_and_indexes
    variable = as_variable(variable, name=name)
  File "/home/santi/.miniforge3/envs/default/lib/python3.9/site-packages/xarray/core/variable.py", line 121, in as_variable
    raise TypeError(
TypeError: Using a DataArray object to construct a variable is ambiguous, please extract the data using the .data property.

System information

  • Operating system: Manjaro XFCE amd64
  • Python installation (Anaconda, system, ETS): Miniforge
  • Version of Python: 3.9.5
  • Version of this package: 1.6.1
  • Version of xarray: 0.19.0
  • If using conda, paste the output of conda list below:
output of conda list

packages in environment at /home/santi/.miniforge3/envs/default:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
affine 2.3.0 py_0 conda-forge
aiohttp 3.7.4 py39h3811e60_0 conda-forge
alsa-lib 1.2.3 h516909a_0 conda-forge
anyio 3.0.1 py39hf3d152e_0 conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
argon2-cffi 20.1.0 py39h3811e60_2 conda-forge
async-timeout 3.0.1 py_1000 conda-forge
async_generator 1.10 py_0 conda-forge
attrs 21.2.0 pyhd8ed1ab_0 conda-forge
babel 2.9.1 pyh44b312d_0 conda-forge
babelfish 0.5.5 pypi_0 pypi
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
beautifulsoup4 4.9.3 pypi_0 pypi
black 21.7b0 pyhd8ed1ab_0 conda-forge
bleach 3.3.0 pyh44b312d_0 conda-forge
bokeh 2.3.2 py39hf3d152e_0 conda-forge
boost-cpp 1.74.0 hc6e9bd1_3 conda-forge
boule 0.2.0 py_0 conda-forge
brotlipy 0.7.0 py39h3811e60_1001 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.17.1 h7f98852_1 conda-forge
ca-certificates 2021.5.30 ha878542_0 conda-forge
cairo 1.16.0 h6cf1ce9_1008 conda-forge
cartopy 0.19.0.post1 py39h3b23250_0 conda-forge
certifi 2021.5.30 py39hf3d152e_0 conda-forge
cffi 1.14.5 py39he32792d_0 conda-forge
cfitsio 3.470 hb418390_7 conda-forge
cftime 1.4.1 py39hce5d2b2_0 conda-forge
chardet 4.0.0 py39hf3d152e_1 conda-forge
click 7.1.2 pyh9f0ad1d_0 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.7.1 pyhd8ed1ab_0 conda-forge
cloudpickle 1.6.0 py_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
cryptography 3.4.7 py39hbca0aa6_0 conda-forge
curl 7.76.1 hea6ffbf_2 conda-forge
cycler 0.10.0 py_2 conda-forge
cytoolz 0.11.0 py39h3811e60_3 conda-forge
dask 2021.5.0 pyhd8ed1ab_0 conda-forge
dask-core 2021.5.0 pyhd8ed1ab_0 conda-forge
dask-labextension 5.1.0 pyhd8ed1ab_0 conda-forge
dataclasses 0.8 pyhc8e2a94_1 conda-forge
dbus 1.13.6 h48d8840_2 conda-forge
decorator 5.0.9 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
distributed 2021.5.0 py39hf3d152e_0 conda-forge
docrep 0.3.2 pyh44b312d_0 conda-forge
entrypoints 0.3 pyhd8ed1ab_1003 conda-forge
expat 2.3.0 h9c3ff4c_0 conda-forge
flake8 3.9.2 pyhd8ed1ab_0 conda-forge
fontconfig 2.13.1 hba837de_1005 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
freexl 1.0.6 h7f98852_0 conda-forge
fsspec 2021.5.0 pyhd8ed1ab_0 conda-forge
future 0.18.2 py39hf3d152e_3 conda-forge
geos 3.9.1 h9c3ff4c_2 conda-forge
geotiff 1.6.0 h2b14fbe_4 conda-forge
gettext 0.19.8.1 h0b5b191_1005 conda-forge
giflib 5.2.1 h36c2ea0_2 conda-forge
gitdb 4.0.7 pyhd8ed1ab_0 conda-forge
gitpython 3.1.17 pyhd8ed1ab_0 conda-forge
glib 2.68.2 h9c3ff4c_0 conda-forge
glib-tools 2.68.2 h9c3ff4c_0 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
gmpy2 2.1.0b1 py39hf9ed83c_1 conda-forge
gst-plugins-base 1.18.4 hf529b03_2 conda-forge
gstreamer 1.18.4 h76c114f_2 conda-forge
guessit 3.3.1 pypi_0 pypi
harmonica 0.2.1 pyhd8ed1ab_0 conda-forge
hdf4 4.2.15 h10796ff_2 conda-forge
hdf5 1.10.6 nompi_h6a2412b_1114 conda-forge
heapdict 1.0.1 py_0 conda-forge
icu 68.1 h58526e2_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
importlib-metadata 4.0.1 py39hf3d152e_0 conda-forge
ipykernel 5.5.5 py39hef51801_0 conda-forge
ipython 7.27.0 py39hef51801_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.6.3 pyhd3deb0d_0 conda-forge
jedi 0.18.0 py39hf3d152e_2 conda-forge
jinja2 3.0.0 pyhd8ed1ab_0 conda-forge
joblib 1.0.1 pyhd8ed1ab_0 conda-forge
jpeg 9d h36c2ea0_0 conda-forge
json-c 0.15 h98cffda_0 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 3.2.0 pyhd8ed1ab_3 conda-forge
jupyter 1.0.0 py39hf3d152e_6 conda-forge
jupyter-server-mathjax 0.2.2 pyhd8ed1ab_0 conda-forge
jupyter-server-proxy 3.0.2 pyhd8ed1ab_0 conda-forge
jupyter_client 6.1.12 pyhd8ed1ab_0 conda-forge
jupyter_console 6.4.0 pyhd8ed1ab_0 conda-forge
jupyter_core 4.7.1 py39hf3d152e_0 conda-forge
jupyter_server 1.7.0 py39hf3d152e_1 conda-forge
jupyterlab 3.1.9 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
jupyterlab_server 2.5.2 pyhd8ed1ab_0 conda-forge
jupyterlab_widgets 1.0.0 pyhd8ed1ab_1 conda-forge
jupytext 1.11.4 pyhd0ecf6b_0 conda-forge
kealib 1.4.14 hcc255d8_2 conda-forge
kiwisolver 1.3.1 py39h1a9c180_1 conda-forge
krb5 1.19.1 hcc1bbae_0 conda-forge
lcms2 2.12 hddcbb42_0 conda-forge
ld_impl_linux-64 2.35.1 hea4e1c9_2 conda-forge
libblas 3.9.0 9_openblas conda-forge
libcblas 3.9.0 9_openblas conda-forge
libclang 11.1.0 default_ha53f305_1 conda-forge
libcurl 7.76.1 h2574ce0_2 conda-forge
libdap4 3.20.6 hd7c4107_2 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.10 hcdb4288_3 conda-forge
libffi 3.3 h58526e2_2 conda-forge
libgcc-ng 9.3.0 h2828fa1_19 conda-forge
libgdal 3.2.1 h38ff51b_7 conda-forge
libgfortran-ng 9.3.0 hff62375_19 conda-forge
libgfortran5 9.3.0 hff62375_19 conda-forge
libglib 2.68.2 h3e27bee_0 conda-forge
libgomp 9.3.0 h2828fa1_19 conda-forge
libiconv 1.16 h516909a_0 conda-forge
libkml 1.3.0 h238a007_1013 conda-forge
liblapack 3.9.0 9_openblas conda-forge
libllvm10 10.0.1 he513fc3_3 conda-forge
libllvm11 11.1.0 hf817b99_2 conda-forge
libnetcdf 4.7.4 nompi_h56d31a8_107 conda-forge
libnghttp2 1.43.0 h812cca2_0 conda-forge
libogg 1.3.4 h7f98852_1 conda-forge
libopenblas 0.3.15 pthreads_h8fe5266_1 conda-forge
libopus 1.3.1 h7f98852_1 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libpq 13.3 hd57d9b9_0 conda-forge
librttopo 1.1.0 h1185371_6 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libspatialite 5.0.1 he52d314_3 conda-forge
libssh2 1.9.0 ha56f1ee_6 conda-forge
libstdcxx-ng 9.3.0 h6de172a_19 conda-forge
libtiff 4.2.0 hbd63e13_2 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libuv 1.41.0 h7f98852_0 conda-forge
libvorbis 1.3.7 h9c3ff4c_0 conda-forge
libwebp-base 1.2.0 h7f98852_2 conda-forge
libxcb 1.13 h7f98852_1003 conda-forge
libxkbcommon 1.0.3 he3ba5ed_0 conda-forge
libxml2 2.9.12 h72842e0_0 conda-forge
livereload 2.6.3 pyh9f0ad1d_0 conda-forge
llvmlite 0.36.0 py39h1bbdace_0 conda-forge
locket 0.2.0 py_2 conda-forge
lz4-c 1.9.3 h9c3ff4c_0 conda-forge
markdown-it-py 1.1.0 pyhd8ed1ab_0 conda-forge
markupsafe 2.0.0 py39h3811e60_0 conda-forge
matplotlib 3.4.2 py39hf3d152e_0 conda-forge
matplotlib-base 3.4.2 py39h2fa2bec_0 conda-forge
matplotlib-inline 0.1.2 pyhd8ed1ab_2 conda-forge
mccabe 0.6.1 py_1 conda-forge
mdit-py-plugins 0.2.8 pyhd8ed1ab_0 conda-forge
mistune 0.8.4 py39h3811e60_1003 conda-forge
mpc 1.1.0 h04dde30_1009 conda-forge
mpfr 4.0.2 he80fd80_1 conda-forge
mpmath 1.2.1 pyhd8ed1ab_0 conda-forge
msgpack-python 1.0.2 py39h1a9c180_1 conda-forge
multidict 5.1.0 py39h3811e60_1 conda-forge
mypy_extensions 0.4.3 py39hf3d152e_3 conda-forge
mysql-common 8.0.23 ha770c72_2 conda-forge
mysql-libs 8.0.23 h935591d_2 conda-forge
nb_conda 2.2.1 py39hf3d152e_4 conda-forge
nb_conda_kernels 2.3.1 py39hf3d152e_0 conda-forge
nbclassic 0.2.8 pyhd8ed1ab_0 conda-forge
nbclient 0.5.3 pyhd8ed1ab_0 conda-forge
nbconvert 6.0.7 py39hf3d152e_3 conda-forge
nbdime 3.1.0 pyhd8ed1ab_0 conda-forge
nbformat 5.1.3 pyhd8ed1ab_0 conda-forge
ncurses 6.2 h58526e2_4 conda-forge
nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge
netcdf4 1.5.6 nompi_py39h36800e2_102 conda-forge
nodejs 15.14.0 h92b4a50_0 conda-forge
notebook 6.4.0 pyha770c72_0 conda-forge
nspr 4.30 h9c3ff4c_0 conda-forge
nss 3.65 hb5efdd6_0 conda-forge
numba 0.53.1 py39h56b8d98_1 conda-forge
numpy 1.21.1 py39hdbf815f_0 conda-forge
olefile 0.46 pyh9f0ad1d_1 conda-forge
openjpeg 2.4.0 hb52868f_1 conda-forge
openssl 1.1.1k h7f98852_0 conda-forge
packaging 20.9 pyh44b312d_0 conda-forge
pandas 1.2.4 py39hde0f152_0 conda-forge
pandoc 2.13 h7f98852_0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.8.2 pyhd8ed1ab_0 conda-forge
partd 1.2.0 pyhd8ed1ab_0 conda-forge
pathspec 0.8.1 pyhd3deb0d_0 conda-forge
pcre 8.44 he1b5a44_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 8.2.0 py39hf95b381_1 conda-forge
pip 21.2.4 pyhd8ed1ab_0 conda-forge
pixman 0.40.0 h36c2ea0_0 conda-forge
pooch 1.5.1 pyhd8ed1ab_0 conda-forge
poppler 0.89.0 h2de54a5_5 conda-forge
poppler-data 0.4.10 0 conda-forge
postgresql 13.3 h2510834_0 conda-forge
proj 7.2.0 h277dcde_2 conda-forge
prometheus_client 0.10.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.18 pyha770c72_0 conda-forge
prompt_toolkit 3.0.18 hd8ed1ab_0 conda-forge
psutil 5.8.0 py39h3811e60_1 conda-forge
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pycodestyle 2.7.0 pyhd8ed1ab_0 conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pyflakes 2.3.1 pyhd8ed1ab_0 conda-forge
pygments 2.9.0 pyhd8ed1ab_0 conda-forge
pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pyproj 3.1.0 py39h50a64a9_3 conda-forge
pyqt 5.12.3 py39hf3d152e_7 conda-forge
pyqt-impl 5.12.3 py39h0fcd23e_7 conda-forge
pyqt5-sip 4.19.18 py39he80948d_7 conda-forge
pyqtchart 5.12 py39h0fcd23e_7 conda-forge
pyqtwebengine 5.12.1 py39h0fcd23e_7 conda-forge
pyrsistent 0.17.3 py39h3811e60_2 conda-forge
pyshp 2.1.3 pyh44b312d_0 conda-forge
pysocks 1.7.1 py39hf3d152e_3 conda-forge
python 3.9.5 h49503c6_0_cpython conda-forge
python-dateutil 2.8.1 py_0 conda-forge
python_abi 3.9 1_cp39 conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
pyyaml 5.4.1 py39h3811e60_0 conda-forge
pyzmq 22.0.3 py39h37b5a0c_1 conda-forge
qt 5.12.9 hda022c4_4 conda-forge
qtconsole 5.1.0 pyhd8ed1ab_0 conda-forge
qtpy 1.9.0 py_0 conda-forge
rasterio 1.2.3 py39h63973eb_0 conda-forge
readline 8.1 h46c0cb4_0 conda-forge
rebulk 3.0.1 pypi_0 pypi
regex 2021.4.4 py39h3811e60_0 conda-forge
requests 2.25.1 pyhd3deb0d_0 conda-forge
requests-cache 0.5.2 pypi_0 pypi
rockhound 0.2.0 py_0 conda-forge
rope 0.19.0 pyhd8ed1ab_0 conda-forge
scikit-learn 0.24.2 py39h4dfa638_0 conda-forge
scipy 1.7.0 py39hee8e79c_1 conda-forge
send2trash 1.5.0 py_0 conda-forge
setuptools 49.6.0 py39hf3d152e_3 conda-forge
shapely 1.7.1 py39ha61afbd_4 conda-forge
simpervisor 0.4 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
smmap 3.0.5 pyh44b312d_0 conda-forge
sniffio 1.2.0 py39hf3d152e_1 conda-forge
snuggs 1.4.7 py_0 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
soupsieve 2.2.1 pypi_0 pypi
sqlite 3.35.5 h74cdb3f_0 conda-forge
subdivx-download 0.6 pypi_0 pypi
sympy 1.8 py39hf3d152e_0 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
terminado 0.9.4 py39hf3d152e_0 conda-forge
testpath 0.5.0 pyhd8ed1ab_0 conda-forge
threadpoolctl 2.1.0 pyh5ca1d4c_0 conda-forge
tiledb 2.2.9 h91fcb0e_0 conda-forge
tk 8.6.10 h21135ba_1 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomli 1.2.1 pyhd8ed1ab_0 conda-forge
toolz 0.11.1 py_0 conda-forge
tornado 6.1 py39h3811e60_1 conda-forge
traitlets 5.0.5 py_0 conda-forge
tvdb-api 3.1.0 pypi_0 pypi
tvnamer 3.0.4 pypi_0 pypi
typed-ast 1.4.3 py39h3811e60_0 conda-forge
typing-extensions 3.7.4.3 0 conda-forge
typing_extensions 3.7.4.3 py_0 conda-forge
tzcode 2021a h7f98852_1 conda-forge
tzdata 2021a he74cb21_0 conda-forge
urllib3 1.26.4 pyhd8ed1ab_0 conda-forge
verde 1.6.1 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 0.57.0 py39hf3d152e_4 conda-forge
wheel 0.36.2 pyhd3deb0d_0 conda-forge
widgetsnbextension 3.5.1 py39hf3d152e_4 conda-forge
xarray 0.19.0 pyhd8ed1ab_1 conda-forge
xerces-c 3.2.3 h9d8b166_2 conda-forge
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.7.0 h7f98852_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h7f98852_1 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xrft 0.3.0 pyhd8ed1ab_0 conda-forge
xz 5.2.5 h516909a_1 conda-forge
yaml 0.2.5 h516909a_0 conda-forge
yarl 1.6.3 py39h3811e60_1 conda-forge
zeromq 4.3.4 h9c3ff4c_0 conda-forge
zict 2.0.0 py_0 conda-forge
zipp 3.4.1 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h516909a_1010 conda-forge
zstd 1.4.9 ha95c52a_0 conda-forge

@leouieda
Copy link
Member

leouieda commented Sep 3, 2021

That does feel like something we should handle automatically. What happens if you always run the variable through np.asarray? It only copies when it needs to and ideally DataArrays should support this conversion without a copy.

@leouieda leouieda added bug Report a problem that needs to be fixed enhancement Idea or request for a new feature labels Sep 3, 2021
@santisoler
Copy link
Member Author

I've been rethinking about this issue and concluded that I blamed Verde too quickly for a bug in Harmonica.
I think that the make_xarray_grid is intended to be used on a Numpy -> Xarray workflow, so letting it accept xarray objects doesn't sound too clever. In fact, Xarray objects offers more robust ways to merge xarray objects into a single Dataset.

The origin of the bug in Harmonica is on the prism_layer function, which is intended to work both with Numpy arrays and with Xarray objects. I'll fix that in a PR in Harmonica.

For this reason I'm closing this issue. If you or anyone thinks otherwise, please feel free to comment and/or reopen it if it's necessary.

@santisoler
Copy link
Member Author

See fatiando/harmonica#243

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Report a problem that needs to be fixed enhancement Idea or request for a new feature
Projects
None yet
Development

No branches or pull requests

2 participants