Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

commented out prefixing of assetId with "projects/earthegine-legacy/assets/" #311

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bbest
Copy link

@bbest bbest commented Jan 12, 2023

Hi @saybar et al,

Thank you for this awesome package! 📦 🚀

I was having trouble uploading vector sf polygons into Earth Engine assets and noticed some odd prefixing of the proposed assetId with "projects/earthengine-legacy/assets/", so I commented that out, and it now works.

Example with my own user-specific paths:

devtools::install_github("ecoquants/offhabr")
if (!require(librarian))
  install.packages("librarian")
librarian::shelf(
  devtools, sf, rgee)
ee_Initialize(gcs = TRUE)

ee_x <- st_read(system.file("shape/nc.shp", package = "sf"))

fc_zones <- rgee::sf_as_ee(
  ee_x, 
  via        = "gcs_to_asset",
  assetId    = "projects/ee-offhab/assets/fc_nc",
  bucket     = "offhab_lyrs",
  proj       = "EPSG:4326",
  monitoring = T,
  quiet      = F)

@bbest
Copy link
Author

bbest commented Jan 13, 2023

So now I see how this is weirdly specified in Asset IDs and names - Manifest Upload  |  Google Earth Engine  |  Google Developers:

The asset name in the manifest needs to be slightly different from the asset ID visible elsewhere in Earth Engine. To upload assets whose asset IDs start with users/some_user or projects/some_project, the asset name in the manifest must have the string projects/earthengine-legacy/assets/ prepended to the ID. For example, EE asset ID users/username/my_geotiff should be uploaded using the name projects/earthengine-legacy/assets/users/username/my_geotiff.

Yes, this means that IDs like projects/some_projects/some_asset get converted into names where projects is mentioned twice: projects/earthengine-legacy/assets/projects/some_projects/some_asset. This is confusing but necessary to conform to the Google Cloud API standards.

However when I updated the path accordingly in the manifest_json it didn't work:

gcs_to_ee_image(
  manifest_json,
  overwrite = T,
  quiet = F)

Result in EE Tasks:

Asset name: projects/earthengine-legacy/assets/projects/ee-offhab/assets/im_zones_v1b
ID: AC2DKC3UHM7OEAVGVYXLUFFS
Phase: Failed
Runtime: 0s (started 2023-01-13 14:21:12 +0100)
Attempted 1 time
Error: Not an assetserver id: 'projects/ee-offhab/assets' (Error code: 3)

So perhaps the GEE policy changed to no longer use that prefix without updating the documentation.

@csaybar
Copy link
Collaborator

csaybar commented Jan 31, 2023

Hi @bbest, sorry for the late reply, and thanks for the PR.

I can reproduce your previous example without errors:

library(rgee)
library(sf)

ee_Initialize(gcs = TRUE)

ee_x <- st_read(system.file("shape/nc.shp", package = "sf"))
#ee_utils_sak_validate(bucket = "rgeedev2")


fc_zones <- sf_as_ee(
  ee_x, 
  via        = "gcs_to_asset",
  assetId    = "users/csaybar/demo2",
  bucket     = "rgeedev2",
  proj       = "EPSG:4326",
  monitoring = T,
  quiet      = F
)

It seems that I'm not considering GEE project users. I will take a deep look on the weekend. Did you find a workaround?. What do you get after running this?:

ee_get_assethome()

@bbest
Copy link
Author

bbest commented Feb 25, 2023

Hi @csaybar,

So sorry for dropping out of touch on this. Yes, I believe you identified the issue with the special treatment of user asset paths (i.e., starting with users/) versus project asset paths (i.e., starting with projects/).

I see how this has a broader systemic implications throughout the code to accomodate. I'd like to help with this, but not sure how soon I can revisit this.

PS Here's the output of:

ee_get_assethome()
[1] "users/ben-ecoquants"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants