unembargo appears to not update access information

[satra]

see this unembargoed dataset for example: https://dandiarchive.org/dandiset/000871

when unembargoing, the access information field of the metadata should be set to open access.

[yarikoptic]

is this a duplicate of #1787 ?

[waxlamp]

is this a duplicate of #1787 ?

Slightly different, but the same underlying cause (probably). This issue is saying, unembargoing must cause the access field to read "open"; the other is pointing out existing resources with incorrect metadata.

I believe the embargo redesign effort (attn: @jjnesbitt) can address both problems.

[satra]

@waxlamp - can we fix the ones which have this issue now (manually) before redesign happens? i'm assuming this is only in the metadata and somewhere in the database the dandiset is stored as open. this was reported by a user who unembargoed.

[waxlamp]

@waxlamp - can we fix the ones which have this issue now (manually) before redesign happens? i'm assuming this is only in the metadata and somewhere in the database the dandiset is stored as open.

Yes.

this was reported by a user who unembargoed.

Which dandiset?

[satra]

Which dandiset?

the one listed above.

[jjnesbitt]

I believe the embargo redesign effort (attn: @jjnesbitt) can address both problems.

The embargo re-design is mostly focused on fixing the structure of the code base and s3 access w.r.t embargoed models and data. What's being described here is a more of a failing or bug in our un-embargo operations, which is probably out of scope for that change (since it's already pretty complex).

[Ahad-Allen]

This issue affects another one of our dandisets that was unembargoed this month as well: https://dandiarchive.org/dandiset/000253?pos=3

[satra]

@mvandenburgh - could we fix these? and check how many slipped through the cracks?

[waxlamp]

@mvandenburgh - could we fix these? and check how many slipped through the cracks?

Working on this -- should be fixed soon.

[waxlamp]

Here's a listing of open Dandisets that were misreporting their embargo status:

[(<Dandiset: 000068>, 'access field length: 0'),
 (<Dandiset: 000070>, 'access field length: 0'),
 (<Dandiset: 000105>, 'access field length: 0'),
 (<Dandiset: 000114>, 'access field length: 0'),
 (<Dandiset: 000115>, 'access field length: 0'),
 (<Dandiset: 000116>, 'access field length: 0'),
 (<Dandiset: 000117>, 'access field length: 0'),
 (<Dandiset: 000121>, 'access field length: 0'),
 (<Dandiset: 000122>, 'access field length: 0'),
 (<Dandiset: 000123>, 'access field length: 0'),
 (<Dandiset: 000125>, 'access field length: 0'),
 (<Dandiset: 000126>, 'access field length: 0'),
 (<Dandiset: 000141>, 'access field length: 0'),
 (<Dandiset: 000143>, 'access field length: 0'),
 (<Dandiset: 000147>, 'access field length: 0'),
 (<Dandiset: 000165>, 'access field length: 0'),
 (<Dandiset: 000166>, 'access field length: 0'),
 (<Dandiset: 000487>, 'access mismatch'),
 (<Dandiset: 000537>, 'access mismatch'),
 (<Dandiset: 000871>, 'access mismatch')]

access field length: 0 means those Dandisets had an empty list as their access metadata; access mismatch means they had access metadata indicating the Dandiset was still embargoed. With @mvandenburgh's help I solved both problems.

To log how we solved this, take a look at this gist.

[waxlamp]

Question for @satra and @yarikoptic: why is the access record a list? Should it not just be the single object that often occupies that list?

[satra]

why is the access record a list? Should it not just be the single object that often occupies that list?

we wanted to potentially allow for mixed access at the dandiset level (e.g. a combination of open and restricted access, etc.,.) eventually that field would simply be a union of the assets contained in it.

[Ahad-Allen]

Here's a listing of open Dandisets that were misreporting their embargo status:

[(<Dandiset: 000068>, 'access field length: 0'),
 (<Dandiset: 000070>, 'access field length: 0'),
 (<Dandiset: 000105>, 'access field length: 0'),
 (<Dandiset: 000114>, 'access field length: 0'),
 (<Dandiset: 000115>, 'access field length: 0'),
 (<Dandiset: 000116>, 'access field length: 0'),
 (<Dandiset: 000117>, 'access field length: 0'),
 (<Dandiset: 000121>, 'access field length: 0'),
 (<Dandiset: 000122>, 'access field length: 0'),
 (<Dandiset: 000123>, 'access field length: 0'),
 (<Dandiset: 000125>, 'access field length: 0'),
 (<Dandiset: 000126>, 'access field length: 0'),
 (<Dandiset: 000141>, 'access field length: 0'),
 (<Dandiset: 000143>, 'access field length: 0'),
 (<Dandiset: 000147>, 'access field length: 0'),
 (<Dandiset: 000165>, 'access field length: 0'),
 (<Dandiset: 000166>, 'access field length: 0'),
 (<Dandiset: 000487>, 'access mismatch'),
 (<Dandiset: 000537>, 'access mismatch'),
 (<Dandiset: 000871>, 'access mismatch')]

access field length: 0 means those Dandisets had an empty list as their access metadata; access mismatch means they had access metadata indicating the Dandiset was still embargoed. With @mvandenburgh's help I solved both problems.

To log how we solved this, take a look at this gist.

Just wanted to check in on this issue, as it seems like one of our dandisets is still suffering from this issue and has an id that was not listed in this report: https://dandiarchive.org/dandiset/000253?pos=3. Thank you for all the help :)

[mvandenburgh]

@Ahad-Allen your dandiset is fixed now. Sorry for the confusion.

[dandibot]

🚀 Issue was released in v0.3.86 🚀