Provenance Files

vol = CloudVolume(...)
vol.provenance.description = "Male Mouse, 3 day old. Test collection of S1."
vol.provenance.owners = [ '[email protected]', '[email protected]', '[email protected]' ] 
vol.commit_provenance()

After the passage of time, it can be difficult to remember why a data layer was created and what parameters were used in its creation. For newcomers, they may not even know whom to ask questions of.

provenance files fill this gap by providing easily discoverable (it's next to the info file) machine and human readable JSON metadata about the data layer. CloudVolume provides facilities for editing these files via the vol.provenance attribute.

Provenance files are read using the json5 library which is less strict than standard JSON to make manual editing easier.

JSON Structure

{
    "description": "",
    "sources": [],
    "owners": [],
    "processing: [],
}

Field	Type	Explanation
description	String	What is this dataset? Why did you make this layer? What organism?
sources	List of Strings	What datasets was this layer derived from? e.g. segmented from which image?
owners	List of Strings	Contact emails for questions.
processing	List of Dicts	What operations have been applied?

Example

# gs://seunglab-test/test_v0/image/provenance
{
  "description": "Test dataset based on SNEMI3D. Do not use for science.",
  "owners": [
    "[email protected]",
  ],
  "processing": [
    {
      "by": "[email protected]",
      "date": "2018-04-09 19:07 EDT",
      "method": {
        "axis": "z",
        "method": "downsample_with_averaging",
        "mip": 0,
        "shape": [
          2048,
          2048,
          50
        ],
        "task": "DownsampleTask"
      }
    },
    {
      "by": "[email protected]",
      "date": "2018-04-27 02:01 EDT",
      "method": {
        "coverage_factor": 0.1,
        "offset": [
          0,
          0,
          0
        ],
        "shape": [
          1024,
          1024,
          100,
          1
        ],
        "src": "gs://seunglab-test/test_v0/image",
        "task": "LuminanceLevelsTask"
      }
    }
  ],
  "sources": []
}