AGROVOC does not pull automatically and does not parse anymore

[jonquet]

Since July 2021 version, AGROVOC does not parse anymore.

Plus, we have a pullLocation for AGROVOC :
http://data.agroportal.lirmm.fr/ontologies/AGROVOC/submissions/16?display=pullLocation
But the ontology never get updated automatically (I have to do it manually each month).

January or February release is expected soon to make new tests.

[syphax-bouazzouni]

AGROV issue diagnostic

State of last submission

See status

See logs

AGROV last submission log file can be found at this path : /srv/ontoportal/data/repository/AGROVOC/16/parsing.log

Conclusion from logs

the error is an java.lang.OutOfMemoryError, like what we see in the screenshot above (where we restarted the parsing of the last submission)

[jonquet]

Relevant post about the issue:
https://stackoverflow.com/questions/52712321/outofmemoryerror-when-joining-a-list-of-strings-in-java

It seems the OWL-API tries to create a string too large.

[jonquet]

Error (OutofMemory) reproduced by @jvendetti when parsing (the nq file) "outside" of AgroPortal stack.
Note: the nt file parse.

[jonquet]

An update :

• In Protégé the .nt version opens
• In AgroPortal the .nt version parses and a 877M owlapi.xrdf file is generated. However next parsing step fails because of next error. This situation was reproduced outside of AgroPortal too.

rapper: Serializing with serializer ntriples
rapper: Error -  - XML parser error: Char 0xFFFF out of allowed range
rapper: Error -  - XML parser error: PCDATA invalid Char value 65535
rapper: Failed to parse file /srv/ncbo/repository/AGROVOC/1/owlapi.xrdf rdfxml content
rapper: Parsing returned 8673139 triples

When generating a RDF/XML file with Protégé and re-opening this same file with Protégé the error shows up again, but this time with a line number :

Which bring us to the URI : http://aims.fao.org/aos/agrovoc/xDef_8f48da66

[jonquet]

Fixing the character allow parsing.
We encounter then another issue described in the log :

Probably linked to the recent changes on indexing fields.

[syphax-bouazzouni]

indexing error fixed here ontoportal-lirmm/goo@ba27011

Agrovoc is now parsed, indexed but we had this issue in the diff process #246

So I don't think that the automatic pull will work, to follow up in the future releases here #251