enable pullLocation to Agrovoc

[syphax-bouazzouni]

No description provided.

[jonquet]

Uploaded new version of AGROVOC (April) with the pull URL today.
We will see early may if this work now.

[syphax-bouazzouni]

It parsed well, it didn't do the DIFF process (see #266)

[syphax-bouazzouni]

Issue report

The issue is that AROVOC could not get updated automatically by the pull location, and after investigation, it seems that it is related to the fact that the source file is a zip file.

I, [2022-05-17T18:00:28.946892 #3504]  INFO -- : New file found for AGROVOC
old: 8c8fd7637484d8b585cea315c959d18f
new: 04a716346d8c7f6b3a489b82f03c5995
E, [2022-05-17T18:00:30.643292 #3504] ERROR -- : The new file for ontology AGROVOC, submission id: 23 did not clear OWLAPI: LinkedData
::Parser::MasterFileMissingException: Master file not provided and input is zipped archive.
/srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/bundler/gems/ontologies_linked_data-8b60742983df/lib/ontologies_linked_
data/parser/owlapi.rb:35:in `setup_environment'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/bundler/gems/ontologies_linked_data-8b60742983df/lib/ontologies
_linked_data/parser/owlapi.rb:153:in `parse'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/ontology_pull.rb:140:in `create_submission'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/ontology_pull.rb:74:in `block in do_remote_ontolog
y_pull'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/ontology_pull.rb:29:in `each'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/ontology_pull.rb:29:in `do_remote_ontology_pull'
        /srv/ontoportal/ncbo_cron/bin/ncbo_cron:273:in `block (3 levels) in <main>'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/scheduler.rb:66:in `block (3 levels) in scheduled_
locking_job'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/scheduler.rb:52:in `fork'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/scheduler.rb:52:in `block (2 levels) in scheduled_
locking_job'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/gems/mlanett-redis-lock-0.2.7/lib/redis-lock.rb:43:in `lock'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/gems/mlanett-redis-lock-0.2.7/lib/redis-lock.rb:234:in `lock'
        /srv/ontoportal/ncbo_cron_deployments/releases/20220516115752/lib/ncbo_cron/scheduler.rb:51:in `block in scheduled_locking_job
'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `trigge
r_block'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:204:in `block 
in trigger'
        /srv/ontoportal/ncbo_cron_deployments/shared/bundle/ruby/2.6.0/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `b
lock in trigger_job'
E, [2022-05-17T18:00:30.643384 #3504] ERROR -- : A new submission has NOT been created.

The cause

It seems that it can't find the master file name and it's because before creating a new submission we check if OWLAPI is able to parse the new file. But this test seems to not work for ZIP/GZIP files.

The solution

Is to update the "isParsable? test" to re-use the same function that is used in parsing step

Related

• AGROVOC submissions api endpoint is not responding  #271

[syphax-bouazzouni]

Fixed with ncbo/ncbo_cron#55