Add an option to bin/fetch to exclude bodies.

[dracos]

No description provided.

[codecov]

Codecov Report

Merging #3804 (c78168f) into master (c626fa7) will decrease coverage by 22.65%.
The diff coverage is 50.00%.

❗ Current head c78168f differs from pull request most recent head 64a1a8c. Consider uploading reports for the commit 64a1a8c to get more accurate results

@@             Coverage Diff             @@
##           master    #3804       +/-   ##
===========================================
- Coverage   82.33%   59.68%   -22.66%     
===========================================
  Files         352      284       -68     
  Lines       23969    19381     -4588     
  Branches     3650     3643        -7     
===========================================
- Hits        19736    11568     -8168     
- Misses       3086     6593     +3507     
- Partials     1147     1220       +73     

Impacted Files	Coverage Δ
perllib/Open311/GetServiceRequests.pm	78.94% <0.00%> (-2.14%)	⬇️
perllib/Open311/UpdatesBase.pm	86.44% <100.00%> (+0.23%)	⬆️
perllib/FixMyStreet/App/Form/Waste/Garden.pm	11.11% <0.00%> (-88.89%)	⬇️
perllib/FixMyStreet/App/Form/Wizard.pm	5.88% <0.00%> (-88.24%)	⬇️
perllib/FixMyStreet/DB/Result/AdminLog.pm	8.16% <0.00%> (-85.72%)	⬇️
perllib/FixMyStreet/App/Controller/Auth/Profile.pm	15.96% <0.00%> (-82.36%)	⬇️
perllib/FixMyStreet/App/Controller/Admin/Roles.pm	16.36% <0.00%> (-81.82%)	⬇️
.../FixMyStreet/App/Controller/Admin/ManifestTheme.pm	18.18% <0.00%> (-81.82%)	⬇️
...erllib/FixMyStreet/App/Controller/Questionnaire.pm	11.45% <0.00%> (-80.16%)	⬇️
...ib/FixMyStreet/DB/Result/ModerationOriginalData.pm	10.00% <0.00%> (-78.75%)	⬇️
... and 188 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c626fa7...64a1a8c. Read the comment docs.

[chrismytton]

I'm afraid this isn't working as expected.

The option in bin/fetch is still called body, but the two .pm scripts are now looking for a bodies option. Similarly the exclude option isn't being passed through to the scripts at all, and would need to be called bodies_exclude at the point it was passed through.

Hopefully easy to fix both of those issues in the my %params declaration.

Would also be good to have a test for the bodies_exclude parameter, if that's not too tricky.

[dracos]

Not sure what I was thinking there! Thanks, fixup pushed with test.

[chrismytton]

Test looks good!

However I'm seeing errors when I run bin/fetch. I think the options passed from bin/fetch need some default values and/or the scripts need to check for undef and handle that correctly.

$ bin/fetch --body 'Rutland County Council' -v --reports
Can't use an undefined value as an ARRAY reference at perllib/Open311/GetServiceRequests.pm line 34.

$ bin/fetch --exclude 'Rutland County Council' -v --reports
Can't use an undefined value as an ARRAY reference at /Users/chrismytton/Code/mysociety/fixmystreet/perllib/Open311/GetServiceRequests.pm line 31.

[chrismytton]

As discussed in Slack it would ultimately be good to push more of the logic in bin/fetch into files in perllib. But that can be done as a separate change. This looks good to me! 🚀