Add logDirectory option to copy logs to WACZ

[tw4l]

Fixes #118

Thanks for taking a look, this is one of the last things needed for Browsertrix Crawler!

[matteocargnelutti]

@tw4l This looks great as usual 🤝 !

One question and one request:

• Is there a common pattern for these log files (extension, content, etc ...)? I would be interested in making that feature less permissive, so that we don't have a "add whatever to WACZ" flag 😅 .
• Would you mind adding docs for this new option to types.js so the IDE / JSDoc knows about this new option?

Thank you!

[tw4l]

* Is there a common pattern for these log files (extension, content, etc ...)? I would be interested in making that feature less permissive, so that we don't have a "add whatever to WACZ" flag 😅 .

The log files from Browsertrix Crawler are JSON-L files with .log as the extension - happy to add extra validation there. Perhaps we can limit to log or txt extensions for now? I think if there was an issue with the correctness of the log files (not that I've seen that happen, but you never know) it'd still be better to include a file with a formatting issue rather than no logs.

* Would you mind adding docs for this new option to `types.js` so the IDE / JSDoc knows about this new option?

Happy to!

[matteocargnelutti]

@tw4l

The log files from Browsertrix Crawler are JSON-L files with .log as the extension - happy to add extra validation there. Perhaps we can limit to log or txt extensions for now? I think if there was an issue with the correctness of the log files (not that I've seen that happen, but you never know) it'd still be better to include a file with a formatting issue rather than no logs.

That makes sense. Let's start by only accepting .log and .txt files. Thank you :)

[matteocargnelutti]

@tw4l Nitpick: Let's use logDir for consistency with pagesDir and the like :)

[tw4l]

Done! I kept the option for class initialization as logDirectory and CLI flag as log-directory for compatibility with py-wacz but internally it now uses this.logDir

[tw4l]

Feel free to change the former though, I don't have a strong opinion, as long as the CLI flag is the same I think we're good :)

[tw4l]

Rebased and updated, thanks!

[matteocargnelutti]

Hey there @tw4l! This is great, thanks!
I pushed a commit to suggest an updated naming convention for all options that expect / accept a path to dir - what do you think?

Cheers,

[tw4l]

Hey there @tw4l! This is great, thanks! I pushed a commit to suggest an updated naming convention for all options that expect / accept a path to dir - what do you think?

Hi @matteocargnelutti , makes sense to me! The consistency is nice :) Thanks for this!