Use an optional .mdbookignore file to specify files to not copy to the output directory

[andrewdavidmackenzie]

Current Behaviour

Currently almost all files in the source directory are copied to the output directory, often copying unneeded or undesired files, which maybe uploaded to gh-pages or otherwise "published"

Proposed Change

Allow an optional “.mdbookignore” file (like .gitignore) to allow specifying patterns of files to ignore and hence NOT get copied during the build.

Implementation

If you provide comments, and the general idea seems acceptable, I would prepare a PR for this.

I would use the existing "ignore" crate from the ripgrep project, and using the provided method to override the name of the ignore file from ".gitignore" to ".mdbookignore"

[ehuss]

I might prefer just making it a field in the config, like exclude = ["src/foo/*"].

But a question is, why add files to the src directory, but don't want them as part of the book? Why not just put those files somewhere else?

[andrewdavidmackenzie]

On the why: It's due to the structure of my book.

My book has a lot of it's content in a docs subfolder, BUT, it also includes README.md and other markdown files from multiple other sub-folders (it's a workspace rust project with multiple crates) so the docs files are mixed in with the code tree structure.

With the fix to avoid the recursive directory copy you merged recently I can now have my book.toml and SUMMARY.md in the root folder of the project, and SUMMARY.md includes entries for those multiple crates in sub-directories.

This leads to all sorts of files being copied into the output folder, such as:
.idea, .iml, .git, .sh, Cargo.toml and .lock, .json, *.rs from source folders, etc

I currently have a make target that deletes all the extra before doing the gh-pages deploy, cutting down the size of the upload considerably.

GH Project: flow
GH Pages Deployed here

[williamd1k0]

But a question is, why add files to the src directory, but don't want them as part of the book? Why not just put those files somewhere else?

In my case, I'm using some helper files in {{#include file:anchor}}. I just need parts of these files to be expanded, I don't need them in the output directory.

I might prefer just making it a field in the config, like exclude = ["src/foo/*"].

That would be enough.

[andrewdavidmackenzie]

To give my own specific example of the files that are copied and undesirable, here is the additiional make target I have had to add to cleanup the output directory before all of these are uploaded to gh_pages:

trim-docs:
	@find target/html -name target -type d | xargs rm -rf {}
	@find target/html -name .idea | xargs rm -rf {}
	@find target/html -name \*.iml | xargs rm -rf {}
	@find target/html -name .git | xargs rm -rf {}
	@find target/html -name .sh | xargs rm -rf {}
	@find target/html -name assets | xargs rm -rf {}
	@find target/html -name Cargo.toml | xargs rm -rf {}
	@find target/html -name manifest.json | xargs rm -rf {}
	@find target/html -name test.err | xargs rm -rf {}
	@find target/html -name \*.rs | xargs rm -rf {}
	@find target/html -name pkg | xargs rm -rf {}
	@find target/html -name \*.dump | xargs rm -rf {}
	@find target/html -name \*.dot | xargs rm -rf {}
	@find target/html -name \*.wasm | xargs rm -rf {}
	@find target/html -name \*.lock  | xargs rm -rf {}
	@cd target/html && rm -f Makefile .crates.toml .DS_Store .gitignore .mdbookignore .travis.yml
	@cd target/html && rm -rf bin
	@rm -rf target/html/flowc/tests/test-flows
	@rm -rf target/html/flowc/tests/test-libs
	@rm -rf target/html/code/debug
	@find target/html -depth -type d -empty -delete

That would be pretty noisy/complex as a set of config file patterns, which made me think of a .ignore file

[andrewdavidmackenzie]

I'm back looking at this.
Just wondering if the proposed use of an additional .mdbook_ignore (or similar) file would fly, and it it's worth me working on a PR for this?
Thanks for letting me know.

[cbrake]

But a question is, why add files to the src directory, but don't want them as part of the book? Why not just put those files somewhere else?

I prefer to have documentation located close to the source files in the repo when possible. This helps ensure that the documentation is maintained.

• https://handbook.tmpdir.org/documentation.html#tips-for-successful-documentation
• https://simonwillison.net/2022/Oct/1/software-engineering-practices/#docs-same-repo

Lately, I've also been using mdbook to pull documentation from multiple repos to build one book -- add all the repos that have docs to a superproject using git submodules, and then the SUMMARY.md simply points to the docs where-ever they might be. Overall, it works pretty well, but I have to clean out the build dir before deploying.

Why does mdbook deploy all files? Would it be possible to just start with *.md files and then only include additional files that are linked from *.md files?

[zacchiro]

But a question is, why add files to the src directory, but don't want them as part of the book? Why not just put those files somewhere else?

Just to mention another use case: an mdbook used to publish student assignments. The solutions to published exercises should be kept in the same place as the assignments, but not published (at least not immediately) together with the exercises. I'd like to have a way to exclude them, and ideally one that is configurable, so that different publication workflows can include/exclude the solutions selectively.

Thanks a lot for mdbook, it's awesome!