A pkgdown website for data.table

[hadley]

jangorecki: WIP status in #3677 (comment)

---

You can see (a manual deploy) of the site at https://nervous-easley-6de120.netlify.com

Note that to build the pkgdown site yourself you'll need the latest version from GitHub (I had to fix a couple of bugs) and build with pkgdown::build_site(devel = FALSE, new_process = FALSE)

This is a very preliminary version presented only in order to have something concrete to respond to. To do:

• If desired, pick a different theme from https://bootswatch.com
• Produce a minimal image of the data.table syntax summary images
• Polish overview paragraphs
• Review structure of reference page
• Prepare usage example

Note that I had to make a few minor changes to data.table sources to work cleanly with pkgdown. I think once you're happy with the basic website, we can discuss in more detail.

Fixes #3675

[codecov]

Codecov Report

Merging #3677 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3677   +/-   ##
=======================================
  Coverage   99.42%   99.42%           
=======================================
  Files          72       72           
  Lines       13498    13498           
=======================================
  Hits        13421    13421           
  Misses         77       77

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1539ef6...7b361bf. Read the comment docs.

[MichaelChirico]

Thanks so much @hadley!

Starting to look things over...

I'm afraid I don't follow you here:

Produce a minimal image of the data.table syntax summary images

Is that something on your end or for us?

---

I notice the mouse-over of 1.12.3 says Released version, but this is the dev version (1.12.2 is on CRAN; we use odd-even versions to distinguish), is that easy to change? Then, on the right, it says Dev status but we have a CRAN badge, so the two things are a bit at odds.

---

Am I right that the Changelog page looks for a foo() pattern and cross-references the package index to link to help pages? Would it be easy to match %foo% as well? In item 8 between() links but %between% doesn't.

between() and %between% are faster for POSIXct, #3519, and now support the .() alias, #2315. Thanks to @Henrik-P for the reports. There is now also support for bit64’s integer64 class and more robust coercion of types, #3517.

---

For the _pkgdown.yml config, we can use any of the \alias{}es to refer to a page right? And all of the non-\keyword{internal} pages must be indexed? Is it OK to comma-separate multiple of a pages aliases (e.g. I would switch IDateTime to IDate, ITime, IDateTime)? Is it OK to have duplicate references to a ref page? It might make sense to have IDate, ITime, IDateTime, then year, month, hour, day separately so that all the functions do get a callout on the reference page (even though they're all in the same Rd)

---

In general if a fix/change/upgrade is required in pkgdown, would be happy to try and help with the caveat that some primer about how things are working would be helpful.

[hadley]

To be precise, I think someone on the data table needs to redraw the minimal syntax image to remove the background colour, remove pdf download link in top right, and refine (or remove) the title.

---

Can you file an issue in pkgdown for this? Currently we assume that dev versions look like 1.0.0.9000, so we'll need to think about how to parameterise that differently.

---

I don't remember the exact details of how the autolinking works, but I think it should be straightforward to also autolink infix operators. Can you please file an issue?

---

There's currently very little control over how the aliases are display on the reference page. So far my position has been that if you think there are too many, then you should consider breaking up the topic into multiple pieces.

---

I don't have pkgdown front of mind at the moment, so I don't have any general advice, sorry 😞

[MichaelChirico]

Re: minimal syntax image -- I had a go at reproducing that in text in my commit. Will leave it up to Matt how to proceed.

Re: issues: filed and filed

[jangorecki]

We need to speed it up, currently it installs remotes, pkgdown and 59! other packages. Plus it requires OS dependency libxml2-dev. As a result this job takes 16 minutes while previously took 5 minutes.
This would help: r-lib/pkgdown#1115

[hadley]

Does gitlab not have caching? So you have to install these packages on every run?

[jangorecki]

GitLab caching is currently in use when mirror suggested dependencies. We do publish all of them together with devel data.table in our R repository, to achieve ultimate reproducibility.
As of now we do not need to use any other package in the pipeline, all CI is made in base R. Thus adding 60 packages related to CI only is pretty big thing. I will try to make a secondary CI-related cache/repo.

Just for reference, mirror-packages job:

data.table/.gitlab-ci.yml

Lines 20 to 37 in 81af9b6

	mirror-packages: # download all recursive dependencies of data.table suggests and integration suggests from inst/tests/tests-DESCRIPTION
	stage: dependencies
	tags:
	- linux
	image: registry.gitlab.com/jangorecki/dockerfiles/r-base-dev
	cache:
	paths:
	- bus/$CI_BUILD_NAME/cran
	variables:
	R_BIN_VERSION: "3.6"
	R_DEVEL_BIN_VERSION: "3.7"
	script:
	- echo 'source("ci.R")' >> .Rprofile
	- mkdir -p bus/$CI_BUILD_NAME/cran/src/contrib
	# mirror R dependencies: source, win.binary
	- Rscript -e 'mirror.packages(dcf.dependencies(c("DESCRIPTION","inst/tests/tests-DESCRIPTION"), "all"), repos=c(Sys.getenv("CRAN_MIRROR"), dcf.repos("inst/tests/tests-DESCRIPTION")), repodir="bus/mirror-packages/cran")'
	- Rscript -e 'sapply(simplify=FALSE, setNames(nm=Sys.getenv(c("R_BIN_VERSION","R_DEVEL_BIN_VERSION"))), function(binary.ver) mirror.packages(type="win.binary", dcf.dependencies("DESCRIPTION", "all"), repos=Sys.getenv("CRAN_MIRROR"), repodir="bus/mirror-packages/cran", binary.ver=binary.ver))'
	<<: *artifacts

[jangorecki]

after recent commits to pkgdown we went down to 13 minutes for this job
so 3/11 of the overhead is fixed

[jangorecki]

Once we agree it is ready to merge I will then update url references from jangorecki.gitlab.io to Rdatatable.gitlab.io and restrict build to master branch only (during dev it also publish for pkgdown).

[MichaelChirico]

Changelog

• It's quite nice to have the internal links to old versions. We also have NEWS.0.md, is there any way to reference that? It so happens that we did link it at the bottom but that's pretty obscure. It might be nice to have the a toggle at the top, or for the last link on the right to redirect to that file on GH? @hadley this might be a pkgdown FR?

• Section headers are also inconsistent @hadley:

For some reason the date as parsed shows up on the left only for v1.12.2?

• I see this, feels a bit redundant, anything to be done? (in development) Unreleased

• Is there a way to add the dates to the links on the right as well?

Manual

• nafill() setnafill() appear twice, is that intentional? If so I would also add copy() to the in-place modification section (and generally double-reference anything that feels kind-of borderline)

Vignettes

• in Intro vignette, in the next vignette at the end -- not totally clear what this is referring to

[jangorecki]

It's quite nice to have the internal links to old versions. We also have NEWS.0.md, is there any way to reference that? It so happens that we did link it at the bottom but that's pretty obscure. It might be nice to have the a toggle at the top, or for the last link on the right to redirect to that file on GH? @hadley this might be a pkgdown FR?

This is already possible, we can make a menu of changelog files, but I don't see it that much important.
NEWS.0.md is more for historical, not much practical, reasons.

date

related issue: r-lib/pkgdown#1118
maybe worth to put your comments there

next vignette

we should explicitly name it, or if possible put a portable link

double referencing

I don't think it is an issue to double reference. What would be useful is to put only setnafill in one group and nafill in another. But pkgdown seems to put all related links from same manual file in the same line.

[mattdowle]

@jangorecki why this change?

[jangorecki]

I am not precisely sure, https://pkgdown.r-lib.org/articles/linking.html vignette mention to use github repo url. I reverted back to use two urls, as initially commited by Hadley

[jangorecki]

I am not sure but we might need to use three urls there, as pkgdown website will be published under: https://Rdatatable.gitlab.io/data.table
now during dev: https://jangorecki.gitlab.io/data.table/