[MS9] Schema and documentation generation from source code

[grobmeier]

Abstract: This PR completes the STF milestone 9 by implementing the infrastructure to auto-generate 1) the Log4j plugin documentation (to be deployed as a part of the Log4j website) and 2) the XSD (for assisting the Log4j runtime configuration, i.e., log4j2.xml, in user applications).

The problem description

Log4j runtime is a composition of 300+ plugins, which is analogous to beans in Java EE and Spring applications. Developers document these plugins in the website. Users assemble them in the log4j2.xml file describing their Log4j runtime configurations.

Currently, several plugins are either not documented (e.g. LOG4J2-3485), or their documentation doesn't match the behaviour of the source code. This not only causes users to misconfigure their applications or not be aware of all options available to them, but can also pose security implications due to unintended (mis)configurations. Next to lacking documentation, the log4j-config.xsd file users use to get assisted while assembling their log4j2.xml configurations is severely outdated. It doesn't cover a very big part of the available plugin ecosystem.

Deliverable 1: Log4j Docgen

Log4j Docgen (Documentation Generator) project aims to attack this problem by generating 1) the documentation, 2) plugin configuration options, and 3) XSD from the Java source code of the Log4j project itself. Effectively, this enables the following improvements:

• Avoid the need to maintain documentation in multiple places – For instance, Log4j JSON Template Layout is composed of several plugins with sufficient Javadoc explaining their behaviour. Next to this we also maintain a big json-template-layout.adoc page for the website, again, documenting each component. Last, but not least, log4j-config.xsd also needs to be updated (which is not!) to document all these whistles and bells too. By generating documentation from the source code, developers can precisely capture the behaviour of their components only in the very code itself that they change. As a result, there will be one and only place of the truth: the source code itself.
• Accurately capture the component configuration – It is pretty common that source code changes are not reflected to documentation and XSD. By generating the documentation from the source code, this discrepancy gets removed.

Currently being released Log4j Tools 0.8.0 bundles several components to implement this end-to-end operation. Refer to the Log4j Tools 0.8.0 website to read about all these new exciting Log4j Docgen features. Implementation details of the new log4j-docgen, log4j-docgen-maven-plugin, and log4j-docgen-asciidoc-extension can be viewed in this diff.

Deliverable 2: Log4j plugin documentation and XSD

Of course, it would not be complete to announce this work as complete without demonstrating its integration into the Log4j project itself. The 2.x-docgen and 3.x-docgen branches of the Log4j repository contain the changes that integrate Log4j Docgen into Log4j 2 and 3, respectively, to generate plugin reference manual to be built along with the website and to generate the plugin XSD (aka. log4j-config.xsd) capturing the type hierarchy of the available set of plugins. (The branch will be merged after the Log4j Tools 0.8.0 release scheduled for 2024-03-21.)

Deliverable 3: Centralizing XML namespace

Log4j project publishes four major XSDs:

1. Log4j Changelog (packaged with Log4j Tools)
2. Log4j 2
3. Log4j 3

Yet there does not exist a central place where users can access to all available versions of all these XSDs with sufficient descriptions. Inspired by the battle-tested principles practiced in Java EE: XML Schemas for Java EE Deployment Descriptors, we have compiled https://logging.apache.org/xml/ns/ for this purpose.

Implementation details

The bulk of the work is accessible in two locations:

1. Log4j Tools 0.8.0 release (the changes incorporated)
2. Add auto-generated plugin reference to the 2.x site #2404
3. Add auto-generated plugin reference to the 3.x site #2407

Other supporting ground work is implemented in following tasks:

• Generate reference documentation and XSD from metadata #1955
• Extract configuration metadata from source code #1956
• Fix the Javadoc comments of all our plugins and generate a "Reference" section in our documentation.
• Check the generated schema with xmllint (cf. Fix regex error in Log4j-config.xsd #2357)

[jvz]

I think I've proposed something along these lines in the past, but it seemed like too much work at the time. However, I love this idea in general. It would be great if we could extract JavaDoc metadata from plugin declarations to help form the documentation. Otherwise, we have a lot of duplicated information about plugins.

Might also be useful to include metadata that specifies some sample source files to inline for examples.

[ppkarwasz]

The plan is to delete all *.md, *.adoc and other files that describe our components' configuration and put everything into Javadoc.

The manual will be split into a Configuration Reference (cf Tomcat's reference) and a User Guide (cf. Tomcat's user guide). The latter will only explain how the system works, how to use the different configuration formats (with examples) and will not be encumbered with lots of technical details.

The Configuration Reference can even collect documentation from third-party plugin developers, if they embed metadata into their jars (or publish on Maven).

[vy]

Note that XmlSchemaTest needs to be removed and its functionality should ideally be replaced with a xml-maven-plugin validation that can take place at the Maven validation phase.

[ppkarwasz]

I have published a preview of a documentation generated from real data. Interesting pages:

• Configuration the description of the global <Configuration> element with links to its possible children,
• Appender and Layout contain a list of known appenders and layouts,
• RollingFileAppender: the king of complex components, with heavily nested subcomponents.

[ppkarwasz]

The descriptions are taken from Javadoc comments, which shows us how many Javadoc comments are missing.

Also the links in the descriptions (which are taken from {@link ...} comments in Javadoc) are usually broken, since the documentation generator currently can not determine the correct link to use.

[vy]

Great work Piotr! 💯 👏
Some remarks:

1. I see many pages are hardly filled. Shall we collapse the entire reference to a single page? WDYT? That is, one page per [Maven] module?
2. Why do some nested components appear as <AppenderRef/> while some ... one Filter element ...? I was expect the latter to be formatted as <Filter/> too.
3. Looking at AsyncAppender, I cannot see
   • if I can provide one or more AppenderRefs
   • what is the default for a nested component
4. We need to document the KeyValuePair component
5. We need to document the value field of the Property component

Also the links in the descriptions (which are taken from {@link ...} comments in Javadoc) are usually broken, since the documentation generator currently can not determine the correct link to use.

What is the plan to address this shortcoming?

[ppkarwasz]

1. Absolutely. It is easier to output a different Asciidoc document per plugin, but it will probably look better as a single page or a couple of pages.
2. This is due to the FreeMarker template I used. If there is only a single choice of plugin (e.g. AppenderRef) it outputs the official name of the plugin, otherwise an expression like ... one Filter element .... I would be against using <Filter/> in the description, since users might literally write <Filter/>, but there is a lot of room for improvement in the template,
3. Again, it is a template problem, the data (multiplicity="*") is there, but is not displayed. Regarding default values for nested components, we should provide that data in the Javadoc of the element: there is no way to reliably guess that.
4. All those missing description must be filled by us (e.g. @grobmeier), but we can do it in parallel with code changes to log4j-docgen.

Also the links in the descriptions (which are taken from {@link ...} comments in Javadoc) are usually broken, since the documentation generator currently can not determine the correct link to use.

What is the plan to address this shortcoming?

As far as I understand we can either:

• leave some placeholders for links in plugins.xml and fill them up with the appropriate link (to our Javadoc, an external Javadoc or our reference manual) during the generation of Asciidoc pages,
• or write an Asciidoc macro that does the same during compilation.

[grobmeier]

Looks good to me, thanks for all the hard work!