Remarks for Kernel developer¶
The sphinx extensions for this build, which are shipped by the kernel source
tree, are placed in the
Documentation/sphinx folder. Some of the LinuxDoc
features are already a part of the Kernel’s
others not (yet). E.g. the sphinx-doc extensions flat-table,
cdomain, kfigure and
kernel-include are merged into Kernel’s source
tree. On the other side, e.g. for parsing kernel-doc comments, the Linux Kernel
build process uses a Perl script while LinuxDoc brings a python module with a
One drawback of the Perl script is, that it fits not very well into
sphinx-extensions which are normally written in python. As a stand-alone
parser it might be good enough, but mostly the parser is driven by the
kernel-doc directive and this can’t be done
in-process when you need a separate process for the Perl interpreter. The only
thing that such a parser can do, is to pipe its results to
stdout, which can
be read by the kernel-doc directive. For a starting point at that time (2016),
the reuse of the given Perl parser in Kernel’s sources was good enough but
nowerdays it has its limits. This was the time, I started a spin-off with a
POC, implementing a python variant of the parser together with it’s
There was also a attempt in 2017 where I send a RFC replacing the Perl parser with the python variant, some parts of the discussion are worth to mention:
Start with lwn article or the ML entry [RFC PATCH v1 0/6] pure python kernel-doc parser and more … there are also helpful remarks and thoughts from the community I won’t miss:
from Jon firstname.lastname@example.org
about using other parser email@example.com
Everything has evolved in the meantime. Meanwhile the LinuxDoc project is also used in other projects and covers more use-cases than only to build Kernel’s documentation. Anyway both implementations, the python and the Perl one support kernel-doc markup while the Python one always serves a superset over the Perl script from Kernel’s source. That’s why it is possible to (drop-in) replace Kernel’s sphinx-extensions and the Perl parser with the LinuxDoc extension. To see how, take a look at chapter Patch Linux Kernel Documentation.
It is worth to mention, that both scripts will produce different reST from the same kernel-doc markup. This is only logical and consistent, because the python variant is further more developed and integrates better into Sphinx.
.. c:function:: int foobar (int arg1, int arg2) short function description of foobar **Parameters** ``int arg1`` Describe the first argument to foobar. ``int arg2`` Describe the second argument to foobar. One can provide multiple line descriptions for arguments. **Description** A longer description, with more discussion of the function :c:func:`foobar()` that might be useful to those using or modifying it. Begins with empty comment line and may include additional embedded empty comment lines. Within, you can refer other definitions (e.g. :c:type:`struct my_struct <my_struct>`, :c:type:`typedef my_typedef <my_typedef>`,``CONSTANT``, $ENVVAR etc.). The longer description can have multiple paragraphs and you can use reST inline markups like *emphasise* and **emphasis strong**. You also have reST block markups like lists or literal available: Ordered List: - item one - item two - literal block:: a + b --> x **Return** Describe the return value of foobar.
As you can see, Perl’s output is a flatten stream of markups and emphasis while the the Python variant produce a structured and valid reST document. One might think, that we can fix the Perl script to be more structural. As far as I know, it is not possible to embed structured reST into the doctree without being in-process where you have access to the in memory doctree. I don’t know where the Kernel development goes in the future, IMO there are three choices:
Stay with Perl implementation of the kernel-doc parser and fumble around with its limitations.
Take over the Python implementation of the parser and it’s kernel-doc directive.
Or simply add LinuxDoc as one external requirement more to the build chain and drop/ignore the extension currently in the source tree (for a POC see patch).