linuxdoc.kernel_doc module

kernel_doc

Implementation of the kernel-doc parser. The kernel-doc parser extracts kernel-doc markup from source code comments.

This module provides an API which could be used by a sphinx-doc generator extension and a command-line interface

Compared with the Perl kernel-doc script used in the Linux kernel, this implementation has additional features like parse options for a smooth integration of reStructuredText (reST) markup in the source code comments. In combination with the (separate) kernel-doc reST directive (which uses this module), the documentation generation becomes more clear and flexible.

The architecture of the parser is simple and consists of three types of objects (three classes).

• Parser: The parser parses the source-file and dumps extracted kernel-doc data.

• subclasses of class TranslatorAPI: to translate the dumped kernel-doc data into output formats. There exists two implementations:

  • NullTranslator: translates nothing, just parse

  • ReSTTranslator: translates dumped kernel-doc data to reST markup

• ParseOptions: a container full with options to control parsing and translation.

With the NullTranslator a source file is parsed only once while different output could be generated (multiple times) just by changing the Translator (e.g. with the ReSTTranslator) and the option container. With parsing the source files only once, the building time is reduced n-times.

class linuxdoc.kernel_doc.Container

Bases: dict

linuxdoc.kernel_doc.DEFAULT_EXP_IDS = ['EXPORT_SYMBOL', 'EXPORT_SYMBOL_GPL', 'EXPORT_SYMBOL_GPL_FUTURE']

Default value of :exp-ids:

linuxdoc.kernel_doc.DEFAULT_EXP_METHOD = 'macro'

Default value of :exp-method:

class linuxdoc.kernel_doc.ListTranslator(list_exported, list_internal_types, *args, **kwargs)

Bases: TranslatorAPI

Generates a list of kernel-doc symbols.

eof()

get_type(name)

output_DOC(sections=None)

output_enum_decl(**kwargs)

output_epilog()

output_function_decl(**kwargs)

output_preamble()

output_prefix()

output_struct_decl(**kwargs)

output_suffix()

output_typedef_decl(**kwargs)

output_union_decl(**kwargs)

class linuxdoc.kernel_doc.NullTranslator

Bases: TranslatorAPI

Null translator, translates nothing, just parse.

HIGHLIGHT_MAP = []

LINE_COMMENT = ('', '')

eof()

output_DOC(*args, **kwargs)

output_enum_decl(*args, **kwargs)

output_epilog(*args, **kwargs)

output_function_decl(*args, **kwargs)

output_preamble(*args, **kwargs)

output_prefix()

output_struct_decl(*args, **kwargs)

output_suffix()

output_typedef_decl(*args, **kwargs)

output_union_decl(*args, **kwargs)

class linuxdoc.kernel_doc.ParseOptions(*args, **kwargs)

Bases: Container

PARSE_OPTIONS = [('highlight', ['on', 'off'], 'setOnOff'), ('INSPECT', ['on', 'off'], 'setINSPECT'), ('markup', ['reST', 'kernel-doc'], 'setVal'), ('SNIP', [], 'setVal'), ('SNAP', [], 'snap')]

PARSE_OPTION_RE = '^/\\*+\\s*parse-%s:\\s*([a-zA-Z0-9_-]*?)\\s*\\*/+\\s*$'

add_filters(parse_options)

dumpOptions()

filter_opt(line, parser)

get_exported_symbols_re()

set_defaults()

class linuxdoc.kernel_doc.Parser(options, translator)

Bases: SimpleLog

kernel-doc comments parser

States:

• 0 - normal code

• 1 - looking for function name

• 2 - scanning field start.

• 3 - scanning prototype.

• 4 - documentation block

• 5 - gathering documentation outside main block (see Split Doc State)

Split Doc States:

• 0 - Invalid (Before start or after finish)

• 1 - Is started (the /** was found inside a struct)

• 2 - The @parameter header was found, start accepting multi paragraph text.

• 3 - Finished (the */ was found)

• 4 - Error: Comment without header was found. Spit a error as it’s not

  proper kernel-doc and ignore the rest.

DOC_TYPES = ['DOC', 'function', 'struct', 'union', 'enum', 'typedef', 'macro']

LOG_FORMAT = '%(fname)s:%(line_no)s: :%(logclass)s: %(message)s\n'

check_return_section(decl_name, return_type)

check_sections(decl_name, decl_type, sectcheck, parameterlist)

close()

create_parameterlist(parameter, splitchar)

debug(message, _line_no=None, **replace)

dump_DOC(name, cont)

dump_enum(proto)

dump_epilog()

dump_function(proto)

dump_preamble()

dump_prefix()

dump_section(name, cont)

Store section’s content under it’s name.

Parameters:

• name (str) – name of the section

• cont (str) – content of the section

Stores the content under section’s name in one of the container. A container is a hash object, the section name is the key and the content is the value.

Container:

• self.ctx.constants: holds constant’s descriptions

• self.ctx.parameterdescs: holds parameter’s descriptions

• self.ctx.sections: holds common sections like “Return:”

There are the following contai

dump_struct(proto)

dump_suffix()

dump_typedef(proto)

dump_union(proto)

error(message, _line_no=None, **replace)

feed(data, eof=False)

classmethod gather_context(src, ctx, opts)

Scan source about context informations.

Scans whole source (e.g. Parser.rawdata) about data relevant for the context (e.g. exported symbols).

Names of exported symbols gathered in ParserContext.exported. The list contains names (symbols) which are exported using the pattern specified in opts.

Hint

A exported symbol does not necessarily have a corresponding source code comment with a documentation.

Module information comes from the MODULE_xxx macros. Module informations are gathered in ParserContext.module_xxx:

• MODULE_AUTHOR("..."): Author entries are collected in a list in ParserContext.mod_authors

• MODULE_DESCRIPTION("..."): A concatenated string in ParserContext.mod_descr

• MODULE_LICENSE("..."): String with comma separated licenses in ParserContext.mod_license.

Hint

While parsing header files, about kernel-doc, you will not find the MODULE_xxx macros, because they are commonly used in the “.c” files.

info(message, _line_no=None, **replace)

output_decl(name, out_type, **kwargs)

parse(src=None)

parse_dump_storage(translator=None, options=None)

prepare_struct_union(proto)

process_state3_function(line)

process_state3_type(line)

push_parameter(p_name, p_type)

reset_state()

sect_title(title)

Normalize common section titles

section_constants = 'Constants'

section_context = 'Context'

section_def = 'Definition'

section_default = 'Description'

section_descr = 'Description'

section_intro = 'Introduction'

section_members = 'Members'

section_return = 'Return'

setOptions(options)

setTranslator(translator)

special_sections = ['Description', 'Definition', 'Members', 'Constants', 'Context', 'Return']

state_0(line)

state: 0 - normal code

state_1(line)

state: 1 - looking for function name

state_2(line)

state: 2 - scanning field start.

state_3(line)

state: 3 - scanning prototype.

state_4(line)

state: 4 - documentation block

state_5(line)

state: 5 - gathering documentation outside main block

syscall_munge(prototype)

tracepoint_munge(prototype)

undescribed = '*undescribed*'

warn(message, _line_no=None, **replace)

exception linuxdoc.kernel_doc.ParserBuggy(parserObj, message)

Bases: RuntimeError

Exception raised when the parser implementation seems buggy.

The parser implementation perform some integrity tests at runtime. This exception type mainly exists to improve the regular expressions which are used to parse and analyze the kernels source code.

In the exception message the last position the parser parsed is stored, this position may, but does not need to be related with the exception (it is only an additional information which might help).

Under normal circumstances, exceptions of this type should never arise, unless the implementation of the parser is buggy.

class linuxdoc.kernel_doc.ParserContext(*args, **kwargs)

Bases: Container

dumpCtx()

new()

class linuxdoc.kernel_doc.RE(*args, **kwargs)

Bases: object

regular expression that stores last match (like Perl’s =~ operator)

match(*args, **kwargs)

search(*args, **kwargs)

class linuxdoc.kernel_doc.ReSTTranslator

Bases: TranslatorAPI

Translate kernel-doc to reST markup.

Variables:

HIGHLIGHT_map (list) – Escape common reST (in-line) markups. Classic kernel-doc comments contain characters and strings like * or trailing _, which are in-line markups in reST. These special strings has to be masked in reST.

BITFIELD = <linuxdoc.kernel_doc.RE object>

FUNC_PTR = <linuxdoc.kernel_doc.RE object>

HEADER_TAGS = '#==--"'

HIGHLIGHT_MAP = [(<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1 \\2 <\\2>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1 \\2 <\\2>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1 \\2 <\\2>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1 \\2 <\\2>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1\\2() <\\1>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`\\1\\2 <\\1>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:func:`\\1`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ ``\\1``\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ ``\\1``\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ ``\\1``\\ '), (<linuxdoc.kernel_doc.RE object>, '\\ :c:type:`struct \\1 <\\1>`\\ '), (<linuxdoc.kernel_doc.RE object>, '\\1')]

INDENT = '    '

LINE_COMMENT = ('.. ', '')

MASK_REST_INLINES = [(<linuxdoc.kernel_doc.RE object>, '\\1\\\\_\\2'), (<linuxdoc.kernel_doc.RE object>, '\\1\\\\_\\2'), (<linuxdoc.kernel_doc.RE object>, '\\\\\\1'), (<linuxdoc.kernel_doc.RE object>, '\\\\\\1'), (<linuxdoc.kernel_doc.RE object>, '\\\\\\1')]

format_block(content)

format the content (string)

highlight(text)

returns highlighted text

output_DOC(sections=None)

output_enum_decl(enum=None, parameterlist=None, parameterdescs=None, sections=None, purpose=None)

output_epilog()

output_function_decl(function=None, return_type=None, parameterlist=None, parameterdescs=None, parametertypes=None, sections=None, purpose=None, decl_type=None)

output_preamble()

output_prefix()

output_struct_decl(decl_name=None, decl_type=None, parameterlist=None, parameterdescs=None, parametertypes=None, sections=None, purpose=None, definition=None)

output_suffix()

output_typedef_decl(typedef=None, sections=None, purpose=None)

write_anchor(refname)

write_definition(term, definition, prefix='')

write_func_param(param, param_type, descr)

write_header(header, sec_level=2)

write_section(header, content, sec_level=2, ID=None)

class linuxdoc.kernel_doc.SimpleLog

Bases: object

LOG_FORMAT = '%(logclass)s: %(message)s\n'

debug(message, **replace)

error(message, **replace)

info(message, **replace)

warn(message, **replace)

class linuxdoc.kernel_doc.TranslatorAPI

Bases: object

Abstract kernel-doc translator.

Variables:

• cls.HIGHLIGHT_MAP (list) – highlight mapping

• cls.LINE_COMMENT (tuple) – tuple with start-/end- comment tags

HIGHLIGHT_MAP = [(<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None), (<linuxdoc.kernel_doc.RE object>, None)]

LINE_COMMENT = ('# ', '')

classmethod comment(cont)

returns commented text

eof()

get_epilog()

get_preamble()

highlight(text)

returns highlighted text

output_DOC(sections=None)

output_enum_decl(enum=None, parameterlist=None, parameterdescs=None, sections=None, purpose=None)

output_epilog()

output_function_decl(function=None, return_type=None, parameterlist=None, parameterdescs=None, parametertypes=None, sections=None, purpose=None, decl_type=None)

output_preamble()

output_prefix()

output_struct_decl(decl_name=None, decl_type=None, parameterlist=None, parameterdescs=None, parametertypes=None, sections=None, purpose=None, definition=None)

output_suffix()

output_typedef_decl(typedef=None, sections=None, purpose=None)

output_union_decl(*args, **kwargs)

setOptions(options)

setParser(parser)

write(*objects)

Write objects to stream.

Write Unicode-values of the objects to :py:attr:self.options.out.

Parameters:

objects – The positional arguments are the objects with the content to write.

write_comment(*objects)

Write objects as comments to stream.

linuxdoc.kernel_doc.highlight_parser(text, map_table)

linuxdoc.kernel_doc.map_row(row, map_table)

linuxdoc.kernel_doc.normalize_id(ID)

substitude invalid chars of the ID with - and mak it lowercase

linuxdoc.kernel_doc.normalize_ws(string)

strip needles whitespaces.

Substitute consecutive whitespaces with one single space and strip trailing/leading whitespaces

linuxdoc.kernel_doc.openTextFile(fname, mode='r', encoding='utf-8', errors='strict')

linuxdoc.kernel_doc.readFile(fname, encoding='utf-8', errors='strict')