Pygments Tutorial

Pygments is a very popular and robust Python package to highlight code snippets. Many well-known sites like Wikipedia, BitBucket, Read the Docs etc. are using it. Here are some of the features of Pygments:

  • Supports a wide range of languages and markup formats.
  • New language and markup can be added easily.
  • Can use used as library or command line utility
  • A number of output formats are available, like HTML, RTF, Latex etc.

To install Pygments enter the following command:

Highlighting using Pygments

To highlight a code snippet using Pygments we follow these steps:

  1. Select lexer.
  2. Select the output format.
  3. Call the highlight() function.

Let’s go through each step in detail.

Selecting the lexer

A lexer is a program which performs the lexical analysis. In other words, it splits the code into tokens (identifier, keyword, literal etc). Every language or markup has its own lexer. To select the lexer use get_lexer_by_name() function from the pygments.lexers package. It takes a single argument named alias which is name of the lexer. For example, to select the lexer for Python do this:

We can see all the available lexers using the get_all_lexers() function:

Selecting the Formatter

Once you have selected the lexer, the next step is to select the Formatter. The Formatter’s job is to take a token stream from the lexer and writes the output, in the format such as HTML, LaTex, RTF, BBCode etc. To see the available formats use the get_all_formatters() function from the pygments.formatters package.

In our case, we are interested in outputting code snippet in HTML. As a result, we will use, HtmlFormatter (third from the last in preceding output). To use HtmlFormatter, import it from pygments.formatters and then instantiate it as follows:

Highlighting Code

Finally, the last step is to call the highlight() function from the pygments package. The highlight() function puts everything together and returns the highlighted output. It takes three arguments, the code to highlight, lexer and the formatter.

If you want highlight() function to save the output to a file rather than returning it, use the outfile argument as follows:

This will create a file named out.html in the current working directory containing the highlighted code.

If open out.html in a browser you will find that code doesn’t appear to be highlighted at all. This is because by default HtmlFormatter generates HTML with CSS classes to marks various tokens, but it doesn’t generate the actual CSS styles. We will see how to fix this in the next section.

Customizing the Output of HtmlFormatter

The following is a lists of some common keyword arguments that you can pass to HtmlFormatter() function to customize the generated output:

Argument Description
full If set to True, HtmlFormatter generates a self-contained document with inline CSS styles. By default, it is set to False
style The color scheme to use. It defaults to default.
linenos If set to True, tells the formatter to generate output with line numbers. By default, it is set to False.
hl_lines Specify the list of lines to be highlighted.
cssclass CSS class for wrapping the code block.

Here is how to use these arguments:

full

In the previous section we have seen that by default, the output generated by HtmlFormatter() only contains HTML markup and CSS classes only, it doesn’t contain the actual CSS styles to format the HTML markup. We can alter this behavior by passing full=True to HtmlFormatter() as follows:

This will re-write the existing out.html file. If you now open out.html in a text editor you will find that in addition to HTML markup and CSS classes, it also contains a list of CSS style in the head section of the document.

Open out.html in your browser and it should look like this:

style

The HtmlFormatter uses a default color scheme named default to highlight the code. Pygments comes with several built-in styles. To view the built-in styles use get_all_styles() function of the pygments.styles package.

Note: To get an instant preview of the available styles visit http://pygments.org/demo.

Once you know the style you want to use set it using the style argument as follows:

This will highlight the code snippet using monokai color scheme.

Open out.html and it will look like this:

linenos

This will add line numbers to the highlighted code:

Open out.html and it will look like this:

hl_lines

This will highlight the first and second line of the code.

Open out.html and it will look like this:

cssclass

By default, HtmlFormatter() generates code block wrapped in <div> tag with
class="highlight". For example:

We can specify any other class name using cssclass argument.

This returns the code block wrapped in <div> with class="code-block".

Generating CSS

We have seen that passing full=True to HtmlFormatter() creates a self-contained document with CSS styles. Although, this approach works, but it generates a lot of redundant CSS styles. A better approach would be to create an external style sheet containing all the CSS classes used in the output.

We can use get_style_defs() method of HtmlFormatter() to get the style definitions used by the formatter as a string.

This returns CSS styles for the default style.

To get the CSS styles for monokai, do this:

We can also pass additional classes to get_style_defs() method that will be prepended to the class. For example:

In case you have specified cssclass argument while creating HtmlFormatter instance then the default selector for get_style_defs() will be this class. For example:

And here is how you can save the generated CSS styles to an external file:

Now open out.html file in a text editor and add the following line at the top of the it:

Open out.html in a browser and it will look like this:

Another way to create CSS styles is to use pygments as a command line tool. For example:

This command will create the CSS styles for the default style. The -S option specifies the style and -f option specifies the formatter.

To generate CSS styles for monokai, use this:

We can also specify the top-level class using -a option:

Leave a Comment