Autodoc - an automated documentation system for Clojure

Introduction

Autodoc is a system for generating HTML documentation for Clojure projects. It is used to create the up-to-the minute published API documentation for Clojure itself, the various contributed libraries, and the statistics package, Incanter.

Autodoc has been created by Tom Faulhaber.

Autodoc leverages two aspects of Clojure in particular:

• Clojure projects are organized into namespaces.
• Clojure supports documentation strings on all variables (including functions and macros) and namespaces.

When run over a project, Autodoc will generate:

1. An overview page (called index.html) with a description of the project, and an overview of each namespace (from the namespace’s metadata) with a list of documented, public variables and functions in that namespace.

   

2. A page for each namespace in the project which has the detailed documentation for each variable, function and macro. This includes name, type, usage and documentation.

   

3. A global index of all the documented variables, their namespace, and a snippet of their documentation.

   

4. A Clojure index of all the namespaces and vars in those namespaces for use by other tools. See the file index.clj in the output directory:

{:namespaces
       ({:source-url nil,
         :wiki-url "noir.cookies-api.html",
         :name "noir.cookies",
         :doc "Stateful access to cookie values"}
        {:source-url nil,
         :wiki-url "noir.core-api.html",
        ...
       :vars
       ({:arglists ([k] [k default]),
         :name "get",
         :namespace "noir.cookies",
         :source-url nil,
         :raw-source-url nil,
         :wiki-url "/noir.cookies-api.html#noir.cookies/get",
         :doc
         "Get the value of a cookie from the request. k can either be a string or keyword.",
         :var-type "function",
         :line 20,
         :file "src/noir/cookies.clj"}
        {:arglists ([sign-key k] [sign-key k default]),
         :name "get-signed",
        ...

By default, Autodoc produces documentation that looks a lot like Clojure’s own API documentation. However, it is possible to customize both the style and layout of the pages generated by Autodoc to your tastes.

Autodoc can be run directly from the command line or can be used from build tools like ant or leiningen.

Things not yet implemented

Most of the features of the Autodoc service have been implemented in the Autodoc command. A couple of features are not set up yet, however:

• Building and linking supporting documentation from Markdown source.
• Automatically updating a GitHub pages repository as part of the build. (See below for a more manual solution to this, however.)

Stay tuned. These features are coming.

Autodoc continues to evolve. If there are features you’d like to see, let me know.

Adding Documentation to your Project

Autodoc uses Clojure’s metadata mechanism to find documentation that you’ve built into your Clojure programs. Autodoc uses the same doc strings that the regular Clojure documentation system uses and adds some extra functionality as well.

Autodoc reads metadata attached to namespaces and vars. Since defined functions are a type of var, their doc strings and other metadata are included automatically.

Every Clojure namespace and var has a Clojure map for its metadata. Metadata is generally added when you use the ns macro or the def macro (or one of its relatives such as defn).

Autodoc uses the following keys in these maps:

Running Autodoc

Getting Autodoc

Unless you are using Autodoc from leiningen, you will need to get a copy of the standalone jar. Download the latest version from github.

If you’re using leiningen, the autodoc code will be pulled automatically from its home on clojars.org.

From the command line

The simplest way to use autodoc is to launch it from the command line in the root directory of your project:

java -jar autodoc-<version>-standalone.jar

This will scan the source in the src/ directory and produce an HTML tree in autodoc/ directory.

To modify the behavior of autodoc, use parameters. For example:

java -jar autodoc-<version>-standalone.jar --source-directory=master

will use the sub-directory master/ as the location of the source files.

As shown above, to set parameters from the command line, simply use options like --<parameter-name>=<value> or --<parameter-name> <value>. The full list of available parameters is below.

If you need extra classpath entries (that is, if your project uses a library other than Clojure or Clojure-Contrib), you won’t be able to run the executable jar directly. Instead, specify your full classpath and the class autodoc.autodoc as follows:

java -cp autodoc-<version>-standalone.jar:<other jars> autodoc.autodoc <autodoc params>

Autodoc parameters

Integrating Autodoc with your build

Building with Leiningen

lein-autodoc provides an Autodoc plug-in for Leiningen. This is the very easiest way to integrate Autodoc into a build process.

There are two ways you can add the plugin to your leiningen environment: add it to the project or add it to a user profile. This is described in the Leiningen plugin documentation. If generating autodoc is a regular artifact of your project, put autodoc in the project.clj. If you want to be able to generate autodoc for arbitrary projects that you may not own, it makes sense to add the plugin to your user profile.

Adding to a project.clj file

The autodoc plugin should always be added to the :dev profile within your project.clj file, as follows:

:profiles {:dev {:plugins [[lein-autodoc "1.0.0"]]}}

Adding to your user profile

To add the plugin to your user profile so that all projects that you work on get autodoc, simply add it to your ~/.lein/profiles.clj file:

{:user
 {:plugins [[lein-autodoc "1.0.0"]
            ...]}
  ...}}

Running autodoc from Leiningen

To build the documentation, simply type:

lein autodoc

Autodoc will set the name, description, and source-path parameters from the Leiningen configuration. Other parameters can be set via a map attached to the :autodoc key:

(defproject bugs "0.1.0-SNAPSHOT"
      :description "Statistical routines for thinking about a bugzilla DB"
      :dependencies [[incanter "1.0-master-SNAPSHOT"]
                     [org.clojure/clojure "1.1.0-alpha-SNAPSHOT"]
                     [org.clojure/clojure-contrib "1.0-SNAPSHOT"]]
      :profiles {:dev {:plugins [[lein-autodoc "1.0.0"]]}}
      :autodoc { :name "Bugs", :page-title "Bugs API Documentation"})

Notes

There are two things to consider with Leiningen integration:

1. Options that take lists (like :load-except-list) are not yet supported.
2. In order to provide more independence from shared dependencies with Leiningen, Autodoc and its dependents are loaded into their own directory and executed in a separate process from Leiningen. This won’t work if you run it for the first time when you’re untethered from the Internet.

Building with Ant

Building with ant is straightforward. Just add an autodoc target to your build.xml file that looks like this:

<!-- Adjust the pathname of the jar to wherever you've stored it and set the version appropriately -->
<property name="autodoc-standalone-jar" location="${user.home}/.clojure/autodoc-<version>-standalone.jar"/>

<!-- Adjust the depends as appropriate -->
<target name="autodoc" depends="build"
        description="Build the HTML documentation">
  		<java classname="autodoc.autodoc" fork="true" failonerror="true">
    <classpath>
      <pathelement location="${autodoc-standalone-jar}"/>
      <!-- other pathelements as required by the project -->
    </classpath>
    <arg value="--name=Bugs"/>
    <arg value="--description=Statistical routines for thinking about a bugzilla DB"/>
    <arg value="--page-title=Bugs API Documentation"/>
  </java>
</target>

The comments refer to the parts that will have to change depending on the configuration of your build system.

Maven

Autodoc currently does not support Maven. It is planned, though. Patches welcome.

Making Autodoc work with source code management

Many users will want the files generated by Autodoc to be excluded in their code repositories. This is easy to do in most SCMs.

For example, in git, create a .gitignore file in the root directory of the project with the following lines:

#ignore generated autodoc files
autodoc/**

Other SCMs support similar mechanisms.

Publishing your documentation on GitHub using GitHub Pages

When the project is using GitHub for sharing source, GitHub pages provide a great way to share documentation. This is how the Clojure core API documentation is hosted at http://clojure.github.com/clojure, for instance.

See http://pages.github.com/ for a complete description of GitHub pages and how they work.

Setting up your Autodoc output to link to the GitHub pages for your project is easy:

1. Add autodoc/** to your .gitignore file as described above.

2. remove any autodoc directory that you’ve already generated.

3. While in your project root directory, create a clone of your private github repository into the autodoc directory:

    git clone git@github.com:<user name>/<project name>.git autodoc
   

4. Create a new root branch in autodoc with the name gh-pages:

    $ cd autodoc
    $ git symbolic-ref HEAD refs/heads/gh-pages
    $ rm .git/index
    $ git clean -fdx
    $ cd ..
   

5. Now build the autodoc using the command line, Leiningen, ant or your preferred build tool.

6. Commit the documentation to GitHub with this set of commands:

    $ cd autodoc
    $ git add -A
    $ git commit -m"Documentation update"
    $ git push origin gh-pages
   

Now, after a few minutes, you should be able to go to http://<user name>.github.com/<project name> and see your documentation rendered for the whole world to see. (Once in a while I’ve had to create an extra checkin to kick GitHub when it didn’t notice my first checkin.

Now that you have GitHub pages set up, updating them is straightforward:

1. Build the autodoc with your normal method.

2. Commit your changes:

    $ cd autodoc
    $ git add -A
    $ git commit -m"Documentation update"
    $ git push origin gh-pages
   

Customizing Autodoc

Autodoc supports customization of page layout, styles and graphical elements.

You can add customized layout templates and styles in the autodoc-params directory of your project.

More documentation about customizing will be coming soon.