Documentation pipeline with Antora

Fabrice Flore-Thebault, Alexander Schwartz

The big picture: shift left and automate

The downstream documentation receives content contributions to the upstream repository, source for the upstream documentation.
The downstream repository has only the required tooling and downstream specific content to publish to the Customer portal.
Everything that can be automated is automated.

Understanding the downstreaming workflow

Figure 1. Downstreaming workflow

Step by step downstreaming workflow

Writers and engineers contribute content to the upstream repository.
On commit, upstream CI publishes the upstream documentation website.
On release, engineers create a tag in the upstream repository.
Writers update the downstream repository content and metadata.
On schedule, downstream CI single-sources content from the upstream repository tag to the downstream repository.
On push, downstream CI publishes the downstream documentation preview.
On release, writers create a release branch in the downstream repository.
Customer Portal CI publishes the downstream documentation to the Customer Portal.

Handling the content variations

Understanding the downstream repository structure

Repository content structure

.
├── docs
│   ├── getting_started
│   │   └── modules
│   │       └── getting_started
│   │           └── examples (1)
│   ├── release_notes
│   │   └── modules
│   │       └── release_notes
│   │           ├── pages (2)
│   │           └── partials (2)
│   └── shared (3)
├── publication (4)
└── tools (5)

1	Editable Getting started downstream specific content: Antora examples, and metadata.
2	Editable Release notes content: Antora module, and metadata.
3	Editable shared downstream specific content: AsciiDoc attributes, shared content.
4	Versioned build artifacts required for publication. Do not edit manually.
5	Automation scripts.

Understanding the CI pipeline

Figure 2. Downstreaming workflow

Single-source: Single-source content from the upstream repository.
Convert: Convert to HTML multi-page, HTML single-page, EPUB, and PDF.
Test: Test language, links.
Push: Commit back changes after verification

AsciiDoc features

Modifying inline content with AsciiDoc attributes

Upstream

Content
```
Configuring {prod}
```
Attributes
```
:prod: Project
```
Output
Configuring Project

Downstream

Attributes
```
:prod: Product
```
Output
Configuring Product

Benefits

Allows variations on simple inline content.
https://docs.asciidoctor.org/asciidoc/latest/attributes/document-attributes/

Modifying block content with AsciiDoc include

Upstream

Content
```
include::{context}-snip.adoc[]
. Run.
```
Partial
```
. Download.
. Build.
```
Output
1. Download.
2. Build.
3. Run.

Downstream

Partial
```
. Install.
```
Output
1. Install.
2. Run.

Benefits

Allows variation on complex block content.
https://docs.asciidoctor.org/asciidoc/latest/directives/include/

Antora features

Collecting multiple content sources with Antora

Upstream

content:
  sources:
    - url: ./ (1)
      branches: HEAD (1)
      start_path: docs (1)

Upstream

Downstream

content:
  sources:
    - url: https://github.com/crc-org/crc (1)
      tags: v2.23.0 (1)
      start_path: docs (1)
    - url: ./ (2)
      branches: HEAD (2)
      start_path: docs (2)

1	Upstream
2	Downstream

Benefits

Collect source files from multiple repositories.
Add downstream specific content (release notes).
https://docs.antora.org/antora/latest/playbook/configure-content-sources/

Combining sources with Distributed Component Versions

Upstream

name: getting_started (1)
title: Getting started (2)
version: ~ (1)
start_page: getting_started:introducing.adoc (2)
nav: (2)
    - modules/getting_started/nav.adoc (2)

1	Same as downstream.
2	Metadata only upstream

Downstream

name: getting_started (1)
version: ~ (1)

1	Same as upstream.

Collect source files that belong to the same component version from multiple repositories.
Add downstream files.
Cannot overwrite files.
https://docs.antora.org/antora/latest/distributed-component-version/

Modifying inline content with Antora

Upstream

asciidoc:
  attributes:
    prod: Project
    version: 1.42.0

Downstream

asciidoc:
  attributes:
    prod: Product
    version: 2.23

Benefits

Use AsciiDoc attributes.
Define default value in upstream component or playbook.
Replace the value in downstream playbook.
https://docs.antora.org/antora/latest/component-attributes/
https://docs.antora.org/antora/latest/playbook/asciidoc-attributes/

Modifying block content with Antora

Upstream

File tree

docs/getting_started/modules/getting_started/
└── examples
    └── project_installing.adoc
└── pages
    └── installing.adoc

installing.adoc

include::example${project-context}_installing.adoc[]
. Run.

project_installing.adoc

. Download.
. Build.

Output

Download.
Build.
Run.

Downstream

File tree

docs/getting_started/modules/getting_started/
└── examples
    └── product_installing.adoc

product_installing.adoc

. Install.

Output

Install.
Run.

Benefits

Including an example defined by a context-dependent attribute.
Provide the example file in a distributed component version.
Allows to handle complex block contents.
https://docs.antora.org/antora/latest/page/include-an-example/

Antora extensions features

Exporting content with Antora Assembler

Downstream

antora-assembler.yml

root_level: 1
component_versions: '*'
section_merge_strategy: fuse
build:
  command: ./tools/assembler2portal.sh
  dir: .cache/assembler
  keep_aggregate_source: true
  process_limit: 1

assembler2portal.sh (simplified)

# Get source file name
while getopts "a:o:f:" opt; do
  case "$opt" in
  o) source=${OPTARG/pdf/adoc}
    ;;
  esac
done
# Get destination file name
destination="portal/$(basename $source)"
# Copy files
cp --recursive "$source" "$destination"

Benefits

This extension assembles content from multiple pages into aggregate documents.
The Customer Portal can ingest the resulting monolithic AsciiDoc files.
https://gitlab.com/antora/antora-assembler

Generating content with Antora Collector

Upstream

Antora playbook

antora:
  extensions:
    - '@antora/collector-extension'

Antora component

ext:
  collector:
  - run:
      command: ./tools/validate_language.sh
  - run:
      command: ./tools/generate_reference.sh
    scan:
      base: modules/reference/examples/reference     #
      dir: build/collector/reference
      files: reference.adoc

Downstream

Antora playbook

antora:
  extensions:
    - '@antora/collector-extension'   #

Benefits

This extension delegates to external commands the ability to generate content from external sources.
Crafting reference guides from source code.
Modifying the outline: removing upstream only content and injecting downstream only content.
https://gitlab.com/antora/antora-collector-extension

Enabling search in Antora website with Antora Lunr

Upstream

Antora playbook

antora:
  extensions:
    - require: '@antora/lunr-extension'   #
      index_latest_only: true
      snippet_length: 142

Benefits

This extension adds a self-hosted search powered by Lunr to an Antora site.
https://gitlab.com/antora/antora-lunr-extension/

Writing your own extension

Antora provides an event-based extension facility you can tap into to augment or influence the functionality of the generator. This extension facility is designed for users of all experience levels.

https://docs.antora.org/antora/latest/extend/extensions/

Testing built site with a custom Antora extension

Downstream

Antora playbook

antora:
  extensions:
    - require: ./extensions/htmltest.js

extensions/htmltest.js

'use strict'
module.exports.register = function () {
    this.on('sitePublished', () => { (1)
        require('child_process').execFile('htmltest', (error, stdout, stderr) => { (2)
            if (error) {
                console.log(stdout + stderr);
                return;
                }
            console.log(stdout);
        })
    })
}

1	Run after building the site.
2	Run the external command `htmltest`.

Inject collector configuration

Downstream

extensions/inject-collector-configuration.js

'use strict'
const ADDITIONAL_COLLECTOR = [
  { run: { command: 'sed --in-place -e="/locally/d" modules/guide/nav.adoc' }}, (1)
  { run: { command: 'rm --force modules/guide/pages/installing-locally.adoc' }}, (2)
  { run: { command: 'mkdir --parents build/collector/downstream' }},(3)
  { run: { command: 'cp --recursive modules/guide/ build/collector/downstream/' }, (4)
    scan: { base: 'modules/guide/', dir: 'build/collector/downstream/guide' }} (4)
]
module.exports.register = function () {
  this.once('contentAggregated', ({ contentAggregate }) => {
    for (const { origins } of contentAggregate) {
      for (const origin of origins) {
        if (origin.descriptor.ext?.collector) {
          origin.descriptor.ext = { collector: ADDITIONAL_COLLECTOR.concat(origin.descriptor.ext?.collector) }
        }
      }
    }
  })
}

1	Remove pages from the navigation.
2	Delete the pages from the module.
3	Create the collector directory.
4	Copy files to the collector directory, and scan the entire modified module.

For the future?

Allows to only build a single reference into a site which has already been published.
https://gitlab.com/antora/antora-atlas-extension
https://github.com/spring-io/antora-extensions#partial-build

Demo

Credits

Antora: https://docs.antora.org/antora/latest/
AsciiDoc: https://docs.asciidoctor.org/asciidoc/latest/
Background photo: https://unsplash.com/photos/dGyshquBzOc
CRC docs: https://crc.dev/crc
CRC repository: https://github.com/crc-org/crc
Eclipse Che docs repository: https://github.com/eclipse-che/che-docs
Eclipse Che docs: https://www.eclipse.org/che/docs/
Red Hat OpenShift Dev Spaces docs: https://access.redhat.com/documentation/en-us/red_hat_openshift_dev_spaces
Red Hat OpenShift Local docs: https://access.redhat.com/documentation/en-us/red_hat_openshift_local