build-docs.py

build-docs.py builds the complete Apache Juneau documentation, including both Docusaurus documentation and Maven-generated Javadocs. This script replicates the documentation generation steps from the GitHub Actions workflow, allowing you to build and test the documentation locally.

What It Does

The script performs the following steps:

1. Installs npm dependencies - Runs npm ci in the juneau-docs directory
2. Compiles Java modules - Runs mvn clean install -DskipTests to build all modules
3. Generates Maven site - Runs mvn site -DskipTests to generate the Maven project site (includes aggregate javadocs)
4. Copies current javadocs - Copies generated javadocs to juneau-docs/static/javadocs/<current-release>/
5. Updates javadocs index - Updates releases.json with version information
6. Copies Maven site - Copies the generated site to juneau-docs/static/site/ (Docusaurus will copy to build)
7. Builds Docusaurus - Runs npm run build to generate the static Docusaurus site (copies static/ to build/)
8. Copies .asf.yaml - Copies the ASF configuration file to the build directory
9. Verifies apidocs - Checks that the javadocs were successfully generated
10. Checks topic links - Validates all documentation links

Usage

Build all documentation

cd /path/to/juneau-docs
python3 scripts/build-docs.py

This will build both the Docusaurus documentation and the Maven site with javadocs.

Skip specific steps

You can skip individual steps if you only need to rebuild part of the documentation:

# Skip npm steps (useful if you only changed Java code)
python3 scripts/build-docs.py --skip-npm

# Skip Maven steps (useful if you only changed Docusaurus docs)
python3 scripts/build-docs.py --skip-maven

# Skip copying step (useful for testing without updating build directory)
python3 scripts/build-docs.py --skip-copy

# Build for staging (sets SITE_URL for staging environment)
python3 scripts/build-docs.py --staging

# Build against a Juneau checkout that isn't a sibling folder
python3 scripts/build-docs.py --juneau-root /path/to/juneau

Command-Line Options

• --skip-npm - Skip npm install and Docusaurus build
• --skip-maven - Skip Maven compilation and site generation
• --skip-copy - Skip copying Maven site to static directory
• --skip-verify - Skip the documentation-integrity, project-page and entry-page checks (verify-docs.py, check-project-pages.py, check-entry-pages.py)
• --staging - Build for staging environment (sets SITE_URL)
• --dry-run - Print every command and copy that would run; perform no work
• --verbose - Print full per-stage banners in plain output (default: one line per stage)
• --juneau-root DIR - Juneau source checkout to build against. Defaults to the JUNEAU_ROOT environment variable, then a sibling ../master/ checkout, then a sibling ../juneau/ checkout
• --console full|board|none - Console output when the live view is available (see below). Defaults to JUNEAU_TERMINAL_CONSOLE, then board

Live View

When the Juneau checkout includes juneau-terminal, the script loads it from <juneau-root>/juneau-terminal/src/main/python and shows each stage as one row of its live view:

Row	Stage
Clean	Prerequisite check and removal of build/
npm	npm ci
Install	mvn clean install -DskipTests, with a bar that fills module by module
Site	mvn site -DskipTests, with the same module bar
Copy	Current javadocs into static/javadocs/<version>/ and the Maven site into static/site/
Docs build	npm run build
Verify	verify-docs.py, check-project-pages.py, check-entry-pages.py
Links	check-topic-links.py, check-ai-artifacts.py, check-code-fences.py, check-java-fences.py --strict and check-guide-snippets.py

A stage turned off by a --skip-* flag (or by a running dev server, for npm and Docs build) shows as a skipped row with the reason. A failing stage turns its row red, with the tool's failure lines under it. --console board (the default) shows just the rows; full adds every tool's output; none prints only the result. Either way, the full output goes to a log under ~/.cache/juneau-terminal/, named in the header. See Console view.

When juneau-terminal can't be found (for example, a CI runner with no Juneau checkout) or can't be imported, the script prints a one-line warning where it applies and falls back to plain output: a banner per stage and each tool's own output, with the same exit codes. --dry-run always uses plain output.

Output Location

After successful execution, the documentation will be available in:

• Docusaurus docs: juneau-docs/build/
• Maven site: juneau-docs/build/site/
• Javadocs: juneau-docs/build/javadocs/ (versioned, copied from static/javadocs) and juneau-docs/build/site/apidocs/ (current)
• Javadocs index: juneau-docs/build/javadocs/index.html (dynamically loads from releases.json)

Requirements

• Python 3.10 or higher
• No external Python dependencies (uses only standard library)
• Node.js 18+ and npm
• Maven 3.6+ and Java 17+

Notes

• The script automatically skips tests (-DskipTests) to speed up the build
• All commands are run from the juneau-docs repository root; the script requires a Juneau checkout (the Java tree, located as described under --juneau-root) to run the Maven steps against, unless --skip-maven is used
• Whenever a Juneau checkout is found, even with --skip-maven, the script regenerates artifact-paths.json and artifact-packages.json from its module pom.xml files; if the checkout has no jar-packaged module, the script fails and leaves the existing artifact-paths.json untouched
• The script will exit immediately if any command fails (unless using --skip-* options)
• The build directory (juneau-docs/build/) will be created if it doesn't exist
• Existing site contents will be replaced when copying

Build Order

The script copies Maven site to static/site/ and updates javadocs in static/javadocs/ before building Docusaurus. Docusaurus automatically copies everything from the static/ directory to the build/ directory during the build process, so the files are available when Docusaurus processes links, preventing broken link warnings.

Generated images

The screenshots of the petstore console and the other product UI in the topic pages are not taken by hand. scripts/docs-images.py regenerates them, and the generated PNGs and SVGs are committed under static/topics/. Re-run it when the UI changes and at each release, then review the changed images before committing them.

python3 scripts/docs-images.py                      # regenerate every image
python3 scripts/docs-images.py --only 'themes-*'    # regenerate matching case ids or image names (globs allowed)
python3 scripts/docs-images.py --source ui          # one source only (ui or terminal)
python3 scripts/docs-images.py --check              # write nothing; report problems, exit 1 on any
python3 scripts/docs-images.py --base-url URL       # shoot an already-running petstore instead of starting one
python3 scripts/docs-images.py --juneau-root DIR    # Juneau checkout (same default as build-docs.py)

Every generated image is a case in scripts/docs-images/ui-cases.json: an id, the image path under static/, the petstore path to open, optional harness actions to stage the page (click, fill, wait for a card), and an optional CSS selector to shoot one element instead of the whole page. Shots are 1280 pixels wide at a device scale factor of 2, with animations and the caret disabled.

Without --base-url, the script reuses a petstore already answering on http://localhost:10000, or starts one with the Juneau checkout's scripts/start-petstore-jetty.py and stops it when done. It drives the Playwright harness juneau-petstore/juneau-petstore-core/src/test/js/petstore-browser.cjs, using the Playwright that the js-tests Maven profile installs under that module's target/js (and installs it there if missing). Images are moved into static/ only after every case of the run succeeded; a case that times out, hits an HTTP error or a script error fails the run and keeps the previous image.

Terminal output is the second source. A case in scripts/docs-images/terminal-cases.json names a fixture command ({juneau_terminal} stands for the Juneau checkout's juneau_terminal.py, {fixtures} for scripts/docs-images/fixtures) and a fixed pane size (columns and rows, 100 by 30 by default). The script runs the command in a detached tmux session on a private tmux server, waits until the pane shows the waitFor regex and has stopped changing, captures it with tmux capture-pane -e -p, and writes an SVG whose text is real <text> elements, in one dark terminal palette that suits both site themes. Anything that varies between runs, such as elapsed times and the spinner frame, is replaced through the case's mask list of [regex, replacement] pairs, so running the script twice gives byte-identical files. A case with panes renders several panes side by side in one image (an agent pane beside a run pane, for example). This source needs tmux and Python only: no Node.js, no petstore.

--check needs no browser and reports, one per line:

• MISSING: a case whose image does not exist.
• ORPHANED: an image under static/topics or static/developer-info that no case produces and that the cases file's unmanaged list does not exempt. Each unmanaged entry records why its images are left as they are.
• DANGLING: an image reference in pages/ to a file that does not exist.
• UNREFERENCED: a case whose image no page uses (a warning only).

To add a screenshot, add a case, run python3 scripts/docs-images.py --only <id>, and reference the image from the page with a Markdown image whose path is /topics/<Page>.<Section>.<n>.png (.svg for a terminal case). Alt text is required.

Integration with CI/CD

This script is used by the documentation release scripts:

• release-docs-stage.py - Calls build-docs.py --staging to build for staging; under the live view, the build's rows join the staging run's
• release-docs.py - Promotes already-built documentation from staging to production