Skip to content

Pumbaa - Architecture Documentation

Overview

Pumbaa is a CLI tool for interacting with the Cromwell workflow engine and WDL files. The project follows Clean Architecture principles with a clear separation between domain logic, application use cases, infrastructure implementations, and user interfaces.

Core Features

  • Workflow Operations: Submit, query, monitor, abort, and debug Cromwell workflows
  • Interactive TUI Dashboard: Real-time workflow monitoring with filtering, status tracking, and run comparison (diff)
  • Debug TUI: Visual tree navigation of workflow execution with call details, failure analysis, and cost breakdown
  • AI Chat Agent: LLM-powered assistant for workflow analysis and troubleshooting (Gemini, Vertex AI, Ollama)
  • WDL Bundle Creation: Package WDL workflows with dependencies into distributable ZIP files
  • WDL Indexing: Fast search and discovery of WDL tasks and workflows
  • Resource Monitoring & Reports: Analyze resource usage from monitoring logs and generate HTML reports with optimization recommendations
  • Telemetry: Optional anonymous usage tracking via a self-hosted Cloudflare Worker endpoint

Project Structure

├── cmd/cli/                    # Application entry point
├── internal/                   # Private application code
│   ├── application/            # Use cases (Application Layer)
│   │   ├── errors.go           # Application layer error types
│   │   ├── ports/              # Port interfaces (Hexagonal Architecture)
│   │   ├── bundle/             # WDL bundle creation
│   │   └── workflow/           # Workflow use cases (flat structure)
│   │       ├── abort.go        # Abort running workflows
│   │       ├── batchlogs.go    # Fetch batch/task logs
│   │       ├── cache_resolver.go # Resolve call-cache hit chains
│   │       ├── compare.go      # Diff two workflow runs
│   │       ├── inputs.go       # Workflow inputs retrieval
│   │       ├── metadata.go     # Metadata retrieval
│   │       ├── monitoring.go   # Resource usage analysis
│   │       ├── outputs.go      # Workflow outputs retrieval
│   │       ├── query.go        # Workflow querying
│   │       ├── resource_report.go # Aggregate resource usage report
│   │       ├── resource_visualization.go # HTML resource report rendering
│   │       ├── submit.go       # Workflow submission
│   │       └── testutil_test.go # Shared test mocks
│   ├── config/                 # Configuration management
│   ├── container/              # Dependency injection container
│   ├── domain/                 # Domain entities (Domain Layer)
│   │   ├── bundle/             # Bundle entities and errors
│   │   ├── wdlindex/           # WDL index entities
│   │   └── workflow/           # Workflow entities and errors
│   ├── infrastructure/         # External services adapters (Infrastructure Layer)
│   │   ├── agents/             # LLM agent infrastructure
│   │   │   ├── llm/            # LLM providers (Gemini, Vertex, Ollama)
│   │   │   └── tools/          # Reusable tools for agents (Cromwell, GCS, local FS, WDL)
│   │   ├── cloudlogging/       # Google Cloud Logging adapter (batch logs)
│   │   ├── cromwell/           # Cromwell API client
│   │   ├── metrics/            # Task metrics TSV reader/writer
│   │   ├── recommendation/     # LLM-based resource recommendations
│   │   ├── session/            # SQLite session management
│   │   ├── storage/            # File storage (local and GCS)
│   │   ├── telemetry/          # Telemetry service (Cloudflare Worker / NoOp)
│   │   ├── templates/          # Embedded HTML templates for reports
│   │   ├── version/            # Version update checker
│   │   └── wdl/                # WDL indexer implementation
│   └── interfaces/             # UI adapters (Interface Layer)
│       ├── cli/handler/        # CLI command handlers
│       ├── cli/presenter/      # Output formatters
│       └── tui/                # Terminal User Interfaces
│           ├── chat/           # Chat agent TUI
│           ├── configwizard/   # Configuration setup wizard
│           ├── dashboard/      # Workflow dashboard TUI
│           └── debug/          # Debug tree navigation TUI
└── pkg/wdl/                    # Public WDL parsing library
    ├── ast/                    # Abstract Syntax Tree
    ├── parser/                 # ANTLR-generated parser
    └── visitor/                # AST visitor

Architecture Layers

1. Domain Layer (internal/domain/)

Contains business entities and value objects. This layer has no dependencies on external frameworks or libraries.

Packages:

  • workflow/: Core workflow entities (Workflow, Call, Status, HealthStatus)
  • errors.go - Domain errors (ErrWorkflowNotFound, ValidationError, APIError)
  • bundle/: WDL bundle entities
  • errors.go - Bundle errors (ErrCircularDependency, DependencyError)
  • wdlindex/: WDL index entities (Index, IndexedTask, IndexedWorkflow)

2. Application Layer (internal/application/)

Contains use cases that orchestrate domain logic, plus the port interfaces they depend on. Each use case is a single business operation with a clear input and output.

Ports (ports/ — Hexagonal Architecture):

  • workflow.go - WorkflowRepository, a composition of smaller focused interfaces (WorkflowQuerier, WorkflowMetadataFetcher, WorkflowSubmitter, WorkflowAborter, HealthChecker, LabelManager, ...). Consumers depend on the smallest interface that meets their needs.
  • storage.go - FileProvider (file content access), FileSizeCache, StorageBackend
  • wdlindex.go - WDLRepository for WDL indexing operations
  • batchlogs.go - BatchLogsRepository for fetching task logs from external log stores
  • metrics.go - TaskMetricsReader / TaskMetricsWriter for task metrics I/O
  • recommendation.go - RecommendationGenerator and LLMDebugWriter for resource optimization recommendations
  • errors.go - Port-level errors (ErrFileNotFound, ErrFileTooLarge, ErrInvalidPath, ErrAccessDenied, FileError)

Error Handling:

  • errors.go - Application-level errors (UseCaseError, InputValidationError)
  • Wraps domain/infrastructure errors with operation context
  • Allows handlers to differentiate error types

Use Cases (flat structure in workflow/):

  • submit.go - Submit workflows to Cromwell with validation
  • metadata.go - Retrieve workflow execution metadata
  • abort.go - Abort running workflows
  • query.go - Query workflows with filters (status, name, dates)
  • inputs.go / outputs.go - Retrieve workflow inputs and outputs
  • compare.go - Compare two workflow runs (metadata diff in the domain layer)
  • batchlogs.go - Fetch batch/task logs
  • cache_resolver.go - Follow call-cache hit chains to the original execution
  • monitoring.go - Analyze resource usage from monitoring logs (CPU, memory, disk)
  • resource_report.go - Aggregate resource usage across tasks into a report
  • resource_visualization.go - Render the resource report as an HTML page
  • bundle/ - Create WDL bundles with dependency resolution

3. Infrastructure Layer (internal/infrastructure/)

Contains implementations of external services and adapters for the application ports.

Implementations:

  • cromwell/: Cromwell REST API client implementing ports.WorkflowRepository
  • HTTP client with timeout configuration
  • JSON marshaling/unmarshaling
  • Error handling and status code mapping

  • Complete workflow lifecycle management

  • agents/: LLM agent infrastructure

  • llm/: LLM provider implementations
    • ollama/ - Local Ollama integration
    • gemini.go - Google Gemini API client
    • vertex.go - Google Vertex AI client
    • genai_stream.go - Streaming support for genai-based providers
    • factory.go - LLM provider factory pattern

  • tools/: Reusable tool registry for agents

    • cromwell/: Query, status, metadata, logs, outputs tools
    • gcs/: Google Cloud Storage file download
    • localfs/: Local file system access (read/write in the working dir)
    • wdl/: WDL search, list, and info tools
    • factory.go: builtinActions table — single source for action registration, descriptions, and schema enum
    • registry.go: Tool registration and schema generation

  • cloudlogging/: Google Cloud Logging client implementing ports.BatchLogsRepository

  • metrics/: TSV-based implementations of ports.TaskMetricsReader / ports.TaskMetricsWriter

  • recommendation/: LLM-backed implementation of ports.RecommendationGenerator for resource optimization suggestions (plus a mock for tests)

  • wdl/: WDL indexer implementing ports.WDLRepository

  • File system traversal
  • ANTLR-based parsing

  • JSON cache persistence

  • storage/: File provider for local and GCS paths

  • Implements ports.FileProvider interface

  • Size limits and validation

  • session/: SQLite-based session storage for chat history

  • Uses Google ADK session interface

  • Persistent conversation state

  • telemetry/: Anonymous usage tracking

  • cloudflare.go - Sends events to a Cloudflare Worker endpoint (endpoint and key injected at build time via -ldflags)
  • noop.go - NoOp implementation for privacy/development

  • service.go - Service interface

  • templates/: Embedded HTML templates (report.html) used by the resource visualization use case

  • version/: Checks for newer pumbaa releases

4. Interface Layer (internal/interfaces/)

Contains adapters for user interaction. This layer depends on the application layer but not on infrastructure.

CLI Handlers:

  • submit, metadata, abort, query, inputs, outputs - Standard workflow operations
  • diff - Compare two workflow runs
  • analyze, resource_report - Resource analysis and reports
  • bundle - WDL packaging
  • debug, dashboard - Launch TUI applications
  • chat - Launch AI chat agent TUI
  • config - Configuration management wizard

TUI Applications (Bubble Tea framework):

  • dashboard/: Real-time workflow monitoring
  • Auto-refresh workflow list
  • Keyboard navigation and filtering
  • Status-based color coding

  • Direct debug mode launch

  • debug/: Interactive debug explorer

  • Tree navigation of workflow calls
  • Call details with inputs/outputs
  • Failure analysis and logs display

  • Timeline visualization

  • chat/: AI chat interface

  • Message streaming
  • Session persistence
  • Markdown rendering

  • Copy-to-clipboard support

  • configwizard/: Interactive configuration setup

  • Directory picker
  • Provider selection
  • Validation and testing

Presenters:

  • cli/presenter/: Format data for CLI output (JSON, table, plain text)

5. Configuration (internal/config/)

Centralized configuration management with file persistence and environment variable support.

Config Fields: - Cromwell host and timeout - LLM provider settings (Gemini, Vertex, Ollama) - WDL directory and index path - Session database path - Telemetry toggle - Client ID for anonymous tracking

6. Dependency Injection (internal/container/)

The container wires all dependencies together following the dependency inversion principle.

Container Responsibilities: - Initialize infrastructure clients (Cromwell, Telemetry) - Create use cases with injected repositories - Build handlers with use cases and presenters - Provide singleton instances for the application lifecycle

Dependency Flow

┌──────────────────────────────────────────────────────┐
│                  Interface Layer                      │
│  (CLI Handlers, TUI Applications, Presenters)        │
└──────────────────┬───────────────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────────────┐
│                 Application Layer                     │
│      (Use Cases + Ports - Business Orchestration)     │
└──────────────────┬───────────────▲───────────────────┘
                   │               │
                   ▼               │ implements ports
┌──────────────────────────────┐   │
│         Domain Layer         │   │
│ (Entities, Business Rules)   │   │
└──────────────────────────────┘   │
                                   │
┌──────────────────────────────────┴───────────────────┐
│              Infrastructure Layer                     │
│    (Cromwell Client, LLM Providers, Storage, etc)    │
└──────────────────────────────────────────────────────┘

          Container (wires everything together)

Key Principles: - Dependencies point inward (toward domain) - Domain has zero external dependencies - Port interfaces live in application/ports/; infrastructure implements them - Interface layer depends only on application layer

Key Design Patterns

1. Ports & Adapters (Hexagonal Architecture)

The application layer defines port interfaces in application/ports/, implemented by adapters in infrastructure/. This inverts dependencies and allows business logic to remain independent of technical details.

2. Use Case Pattern

Each business operation is encapsulated in a use case with: - Clear input/output DTOs - Single responsibility - No framework dependencies

3. Dependency Injection

The container.Container manages object lifecycle and dependency wiring using constructor injection.

4. Factory Pattern

  • llm.NewLLM() creates appropriate LLM provider based on configuration
  • tools.GetAllTools() builds tool registry from available components

5. Strategy Pattern

Different LLM providers implement a common interface, allowing runtime provider selection.

6. Adapter Pattern

Infrastructure adapters translate between external APIs and application ports.

Testing Strategy

  • Domain Layer: Pure unit tests with no mocks
  • Application Layer: Test use cases with mock repositories
  • Infrastructure Layer: Integration tests with test servers/fixtures
  • Interface Layer: UI component tests with mock use cases

Known Architecture Considerations

Design Decisions

  • Centralized Ports Package: All port interfaces are defined in application/ports/ following Hexagonal Architecture. This makes it easy to see all external dependencies and maintains a clear boundary between application logic and infrastructure.

  • Composed WorkflowRepository: WorkflowRepository is a composition of small, focused interfaces (Interface Segregation Principle). Use cases depend on the narrowest interface they need (e.g. WorkflowMetadataReader), while the Cromwell client satisfies the full contract.

  • FileProvider abstraction: Single interface (ports.FileProvider) for file access, allowing implementations for local files, GCS, S3, or any other storage backend. Application layer depends only on the port.

  • WDLRepository: Dedicated port for WDL indexing operations, separating concerns between workflow execution (Cromwell) and workflow discovery (WDL index).

  • Chat agent tools in infrastructure: Tools are adapters to external services (Cromwell API, GCS, local FS, WDL files), implementing application ports where appropriate.

  • Session management: Delegates to Google ADK interfaces for compatibility with future storage backends.

  • Telemetry service: Interface-based design with a Cloudflare Worker backend and a NoOp implementation for privacy and development. The endpoint and API key are injected at build time, so development builds send nothing.

CI/CD Pipeline

The project uses GitHub Actions for continuous integration and releases:

Workflows

  • ci.yml: Runs on PRs and pushes to main
  • Go linting and formatting checks
  • Unit and integration tests with coverage
  • Build verification for multiple platforms

  • Uploads coverage and build artifacts

  • release.yml: Triggered on version tags (v*)

  • Runs full test suite
  • Uses GoReleaser to build multi-platform binaries
  • Creates GitHub releases with signed artifacts
  • Publishes binaries for Linux, macOS, Windows (amd64, arm64)

Release Process

  1. Tag a new version: git tag v1.2.3 && git push origin v1.2.3
  2. GitHub Actions builds and tests
  3. GoReleaser creates release with artifacts
  4. Users download from GitHub Releases or use install.sh script

External Dependencies

Core Libraries

  • urfave/cli/v2: CLI framework and command routing
  • charmbracelet/bubbletea + bubbles: TUI framework (Elm architecture) and components
  • charmbracelet/lipgloss: TUI styling and layout
  • charmbracelet/glamour: Markdown rendering in the chat TUI
  • antlr4-go/antlr/v4: WDL parsing (via generated parser)

Cloud & AI

  • google.golang.org/genai: Gemini and Vertex AI clients (single SDK for both)
  • cloud.google.com/go/storage: GCS file access
  • cloud.google.com/go/logging: Google Cloud Logging (batch logs)
  • Ollama: Local LLM via HTTP API

Infrastructure

  • google.golang.org/adk: Agent Development Kit (sessions, tools)
  • modernc.org/sqlite: Session storage (pure-Go SQLite driver, no CGO)
  • Telemetry: In-house HTTP client posting to a Cloudflare Worker (no third-party tracking SDK)

Last Updated: 2026-07-07 Document Version: 2.2 Project Version: See releases