Automated PR Review with AOF

This tutorial shows you how to set up automated, AI-powered pull request reviews using AOF. Build a multi-agent review system that checks PRs for security vulnerabilities, performance issues, and code quality—all automatically triggered by GitHub webhooks.

What You'll Build

An automated PR review system that:

• Triggers automatically when PRs are opened or updated
• Reviews code from multiple perspectives (security, performance, quality)
• Uses consensus-based fleet coordination for reliable results
• Posts comprehensive review comments on GitHub
• Labels PRs based on findings
• Runs entirely automated—no manual intervention required

Example workflow: Developer opens PR → AOF detects it → Fleet reviews code in parallel → Results posted as PR comment with labels.

Prerequisites

• AOF installed (curl -sSL https://docs.aof.sh/install.sh | bash)
• GitHub account with repository access
• GitHub Personal Access Token (PAT) or GitHub App
• Public HTTPS endpoint (use cloudflared, ngrok, or a server)
• API key for your LLM provider (we'll use Google Gemini in examples)

Architecture Overview

GitHub PR Event → Webhook → AOF Daemon → Trigger → Fleet → Review Agents
                                                      ↓
                                            (Consensus + Aggregation)
                                                      ↓
                                         Post Review Comment + Labels

Step 1: Create the GitHub Trigger

The trigger listens for GitHub webhook events and routes them to agents, fleets, or flows. All command routing is defined in the Trigger—the DaemonConfig only enables platforms.

Create triggers/github-pr-trigger.yaml:

apiVersion: aof.dev/v1
kind: Trigger
metadata:
  name: github-pr-bot
  labels:
    platform: github
    purpose: pr-automation

spec:
  type: GitHub

  config:
    # Webhook secret for signature verification
    webhook_secret: ${GITHUB_WEBHOOK_SECRET}

    # GitHub events to listen for
    github_events:
      - pull_request           # PR lifecycle events
      - pull_request_review    # Review submissions
      - issue_comment          # Comments on PRs (for /commands)

    # Optional: Filter by repository
    repositories:
      - "myorg/myrepo"

    # Optional: Filter by target branch
    branches:
      target:
        - main
        - develop

  # Command mappings - route events and slash commands to handlers
  commands:
    # Automatic triggers on PR events
    pull_request.opened:
      fleet: code-review-team
      params:
        review_type: comprehensive
        auto_comment: true

    pull_request.synchronize:
      fleet: code-review-team
      params:
        review_type: incremental  # Only review changed files

    # Slash commands (triggered via PR comments)
    /review:
      fleet: code-review-team
      description: "Run comprehensive code review"
      response: |
        🔍 Starting comprehensive code review...

    /review security:
      agent: security-reviewer
      description: "Security-focused review"
      response: "🔒 Running security scan..."

    /review performance:
      agent: performance-reviewer
      description: "Performance analysis"
      response: "⚡ Analyzing performance..."

    /lgtm:
      agent: approval-agent
      description: "Approve PR"
      response: "✅ LGTM! Adding approval..."

    /deploy staging:
      flow: deploy-staging-flow
      description: "Deploy to staging"
      response: "🚀 Deploying to staging..."

    /help:
      agent: help-bot
      response: |
        🤖 **Available Commands:**
        - `/review` - Comprehensive code review
        - `/review security` - Security-focused review
        - `/review performance` - Performance analysis
        - `/lgtm` - Approve PR
        - `/deploy staging` - Deploy to staging

  # Default for unmatched events
  default_agent: devops

  enabled: true

Key points:

• commands section maps both events (pull_request.opened) and slash commands (/review) to handlers
• Each command can route to an agent, fleet, or flow
• response provides immediate feedback to the user
• This design keeps all routing logic in one place, making it easy to manage

Step 2: Create Review Agents

Create three specialized agents, each focused on a specific review aspect.

Security Reviewer

Create agents/security-reviewer.yaml:

apiVersion: aof.dev/v1
kind: Agent
metadata:
  name: security-reviewer
  labels:
    role: reviewer
    specialty: security

spec:
  model: google:gemini-2.5-flash

  model_config:
    temperature: 0  # Deterministic for security checks
    max_tokens: 4096

  instructions: |
    You are a security engineer reviewing code for vulnerabilities.

    Your focus areas:
    - SQL injection, XSS, CSRF vulnerabilities
    - Authentication and authorization flaws
    - Insecure cryptography usage
    - Secrets or credentials in code
    - Input validation issues
    - Race conditions and concurrency bugs
    - Security misconfigurations
    - Dependency vulnerabilities

    For each issue found:
    1. Severity: CRITICAL, HIGH, MEDIUM, LOW
    2. Location: File and line number
    3. Description: What's vulnerable
    4. Impact: What could happen if exploited
    5. Fix: How to resolve it (with code example if helpful)
    6. Reference: Link to CWE, OWASP, or CVE if applicable

    Be thorough but practical. Focus on real security risks, not theoretical edge cases.
    If code is secure, say so clearly and praise good security practices.

  tools:
    - shell
    - read_file
    - list_directory

  memory: "InMemory"

Performance Reviewer

Create agents/performance-reviewer.yaml:

apiVersion: aof.dev/v1
kind: Agent
metadata:
  name: performance-reviewer
  labels:
    role: reviewer
    specialty: performance

spec:
  model: google:gemini-2.5-flash

  model_config:
    temperature: 0.2
    max_tokens: 4096

  instructions: |
    You are a performance engineer reviewing code for efficiency issues.

    Your focus areas:
    - Algorithmic complexity (O(n²) loops, inefficient algorithms)
    - Database query optimization (N+1 queries, missing indexes, full table scans)
    - Memory leaks and resource management
    - Caching opportunities
    - Lazy loading vs eager loading trade-offs
    - Unnecessary computations or redundant work
    - Inefficient data structures
    - Network round trips and API call batching

    For each issue found:
    1. Impact: Estimated performance degradation (response time, memory, CPU)
    2. Location: File and line number
    3. Current: What's happening now (with complexity analysis)
    4. Improvement: Specific optimization suggestion
    5. Trade-offs: Any downsides to the fix
    6. Priority: HIGH (user-facing), MEDIUM (backend), LOW (negligible impact)

    Focus on issues that matter in production. Avoid micro-optimizations unless
    they're in hot paths. If performance looks good, say so and highlight good practices.

  tools:
    - shell
    - read_file
    - list_directory

  memory: "InMemory"

Quality Reviewer

Create agents/quality-reviewer.yaml:

apiVersion: aof.dev/v1
kind: Agent
metadata:
  name: quality-reviewer
  labels:
    role: reviewer
    specialty: quality

spec:
  model: google:gemini-2.5-flash

  model_config:
    temperature: 0.3
    max_tokens: 4096

  instructions: |
    You are a senior engineer reviewing code for quality and maintainability.

    Your focus areas:
    - Code clarity and readability
    - Naming conventions (descriptive, consistent)
    - Function/method length and complexity
    - DRY principle violations (code duplication)
    - SOLID principle adherence
    - Error handling completeness and clarity
    - Test coverage and test quality
    - Documentation quality (docstrings, comments)
    - API design and interface contracts
    - Type safety and null safety

    For each issue found:
    1. Category: Readability, Maintainability, Design, Testing, Documentation
    2. Location: File and line number
    3. Issue: What could be better
    4. Suggestion: Specific improvement with reasoning
    5. Example: Show improved code if helpful (brief snippet)
    6. Priority: MUST (blocks merge), SHOULD (important), COULD (nice-to-have)

    Be constructive and educational. Praise good patterns when you see them.
    Focus on issues that affect long-term maintainability, not just style preferences.
    If code quality is excellent, say so and highlight what makes it good.

  tools:
    - shell
    - read_file
    - list_directory

  memory: "InMemory"

Step 3: Create the Review Fleet

The fleet coordinates the three reviewers using peer mode with consensus.

Create fleets/code-review-team.yaml:

apiVersion: aof.dev/v1
kind: AgentFleet
metadata:
  name: code-review-team
  description: "Multi-perspective code review team"
  labels:
    team: engineering
    purpose: quality-assurance

spec:
  # All agents work in parallel as peers
  agents:
    - name: security-reviewer
      role: specialist
      weight: 1.5  # Security findings count more in consensus
      spec:
        ref: agents/security-reviewer.yaml

    - name: performance-reviewer
      role: specialist
      weight: 1.0
      spec:
        ref: agents/performance-reviewer.yaml

    - name: quality-reviewer
      role: specialist
      weight: 1.0
      spec:
        ref: agents/quality-reviewer.yaml

  # Coordination: All agents review in parallel
  coordination:
    mode: peer
    distribution: round-robin

    # Consensus for final recommendation
    consensus:
      algorithm: weighted
      min_votes: 2
      timeout_ms: 90000  # 90 seconds
      allow_partial: true  # Don't fail if one agent is slow
      min_confidence: 0.7

      # Weights for consensus
      weights:
        security-reviewer: 1.5
        performance-reviewer: 1.0
        quality-reviewer: 1.0

  # Shared memory for cross-agent context
  shared:
    memory:
      type: in_memory

  # Resource limits
  resources:
    max_tokens_per_agent: 4096
    timeout: 120s

Key features:

• mode: peer - All agents review in parallel
• consensus.algorithm: weighted - Security has higher weight (1.5x)
• min_votes: 2 - At least 2 agents must agree
• allow_partial: true - Continue even if one agent fails

Step 4: Create the Workflow (Optional)

For advanced use cases, create a workflow that handles the full PR lifecycle.

Create flows/pr-review-workflow.yaml:

apiVersion: aof.dev/v1alpha1
kind: AgentFlow
metadata:
  name: pr-review-workflow
  description: "Full PR review lifecycle"

spec:
  # Triggered by GitHub PR events
  trigger:
    platform: github
    events:
      - pull_request.opened
      - pull_request.synchronize

  # Input context from GitHub webhook
  input:
    pr_number: "{{event.pull_request.number}}"
    repo: "{{event.repository.full_name}}"
    author: "{{event.pull_request.user.login}}"
    head_sha: "{{event.pull_request.head.sha}}"
    files_changed: "{{event.pull_request.changed_files}}"

  # Skip drafts and bot PRs
  conditions:
    - "{{not event.pull_request.draft}}"
    - "{{event.pull_request.user.type != 'Bot'}}"

  steps:
    # Step 1: Fetch changed files
    - name: get-files
      agent: github-helper
      action: get_pr_files
      input:
        repo: "{{input.repo}}"
        pr_number: "{{input.pr_number}}"

    # Step 2: Run fleet review
    - name: review
      fleet: code-review-team
      input:
        files: "{{steps.get-files.output.files}}"
        pr_context:
          number: "{{input.pr_number}}"
          author: "{{input.author}}"
          repo: "{{input.repo}}"

    # Step 3: Aggregate results
    - name: aggregate
      action: merge_results
      input:
        security: "{{steps.review.security-reviewer.output}}"
        performance: "{{steps.review.performance-reviewer.output}}"
        quality: "{{steps.review.quality-reviewer.output}}"

    # Step 4: Post review comment
    - name: post-comment
      agent: github-helper
      action: post_comment
      input:
        repo: "{{input.repo}}"
        issue_number: "{{input.pr_number}}"
        body: |
          ## 🤖 Automated Code Review

          Hey @{{input.author}}! I've reviewed your changes.

          ### Summary
          - **Files Changed**: {{input.files_changed}}
          - **Reviewers**: Security, Performance, Quality

          ### 🛡️ Security Review
          {{steps.aggregate.output.security.summary}}

          ### ⚡ Performance Review
          {{steps.aggregate.output.performance.summary}}

          ### ✨ Quality Review
          {{steps.aggregate.output.quality.summary}}

          ### Overall Assessment
          {{steps.aggregate.output.consensus.recommendation}}

          ---
          *Automated review by [AOF](https://docs.aof.sh)*

    # Step 5: Add labels
    - name: add-labels
      agent: github-helper
      action: add_labels
      input:
        repo: "{{input.repo}}"
        issue_number: "{{input.pr_number}}"
        labels: "{{steps.aggregate.output.suggested_labels}}"

Step 5: Configure the Daemon

The DaemonConfig is minimal—it enables platforms and points to resource directories. All command routing is defined in your Triggers, not here.

Create config/daemon.yaml:

apiVersion: aof.dev/v1
kind: DaemonConfig
metadata:
  name: aof-daemon

spec:
  # Server configuration
  server:
    port: 3000
    host: "0.0.0.0"
    cors: true
    timeout_secs: 60

  # Enable platforms (webhook endpoints)
  # NOTE: Command routing is defined in Triggers, not here
  platforms:
    github:
      enabled: true
      token_env: GITHUB_TOKEN
      webhook_secret_env: GITHUB_WEBHOOK_SECRET
      bot_name: aofbot  # Optional: for @mentions

  # Resource discovery paths
  triggers:
    directory: "./triggers"
    watch: true  # Hot-reload on changes

  agents:
    directory: "./agents"
    watch: false

  fleets:
    directory: "./fleets"
    watch: false

  flows:
    directory: "./flows"
    enabled: true
    watch: false

  # Runtime limits
  runtime:
    max_concurrent_tasks: 10
    task_timeout_secs: 300
    default_agent: devops

Key points:

• platforms.github enables the /webhook/github endpoint
• token_env and webhook_secret_env reference environment variables (not values)
• triggers.directory points to where your Trigger files live
• Command routing is NOT in DaemonConfig—it's in your Trigger files
• Webhook endpoint: http://your-domain:3000/webhook/github

Step 6: Set Up Environment Variables

Create a .env file (never commit this to git):

# GitHub webhook secret (generate with: openssl rand -hex 32)
export GITHUB_WEBHOOK_SECRET="your-webhook-secret-here"

# GitHub Personal Access Token (for posting comments)
export GITHUB_TOKEN="ghp_your_github_token_here"

# LLM API Key (using Google Gemini in this example)
export GOOGLE_API_KEY="your-google-api-key-here"

Load the environment:

source .env

GitHub Token Permissions Required:

• Repository: Contents (Read)
• Repository: Pull requests (Read & Write)
• Repository: Issues (Read & Write) - for labels

Step 7: Start the AOF Daemon

# Start the webhook server
aofctl serve --config config/daemon.yaml

# Expected output:
# ✓ Loaded 3 agents
# ✓ Loaded 1 fleet
# ✓ Loaded 1 trigger
# ✓ GitHub webhook endpoint: /webhooks/github
# ✓ Server listening on http://0.0.0.0:3000

Step 8: Expose Your Webhook Endpoint

Choose one of these options:

Option A: Cloudflared Tunnel (Recommended for Testing)

# Install cloudflared
brew install cloudflare/cloudflare/cloudflared

# Create tunnel
cloudflared tunnel --url http://localhost:3000

# Note the public URL (e.g., https://random-name.trycloudflare.com)

Option B: ngrok

# Install ngrok
brew install ngrok

# Start tunnel
ngrok http 3000

# Note the public URL

Option C: Production Server

Deploy to a server with a public domain and HTTPS:

# Example with systemd
sudo systemctl enable aof-daemon
sudo systemctl start aof-daemon

# Use your domain: https://aof.example.com

Step 9: Configure GitHub Webhook

1. Go to your repository on GitHub
2. Navigate to Settings → Webhooks → Add webhook
3. Configure:
   • Payload URL: https://your-domain.com/webhooks/github
   • Content type: application/json
   • Secret: Use the same value as GITHUB_WEBHOOK_SECRET
   • Which events: Select "Pull requests"
   • Active: ✓ Checked
4. Click Add webhook

Verify the webhook:

• GitHub will send a ping event
• Check the "Recent Deliveries" tab for a green checkmark
• Check AOF logs: tail -f logs/aof-daemon.log

Step 10: Test It!

Create a test PR to verify everything works:

1. Create a Test PR

# In your repository
git checkout -b test-pr-review
echo "console.log('Hello, world!');" > test.js
git add test.js
git commit -m "Add test file"
git push origin test-pr-review

# Open PR on GitHub
gh pr create --title "Test PR Review" --body "Testing automated review"

2. Watch the Magic Happen

Within seconds, you should see:

1. AOF logs show webhook received:

   ✓ Received GitHub webhook: pull_request.opened
   ✓ Routing to fleet: code-review-team
   ✓ Starting 3 agents in parallel...

2. GitHub PR gets a comment:

   ## 🤖 Automated Code Review
   
   Hey @alice! I've reviewed your changes.
   
   ### Summary
   - **Files Changed**: 1
   - **Reviewers**: Security, Performance, Quality
   
   ### 🛡️ Security Review
   ✅ No security issues found.
   
   ### ⚡ Performance Review
   ✅ No performance concerns.
   
   ### ✨ Quality Review
   ⚠️ Missing error handling for console.log
   
   ### Overall Assessment
   ✅ Approved - Minor suggestions for improvement

3. Labels are automatically added:

   • automated-review
   • lgtm (if approved)

Expected Output Example

Here's what a comprehensive review looks like:

## 🤖 Automated Code Review

Hey @developer! I've reviewed your changes.

### Summary
- **Files Changed**: 3
- **Reviewers**: Security, Performance, Quality

---

### 🛡️ Security Review

**Status**: ⚠️ 2 issues found

#### HIGH: SQL Injection Vulnerability
- **Location**: `src/api/users.go:45`
- **Issue**: User input directly concatenated into SQL query
- **Impact**: Attacker could extract or modify database contents
- **Fix**: Use parameterized queries
  ```go
  // Bad
  query := "SELECT * FROM users WHERE id = " + userID

  // Good
  query := "SELECT * FROM users WHERE id = $1"
  db.Query(query, userID)

• Reference: CWE-89

MEDIUM: Hardcoded Secret

• Location: config/app.yaml:12
• Issue: API key hardcoded in config file
• Fix: Use environment variables
• Reference: OWASP A02

---

⚡ Performance Review

Status: ✅ No critical issues

Suggestion: Database Index

• Location: src/api/search.go:78
• Current: Full table scan on users.email
• Improvement: Add index on email column
• Impact: ~100x faster for user lookups
• Priority: MEDIUM

---

✨ Quality Review

Status: ✅ Good code quality

Positive highlights:

• ✅ Excellent test coverage (92%)
• ✅ Clear function names and documentation
• ✅ Consistent error handling

Suggestion: Extract Function

• Location: src/api/auth.go:120-180
• Issue: Function is 60 lines long, does multiple things
• Suggestion: Extract validation logic into separate function
• Priority: SHOULD

---

Overall Assessment

⚠️ Changes Requested - Please address HIGH security issues before merging.

Summary:

• Security: 1 HIGH, 1 MEDIUM
• Performance: No critical issues
• Quality: Excellent

Recommendation: Fix SQL injection vulnerability, consider other suggestions.

---

Automated review by AOF • Re-run review

## Customization

### Adjust Review Criteria

Modify agent instructions to match your standards:

```yaml
# In security-reviewer.yaml
spec:
  instructions: |
    ADDITIONAL CHECKS:
    - Check for use of deprecated libraries
    - Verify HTTPS is used for all external calls
    - Ensure secrets are never logged

Change Consensus Requirements

Adjust the fleet's consensus settings:

# In code-review-team.yaml
spec:
  coordination:
    consensus:
      algorithm: unanimous  # All agents must agree
      min_votes: 3          # All 3 reviewers required

Add More Reviewers

Add specialized reviewers to the fleet:

# Add accessibility reviewer
agents:
  - name: a11y-reviewer
    role: specialist
    spec:
      model: google:gemini-2.5-flash
      instructions: |
        Review for accessibility (WCAG 2.1):
        - Semantic HTML
        - ARIA labels
        - Keyboard navigation
        - Color contrast

Custom Label Logic

Modify what labels get added:

# In the workflow, adjust label logic
- name: add-labels
  input:
    labels:
      - "automated-review"
      # Add labels based on findings
      - label: "security-critical"
        condition: "{{security.critical_count > 0}}"
      - label: "needs-refactoring"
        condition: "{{quality.complexity_issues > 3}}"
      - label: "performance-concern"
        condition: "{{performance.high_count > 0}}"

Skip Certain Files

Ignore auto-generated or vendor files:

# In the workflow
conditions:
  # Skip if only generated files changed
  - "{{not (files | all_match('.*_gen.go$'))}}"
  # Skip if only vendor directory changed
  - "{{not (files | all_match('^vendor/.*'))}}"

Integrate with CI/CD

Block PR merges if review fails:

# In the workflow
- name: create-check-run
  agent: github-helper
  action: create_check_run
  input:
    name: "AOF Code Review"
    conclusion: "{{steps.aggregate.output.status}}"  # success/failure
    # This will show as a required status check

Then in GitHub:

• Settings → Branches → Branch protection rules
• Add rule for main
• Require status check: "AOF Code Review"

Troubleshooting

Webhook Not Received

Check 1: Webhook Delivery

• GitHub → Settings → Webhooks → Edit
• Click "Recent Deliveries"
• Look for green checkmarks
• If red X, check the error message

Check 2: Signature Verification

# Verify secret matches in both places
echo $GITHUB_WEBHOOK_SECRET  # Local environment
# Compare with GitHub webhook configuration

Check 3: Server Logs

tail -f logs/aof-daemon.log
# Should see: "Received GitHub webhook: pull_request.opened"

Review Not Posted

Check 1: GitHub Token Permissions

# Test token
curl -H "Authorization: token $GITHUB_TOKEN" \
  https://api.github.com/repos/myorg/myrepo

# Should return 200 OK

Check 2: Agent Errors

# Check agent logs for errors
grep -i error logs/aof-daemon.log

Check 3: LLM API Key

# Verify API key works
curl -H "x-goog-api-key: $GOOGLE_API_KEY" \
  https://generativelanguage.googleapis.com/v1beta/models

Review is Slow

Optimize fleet settings:

coordination:
  consensus:
    timeout_ms: 60000  # Reduce from 90s to 60s
    allow_partial: true  # Don't wait for slow agents

resources:
  max_tokens_per_agent: 2048  # Reduce from 4096

Use faster model:

spec:
  model: google:gemini-2.0-flash-exp  # Faster variant

Inconsistent Reviews

Increase temperature for consistency:

model_config:
  temperature: 0  # More deterministic

Use unanimous consensus:

consensus:
  algorithm: unanimous  # All agents must agree

Next Steps

Extend to More Events

Add more GitHub automation:

# In trigger config
github_events:
  - pull_request
  - push
  - issues
  - deployment

# Route different events
commands:
  /deploy:
    flow: deploy-flow
  /triage:
    agent: issue-triage-agent

Add Deployment Automation

See: GitHub Automation Tutorial

Integrate with More Platforms

• Slack Bot Tutorial - Post reviews to Slack
• Telegram Bot - Mobile notifications

Advanced Patterns

• Multi-Model RCA - Use multiple LLMs for consensus
• Incident Response - Automated incident workflows

Summary

You've built an automated PR review system that:

✅ Triggers on GitHub PR events via webhooks ✅ Reviews code from 3 perspectives (security, performance, quality) ✅ Uses fleet consensus for reliable results ✅ Posts comprehensive reviews as PR comments ✅ Automatically labels PRs based on findings ✅ Runs entirely hands-off

Total setup time: ~15 minutes Cost per review: ~$0.01 (using Gemini Flash) Review time: 30-60 seconds

The beauty of AOF is composability—swap models, add more reviewers, integrate with CI/CD, and customize review criteria to match your team's standards.

Happy automating! 🚀