A cross-platform desktop app and CLI that turns AI coding sessions into first-class, reviewable Git branches. Create isolated sessions, chat with Amp interactively, track real-time metrics, and merge changes cleanly to your main branch.
Key Features:
- Production & Development Support: Switch between production Amp and local development environments
- Live Mode Switching: Change Amp configurations without restarting the application
- Comprehensive Testing: Run evaluations and benchmarks against both environments
git clone https://github.com/sjarmak/amp-session-orchestrator.git
cd amp-session-orchestrator
pnpm install && pnpm build
cd apps/desktop && pnpm devPrerequisites: Git ≥2.38, Node.js ≥18, pnpm ≥8, authenticated Amp CLI
The Amp Session Orchestrator supports both production and local development Amp environments, allowing you to test changes and run evaluations against your development setup.
If you have a local Amp development environment:
- Configure Local Mode: Click the lefthand grid icon in the desktop app header
- Select Amp Mode:
- Production: Use the standard Amp service (default)
- Local CLI: Use a local Amp CLI binary for development
- Local Server: Connect to your local Amp development server
- Set Paths: Configure CLI path to your local Amp CLI path (e.g, /Users/username/amp/cli/dist/main.js)
- Authentication:
- Production: Ensure
amp loginis authenticated in your terminal - Local: Set up your local development environment authentication
- Production: Ensure
You can switch between production and local development modes at any time:
- Open Settings: Click the lefthand grid icon in the app header
- Change Mode: Select your desired Amp environment
- Update Paths: Adjust CLI path to your local Amp CLI path (e.g, /Users/username/amp/cli/dist/main.js)
- Apply: Settings take effect immediately without restarting and you can swap between production and development without restarting the app
Testing Local Changes:
- Switch to local mode to test your Amp modifications
- Run sessions and benchmarks against your development environment
- Switch back to production for normal usage
Evaluation and Testing:
- Use local mode for extensive testing without production usage limits
- Run benchmarks against your local development setup
- Compare results between local and production environments
- Launch the Desktop App: Run
pnpm devfrom theapps/desktopdirectory - Navigate to Sessions Tab: Main interface for session management
- Create New Session:
- Click "New Session" to open the creation modal
- Select Repository: Use the file browser to choose your project directory
- Configure Session:
- Session name (descriptive identifier)
- Base branch (default:
main) - Model selection (Default currently Claude Sonnet 4, GPT-5, Alloy)
- Click "Create" to initialize the session worktree and branch
The Interactive tab provides real-time communication with Amp:
- Starting Conversations: Type your coding request or question
- Continuing Threads: Previous messages maintain conversation context
- Thread Management:
- Each session maintains its own conversation thread
- Switch between sessions to continue different conversations
- All chat history is preserved and reviewable
- Real-time Streaming: Watch Amp's responses appear live
- Tool Call Execution: See detailed tool usage and results
- Connection Status: Monitor Amp connectivity and authentication
The Overview tab displays comprehensive session analytics:
- Performance Metrics:
- Tokens per second processing rate
- Cost per minute tracking
- Real-time session progress
- Token Usage Breakdown:
- Input/output tokens by model
- Total session consumption
- Cost analysis and budgeting
- Tool Usage Statistics:
- Tool call frequency and success rates
- File modification tracking
- Error analysis and debugging
- Session Timeline:
- Iteration history and progress
- Commit timeline with messages
- Status changes and events
- Export Options: Download metrics as JSON, CSV, or JSONL
The Git tab provides full version control operations:
- File Status Monitor:
- View staged and unstaged changes
- Preview file modifications with diff viewer
- Track which files Amp has modified
- Staging Operations:
- Stage individual files or all changes
- Unstage files before committing
- Review changes before finalizing
- Commit Workflow:
- Create commits with custom messages
- View commit history and timeline
- All Amp commits automatically prefixed with
amp:
- Merge to Main:
- Squash: Combine all
amp:commits into single commit - Rebase: Apply changes onto latest main branch
- Conflict Resolution: Guided workflow for merge conflicts
- Final Merge: Clean integration back to main branch
- Squash: Combine all
- Real-time Status: Track session state (idle, running, awaiting-input, error, done)
- Background Processing: Monitor long-running operations
- Notifications: Desktop alerts for session completion or errors
- Progress Tracking: Visual indicators for iteration progress
- Navigate to Batches Tab: Access batch processing interface
- Create Batch Configuration:
- Upload YAML configuration file or create inline
- Define multiple repositories and prompts
- Set concurrency limits and validation scripts
- Monitor Execution:
- Real-time progress tracking across all sessions
- Individual session status updates
- Aggregate metrics and success rates
- Review Results: Export batch results and analyze outcomes
# Execute batch (production)
pnpm cli batch start --file batch-config.yaml
# Execute batch (local development)
pnpm cli batch start --file batch-config.yaml --amp-server https://localhost:7002
# Monitor progress
pnpm cli batch status batch-123 --follow
# Export results
pnpm cli batch export batch-123 --format csvAll CLI commands support local Amp development environments:
# Session commands with local Amp
pnpm cli new --repo ./my-project --name "test" --prompt "Hello" --amp-server https://localhost:7002
pnpm cli iterate session-id --amp-cli /path/to/local/amp/cli/dist/main.js
# Benchmark commands with local Amp
pnpm cli benchmark eval.yaml --amp-server https://localhost:7002
# Use environment variables
export AMP_SERVER_URL="https://localhost:7002"
export AMP_CLI_PATH="/path/to/local/amp/cli/dist/main.js"
pnpm cli benchmark eval.yaml# batch-config.yaml - Complete configuration example
runId: "custom-batch-id" # Optional: Custom batch identifier
concurrency: 3 # Required: Max parallel sessions
# Default settings applied to all matrix items
defaults:
baseBranch: "main" # Required: Default base branch
scriptCommand: "npm test" # Optional: Default test/validation script
model: "claude-3-5-sonnet" # Optional: Default AI model
jsonLogs: true # Optional: Enable structured logging
timeoutSec: 600 # Optional: Timeout per session (seconds)
retries: 2 # Optional: Retry attempts on failure
mergeOnPass: false # Optional: Auto-merge successful sessions
# Matrix of batch items (required, minimum 1 item)
matrix:
- repo: "./frontend" # Required: Repository path
prompt: "Implement JWT authentication" # Required: Task description
baseBranch: "develop" # Optional: Override default branch
scriptCommand: "pnpm test:auth" # Optional: Override default script
model: "gpt-5" # Optional: Override default model
timeoutSec: 900 # Optional: Override default timeout
mergeOnPass: true # Optional: Override default merge behavior
- repo: "./backend"
prompt: "Add refresh token rotation"
# Uses defaults for unspecified options
- repo: "./mobile"
prompt: "Update authentication flow"
model: "alloy" # Use --blend alloy-random mode
scriptCommand: "flutter test"Configuration Options:
| Field | Type | Required | Description |
|---|---|---|---|
runId |
string | No | Custom batch identifier (auto-generated if omitted) |
concurrency |
number | Yes | Maximum parallel sessions to run |
defaults.baseBranch |
string | Yes | Default Git branch for all repositories |
defaults.scriptCommand |
string | No | Default test/validation command |
defaults.model |
string | No | Default AI model (claude-3-5-sonnet, gpt-5, alloy) |
defaults.jsonLogs |
boolean | No | Enable structured JSON logging for metrics |
defaults.timeoutSec |
number | No | Timeout per session in seconds |
defaults.retries |
number | No | Number of retry attempts on failure |
defaults.mergeOnPass |
boolean | No | Auto-merge sessions that pass validation |
matrix[].repo |
string | Yes | Path to repository for this item |
matrix[].prompt |
string | Yes | Task description for Amp |
matrix[].baseBranch |
string | No | Override default base branch |
matrix[].scriptCommand |
string | No | Override default validation script |
matrix[].model |
string | No | Override default AI model |
matrix[].timeoutSec |
number | No | Override default timeout |
matrix[].mergeOnPass |
boolean | No | Override default merge behavior |
- Navigate to Benchmarks Tab: Access evaluation interface
- Configure Benchmark:
- Select SWE-bench dataset or custom problems
- Choose repositories for testing
- Set evaluation criteria and scripts
- Monitor Execution: Track benchmark progress and results
- Analyze Results: View success rates, performance metrics, and detailed reports
# Run SWE-bench evaluation
pnpm cli bench swe-bench-lite --dataset ./swe-bench.jsonl
# Custom benchmark
pnpm cli bench custom --config ./custom-benchmark.yaml
# Export results
pnpm cli bench export bench-run-123 --format csv# benchmark-config.yaml - Complete configuration example
version: 1 # Required: Configuration version
name: "Custom Benchmark" # Required: Benchmark name
description: "Evaluation of specific tasks" # Optional: Benchmark description
# Global defaults applied to all suites and cases
defaults:
base_branch: "main" # Default Git branch
parallel: 4 # Parallel execution limit
max_iterations: 10 # Maximum iterations per case
timeout_sec: 900 # Timeout per case (15 minutes)
json_logs: true # Enable structured logging
merge_on_pass: false # Never auto-merge (measurement only)
# Model configurations for testing different AI models
models:
default:
name: "default" # Use Amp's default model
gpt5:
name: "gpt-5"
amp_args: ["--try-gpt5"] # CLI arguments for GPT-5
alloy:
name: "alloy"
amp_args: ["--blend", "alloy-random"] # CLI arguments for Alloy
# Metrics to collect and analyze
metrics:
- success_rate # Pass rate percentage
- avg_iterations # Average iterations to completion
- avg_prompt_tokens # Average input tokens used
- avg_completion_tokens # Average output tokens generated
- avg_total_tokens # Average total tokens consumed
- total_cost_usd # Total cost in USD
- total_runtime_sec # Total execution time
# Test suites containing cases
suites:
- id: "smoke_tests" # Required: Suite identifier
description: "Quick validation tasks" # Optional: Suite description
max_iterations: 5 # Optional: Override default max iterations
cases:
- id: "readme_creation" # Required: Case identifier
repo: "octocat/Hello-World" # Required: Repository (GitHub slug or path)
prompt: | # Required: Task description
Create a README.md file explaining how to run the project
script_command: "grep -q 'Hello' README.md" # Optional: Validation script
- id: "function_addition"
repo: "scratch/math-lib"
setup_script: | # Optional: Pre-execution setup
echo 'def subtract(a,b): return a-b' > math.py
prompt: |
Add an add(a: int, b: int) -> int function and test file
follow_up_prompts: # Optional: Multi-turn conversation
- "Add type hints to all functions"
- "Optimize for performance"
script_command: "pytest -q"
- id: "swe_bench_cases" # SWE-bench integration
description: "Real bug fixing evaluation"
swebench_cases_dir: "eval_data/swebench/easy" # Directory with *.json case files
max_iterations: 8Configuration Options:
| Field | Type | Required | Description |
|---|---|---|---|
version |
number | Yes | Configuration schema version (currently 1) |
name |
string | Yes | Benchmark name for identification |
description |
string | No | Detailed benchmark description |
defaults.base_branch |
string | No | Default Git branch for repositories |
defaults.parallel |
number | No | Maximum parallel case execution |
defaults.max_iterations |
number | No | Maximum iterations per case |
defaults.timeout_sec |
number | No | Timeout per case in seconds |
defaults.json_logs |
boolean | No | Enable structured logging |
defaults.merge_on_pass |
boolean | No | Auto-merge successful cases |
models |
object | No | AI model configurations to test |
models.*.name |
string | Yes | Model identifier |
models.*.amp_args |
array | No | Additional CLI arguments for model |
metrics |
array | No | List of metrics to collect |
suites[].id |
string | Yes | Unique suite identifier |
suites[].description |
string | No | Suite description |
suites[].max_iterations |
number | No | Override default max iterations |
suites[].cases |
array | No | List of test cases (if not using SWE-bench) |
suites[].cases[].id |
string | Yes | Unique case identifier |
suites[].cases[].repo |
string | Yes | Repository path or GitHub slug |
suites[].cases[].prompt |
string | Yes | Task description for Amp |
suites[].cases[].script_command |
string | No | Validation/test command |
suites[].cases[].setup_script |
string | No | Pre-execution setup commands |
suites[].cases[].follow_up_prompts |
array | No | Multi-turn conversation prompts |
suites[].swebench_cases_dir |
string | No | Path to SWE-bench case files directory |
SWE-bench Integration:
- Set
swebench_cases_dirto directory containing*.jsoncase files - Each JSON file should contain
id,repo,bugCommit,fixCommit,testPath, andprompt - Cases automatically use repository setup and validation from SWE-bench format
Configure the app through ~/.ampsm/config.yaml:
git:
default_base_branch: "main"
commit_prefix: "amp:"
amp:
default_model: "claude-3-5-sonnet"
timeout: "300s"
notifications:
desktop: true
telemetry:
enabled: true
export_formats: ["jsonl", "csv"]
ui:
theme: "gruvbox"
auto_update: trueexport AMP_API_KEY="your-amp-api-key" # Required
export AMPSM_DB_PATH="/custom/path/sessions.db" # Optional
export AMPSM_LOG_LEVEL="debug" # Optional| Issue | Solution |
|---|---|
| Desktop app won't start | pnpm store prune && rm -rf node_modules && pnpm install |
| IPC handler errors | Restart desktop app; handlers register at startup |
| Git worktree issues | Ensure repo is clean and base branch exists |
| Authentication errors | Run amp login and verify with amp whoami |
pnpm cli doctor # System health check
pnpm cli verify-amp # Test Amp integration
pnpm cli clean-environment # Clean orphaned worktrees- packages/core: Session engine, Git operations, SQLite persistence
- packages/cli: Command-line interface (
@ampsm/cli→amp-sessions) - packages/types: Shared TypeScript contracts
- apps/desktop: Electron + React + TypeScript + Vite + Tailwind
See ARCHITECTURE.md for detailed documentation.
MIT License - see LICENSE for details.
Support: GitHub Issues | Discussions