Skip to content

feat(cost): attribution, pricing, ledger, budgets and CostManager - #999

Draft
induwara-yaala wants to merge 12 commits into
feature/971-cost-management-llm-gatewaysfrom
feature/972-cost-management-budgets
Draft

induwara-yaala wants to merge 12 commits into
feature/971-cost-management-llm-gatewaysfrom
feature/972-cost-management-budgets

Conversation

@induwara-yaala

Copy link
Copy Markdown
Contributor

Description

Budgets and usage reporting for cost management (#954). This PR adds the pieces behind budgets and usage reports:

  • a pluggable Attributor, PricingSource, SpendLedger (in-memory, Redis, Valkey, DynamoDB, CosmosDB, Firestore),
    BudgetPolicy and UsageView, each with a contract suite;
  • NativeBudgetSync;
  • CostManager, which builds them all from the cost block, checks them against the gateway's capabilities, and runs
    the startup probe and sync.

Nothing runs yet. RunMeterFactory.get() still returns the no-op meter (tested), so behaviour is unchanged with or
without a cost block. Wiring CostManager into runs, the gateway client and the usage route is #973. Spec:
docs/specs/954-pluggable-cost-management/spec.md §9–§13 and §15, deviations 28–31.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update
  • CI/CD update
  • Other (please describe):

Related Issues

Fixes #972
Relates to #969, #954
Depends on #970 (#986) and #971 (#987)

Changes Made

Commits follow the ticket's iterations.

  • 3.1 Attributor (cost/attribution.py). DefaultAttributor bills the principal's subject, or anonymous, and
    never the acting agent. Bring-your-own attributors are loaded by dotted path. AttributorContract includes the
    acting-agent rule.
  • 3.2 PricingSource (cost/pricing.py).
    • Passthrough, and static rates in USD per 1M tokens with an optional markup.
    • Cached tokens are charged once. If cached exceeds input, the uncached count is clamped at 0, so no charge goes
      negative.
    • bill_total equals the sum of bill over the same rows.
  • 3.3 Ledger core (cost/ledger/).
    • _RedisLikeDriver.compare_and_set is one Lua script, checked against a real Redis.
    • LedgerDocument and SpendLedger (one document per user, written by compare-and-set), the in-memory backend, the
      factory and SpendLedgerContract.
  • 3.4 Ledger backends. Redis, Valkey, DynamoDB (a conditional put_item), CosmosDB (etag) and Firestore (an
    update-time precondition).
    • DynamoDB, Firestore and CosmosDB leave expired documents in place. A write from version 0 may now replace an expired
      document; without that, the user's ledger would be stuck (deviation 28).
  • 3.5 BudgetPolicy (cost/budget.py).
    • StaticBudgetPolicy reads spend through SpendReader, bills it, and allows, warns or denies.
    • Reads are bounded at 10 s. Read failures follow budgets.on_error, and the logs carry the user and the error type,
      never the error text.
  • 3.6 LedgerSpendFlow.
    • Rows older than 10 minutes are committed to the ledger batch by batch. The last 10 minutes are re-read from the
      gateway on every check and never stored.
    • The catch-up is limited to 5 s. If it runs out, it raises LedgerBehindError and keeps its progress.
    • Two fixes to edge cases (deviation 29):
      • A row exactly at the settle line was counted twice.
      • After an idle gap, a user's catch-up would have replayed rows no current budget can count.
  • 3.7 NativeBudgetSync.
    • Startup writes the budget definitions and the ak-system row while the caller waits. A bad admin key or a rejected
      shape fails startup.
    • Per-user rows go to the ak-cost-budget-sync thread, with backoff. Users on the default budget are upserted lazily.
    • Covered users skip the spend read.
    • §15 assumed run_async_sync works inside a running loop. It doesn't, so the startup sync runs on a one-shot thread
      with its own loop (deviation 30).
  • 3.8 UsageView and CostManager.
    • The view authorises through the attributor, applies on the client any filter the gateway can't, and splits into
      days and then hours only when the gateway truncates.
    • CostManager runs the capability checks, builds a ledger only for gateways without windowed spend, and probes the
      gateway, retrying an unavailable one in the background (budget checks follow on_error until it answers).
    • Its refusal walk follows exception causes and contexts, and always ends within 16 links (deviation 31).

Testing

  • Unit tests pass locally: full suite, 7248 passed, 0 failed.
  • Integration tests pass locally: the Redis compare-and-set script and the RedisSpendLedger contract against a
    throwaway redis:7 container. The run is gated on AK_TEST_REDIS_URL and skipped in CI.
  • Manual testing completed
  • New tests added for changes:
    • New files: test_cost_attribution.py, test_cost_pricing.py, test_cost_ledger.py, test_cost_budget.py,
      test_cost_view.py and test_cost_manager.py.
    • Extended: test_shared_drivers.py.
    • The five contract suites run against every built-in, plus a bring-your-own example where it applies.
    • The ledger backends run the contract against fakes of their native clients that honour the conditions the code
      sends.
    • In every iteration I broke the code on purpose, and each break failed a test. One of them (removing the cycle
      guard) first made a test hang; the walk now has its own bound, so a missing guard fails instead.

Acceptance criteria:

  • the six named test files pass;
  • two concurrent ledger adds with the same cursor: exactly one succeeds;
  • RunMeterFactory.get() returns the no-op meter;
  • make lint-check-all passes.

mypy reports nothing new beyond the factories' type-abstract on resolve_dotted(..., base=<ABC>), which the
existing factories also have.

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant