Repository navigation
feat(cost): attribution, pricing, ledger, budgets and CostManager - #999
Draft
induwara-yaala wants to merge 12 commits into
Draft
induwara-yaala wants to merge 12 commits into
induwara-yaala wants to merge 12 commits into
Conversation
…ttributor classes
…and Static pricing
…ver and add SpendLedger classes
…, DynamoDB, CosmosDB, Firestore, and Valkey
…and versioning logic
…r, and related classes
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Budgets and usage reporting for cost management (#954). This PR adds the pieces behind budgets and usage reports:
Attributor,PricingSource,SpendLedger(in-memory, Redis, Valkey, DynamoDB, CosmosDB, Firestore),BudgetPolicyandUsageView, each with a contract suite;NativeBudgetSync;CostManager, which builds them all from thecostblock, checks them against the gateway's capabilities, and runsthe startup probe and sync.
Nothing runs yet.
RunMeterFactory.get()still returns the no-op meter (tested), so behaviour is unchanged with orwithout a
costblock. WiringCostManagerinto runs, the gateway client and the usage route is #973. Spec:docs/specs/954-pluggable-cost-management/spec.md§9–§13 and §15, deviations 28–31.Type of Change
Related Issues
Fixes #972
Relates to #969, #954
Depends on #970 (#986) and #971 (#987)
Changes Made
Commits follow the ticket's iterations.
Attributor(cost/attribution.py).DefaultAttributorbills the principal's subject, oranonymous, andnever the acting agent. Bring-your-own attributors are loaded by dotted path.
AttributorContractincludes theacting-agent rule.
PricingSource(cost/pricing.py).negative.
bill_totalequals the sum ofbillover the same rows.cost/ledger/)._RedisLikeDriver.compare_and_setis one Lua script, checked against a real Redis.LedgerDocumentandSpendLedger(one document per user, written by compare-and-set), the in-memory backend, thefactory and
SpendLedgerContract.put_item), CosmosDB (etag) and Firestore (anupdate-time precondition).
document; without that, the user's ledger would be stuck (deviation 28).
BudgetPolicy(cost/budget.py).StaticBudgetPolicyreads spend throughSpendReader, bills it, and allows, warns or denies.budgets.on_error, and the logs carry the user and the error type,never the error text.
LedgerSpendFlow.gateway on every check and never stored.
LedgerBehindErrorand keeps its progress.NativeBudgetSync.ak-systemrow while the caller waits. A bad admin key or a rejectedshape fails startup.
ak-cost-budget-syncthread, with backoff. Users on the default budget are upserted lazily.run_async_syncworks inside a running loop. It doesn't, so the startup sync runs on a one-shot threadwith its own loop (deviation 30).
UsageViewandCostManager.days and then hours only when the gateway truncates.
CostManagerruns the capability checks, builds a ledger only for gateways without windowed spend, and probes thegateway, retrying an unavailable one in the background (budget checks follow
on_erroruntil it answers).Testing
RedisSpendLedgercontract against athrowaway
redis:7container. The run is gated onAK_TEST_REDIS_URLand skipped in CI.test_cost_attribution.py,test_cost_pricing.py,test_cost_ledger.py,test_cost_budget.py,test_cost_view.pyandtest_cost_manager.py.test_shared_drivers.py.sends.
guard) first made a test hang; the walk now has its own bound, so a missing guard fails instead.
Acceptance criteria:
adds with the same cursor: exactly one succeeds;RunMeterFactory.get()returns the no-op meter;make lint-check-allpasses.mypy reports nothing new beyond the factories'
type-abstractonresolve_dotted(..., base=<ABC>), which theexisting factories also have.
Checklist
Additional Notes
feature/971-cost-management-llm-gateways, so the diff shows only [TASK] Cost management: attribution, pricing, ledger, budgets and CostManager #972. Retarget it as feat(cost): cost management foundation and core changes #986 andfeat(cost): LLM gateways (LiteLLM, Agentgateway, Langfuse) #987 merge.
fakes, not real services or emulators.
shortcut:in the code:NativeBudgetSync's per-process user set is never trimmed.end_userbudgets read through the ledger: about two gateway reads, threeledger reads and at most one write per check in steady state. Covered native users read nothing.
🤖 Generated with Claude Code