Repository navigation
docs(nexus-pdp): publish measured check latency and capacity, and correct memory and disk sizing (early access) - #670
EliMoshkovich wants to merge 13 commits into
Conversation
…ly access) PER-16873. From the PER-16040 staging scale test (nexus-pdp 0.7.0-beta.26, default storage-engine settings, 2.0M / 10.8M / 20.0M facts). Memory grows with the data set and peaks during a cold start (~6.5 GiB at 20M facts), so 'memory you configure' and a flat 4 GiB are corrected. No latency or throughput figures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
✅ Deploy Preview for permitio-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
zeevmoney
left a comment
There was a problem hiding this comment.
Changes requested — 1 HIGH, 9 MEDIUM, 5 LOW.
Blocking:
- HIGH
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:147— no memory floor; below about 2.5 GiB Nexus PDP can stop applying updates without an error
Non-blocking:
- MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:142— 2.0M cold-start memory isn't from the measured build; that run peaked under load - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:144— 20M serving memory "~2.7 GiB" isn't in the measurements (1.7–3.2 GiB) - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:146— "+0.5" in the cold-start formula has no source; rebuilds not covered - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:140— peaks are sampled lower bounds; runs had 8/32 GiB limits - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:150— 5.8 GiB reading doesn't show the disk guidance holds - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:128— sentence above the caution still says memory depends on cache sizes you set - MEDIUM
docs/concepts/pdp/nexus-pdp.mdx:61— the memory limit, not a setting, sizes the serving cache in this build - MEDIUM
docs/concepts/pdp/nexus-pdp-deployment.mdx:25— pod spec sets a memory request but no limit - MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:131— configuration reference and PDP overview still give the old memory guidance - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:148— cold-start growth claim needs its CPU caveat - LOW
docs/concepts/pdp/nexus-pdp-deployment.mdx:31— name the startup-probe endpoint and the CPU count - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:152— "deliberately heavier than a typical one" has no source - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:123— "grows far more slowly" has no like-for-like measurement - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:138— small consistency fixes in the new section
Details are in the inline comments on each line.
…873) - Add the 2.5 GiB memory floor (PER-16808) to the caution, the sizing bullet, the deployment Memory row and the configuration reference. - Describe the serving cache as sized from the container memory limit (SurrealDB 3.2.4 defaults); scope the SURREAL_ROCKSDB_* variables to the snapshot load; add a memory limit to the pod spec. - 2.0M row: cold-start peak "Not recorded"; serving ranges from the 10 s container working set (2.1-2.9 GiB at 10.8M, 1.7-3.2 GiB at 20M); note the peaks are sampled lower bounds and the runs' 8/32 GiB limits. - Scope "peaks during a cold start" to large data sets; drop the unsourced +0.5 GiB end of the formula; note a rebuild was not measured. - Disk: report measured use (0.33 / 1.6 / up to 5.8 GiB) and size the volume from the snapshot (3x cold start, 4x rebuild, unbounded event store). - Cold start: CPU caveat, /health/ready startup probe, the node wait. - Cite the container PDP's 6 KB-per-object estimate as an estimate. - Consistency: Google Drive-style, role assignments in the change stream, image digest, facts link on the deployment page. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…-sizing' into eli/per-16873-nexus-pdp-measured-sizing
…ecovery (PER-16873) - 2304 MiB (2.25 GiB) is the stall point; 2.5 GiB the minimum to run at. - Nexus PDP keeps answering from the data it already has; beta.26 reports nothing, and at 2 CPUs or fewer the process can stop answering. - Restart to recover; decimal memory units count as written. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Follow-up on the two points that were open before customer-facing use. Both are now backed by a test or the owner's ticket, not only the source. 1. The memory limit sizes the serving database;
This matches the table in the configuration reference, and PER-16808's bisect ("the embedded SDK ignores 2. The stall caution now matches PER-16808 and @omer9564's draft customer page (cloud-pdp #270,
When a release carries cloud-pdp #306 (refuse to start below the floor) and #292 ( 🤖 Generated with Claude Code |
zeevmoney
left a comment
There was a problem hiding this comment.
Approved — no CRITICAL or HIGH issues found.
Non-blocking:
- MEDIUM
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:150— the recommended limits cover a first boot, not a rebuild - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:144— the 2.0M row's serving range mixes two builds - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:131— the snapshot load isn't what stalls, and the floor is worded differently across pages - LOW
docs/concepts/pdp/nexus-pdp-configuration.mdx:84— theSURREAL_ROCKSDB_*variables barely affect the snapshot load either - LOW
docs/concepts/pdp/overview.mdx:53— the PDP overview doesn't say that Nexus PDP memory still grows with the data set - LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:162— disk sizing assumes a known snapshot size, and a killed cold start leaves its copies behind - LOW
docs/concepts/pdp/nexus-pdp-deployment.mdx:31— the startup probe times are from 8 CPUs only
Details are in the inline comments on each line.
- Size memory for a rebuild: 4x the snapshot + 1 GiB (about 7 / 12 GiB at 10M / 20M facts), labelled estimates, on all three pages. - 2.0M serving memory: up to 1.5 GiB (the measured build only). - The stall comes from applying changes, not the SST snapshot load; the 2304 MiB floor is worded the same everywhere; raise the limit to recover. - SURREAL_ROCKSDB_* barely affect the snapshot load either. - PDP overview: memory still grows with the data set. - Disk: snapshot size per million facts; a killed cold start leaves copies. - Startup probe: fewer CPUs take longer; startupProbe in the pod spec. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e from them Add a Nexus PDP Benchmarks page in the Cloud PDP Benchmarks format: server-side check latency at a fixed rate for RBAC and ReBAC depth 0-3, per-instance capacity at 2 and 4 vCPU and with about 20 million facts, scale-out to 3 instances, a short method section, the limits of the results, and step-by-step sizing guidance with a worked example. Replace the "no published throughput figures" caution with a link to the page, add a CPU row to the deployment requirements, add the measured serving memory to the resource footprint, and link the page from the Nexus PDP pages, the PDP overview, the Cloud PDP benchmarks and the sidebar. Update the route inventory baseline for the new route. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review follow-up on the Nexus PDP benchmarks page: - Capacity method: a separate open-model run for each scenario, starting at the latency-run rate, +25% every 2 minutes, up to 200 concurrent requests. State the latency bound's source per policy model. - Correctness: describe the pre-run and in-load checks; no run had a failed request or a wrong decision. - Access patterns: count the parent relationships for each depth, and say the Cloud PDP page is not a row-by-row comparison. - Data sets: define facts and give the full small data set. - Key results: scope the latency results to the measured rate, say the large data set shows total size only, name which patterns lost a step, state the CPU observation and the 1.56-1.95x figure, explain the per-connection balancing with the per-instance shares, and give the memory range with its read-cache caveat. - Sizing: mark the steps as guidance, add headroom before dividing, add a mixed-traffic formula and a depth 3 example. - Limits: x86 only (arm64 not measured), ABAC conditions not measured. - Sentence-case title, consistent P99 casing. Elsewhere: the CPU wording on How It Works and Deployment, the memory caveat on How It Works, "last verified on 0.7.0-beta.26" for the stall caution, the 2-CPU stop-answering condition tied to the memory floor, and the early-access callouts dated October 2026. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The deployment requirements now list 2 vCPU to start, with the request equal to the limit, matching the benchmark setup. Add --cpus 2 to the docker run sample and cpu: "2" to the pod spec's requests and limits. Code-sample change in its own commit, per the style guide. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@zeevmoney @omer9564 this is a deep rewrite with the measured Nexus PDP check benchmark: a new page, |
…mory and disk - Remove the per-connection balancing explanation of the RBAC scale-out figures; the measured ratios only resolve to whole 25% steps. State the 2- and 3-instance figures against linear scaling within that resolution, and keep load spreading as general sizing practice. - Add the measured snapshot-load memory peak (about 5.3 to 6.7 GiB) and data volume use (about 7.4 GiB) with about 20 million facts, on the benchmarks page and in the resource footprint. - Say the added facts were of resource types the measured checks don't query, so the large data set shows the effect of a large environment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
zeevmoney
left a comment
There was a problem hiding this comment.
Approved — no CRITICAL or HIGH issues found.
Non-blocking:
- MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:107— "not 2 times" isn't established for ReBAC depth 2; "CPU was at its limit" is an estimate - MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:108— scale-out bullet misstates the depth 2 arithmetic and understates the 2-instance RBAC gap - MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:106— the capacity drop with 20 million facts reads as measurement noise - MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:48— the large data set isn't the small data set plus 20 million facts - LOW
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:109— small-data-set memory range doesn't match the measurements - LOW
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:120— sizing assumes linear scale-out past 3 instances; step 3's headroom reason - LOW
docs/concepts/pdp/nexus-pdp-deployment.mdx:57— pod-spec intro asks for a startup probe the excerpt already has - LOW
docs/concepts/pdp/nexus-pdp-configuration.mdx:101— Next steps lists go past the style guide's five links
Still open from the previous review:
- LOW
docs/concepts/pdp/nexus-pdp-how-it-works.mdx:140— disk is sized in multiples of a snapshot size readers can't see; the per-million-facts conversion left with the removed measurements
Process note: no explicit OK of the published latency and capacity figures from omer9564 is on record yet; his earlier approval predates the figures. The source results in pdp-benchmarks are also still in review.
Details are in the inline comments on each line.
- CPU: state the 4 vCPU gain per pattern in 25% steps (below 2x for RBAC and ReBAC depth 0, 1 and 3; consistent with 2x for depth 2), and say the instance was throttled at its CPU limit during capacity runs. - Scale-out: correct the linear-scaling arithmetic; RBAC with 2 instances is about one step below 2x, cause not established; measured up to 3 instances. - Large data set: describe the checks' own, smaller data and the 10 GiB memory limit as differences besides size; the capacity drop for RBAC and ReBAC depth 0 and 1 is real, one 20% step. - Small-data-set memory: about 0.15 to 0.2 GiB. - Sizing: headroom covers below-linear scale-out instead of the step resolution; load test beyond 3 instances. - Disk: give a measured volume data point for a first estimate. - Deployment: the excerpt's startup probe is a placeholder to set from a timed cold start. - Next steps lists capped at five links. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@zeevmoney thanks. All your comments are fixed in a8e09ac, with a reply on each thread. The older still-open item (how-it-works: disk sized from a snapshot size readers can't see) is addressed with a measured data point instead of the removed per-million-facts conversion. The Disk section now says you don't see the snapshot size before the first cold start, and gives the benchmark figure as a first estimate: with about 20 million facts, the data volume used up to about 7.4 GiB. It adds that no rebuild ran in those measurements and a rebuild needs room for one more database copy, so give the volume more than that and monitor free space after the first cold start. The build passes (0 bad links or anchors) and cspell is clean. Not merging: per Omer's rule, @omer9564's explicit OK on the published figures is still needed. |
…view) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
zeevmoney
left a comment
There was a problem hiding this comment.
Changes requested — 1 HIGH, 2 MEDIUM, 2 LOW.
Blocking:
- HIGH
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:2— conflicts with master, where #674 already published a Nexus PDP benchmarks page at this path
Resolve the conflict with master before this lands. Eli, Or and Omer need to agree on two things: one Nexus PDP benchmarks page, and one wording for the large-data-set disclaimer. #674 keeps the "resource types the checks never read" line that ddc0105 removed here.
Which page becomes the base hasn't been decided yet. Either way, keep what this PR adds:
- the sizing section with its worked example;
- the correctness method;
- the limits: region and instance family, arm64, synthetic data, and serving checks only;
- the corrections on the other pages:
- the 2.5 GiB memory floor;
- the cache sized from the memory limit;
- the scope of
SURREAL_ROCKSDB_*; - the CPU row and the CPU in the Docker and pod-spec samples;
- the pod-spec memory limit and startup probe;
- the disk sizing.
Non-blocking:
- MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:48— the large data set's model has no drives or files - MEDIUM
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:141— the limits leave out three conditions that make capacity optimistic for a default deployment: logging, a warm working set, and one run per configuration - LOW
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:17— the measured build is named only by a tag that can move - LOW
docs/concepts/pdp/nexus-pdp-benchmarks.mdx:107— the throttling clause reads as more than the runs show (in the reply on the earlier CPU thread)
Each thread from the previous review now has a reply saying it was verified at ddc0105. The details are in the inline comments.
| @@ -0,0 +1,152 @@ | |||
| --- | |||
| title: Nexus PDP benchmarks | |||
There was a problem hiding this comment.
[HIGH] This PR no longer merges: master already has a different page at this path (#674)
Problem:
- docs: add Nexus PDP benchmarks page #674 (merged 2026-10-20) added its own
nexus-pdp-benchmarks.mdxfrom the same benchmark run. It also edited the how-it-works info box, the deployment Memory row, the PDP overview's links, the Nexus PDP overview's Next steps and the sidebar. - Merging master into this branch gives an add/add conflict on this page, and content conflicts in
nexus-pdp-deployment.mdx,nexus-pdp-how-it-works.mdxandoverview.mdx. - Two files merge with no conflict marker but end up with duplicates:
sidebars.jslistsconcepts/pdp/nexus-pdp-benchmarkstwice, and the Next steps innexus-pdp.mdxlink to the benchmarks page twice. - Master's deployment Memory row links to
/concepts/pdp/nexus-pdp-benchmarks#memory. This page has nomemoryanchor, so keeping that row next to this page fails the build's anchor check. - The two pages differ in content:
- Only master's page states the image digest, the warm-working-set, logging and one-run-per-configuration caveats, and the 2-instance request splits.
- Master's page still says the 20 million facts were on resource types the checks never read, which ddc0105 removed here.
- Only this page has the sizing guidance, the correctness method, and the arm64 and synthetic-data limits.
Suggestion: Rebase on master and keep a single page that carries the content of both. Agree with #674's author on one wording for the 20 million facts. Drop the duplicate sidebar entry and the duplicate Next steps link. For the anchor, either add {#memory} to a memory heading or point the deployment row at #key-results.
| | Data set | Contents | | ||
| | --- | --- | | ||
| | Small | 1,750 users, 12,610 resource instances, 12,000 relationship tuples, and 2,223 role assignments, in 11 tenants | | ||
| | Large | About 20 million facts (users, resource instances, relationship tuples, and role assignments) in a file-sharing model of drives, folders, and files, plus the checks' own data: 1,100 users, 1,500 resource instances, 900 relationship tuples, and 1,573 role assignments | |
There was a problem hiding this comment.
[MEDIUM] The large data set's model has no drives or files
Problem: In the large-data-set runs, the 20 million facts come from a document-sharing data set with these resource types:
- nested folders;
- documents;
- user groups, which hold shares.
It has no drives and no files. My earlier suggestion for this row used the same "drives, folders, and files" wording.
ddc0105 removed the "checks don't query" clause. Without it, "folders, and files" matches the access-pattern table's "a role on a folder that contains the file", so readers can take the row to mean the measured checks walked the 20-million-fact folder tree. The runs didn't measure that.
Suggestion: "About 20 million facts (users, resource instances, relationship tuples, and role assignments) in a document-sharing model of nested folders, documents, and user groups, plus the checks' own data: 1,100 users, 1,500 resource instances, 900 relationship tuples, and 1,573 role assignments"
| - **One test environment.** All runs used one AWS region and one instance family: x86 (AMD EPYC, `m6a`) only. The `arm64` image was not measured. Other CPUs, storage, and Kubernetes setups can give different results. | ||
| - **Synthetic data and policies.** The data sets and the RBAC and ReBAC policies were generated for the test. Your policy, the number of roles and relationships, and the shape of your permission graph change the results. | ||
| - **The access patterns listed.** The runs measured single permission checks for the patterns in [Access patterns](#access-patterns), with one access path for each user. Users who reach a resource through many paths, ABAC conditions, and other endpoints such as bulk checks or user permissions were not measured. | ||
| - **Serving checks only.** The runs measured checks against a running instance. They didn't measure checks while Nexus PDP applies a large burst of data changes, or the time to start a new instance. |
There was a problem hiding this comment.
[MEDIUM] The limits leave out three conditions that make the capacity figures optimistic for a default deployment
Problem:
- Logging: the benchmarked instances ran at
warnlog level, exported only the request trace span, and sent no decision logs. The configuration reference gives the defaults asRUST_LOG=infoandEDGE_DECISION_LOG_OPT_OUT=false, which means decision logs are on. A default deployment therefore spends CPU on each check that these runs didn't, and step 4 of the sizing guidance divides by capacity figures that don't include that cost. - Warm working set: checks were drawn at random from a fixed set of 1,000 checks for each data set. Every decision repeated many times, so any caching worked on a warm working set.
- One run per configuration: each figure comes from a single run. That includes the 20-million-fact capacity drop, which Key results describes as lower.
The benchmark results list the first two as notes to publish with the figures. Master's version of this page (#674) includes all three.
Suggestion: Add three bullets to "What these results don't cover":
- "Logging. Nexus PDP ran at
warnlog level and sent no decision logs. The defaults (RUST_LOG=info, decision logs on) use more CPU for each check, so load test with the logging you run in production." - "A repeated set of checks. Checks were drawn from a fixed set of 1,000 for each data set, so every decision repeated and caches stayed warm."
- "One run per configuration. Each figure comes from a single run."
| ## Methodology | ||
|
|
||
| - **Date:** October 2026. | ||
| - **Build:** `permitio/nexus-pdp` `0.7.0-beta.50`. |
There was a problem hiding this comment.
[LOW] The measured build is named only by a tag that can move
Problem: The page names the measured build by its tag alone, 0.7.0-beta.50. The deployment page in this PR says a numbered beta tag can be published again for a later build, so the tag doesn't pin down which build was measured.
Suggestion: "Build: permitio/nexus-pdp 0.7.0-beta.50 (sha256:43b0f5c6d422558555e47ff4e84b99920d00ac832978f245e57b28d2d0d5eec6)."
|
Direction for the benchmarks page conflict with #674 (Zeev's call): rebase this PR with #674's page as the base. #674's page is
The other findings from the last review still apply where the rebased page has the same text: the data-set model naming, the Limits points (warm working set, logging levels used in the runs vs the defaults), the image digest, and throttling per step. Re-request review after the rebase. |
PER-16873. The figures come from the Nexus PDP check benchmark of Oct 9–10, 2026.
This PR does two things on the early-access Nexus PDP pages.
1. New Nexus PDP benchmarks page (
/concepts/pdp/nexus-pdp-benchmarks), in the Cloud PDP Benchmarks page's format:0.7.0-beta.50, AWS us-east-2. No cold-start figures.The page is written to be shareable with any prospect as-is: generic, with no customer named and no internal details.
The page is linked from the Nexus PDP overview, How It Works (replacing the "no published throughput figures" caution), Deployment, Configuration, the PDP overview, the Cloud PDP Benchmarks page and the sidebar. Deployment also gets a new CPU requirement row, and 2 CPUs in the Docker and pod-spec samples (separate commit). The route inventory baseline includes the new route, and the early-access callouts are dated October 2026.
2. Memory and disk corrections, kept from earlier revisions:
0.7.0-beta.26).SURREAL_ROCKSDB_*variables only affect the snapshot step.How it was checked:
npm run buildpasses (0 bad links or anchors; routes and sidebars OK). cspell is clean on the Nexus PDP pages.Sign-off on the figures: omer9564 approved the version with the published latency and capacity figures at a8e09ac, and ddc0105 applies the one comment from that review.
Description updated during review to match ddc0105 (the benchmark source line, the sign-off line, and the ddc0105 change).
🤖 Generated with Claude Code