Repository navigation
Conversation
|
🌿 Preview your docs: https://nvidia-preview-pr-5756.docs.buildwithfern.com/openshell |
|
/ok to test c17f546 |
|
/ok to test abd2964 |
|
Local review: two findings. Both still apply at
Validation on the originally reviewed commit, |
|
Route list counted in the receive limit: "Fixed in b989be8: the supervisor receive limit now adds room for the whole cluster route list at 131072 sandboxes (128 bytes each), with a test that a supervisor limited to 1 sandbox can still accept that list." -f values-scale-10k.yaml on a published chart: "Fixed in 131fb46: the guide now runs helm pull --untar first and passes -f helm-chart/values-scale-10k.yaml from the extracted chart." |
|
/ok to test b989be8 |
|
I have read the DCO document and I hereby sign the DCO |
…imits Add [openshell.gateway.supervisor_pool] max_instances_per_member (16-1024, default 128) and max_members_per_gateway (1-64, default 32). The pool cap is live gateway replicas * members * instances, never above 131072. Placement picks the least-loaded member with room and never charges a member past its cap. Supervisors take the same cap through --pool-max-instances and size their gateway stream buffers from it. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
Render supervisor.pool.maxInstancesPerMember and maxMembersPerGateway into the gateway config and supervisor env, validate their ranges, and fail the render when supervisor replicas or HPA maxReplicas exceed gateway replicas * maxMembersPerGateway. Default to two supervisors with configurable resources (250m/768Mi requests, 1.5Gi memory limit) and add values-scale-100k.yaml. Closes #5748 Signed-off-by: divesh <dgude@nvidia.com>
Lower max_instances_per_member and supervisor.pool.maxInstancesPerMember to 1..=1024. Gateway message and status-report bounds keep the floor a cap of 16 gave, since snapshots also carry the pool-wide route directory. Document how to add gateway replicas for more supervisors. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
…ool limits plumbing Only sandboxes per supervisor stays bounded (1-1024). Members per gateway accepts any positive value; the 131072 pool ceiling still applies. Placement refusals now say "sandbox capacity reached" and name the limit that was hit. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
…imits Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
Two gateways with up to 40 supervisors each, at 128 sandboxes per supervisor, hold 10,240 sandboxes. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
…tion Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
Installing the chart no longer reserves supervisor CPU and memory. The values file, the HA guide, and the 10k profile carry the recommended requests and limits instead. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
…file Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
…tory Every snapshot lists where each instance in the pool runs, so a member with a small cap rejected snapshots of a large pool. The decode bound now adds room for the route directory of the largest pool. Refs #5748 Signed-off-by: divesh <dgude@nvidia.com>
b989be8 to
0077bb7
Compare
Summary
Make shared supervisor pool capacity configurable from Helm, so operators can size a cluster by gateways, supervisors, and sandboxes per supervisor.
Related Issue
Closes #5748
Changes
supervisor.pool:maxInstancesPerMember: sandboxes per supervisor, 1–1024, default 128maxMembersPerGateway: supervisors per gateway, 1 or more, default 32sandbox createfails right away withFAILED_PRECONDITION:sandbox capacity reached; add supervisors or gateway replicas, or raise the supervisor pool limits. The gateway log says which limit was hit.supervisor.replicas: 1(unchanged)maxReplicas: 4maxReplicas) exceed gateway replicas ×maxMembersPerGateway.values-scale-10k.yamlprofile: 2 gateways × up to 40 supervisors each × 128 sandboxes = 10,240.Release Notes
Helm users can set shared supervisor capacity with
supervisor.pool.maxInstancesPerMemberandsupervisor.pool.maxMembersPerGateway. Creates beyond capacity fail with "sandbox capacity reached".Testing
openshell-supervisor-control, the shared supervisor host, server config, and placement; clippy cleanhelm unittest(503 tests),mise run helm:lint,test-gateway-config-parser.sh,test-supervisor-pool-scaling.shChecklist