Skip to content

Why do CUDA device allocations appear to produce a nearly 1:1 increase in WSL host committed memory? Is this expected behavior related to CUDA/WSL2 GPU memory management, or an actual memory-accounting issue? #42294

Open

Description

Windows Version

Microsoft Windows [Version 10.0.26200.7840]

WSL Version

3.0.1.0

Are you using WSL 1, WSL 2, or WSLC?

  • WSL 2
  • WSL 1
  • WSLC

Kernel Version

6.18.40.1-microsoft-standard-WSL2

Distro Version

Ubuntu 24.04.3 LTS (Noble Numbat)

Other Software

Hardware:

  • Primary GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM, sm_120)
  • Secondary GPU: NVIDIA GeForce RTX 4090 (24 GB VRAM)
  • NVIDIA Windows Driver: 617.42
  • System RAM: 128 GiB

Software:

  • Windows 11, Build 26200.7840
  • WSL: 3.0.1.0
  • WSL2 Kernel: 6.18.40.1-microsoft-standard-WSL2
  • Ubuntu 24.04.3 LTS (Noble Numbat)
  • PyTorch: 2.9.0+cu128 (verified in native Windows environment; WSL environment version not yet confirmed)
  • CUDA runtime (Windows PyTorch): 12.8

WSL2 configuration (%UserProfile%\.wslconfig):

[wsl2]
networkingMode=nat
guiApplications=false
memory=64GB
swap=32GB

[experimental]
autoMemoryReclaim=gradual

The issue occurs during CUDA GPU allocations under WSL2 and can be reproduced using a minimal PyTorch script without a full model or training framework.

Repro Steps

Steps to reproduce

1. Configure WSL2

Configure .wslconfig as described above, then restart WSL using:

wsl --shutdown

Launch Ubuntu and activate a Linux Python environment with CUDA-enabled PyTorch installed.

2. Record initial WSL memory information

Run from Windows PowerShell:

wsl -d Ubuntu -- sh -c 'grep -E "^(MemTotal|MemFree|MemAvailable|Cached|SwapTotal|SwapFree|Committed_AS):" /proc/meminfo'

Observed WSL memory statistics:

MemTotal:       65845096 kB
MemFree:        37435408 kB
MemAvailable:   50050264 kB
Cached:         11298936 kB
SwapTotal:      33554432 kB
SwapFree:       33554432 kB
Committed_AS:   19508596 kB

3. Run a Windows-side commit monitor

Open a separate Windows PowerShell terminal:

while ($true) {
    $vm = Get-Process vmmemWSL -ErrorAction SilentlyContinue
    $sys = (Get-Counter '\Memory\Committed Bytes').CounterSamples[0].CookedValue

    "{0:HH:mm:ss} System={1:N2}GiB WSLCommit={2:N2}GiB WSLWorkingSet={3:N2}GiB" -f `
        (Get-Date),
        ($sys / 1GB),
        ($vm.PrivateMemorySize64 / 1GB),
        ($vm.WorkingSet64 / 1GB)

    Start-Sleep -Seconds 3
}

This monitors:

  • Total Windows system committed memory
  • vmmemWSL private committed memory
  • vmmemWSL working set

4. Execute the minimal CUDA allocation test inside Ubuntu WSL2

Create repro_cuda_commit.py:

import gc
import time
import torch

print("GPU:", torch.cuda.get_device_name(0), flush=True)

chunks = []

def measure(stage):
    torch.cuda.synchronize()

    print(
        f"{stage}: GPU allocated="
        f"{torch.cuda.memory_allocated()/1024**3:.2f} GiB",
        flush=True
    )

    with open("/proc/meminfo") as f:
        for line in f:
            if line.startswith(("MemAvailable:", "Committed_AS:")):
                print(line.strip(), flush=True)

    time.sleep(15)

measure("CUDA initialized")

for i in range(3):
    x = torch.empty(
        4 * 1024**3,
        device="cuda:0",
        dtype=torch.uint8
    )
    x.fill_(1)
    chunks.append(x)

    measure(f"{(i+1)*4} GiB GPU allocation")

chunks.clear()
del x
gc.collect()
torch.cuda.empty_cache()

measure("GPU allocations released")

Run:

python repro_cuda_commit.py

5. Observe Windows host commit

While the script executes, observe the WSLCommit column in the Windows PowerShell monitor.

GPU memory is allocated in three consecutive 4 GiB increments (4, 8, and 12 GiB total), followed by complete release.

Compare the increase in GPU VRAM allocation against the corresponding increase in vmmemWSL private committed memory.

6. Check Windows page file usage

Get-CimInstance Win32_PageFileUsage |
    Format-Table Name, AllocatedBaseSize, CurrentUsage, PeakUsage

This helps distinguish Windows commit accounting from actual page file occupancy.

Expected Behavior

CUDA device-only allocations (torch.empty(..., device="cuda:0")) should primarily consume GPU device memory, without requiring an approximately equal increase in Windows host committed memory unless this is an intentional, documented requirement of WSL2 GPU virtualization.

Specifically:

  1. Allocating 4 GiB of GPU VRAM should not unexpectedly require approximately 4 GiB of additional Windows system commit.
  2. Large GPU workloads should not encounter host commit-limit pressure solely because GPU VRAM allocations are being mirrored into Windows commit accounting, unless such accounting is required for the virtualization architecture.
  3. The relationship between device VRAM allocations, vmmemWSL private committed memory, and Windows commit limits should be documented if this behavior is expected.
  4. Users should not need to continuously increase the Windows page file size simply to accommodate GPU allocations without a corresponding explanation of the host commit requirement.

The goal is to understand whether this near-1:1 host commit growth is expected WSL2/CUDA virtualization behavior or an unnecessary memory-accounting overhead that can be corrected.

Actual Behavior

CUDA device allocations produce approximately 1:1 Windows host commit growth

Running the minimal reproduction on the NVIDIA RTX PRO 6000 Blackwell under WSL2 produces the following measurements:

GPU VRAM allocated | vmmemWSL private commit | Increase from baseline -- | -- | -- 0 GiB | 8.99 GiB | — 4 GiB | 13.12 GiB | +4.13 GiB 8 GiB | 17.13 GiB | +8.14 GiB 12 GiB | 21.10 GiB | +12.11 GiB Released | 9.28 GiB | +0.29 GiB

Findings

1. Nearly exact 1:1 relationship

Every additional 4 GiB of CUDA device allocation produces approximately 4 GiB of additional vmmemWSL private committed memory.

At 12 GiB of GPU allocations, WSL host commit has increased by 12.11 GiB.

This strongly suggests the GPU allocation size is directly associated with the additional Windows host commit charge.

2. Memory commitment is reversible

After executing:

chunks.clear()
del x
gc.collect()
torch.cuda.empty_cache()

WSL committed memory falls from 21.10 GiB to 9.28 GiB, close to the initial 8.99 GiB baseline.

This makes the behavior reproducible and allocation-dependent rather than evidence of a monotonically growing memory leak.

3. Windows page file occupancy remains low

Separately observed Windows page file statistics:

Name             AllocatedBaseSize CurrentUsage PeakUsage
----             ----------------- ------------ ---------
C:\pagefile.sys              32768          468       850

All sizes are in MiB.

The page file has 32 GiB allocated, but only approximately 468 MiB was in use at the time of measurement. This illustrates that the observed increase in committed memory is not equivalent to an equal increase in actual page file occupancy.

4. Impact during model training

During larger PyTorch model training workloads on the RTX PRO 6000 Blackwell, total Windows system committed memory has exceeded 70 GB.

The controlled test demonstrates that CUDA VRAM allocations contribute directly to vmmemWSL host commit.

This reduces available Windows commit headroom and can make the system dependent on larger page file configurations even when the additional memory is not actively being paged to disk.

Suspected area

The measurements suggest an interaction between CUDA GPU device allocations, WSL2 GPU virtualization, and Windows host commit accounting.

Possible areas for investigation include CUDA allocation backing or reservation behavior in the WSL2 GPU paravirtualization path.

The test does not establish whether physical host RAM is duplicated or whether the additional charge represents reserved virtual memory.

Question: Is the approximately 1:1 host commit charge for CUDA device allocations an intentional part of WSL2's GPU memory model, or can it be eliminated or reduced?

Diagnostic Logs

System information

Windows Version: 10.0.26200.7840
WSL Version: 3.0.1.0
WSL2 Kernel: 6.18.40.1-microsoft-standard-WSL2
Distribution: Ubuntu 24.04.3 LTS

GPU 0: NVIDIA RTX PRO 6000 Blackwell Workstation Edition
GPU 1: NVIDIA GeForce RTX 4090
NVIDIA Driver: 617.42

Windows PyTorch: 2.9.0+cu128
Windows PyTorch CUDA: 12.8

Windows page file information

Name             AllocatedBaseSize CurrentUsage PeakUsage
----             ----------------- ------------ ---------
C:\pagefile.sys              32768          468       850

Controlled GPU allocation results

GPU allocated    WSL private commit    Delta
0 GiB             8.99 GiB            Baseline
4 GiB            13.12 GiB            +4.13 GiB
8 GiB            17.13 GiB            +8.14 GiB
12 GiB           21.10 GiB            +12.11 GiB
Released          9.28 GiB            +0.29 GiB

Related issue

This report isolates a different observable behavior from the related CUDA context overhead issue: proportional Windows host commit growth during explicit CUDA device allocations, independently of model loading or inference frameworks.

The minimal PyTorch reproduction is included in the Repro Steps section.

Additional WSL-side PyTorch version information and WSL diagnostic logs can be provided if requested.

Activity

  1. github-actions commented on Nov 17, 2026

    @github-actions
    Contributor

    Logs are required for review from WSL team

    If this a feature request, please reply with '/feature'. If this is a question, reply with '/question'.
    Otherwise, please attach logs by following the instructions below, your issue will not be reviewed unless they are added. These logs will help us understand what is going on in your machine.

    How to collect WSL logs

    Download and execute collect-wsl-logs.ps1 in an administrative powershell prompt:

    Invoke-WebRequest -UseBasicParsing "https://raw.githubusercontent.com/microsoft/WSL/master/diagnostics/collect-wsl-logs.ps1" -OutFile collect-wsl-logs.ps1
    Set-ExecutionPolicy Bypass -Scope Process -Force
    .\collect-wsl-logs.ps1
    

    The script will output the path of the log file once done.

    If this is a networking issue, please use .\collect-wsl-logs.ps1 -LogProfile networking instead, following the instructions in Collect WSL logs for networking issues

    Once completed please upload the output files to this GitHub issue.

    See Collect WSL logs (recommended method).

    If you choose to email these logs instead of attaching them to the bug, please send them to wsl-gh-logs@microsoft.com with the GitHub issue number in the subject, and include a link to your GitHub issue comment in the message body, and reply with '/emailed-logs'.

  2. IISuperluminaLII commented on Nov 17, 2026

    @IISuperluminaLII
    Author

    The issue already includes a complete, minimal, deterministic reproduction using PyTorch CUDA allocations, alongside Windows host commit measurements at each allocation stage.

    The results demonstrate approximately 1:1 growth in vmmemWSL private committed memory with CUDA device allocations, followed by a return to baseline when the allocations are released.

    This should be independently reproducible on a comparable WSL2/CUDA configuration without requiring machine-specific diagnostic logs.

    The primary question is whether this behavior is an intentional consequence of WSL2's GPU memory management architecture or an unnecessary host commit charge.

    Could the WSL team first confirm whether this behavior is expected by design? If additional diagnostic traces are necessary to investigate a specific aspect that cannot be established using the reproduction, please clarify what information is missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions