Skip to content

Small-integer inputs produce incorrect regression metric values or runtime errors #9246

Open

Description

@leecoring

Describe the bug

With pairs of uint8, int8, or int16 images, MONAI regression metrics can subtract in the integer dtype and overflow or underflow before any subsequent floating-point conversion. A later conversion cannot recover the correct difference. For example, 0 - 255 wraps to 1 with uint8 inputs, so MSE, RMSE, PSNR, and MAPE can return incorrect values. MAE can also fail when computing the mean of an integer tensor.

To Reproduce

import torch
from monai.metrics import MAEMetric, MAPEMetric, MSEMetric, PSNRMetric, RMSEMetric

prediction = torch.full((1, 1, 2, 3, 4), 255, dtype=torch.uint8)
reference = torch.zeros_like(prediction)
for name, metric in (
    ("MSE", MSEMetric()),
    ("RMSE", RMSEMetric()),
    ("MAE", MAEMetric()),
    ("PSNR", PSNRMetric(max_val=255)),
):
    try:
        print(name, metric(prediction, reference).item())
    except Exception as error:
        print(name, type(error).__name__, str(error))

# Use a nonzero reference for a simple percentage-error example.
print("MAPE", MAPEMetric()(prediction, torch.ones_like(reference)).item())
Metric Expected Observed before the fix
MSE 65025 1
RMSE 255 1
MAE 255 NotImplementedError from integer mean
PSNR (max_val=255) 0 dB approximately 48.1308 dB
MAPE (reference = 1) 25400% 200%

Expected behavior

Small-integer images should be converted to a suitable floating-point calculation dtype before subtraction. Existing floating-point inputs, including float64, should keep their precision.

Environment

Reproduced on upstream dev commit 827fa24fef4f474f70ead141412a2a0675990684, using macOS CPU, Python 3.14.7, PyTorch 2.15.0, and NumPy 2.5.4. A separate unmodified checkout was used to check the failure before applying the local patch.

Additional context

The local patch handles pairs of uint8/int8/int16 inputs at the shared RegressionMetric entry point. It does not claim to solve all integer or mixed-dtype cases and does not change direct functional helper calls. Large int32/int64 values need separate care because an unconditional floating-point cast can erase small residuals.

No patient data are used. All examples and test inputs are synthetic.

Tool assistance

The submitter provided the explanation of the problem and proposed fix. OpenAI Codex assisted with the investigation, English translation, and preparation of this report. Codex performed the local investigation, implementation, and reported checks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions