clean_contents(sanitize=True) drops glyph positioning, duplicating an overprinted leading glyph (regression in 1.28.0 from 1.27.2.3)

Description of the bug

Page.apply_redactions() corrupts unredacted text: the leading glyph of a
text span is duplicated when the page content stream is regenerated. It
reproduces even when the redaction removes nothing — a dummy annotation over an
empty area with every removal flag set to *_NONE — so it is the content-stream
regeneration itself, not the removal, that introduces the extra glyph.

The trigger is a span drawn with an "overprint" construct: a glyph is shown, then
0 0 TD moves back to the line-matrix origin, then a word is shown starting at
that same x. The two draws overlap exactly, so the page correctly reads faeces.
After apply_redactions() regenerates the stream, the draws no longer coincide
and both become visible, yielding ffaeces.

The duplication is not cosmetic: after redaction the two f glyphs render at
distinct x-origins (confirmed via get_text("rawdict")), so both the rendered
page and the extracted text show the extra letter.

This is a regression — correct in every release from 1.24.14 through
1.27.2.3, broken only in 1.28.0 (MuPDF 1.29.0):

PyMuPDF	MuPDF	text after no-op `apply_redactions()`
1.24.14	1.24.11	`faeces` (correct)
1.26.7	1.26.12	`faeces` (correct)
1.27.1	1.27.1	`faeces` (correct)
1.27.2	1.27.2	`faeces` (correct)
1.27.2.3	1.27.2	`faeces` (correct)
1.28.0	1.29.0	`ffaeces` (leading glyph duplicated)

Environment: PyMuPDF 1.28.0 / MuPDF 1.29.0 (broken), last good 1.27.2.3 /
MuPDF 1.27.2; Python 3.13.13; Linux x86-64. The construct is emitted by
real-world producers (first hit on an InDesign-generated PDF, where the extra
glyph landed inside a math formula). Looks related to the MuPDF 1.29.0 text/font
content-stream changes also behind the other 1.28.0 regressions (#5049, #5052).

How to reproduce the bug

Fully self-contained. Step 1 builds a ~600-byte PDF with the standard library
only (Helvetica base-14, no embedded fonts, no third-party content); step 2 needs
PyMuPDF.

Step 1 — make_repro_pdf.py (writes overprint.pdf):

"""Write a minimal, content-free PDF that triggers the redaction glyph-duplication bug.

The page draws the letter 'f' twice using the "overprint" construct:

    (f)Tj        # draw a lone 'f' at the line-matrix origin
    0 0 TD       # TD moves relative to the LINE matrix, not the pen -> back to origin
    (faeces)Tj   # draw 'faeces' starting at the SAME x -> its 'f' overprints the lone 'f'

Both f's paint at the same position, so the page shows a single, correct
"faeces" and text extraction returns "faeces".
"""

import sys

CONTENT = b"BT\n/F1 24 Tf\n1 0 0 1 100 700 Tm\n(f)Tj\n0 0 TD\n(faeces)Tj\nET\n"

OBJECTS = [
    b"<< /Type /Catalog /Pages 2 0 R >>",
    b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>",
    b"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 300 800] "
    b"/Resources << /Font << /F1 5 0 R >> >> /Contents 4 0 R >>",
    b"<< /Length %d >>\nstream\n%s\nendstream" % (len(CONTENT), CONTENT),
    b"<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>",
]


def build() -> bytes:
    pdf = b"%PDF-1.7\n"
    offsets = []
    for i, body in enumerate(OBJECTS, start=1):
        offsets.append(len(pdf))
        pdf += b"%d 0 obj\n%s\nendobj\n" % (i, body)
    xref_pos = len(pdf)
    pdf += b"xref\n0 %d\n" % (len(OBJECTS) + 1)
    pdf += b"0000000000 65535 f \n"
    for off in offsets:
        pdf += b"%010d 00000 n \n" % off
    pdf += b"trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n" % (len(OBJECTS) + 1, xref_pos)
    return pdf


if __name__ == "__main__":
    out = sys.argv[1] if len(sys.argv) > 1 else "overprint.pdf"
    with open(out, "wb") as fh:
        fh.write(build())
    print(f"wrote {out}")

Step 2 — repro.py (extract, no-op redact, extract again):

import pymupdf

doc = pymupdf.open("overprint.pdf")
page = doc[0]

before = page.get_text().strip()

# A redaction that targets nothing: empty corner, every removal flag NONE.
page.add_redact_annot(pymupdf.Rect(2, 2, 5, 5))
page.apply_redactions(
    text=pymupdf.PDF_REDACT_TEXT_NONE,
    images=pymupdf.PDF_REDACT_IMAGE_NONE,
    graphics=pymupdf.PDF_REDACT_LINE_ART_NONE,
)
after = page.get_text().strip()

print(f"PyMuPDF {pymupdf.version[0]} / MuPDF {pymupdf.version[1]}")
print(f"  before redaction: {before!r}")
print(f"  after  redaction: {after!r}")

Run:

python make_repro_pdf.py overprint.pdf
python repro.py

Observed on 1.28.0 (expected: after stays 'faeces'):

PyMuPDF 1.28.0 / MuPDF 1.29.0
  before redaction: 'faeces'
  after  redaction: 'ffaeces'

PyMuPDF version

1.28.0

Operating system

Linux

Python version

3.13

Thanks for submitting this. Confirming the behavior.
Please note:
The problem has nothing to do with redactions! ~~I am not sure whether it is an actual problem.~~

import pymupdf
doc=pymupdf.open("overprint.pdf")
page=doc[0]
print(page.read_contents().decode())  # print the page's Contents stream
BT
/F1 24 Tf
1 0 0 1 100 700 Tm
(f)Tj
0 0 TD
(faeces)Tj
ET

page.clean_contents()  # clean / rebuild the Contents stream
print(page.read_contents().decode())  # print it again:
q BT/F1 24 Tf 100 700 TD (f)Tj (faeces)Tj ET Q

Applying redactions inevitably rebuilds a page's Contents stream - just like method .clean_contents(sanitize=True).
Rebuilding / sanitizing a Contents stream should not cause text extractions produce different results.
In its shortest form, the actual problem is this:

doc=pymupdf.open("overprint.pdf")
page=doc[0]
page.get_text()
'faeces\n'
page.clean_contents(sanitize=True)  # "sanitize" is the default
page.get_text()
'ffaeces\n'

This is the link to the connected MuPDF issue: https://bugs.ghostscript.com/show_bug.cgi?id=709556

Would you agree to change the issue title? Redactions are irrelevant, mentioning them blurs the actual cause. It should rather read something like
"Clean-Sanitize removes glyph positions ..."
This would help identifying the problem at a later time.

Thanks for the quick response Jorj. Sure, I have renamed the title. Let me know if it makes sense now or if you would prefer any other.
And thanks for the package !!!

The MuPDF team has developed a fix. It will be included in PyMuPDF with the first release generated after availability of a MuPDF version having this fix.

Glad to hear it. Thanks again for the whole library and ecosystem 😀

Current situation:

The mupdf fix is on mupdf banch master.
We have a Pymupdf test tests/test_general.py:test_5054().
This expects failure unless mupdf version is >= 1.29, which currently means mupdf's master branch.

tests/: fixed some tests with current mupdf-1.28.x version 1.28.5.

Fixed in PyMuPDF-2.0.

Navigation Menu

Repository navigation

clean_contents(sanitize=True) drops glyph positioning, duplicating an overprinted leading glyph (regression in 1.28.0 from 1.27.2.3) #5054

Description of the bug

How to reproduce the bug

PyMuPDF version

Operating system

Python version

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

clean_contents(sanitize=True) drops glyph positioning, duplicating an overprinted leading glyph (regression in 1.28.0 from 1.27.2.3) #5054

Description

Description of the bug

How to reproduce the bug

PyMuPDF version

Operating system

Python version

Activity

JorjMcKie commented on Jul 20, 2026

JorjMcKie commented on Jul 20, 2026

JorjMcKie commented on Jul 20, 2026

dmontaner-sme commented on Jul 20, 2026

JorjMcKie commented on Jul 27, 2026

dmontaner-sme commented on Jul 27, 2026

julian-smith-artifex-com commented on Sep 4, 2026

julian-smith-artifex-com commented on Oct 19, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions