doc.xref_stream(xref).decode().splitlines() does NOT split the line #3419

Activity

1. JorjMcKie commented on Apr 27, 2024

   

   JorjMcKie

   on Apr 27, 2024

   Collaborator

   More actions

   This is not a bug.
   MuPDF has changed to no longer generate line breaks between PDF commands, but spaces.
2. JorjMcKie

   added

   not a bugnot a bug / user error / unable to reproducenot a bug / user error / unable to reproduce

   on Apr 27, 2024
3. JorjMcKie

   closed this as completedon Apr 27, 2024
4. JorjMcKie commented on Apr 28, 2024

   

   JorjMcKie

   on Apr 28, 2024

   Collaborator

   More actions

   @huahua74 There still is a way to get a page's contents with line breaks. It is with less comfort - on the other hand, MuPDF team had good reasons for their change.

   Line breaks between PDF commands are generated as part of saving the file with the option pretty=True. So if you use this little function with the page as parameter, it will return you the contents as desired:

   def pretty_contents(page):
       """Invoke with some PDF page."""
       doc = page.parent  # owing document
       new = fitz.open()  # make temp PDF, copy page into it
       new.insert_pdf(doc, from_page=page.number, to_page=page.number)
   
       # save temp pdf with pretty option
       pdfdata = new.tobytes(pretty=True, clean=True)
       new.close()  # close and re-open
       new = fitz.open("pdf", pdfdata)
   
       # read and return /Contents ... now with linebreaks
       contents = new[0].read_contents()
       new.close()
       return contents
   
   contents_lines = pretty_contents(page).splitlines()
5. ragebear00 commented on May 8, 2024

   

   ragebear00

   on May 8, 2024

   Last edited by ragebear00

   Author

   More actions

   many thanks! it is extremely helpful! I also find your reply earlier without "splitline()". Runing the script below under PyMuPDF 1.22.5 and 1.24 will end with different results. The following work smooth with 1.24 but not 1.22.5.

     page.clean_contents()
     xref = page.get_contents()[0]  # get xref of resulting /Contents object
     cont = bytearray(page.read_contents())  # read the contents source as a (modifyable) bytearray
     if cont.find(b"/Subtype/Watermark") > 0:  # this will confirm a marked-content watermark is present
         print("marked-content watermark present")
     while True:
         i1 = cont.find(b"/Artifact")  # start of definition
         if i1 < 0: break  # none more left: done
         i2 = cont.find(b"EMC", i1)  # end of definition
         cont[i1-2 : i2+3] = b""  # remove the full definition source "q ... EMC"
     doc.update_stream(xref, cont)  # replace the original source
     doc.ez_save("x.pdf")  # save to new file
   
6. firezym commented on Dec 4, 2024

   

   firezym

   on Dec 4, 2024

   More actions

   I comment out the page.clean_contents(), then the splitlines works ok now.

   The code iss revised a little bit from previous code in this discussion #1855. It works ok on current pymupdf release.

   pip install PyMuPDF

   import pymupdf
   
   def process_page(page : pymupdf.Page):
       """Process one page."""
       # doc = page.parent  # the page's owning document
       # page.clean_contents()  # clean page painting syntax
       xref = page.get_contents()[0]  # get xref of resulting /Contents
       changed = 0  # this will be returned
       # read sanitized contents, splitted by line breaks
       cont_lines = page.read_contents().splitlines()
       print(len(cont_lines))
       # print(cont_lines)
       for i in range(len(cont_lines)):  # iterate over the lines
           line = cont_lines[i]
           # print(line)
           if not (line.startswith(b"/Artifact") and b"/Watermark" in line):
               continue  # this was not for us
           # line number i starts the definition, j ends it:
           print(line)
           j = cont_lines.index(b"EMC", i)
           for k in range(i, j):
               # look for image / xobject invocations in this line range
               do_line = cont_lines[k]
               if do_line.endswith(b"Do"):  # this invokes an image / xobject
                   cont_lines[k] = b""  # remove / empty this line
                   changed += 1
       if changed > 0:  # if we did anything, write back modified /Contents
           doc.update_stream(xref, b"\n".join(cont_lines))
       return changed
   
   fpath = 'your_pdf_file_path/file_name.pdf'
   doc = pymupdf.open(fpath)
   changed = 0  # indicates successful removals
   for page in doc:
       changed += process_page(page)  # increase number of changes
   if changed > 0:
       x = "s" if doc.page_count > 1 else ""
       print(f"{changed} watermarks have been removed on {doc.page_count} page{x}.")
       doc.ez_save(doc.name.replace(".pdf", "-nowm.pdf"))
   else:
       print("Nothing to change")
7. JorjMcKie commented on Dec 4, 2024

   

   JorjMcKie

   on Dec 4, 2024

   Collaborator

   More actions

   There have been changes to the underlying MuPDF code for clean_contents():
   It does no longer generate line breaks between PDF contents commands, but instead 1 space.
   This has happened for an overall improved compressibility of content streams.

   Here is a little trick to get back these line breaks:

   import pymupdf
   
   
   def pretty_contents(page):
       doc = page.parent
       new = pymupdf.open()
       new.insert_pdf(doc, from_page=page.number, to_page=page.number)
       pdfdata = new.tobytes(pretty=True, clean=True)
       new.close()
       new = pymupdf.open("pdf", pdfdata)
       xrefs = new[0].get_contents()
       if not xrefs:
           return b""
       contents = new.xref_stream(xrefs[0])
       new.close()
       return contents
   
   
   doc = pymupdf.open("input.pdf")
   page = doc[0]
   cont = pretty_contents(page)
   # "cont" has line breaks again