float values mangled by scrub() with clean_pages enabled (via a bug in the underlying mupdf "clean" function)

[johnkw]

Description of the bug

Float values mangled by scrub() with clean_pages enabled. The save() with clean enabled also does this.

This only seems to occur in certain directions. I managed to pare this down to a very small input PDF.

Repro in python:
n = pymupdf.Document(stream=open('in.pdf','rb').read()) ; n.save('out.pdf', no_new_id=True, clean=True)

Then:

in.pdf

$ diff -aU0 in.pdf out.pdf | cat -e | grep ' 75 '
-116.8 9.66 3.2 75 re$
-120.8 9.66 0.8 75 re$
-133.6 9.66 0 75 re f Q$
+q 116.8 9.66 3.199997 75 re 120.8 9.66 .80000308 75 re 133.6 9.66 0 75 re f Q$

In this case 3 of the floats are mangled, but others are fine.

PyMuPDF version

1.26.3

[JorjMcKie]

1. That is not a bug and leads to zero consequential effect in whatever context the resulting PDF may be used. The value differences are in the range of 3e-6 - much smaller than 1e-5, the epsilon value below which no significant effect in PDF can be observed.
2. As you already noted, the effect is not caused by method .scrub() itself. In so far the title of the issue is misleading. It happens inside MuPDF's content stream recreation invoked by page.clean_contents(sanitize=True) and also by the corresponding .save() option. In both cases this is outside of PyMuPDF's control.

[johnkw]

It does cause visual impacts, and can result in the difference between legible and illegible text apparently due to some weird rounding in the PDF viewing? Even if it wasn't a visual bug though the file size bloat alone would be worth fixing.

You're saying this should be moved to https://github.com/ArtifexSoftware/mupdf though?

[JorjMcKie]

You're saying this should be moved to https://github.com/ArtifexSoftware/mupdf though?

Yes - reproduce this by their CLI tool:

$ mutool clean -gggdsi in.pdf out.pdf
syntax error: expected generation number (0 ? obj)
warning: trying to repair broken xref
warning: repairing PDF document

As you can see, the input file contains errors (which are repaired). The output looks like this:

[robinwatts]

It does cause visual impacts, and can result in the difference between legible and illegible text apparently due to some weird
rounding in the PDF viewing?

I'm one of the MuPDF authors.

Please provide evidence for this. A file before the rewrite, and a file after the rewrite that look different would be suitable. Thanks.

Even if it wasn't a visual bug though the file size bloat alone would be worth fixing.

We already do our best to minimise such changes, but they are very hard to avoid due to the nature of floating point representation.

We are not reading "3.2" as an ascii string and then rewriting it out. We convert to a float, and then write that float out again. Unfortunately, this can result in changes to things such as "3.1999997".

[JorjMcKie]

@johnkw as Robin explained: these are inevitable effects of how floats are stored internally. Have you ever tried the following in Python:

or this

[JorjMcKie]

Closing this for now.
@johnkw please feel free to reopen with an example file that shows visual differences because of the described effect.

[johnkw]

Yes, floats are fine for situations where precision and accuracy are unimportant, and close approximations are fine. But this bug is definitely not "inevitable" -- libraries such as python's decimal can be used to prevent bugs of this classification.

I cannot provide the current PDF due to proprietary information. If this bug is fixed and the visual artifacts remain, I will provide another sample.

For now I would advise addressing this bug with the file bloat and rounding issues.

The file size bloat is unarguably a bug in and of itself, regardless of the visual artifacts.