Pillow 11.1.0 PNG file differences in Linux and MacOS

[henu]

We noticed with my coworker that newest Pillow version produces different PNG files when ran on MacOS, and this is currently breaking our tests.

We made this example to demonstrate the problem:

import PIL
from PIL import Image
ver = PIL.__version__
with open(f"/tmp/pillow_{ver}.png", "wb") as f:
    image = Image.new("RGBA", size=(10, 10), color=(256, 0, 0))
    image.save(f, format="png")

When ran on my Linux machine:

(venv) henu@nugetti:/tmp$ shasum /tmp/pillow_*.png
72f475411f224c867838cc01c1abef1508f647f4  /tmp/pillow_11.0.0.png
72f475411f224c867838cc01c1abef1508f647f4  /tmp/pillow_11.1.0.png

When ran on my coworker's MacOS machine:

$  shasum /tmp/pillow_11.*png
72f475411f224c867838cc01c1abef1508f647f4  /tmp/pillow_11.0.0.png
504925807f3cdd56c0f5ab49978257678da42f0a  /tmp/pillow_11.1.0.png

He also provided this extra info, in case it helps:

$  cmp -b /tmp/pillow_11.*png
/tmp/pillow_11.0.0.png /tmp/pillow_11.1.0.png differ: char 37, line 5 is  30   33

[radarhere]

Hi. This is a duplicate of #8749. Here are the key points.

#8749 (comment)

you're experiencing the effects of #8500, where we switched from zlib to zlib-ng in our wheels. See also https://pillow.readthedocs.io/en/stable/releasenotes/11.1.0.html#zlib-ng-in-wheels. As noted in the PR, this changes the output, but improves performance.

#8749 (comment)

I don’t think that file bytes comparison is a valid test for PNG files. There are several different ways that a PNG can be encoded losslessly into different bytes — there are different filter methods that can be chosen arbitrarily by the encoder to improve compression on a scan line by scan line basis. On top of that, the underlying compression can be done that are bit different, but still decompress to the same stream of bits.

So, all in all, I think that bit differences in png files is an invalid failure for a test.

#8749 (comment)

I'm not sure what the purpose of the test is, but I would suggest you compare the pixels in the images, rather than the bytes.

Let us know if you have any further questions.

[henu]

Ah, it indeed looks like identical problem. Thanks for finding it for me.

And you are right, the bit comparison of images is kind of stupid... Well, I just hope it would get more deterministic in the future 😅️ In the mean time I guess I have to compare it pixel by pixel.

Thanks for all the help!