Managed ComfyUI can't handle custom nodes that print emojis

[arcusmaximus]

First of all, thank you for creating and maintaining this software.

I discovered today that custom nodes that print special characters cause the managed server to fail to start (at least on Windows). One example is SeedVR2 which does the following:

print(f"⚠️  SeedVR2 optimizations check: SageAttention {sage_status} | Flash Attention {flash_status} | Triton {triton_status}")

This works fine when ComfyUI is launched manually from the command line, but when it's launched by the Krita plugin, it fails with the following error:

INFO File "D:\Krita\sd\ComfyUI\custom_nodes\seedvr2_videoupscaler\src\optimization\compatibility.py", line 658, in <module>
INFO File "D:\Krita\sd\ComfyUI\app\logger.py", line 32, in write
INFO super().write(data)
INFO UnicodeEncodeError: 'cp932' codec can't encode character '\u26a0' in position 0: illegal multibyte sequence

It turns out that this is a Python problem: asyncio.create_subprocess_exec() apparently pipes the process's stdout - which is normally UTF-16 - using the system codepage, which is of course a legacy feature for programs that predate the invention of Unicode.

Fortunately, it's possible to override this by running Python in "UTF-8 mode." Going by this comment in the plugin's source code, I'm guessing Krita does this already. The plugin could do it for ComfyUI by launching it with the PYTHONUTF8 environment variable set to 1. This would also make the determine_system_encoding() function unnecessary.

(I know we're not really supposed to install extra custom nodes into the managed server, but at the same time, setting up a whole custom server just for this one upscaler seemed overkill. And who knows, maybe a prepackaged custom node - or even ComfyUI itself - will print emojis too in the future. As such, please consider addressing this.)

[Acly]

It was running in utf-8 mode in the past, I removed it here: 8ff40c7

It just causes other issues instead. Since utf-8 is not the default on windows, in my experience it is more often not expected and cause of mysterious errors.

which is normally UTF-16 - using the system codepage, which is of course a legacy feature for programs that predate the invention of Unicode.

should be legacy perhaps, but afaik even after decades it's not. unicode is not the default on windows, you have to manually enable it.

[arcusmaximus]

You're right that the Windows codepage is not UTF-8 by default. What I meant is that the very concept of codepages was supposed to be a thing of the past, irrelevant for applications written after the year 2000 - they can (and should) leave codepage-dependent 8-bit characters behind and work with UTF-16 just like the operating system itself.

And yet, Python apparently goes out of its way to keep doing things 90's style:

Non-character devices such as disk files and pipes use the system locale encoding (i.e. the ANSI codepage).

But alright, I guess they have their historical baggage and can't change this default anymore.

If PYTHONUTF8 (or the equivalent -X utf8 that you used to have) causes collateral damage, what about setting PYTHONIOENCODING to utf-8? That one only affects stdin/stdout, not the entire Python filesystem encoding.

For as far as I can tell, this should result in a configuration that's identical to running ComfyUI directly from the command prompt. After all, the documentation for sys.stdout also says this:

On Windows, UTF-8 is used for the console device. [Regardless of the system codepage]

[Acly]

That might be better, but I'm still hesitant to risk a change here. It's practically impossible to test and I never fully understood the codepage related errors some people had in the past (haven't seen any in a long time now).

But you can always change that for yourself by setting the environment variable globally on your system?

[arcusmaximus]

But you can always change that for yourself by setting the environment variable globally on your system?

Yes, I did just that. The main reason I logged this ticket was to proactively prevent the error for others users.

I'm still hesitant to risk a change here.

I understand. I'll leave it up to you then whether to try finding further options or just close this issue.