Automatic unloading of models after inactivity

[Seven-94]

Hi,

Thank you for the new features, especially the model router in the web UI. It’s been really useful to switch models on the fly.

I was wondering: is there an existing way to automatically unload model after a period of inactivity? I’ve noticed that model stay loaded in VRAM until manually unloaded or when switching model. Since I use llama.cpp on my main machine, this sometimes causes issues when switching to other applications that use VRAM and i forget to manually unload it. Maybe I missed a setting or a command that enables this?

I acutally use the preset.ini method --models-max 1.

If it’s not already possible, would it be feasible to add an option to unload models automatically after, say, 5 minutes of inactivity? This could help free up VRAM when the app is not in use.

Thanks for your work