Rough API Usage Estimates for evaluation on FutureX

[jmnian]

Hi MiroFlow team, thank you for open sourcing MiroFlow and it's a great repo! I'm reading your code and trying to run the FutureX benchmark and am wondering the rough API costs. For example if I were to run the ./scripts/run_evaluate_multiple_runs_futurex.sh for majority voting over 3 runs, what would the rough cost be for OpenRouter, search API, and others combined (assuming I use agent_quickstart_1.yaml and I add tool-searching, o3_hint: true, and o3_final_answer: true)?

Or could you please share how much money did it cost you to run the submission you gave to FutureX? The "GPT-5 (MiroFlow-preview DR)" currently on FutureX's leaderboard.

Any insights are highly appreciated as I am paying out of pocket for the API credits.

[BinWang28]

Hi @jmnian, Thanks for your interest.

For the cost, we did not do a thorough test. It also varies due to the difficulty of weekly released tasks from FutureX. It is suggested to run a few QA to try first and estimate the cost.

Also, note that the scripts are just for a quick start and not for obtaining optimal results. It requires more tools to obtain comprehensive results; definitely, searching is an important one.

(Looping @JubSteven as well)