Absolute-sized orders canceled for insufficient margin without warning (relative-sized orders do warn) #1334
Description
Expected behavior
Summary
When placing an order with an absolute size (e.g. size=916) that slightly exceeds available margin once commission / adjusted price are considered, the broker silently cancels the order:
Strategy.buy() returns an Order object
_Broker._process_orders() later removes it due to insufficient margin
No warning or exception is raised
Result: 0 trades, no user-visible feedback
For relative-sized orders (0 < abs(size) < 1), the same “insufficient margin” condition does raise a UserWarning:
Broker canceled the relative-sized order due to insufficient margin.
This behaviour is inconsistent between sizing modes and makes debugging absolute-sized strategies harder. From the Strategy’s point of view, buy() appears to succeed, but nothing ever executes and there is no indication why.
Environment
backtesting.py version: 0.6.5
Python: 3.12.3
OS: Ubuntu 24.04.2 LTS (noble)
Code sample
"""
Minimal example 2: 915 vs 916 comparison
The following test runs the same strategy twice on the same data, same cash, same commission, only changing size from 915 to 916.
"""
from backtesting import Backtest, Strategy
import pandas as pd
sizes = (915, 916) # 915: trade executes, 916: canceled for margin & silent
def run_strat(size):
# simple dummy data: flat price ~1.09 for a few bars
data = pd.DataFrame(
{
"Open": [1.0916] * 10,
"High": [1.0918] * 10,
"Low": [1.0912] * 10,
"Close": [1.0916] * 10,
},
index=pd.date_range("2024-01-01", periods=10, freq="1min"),
)
class BigSize(Strategy):
def init(self):
pass
def next(self):
if self.position.size == 0:
# Use the size passed into run_strat(...)
self.buy(size=size)
bt = Backtest(
data,
BigSize,
cash=1000,
commission=0.0002,
finalize_trades=True, # ensure open trades are closed into stats
)
stats = bt.run()
trades = stats["_trades"]
print(trades)
print("n_trades:", len(trades))
if trades.empty:
print("WARNING: no trades executed")
for s in sizes:
print('----------------------------')
print(f"run strategy with size {s}")
run_strat(s)
print()Actual behavior
Output of script above:
run strategy with size 915
Size EntryBar ExitBar EntryPrice ExitPrice SL TP PnL
0 915 2 9 1.0916 1.0916 None None -0.399526
Commission ReturnPct EntryTime ExitTime
0 0.399526 -0.0004 2024-01-01 00:02:00 2024-01-01 00:09:00
Duration Tag
0 0 days 00:07:00 None
n_trades: 1
run strategy with size 916
Empty DataFrame
Columns: [Size, EntryBar, ExitBar, EntryPrice, ExitPrice, SL, TP, PnL,
Commission, ReturnPct, EntryTime, ExitTime, Duration, Tag]
Index: []
n_trades: 0
WARNING: no trades executed
#-----------------------------
So:
size = 915 → 1 trade in _trades as expected.
size = 916 → 0 trades, no exception, no UserWarning about margin.
Given cash=1000, commission=0.0002, and price ≈ 1.09:
size = 915 is just under the true margin limit and executes.
size = 916 is just over the true margin limit once commission and adjusted price are considered, so the broker cancels it internally as “insufficient margin.”
From the user’s perspective, though, both self.buy(size=915) and self.buy(size=916) “succeed” (they return an Order), but only the former ever results in a trade, and only the relative-size path in the broker emits a warning.
Additional info, steps to reproduce, full crash traceback, screenshots
Internal behaviour (from instrumentation)
With additional logging added to _Broker._process_orders, the following line appears for the failing size=916 case:
[Broker._process_orders] canceling order=<Order size=916.0, ...> due to insufficient margin:
need_size=916 adj_price_plus_comm=1.09181 margin_avail=1000.00 leverage=1.00
So the logic itself is correct:
notional = 916 * 1.09181 ≈ 1000.098
limit = margin_avail * leverage = 1000 * 1.0 = 1000
notional > limit ⇒ margin violated ⇒ order canceled
The issue is that this cancellation is silent for absolute sizes:
The order is created and then removed,
No warning is raised,
The user just sees no trades and no explanation.
By contrast, in the relative-size branch (when -1 < size < 1), the code does:
size = int(margin_available * leverage * abs(size) / adj_price_plus_commission)
if not size:
warnings.warn(
f"time={self._i}: Broker canceled the relative-sized "
f"order due to insufficient margin.",
category=UserWarning,
)
self.orders.remove(order)
continue
So the relative-size path surfaces a clear UserWarning, the absolute-size path does not.
Expected / suggested behaviour
The margin logic itself is fine: the broker should reject orders whose notional exceeds margin_available * leverage.
The problem is the lack of feedback for absolute-sized orders, while relative-sized orders already warn in this situation.
A minimal, backwards-compatible improvement would be to mirror the relative-size behaviour in the absolute-size margin check inside _Broker._process_orders, e.g.:
import warnings
if abs(need_size) * adjusted_price_plus_commission > self.margin_available * self._leverage:
warnings.warn(
f"time={self._i}: Broker canceled the absolute-sized order due to "
f"insufficient margin (need_size={need_size}, "
f"adj_price_plus_comm={adjusted_price_plus_commission:.5f}, "
f"margin_avail={self.margin_available:.2f}, "
f"leverage={self._leverage:.2f}).",
category=UserWarning,
)
self.orders.remove(order)
continue
This would:
Keep the current margin and trading logic unchanged,
Maintain backward compatibility,
Make absolute-sized orders consistent with relative-sized ones in terms of user feedback,
Greatly reduce confusion in cases where buy() appears to work but no trades are ever opened.
Software versions
for pkg in ('backtesting', 'pandas', 'numpy', 'bokeh'):
print('-', pkg, getattr(import(pkg), 'version', 'git'))
-
backtesting 0.6.5
-
pandas 2.3.2
-
numpy 2.3.2
-
bokeh 3.8.1
-
OS:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS"