Mixture-of-Experts for Cryptocurrency Order Execution: Training Stability, Tail Risk, and Failure Modes

What it claims

This is a repeated-seed evaluation, not a new architecture. It runs 100 independently trained runs per arm. At K=8 the mean IS difference against DDQL is −0.03 bps, 95% CI [−0.15, +0.09], and the authors note this does not establish equivalence. Both simple baselines beat every learned configuration: TWAP at 0.39 bps and immediate liquidation at 0.21 bps.

The MoE's apparent stability gain is narrow and comes with a cost:

• Complete-run collapse. Collapse means no trading before forced liquidation in all 500 evaluation episodes. It hits 11/100 DDQL runs and 0/100 runs at both K=4 and K=8 (Fisher p=0.0007 each).
• Inactivity per episode rises with K. It goes from 11.1% (DDQL) to 24.9% (K=8). This figure comes from an IS-cost-matching proxy, not from counted actions. 84 of 100 K=8 runs contain episodes that never trade. The MoE hides collapse at the run level while spreading it across episodes.
• Tail risk worsens monotonically with K. Per-run CVaR-95 at K=8 is +11.5 bps worse than DDQL, 95% CI [+3.7, +19.7]. Across-seed dispersion is lowest at K=8, but the pattern is non-monotone and does not survive family-wise adjustment.

How it works

• Episode. Each episode sells q₀=20 units in at most N=10 five-minute steps, about 50 minutes. The start bar is sampled uniformly at random.
• Order size. Median visible 10-level depth is 20.9 units, so the parent order is roughly the whole visible book.
• Actions. Each step is a market order for v ∈ {0,…,q_t} units, filled by walking the 10 bid levels. Because the action set always includes 0 and leftover inventory is force-liquidated, a never-trade policy is feasible.
• Terminal penalty. In the last two steps the book walk starts at the 5th-best bid. This mechanically favours early selling.
• Network. The DDQL network has 5 fully connected layers, 30 hidden nodes and LeakyReLU, following Macrì & Lillo.
• State. The state is 4-dimensional: normalised inventory, normalised time, normalised mid-price and bid-ask spread fraction. It contains no depth-of-book sizes.
• MoE. The MoE uses K copies of the same network. Routing is hard nearest-centroid K-Means, fitted on features from 300 TWAP warm-up episodes:
• realised volatility
• relative spread
• book imbalance
• a constant, so effectively 3-D

flowchart LR
  W[300 TWAP warm-up episodes] --> KM[Fit K-Means on features]
  KM --> R{Nearest centroid}
  P[Previous episode features] --> R
  R --> E[Expert k acts, own replay buffer]
  E -->|stopping time shapes features| P

Routing uses the previous episode's features. Episodes are sampled independently, so the router carries no information about the current episode's regime. Those features also depend on when the previous policy stopped trading. The authors conclude that regime conditioning cannot explain anything the MoE does.

Evidence

The authors decomposed the training specification on a device-matched CPU baseline:

• Interactions are non-additive. Annealing removes collapse, but combining it with the aligned reward brings collapse back.
• K=8 across six specifications. No K=8 run is fully inactive under any of them. In five of the six, 1.1–28.7% of episodes never trade.
• Seed count. Significance of the collapse-suppression result changes between 30 and 100 seeds.
• Simulator check. The fill pseudocode was checked against an independent implementation to machine precision, across 500 episodes and six probe policies.

Limits

• Simulator. It is frictionless historical replay with no price impact or reactive order flow. The 5th-best-bid terminal rule makes early liquidation nearly costless, which explains why the naive baselines win.
• Confounded arms. The dense loop halves its 15,000-transition replay buffer when full, and the MoE loop never does. The arms therefore differ in replay retention as well as in partitioning.
• Router-lag bug. The authors found and corrected one. Effects of routing through training remain unmeasured; only evaluation-time routing changes are bounded.
• Data. The source quantity unit is unverified. The data is 5-minute mean-aggregated BTC/USDT from Binance only.
• Untested. The soft-gating variant is not evaluated.
• Truncated page. The fetched text cuts off in the DDQL section. The reward definition, hyperparameters and data window are not shown here, and no code or licence is mentioned.