Chapter 15: Strategy Risk¶

AFML Ch. 15 -- Understanding the risk of strategy failure.

A strategy's profitability depends on three levers: precision (hit rate), trading frequency, and the average win/loss ratio. This chapter shows how to derive the Sharpe ratio from these components, compute the minimum precision or frequency needed for a target Sharpe, and estimate the probability that a strategy will fail in production.

This notebook demonstrates:

• Sharpe ratio from precision, frequency, and win/loss ratio
• Implied precision for a target Sharpe ratio
• Implied frequency for a target Sharpe ratio
• Strategy failure probability

In [1]:

import numpy as np
import matplotlib.pyplot as plt
import pymlfinance

%matplotlib inline
plt.rcParams['figure.figsize'] = (14, 6)
plt.rcParams['figure.dpi'] = 150
plt.rcParams['font.size'] = 15
plt.rcParams['axes.titlesize'] = 18
plt.rcParams['axes.labelsize'] = 15
plt.rcParams['xtick.labelsize'] = 13
plt.rcParams['ytick.labelsize'] = 13
plt.rcParams['legend.fontsize'] = 13
np.random.seed(42)

import numpy as np import matplotlib.pyplot as plt import pymlfinance %matplotlib inline plt.rcParams['figure.figsize'] = (14, 6) plt.rcParams['figure.dpi'] = 150 plt.rcParams['font.size'] = 15 plt.rcParams['axes.titlesize'] = 18 plt.rcParams['axes.labelsize'] = 15 plt.rcParams['xtick.labelsize'] = 13 plt.rcParams['ytick.labelsize'] = 13 plt.rcParams['legend.fontsize'] = 13 np.random.seed(42)

Sharpe Ratio from Precision¶

Given a strategy's hit rate (precision), trading frequency, and average win/loss ratio, we can derive the expected annualized Sharpe ratio analytically (AFML Snippet 15.1). This avoids the need for a full backtest to estimate performance.

In [2]:

# Base case: symmetric bets, daily trading, 55% hit rate
precision = 0.55
freq = 252.0       # daily trading
win_loss = 1.0      # symmetric payoff

sr = pymlfinance.backtesting.sr_from_precision(precision, freq, win_loss)
print(f"Precision: {precision:.0%}, Frequency: {freq:.0f}/yr, W/L: {win_loss:.1f}")
print(f"Expected Sharpe Ratio: {sr:.4f}")

# Base case: symmetric bets, daily trading, 55% hit rate precision = 0.55 freq = 252.0 # daily trading win_loss = 1.0 # symmetric payoff sr = pymlfinance.backtesting.sr_from_precision(precision, freq, win_loss) print(f"Precision: {precision:.0%}, Frequency: {freq:.0f}/yr, W/L: {win_loss:.1f}") print(f"Expected Sharpe Ratio: {sr:.4f}")

Precision: 55%, Frequency: 252/yr, W/L: 1.0
Expected Sharpe Ratio: 1.5954

In [3]:

# Explore how Sharpe changes with precision
precisions = np.linspace(0.40, 0.70, 31)
srs = [pymlfinance.backtesting.sr_from_precision(p, freq, win_loss) for p in precisions]

print(f"{'Precision':>10} {'Sharpe':>10}")
for p, s in zip(precisions[::5], srs[::5]):
    print(f"{p:>10.0%} {s:>10.4f}")

# Explore how Sharpe changes with precision precisions = np.linspace(0.40, 0.70, 31) srs = [pymlfinance.backtesting.sr_from_precision(p, freq, win_loss) for p in precisions] print(f"{'Precision':>10} {'Sharpe':>10}") for p, s in zip(precisions[::5], srs[::5]): print(f"{p:>10.0%} {s:>10.4f}")

 Precision     Sharpe
       40%    -3.2404
       45%    -1.5954
       50%     0.0000
       55%     1.5954
       60%     3.2404
       65%     4.9923
       70%     6.9282

Visualisation: Sharpe Surface¶

The Sharpe ratio depends jointly on precision and the win/loss ratio. This heatmap shows how different combinations produce different expected Sharpe ratios at daily trading frequency.

In [4]:

precisions_grid = np.linspace(0.40, 0.70, 31)
wl_ratios = np.linspace(0.5, 2.5, 21)

sr_grid = np.zeros((len(wl_ratios), len(precisions_grid)))
for i, wl in enumerate(wl_ratios):
    for j, p in enumerate(precisions_grid):
        sr_grid[i, j] = pymlfinance.backtesting.sr_from_precision(p, freq, wl)

fig, ax = plt.subplots(figsize=(12, 7))
im = ax.imshow(sr_grid, aspect='auto', origin='lower', cmap='RdYlGn',
               extent=[precisions_grid[0], precisions_grid[-1],
                       wl_ratios[0], wl_ratios[-1]],
               vmin=-2, vmax=4)
ax.contour(precisions_grid, wl_ratios, sr_grid,
           levels=[0, 1, 2, 3], colors='black', linewidths=1.0)
cs = ax.contour(precisions_grid, wl_ratios, sr_grid,
                levels=[0], colors='red', linewidths=2.0)
ax.clabel(cs, fmt='SR=0')
cb = plt.colorbar(im, ax=ax)
cb.set_label('Sharpe Ratio')
ax.set_xlabel('Precision (Hit Rate)')
ax.set_ylabel('Win/Loss Ratio')
ax.set_title('Expected Sharpe Ratio Surface (freq=252)')
plt.tight_layout()
plt.show()

precisions_grid = np.linspace(0.40, 0.70, 31) wl_ratios = np.linspace(0.5, 2.5, 21) sr_grid = np.zeros((len(wl_ratios), len(precisions_grid))) for i, wl in enumerate(wl_ratios): for j, p in enumerate(precisions_grid): sr_grid[i, j] = pymlfinance.backtesting.sr_from_precision(p, freq, wl) fig, ax = plt.subplots(figsize=(12, 7)) im = ax.imshow(sr_grid, aspect='auto', origin='lower', cmap='RdYlGn', extent=[precisions_grid[0], precisions_grid[-1], wl_ratios[0], wl_ratios[-1]], vmin=-2, vmax=4) ax.contour(precisions_grid, wl_ratios, sr_grid, levels=[0, 1, 2, 3], colors='black', linewidths=1.0) cs = ax.contour(precisions_grid, wl_ratios, sr_grid, levels=[0], colors='red', linewidths=2.0) ax.clabel(cs, fmt='SR=0') cb = plt.colorbar(im, ax=ax) cb.set_label('Sharpe Ratio') ax.set_xlabel('Precision (Hit Rate)') ax.set_ylabel('Win/Loss Ratio') ax.set_title('Expected Sharpe Ratio Surface (freq=252)') plt.tight_layout() plt.show()

Implied Precision¶

Given a target Sharpe ratio, what minimum hit rate does the strategy need? This inverts the Sharpe formula to solve for precision (AFML Snippet 15.3).

In [5]:

target_srs = [0.5, 1.0, 1.5, 2.0, 2.5]

print(f"{'Target SR':>12} {'Implied Precision':>20}")
print(f"{'-'*12:>12} {'-'*20:>20}")
for tsr in target_srs:
    ip = pymlfinance.backtesting.implied_precision(tsr, freq, win_loss)
    print(f"{tsr:>12.1f} {ip:>20.2%}")

print(f"\nAssumptions: freq={freq:.0f}/yr, W/L={win_loss:.1f}")

target_srs = [0.5, 1.0, 1.5, 2.0, 2.5] print(f"{'Target SR':>12} {'Implied Precision':>20}") print(f"{'-'*12:>12} {'-'*20:>20}") for tsr in target_srs: ip = pymlfinance.backtesting.implied_precision(tsr, freq, win_loss) print(f"{tsr:>12.1f} {ip:>20.2%}") print(f"\nAssumptions: freq={freq:.0f}/yr, W/L={win_loss:.1f}")

   Target SR    Implied Precision
------------ --------------------
         0.5               51.57%
         1.0               53.14%
         1.5               54.70%
         2.0               56.25%
         2.5               57.78%

Assumptions: freq=252/yr, W/L=1.0

In [6]:

# How win/loss ratio affects required precision
fig, ax = plt.subplots(figsize=(10, 6))
wl_values = [0.5, 1.0, 1.5, 2.0]
target_sr_range = np.linspace(0.1, 3.0, 30)

for wl in wl_values:
    implied_p = [pymlfinance.backtesting.implied_precision(tsr, freq, wl)
                 for tsr in target_sr_range]
    ax.plot(target_sr_range, implied_p, linewidth=2, label=f'W/L = {wl:.1f}')

ax.set_xlabel('Target Sharpe Ratio')
ax.set_ylabel('Required Precision')
ax.set_title('Implied Precision vs Target Sharpe Ratio')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(0.3, 1.0)
ax.axhline(0.5, color='gray', linestyle=':', alpha=0.5, label='50% (coin flip)')
plt.tight_layout()
plt.show()

# How win/loss ratio affects required precision fig, ax = plt.subplots(figsize=(10, 6)) wl_values = [0.5, 1.0, 1.5, 2.0] target_sr_range = np.linspace(0.1, 3.0, 30) for wl in wl_values: implied_p = [pymlfinance.backtesting.implied_precision(tsr, freq, wl) for tsr in target_sr_range] ax.plot(target_sr_range, implied_p, linewidth=2, label=f'W/L = {wl:.1f}') ax.set_xlabel('Target Sharpe Ratio') ax.set_ylabel('Required Precision') ax.set_title('Implied Precision vs Target Sharpe Ratio') ax.legend() ax.grid(True, alpha=0.3) ax.set_ylim(0.3, 1.0) ax.axhline(0.5, color='gray', linestyle=':', alpha=0.5, label='50% (coin flip)') plt.tight_layout() plt.show()

Implied Frequency¶

How often must a strategy trade to achieve a target Sharpe, given its precision and win/loss ratio? More frequent trading compensates for a lower edge (AFML Snippet 15.4).

In [7]:

precisions_list = [0.51, 0.52, 0.55, 0.60]
target_sr = 1.0

print(f"Target Sharpe Ratio: {target_sr:.1f}, W/L: {win_loss:.1f}")
print(f"\n{'Precision':>12} {'Implied Freq (trades/yr)':>26}")
print(f"{'-'*12:>12} {'-'*26:>26}")
for p in precisions_list:
    implied_f = pymlfinance.backtesting.implied_frequency(target_sr, p, win_loss)
    print(f"{p:>12.0%} {implied_f:>26.1f}")

precisions_list = [0.51, 0.52, 0.55, 0.60] target_sr = 1.0 print(f"Target Sharpe Ratio: {target_sr:.1f}, W/L: {win_loss:.1f}") print(f"\n{'Precision':>12} {'Implied Freq (trades/yr)':>26}") print(f"{'-'*12:>12} {'-'*26:>26}") for p in precisions_list: implied_f = pymlfinance.backtesting.implied_frequency(target_sr, p, win_loss) print(f"{p:>12.0%} {implied_f:>26.1f}")

Target Sharpe Ratio: 1.0, W/L: 1.0

   Precision   Implied Freq (trades/yr)
------------ --------------------------
         51%                     2499.0
         52%                      624.0
         55%                       99.0
         60%                       24.0

In [8]:

# Trade-off between precision and frequency
fig, ax = plt.subplots(figsize=(10, 6))
target_srs_plot = [0.5, 1.0, 1.5, 2.0]
prec_range = np.linspace(0.505, 0.65, 30)

for tsr in target_srs_plot:
    freqs = [pymlfinance.backtesting.implied_frequency(tsr, p, win_loss)
             for p in prec_range]
    ax.plot(prec_range, freqs, linewidth=2, label=f'Target SR = {tsr:.1f}')

ax.set_xlabel('Precision (Hit Rate)')
ax.set_ylabel('Required Trades / Year')
ax.set_title('Implied Trading Frequency')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_yscale('log')
ax.set_ylim(1, 100000)
ax.axhline(252, color='gray', linestyle=':', alpha=0.5)
ax.text(0.62, 280, 'Daily', color='gray', fontsize=12)
plt.tight_layout()
plt.show()

# Trade-off between precision and frequency fig, ax = plt.subplots(figsize=(10, 6)) target_srs_plot = [0.5, 1.0, 1.5, 2.0] prec_range = np.linspace(0.505, 0.65, 30) for tsr in target_srs_plot: freqs = [pymlfinance.backtesting.implied_frequency(tsr, p, win_loss) for p in prec_range] ax.plot(prec_range, freqs, linewidth=2, label=f'Target SR = {tsr:.1f}') ax.set_xlabel('Precision (Hit Rate)') ax.set_ylabel('Required Trades / Year') ax.set_title('Implied Trading Frequency') ax.legend() ax.grid(True, alpha=0.3) ax.set_yscale('log') ax.set_ylim(1, 100000) ax.axhline(252, color='gray', linestyle=':', alpha=0.5) ax.text(0.62, 280, 'Daily', color='gray', fontsize=12) plt.tight_layout() plt.show()

Strategy Failure Probability¶

Even if a strategy has been profitable in the past, there is a probability that its true precision is below the break-even threshold. Using a binomial test, we estimate the likelihood that the observed hit rate is a statistical fluke (AFML Snippet 15.5).

In [9]:

# Break-even precision for symmetric bets: 50%
break_even = 1.0 / (1.0 + win_loss)  # = 0.5 for W/L=1

# Strategy with 55% observed precision over different sample sizes
obs_precision = 0.55
sample_sizes = [50, 100, 200, 500, 1000, 2000]

print(f"Observed precision: {obs_precision:.0%}, Break-even: {break_even:.0%}")
print(f"\n{'N Trades':>10} {'Failure Prob':>15}")
print(f"{'-'*10:>10} {'-'*15:>15}")
for n in sample_sizes:
    fp = pymlfinance.backtesting.strategy_failure_probability(
        obs_precision, n, break_even
    )
    print(f"{n:>10d} {fp:>15.4%}")

# Break-even precision for symmetric bets: 50% break_even = 1.0 / (1.0 + win_loss) # = 0.5 for W/L=1 # Strategy with 55% observed precision over different sample sizes obs_precision = 0.55 sample_sizes = [50, 100, 200, 500, 1000, 2000] print(f"Observed precision: {obs_precision:.0%}, Break-even: {break_even:.0%}") print(f"\n{'N Trades':>10} {'Failure Prob':>15}") print(f"{'-'*10:>10} {'-'*15:>15}") for n in sample_sizes: fp = pymlfinance.backtesting.strategy_failure_probability( obs_precision, n, break_even ) print(f"{n:>10d} {fp:>15.4%}")

Observed precision: 55%, Break-even: 50%

  N Trades    Failure Prob
---------- ---------------
        50        23.9750%
       100        15.8655%
       200         7.8650%
       500         1.2674%
      1000         0.0783%
      2000         0.0004%

Visualisation: Failure Probability vs Sample Size¶

With more observations, we become more confident that the observed precision is real. The failure probability decays as the sample grows.

In [10]:

n_range = np.arange(20, 2001, 10)
obs_precs = [0.52, 0.55, 0.60]

fig, ax = plt.subplots(figsize=(12, 6))
for op in obs_precs:
    fp_values = [pymlfinance.backtesting.strategy_failure_probability(op, int(n), break_even)
                 for n in n_range]
    ax.plot(n_range, fp_values, linewidth=2, label=f'Precision = {op:.0%}')

ax.axhline(0.05, color='red', linestyle='--', linewidth=1.5, label='5% threshold')
ax.set_xlabel('Number of Trades')
ax.set_ylabel('Failure Probability')
ax.set_title('Strategy Failure Probability vs Sample Size')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(-0.02, 0.55)
plt.tight_layout()
plt.show()

n_range = np.arange(20, 2001, 10) obs_precs = [0.52, 0.55, 0.60] fig, ax = plt.subplots(figsize=(12, 6)) for op in obs_precs: fp_values = [pymlfinance.backtesting.strategy_failure_probability(op, int(n), break_even) for n in n_range] ax.plot(n_range, fp_values, linewidth=2, label=f'Precision = {op:.0%}') ax.axhline(0.05, color='red', linestyle='--', linewidth=1.5, label='5% threshold') ax.set_xlabel('Number of Trades') ax.set_ylabel('Failure Probability') ax.set_title('Strategy Failure Probability vs Sample Size') ax.legend() ax.grid(True, alpha=0.3) ax.set_ylim(-0.02, 0.55) plt.tight_layout() plt.show()

Combined Analysis: Strategy Viability Assessment¶

Putting it all together: given observed strategy characteristics, we assess viability from multiple angles.

In [11]:

# Simulate a realistic strategy
obs_prec = 0.54
obs_freq = 150.0   # ~150 trades/year
obs_wl = 1.2       # wins 20% larger than losses on average
n_obs = 300         # 2 years of track record

# Break-even precision
be_prec = 1.0 / (1.0 + obs_wl)

# Expected Sharpe
expected_sr = pymlfinance.backtesting.sr_from_precision(obs_prec, obs_freq, obs_wl)

# What precision would we need for SR=2?
prec_for_sr2 = pymlfinance.backtesting.implied_precision(2.0, obs_freq, obs_wl)

# What frequency would we need for SR=2?
freq_for_sr2 = pymlfinance.backtesting.implied_frequency(2.0, obs_prec, obs_wl)

# Failure probability
fail_prob = pymlfinance.backtesting.strategy_failure_probability(obs_prec, n_obs, be_prec)

print("Strategy Assessment")
print("=" * 50)
print(f"Observed precision:    {obs_prec:.0%}")
print(f"Trading frequency:     {obs_freq:.0f} trades/year")
print(f"Win/loss ratio:        {obs_wl:.1f}")
print(f"Track record:          {n_obs} trades")
print(f"\nBreak-even precision:  {be_prec:.2%}")
print(f"Expected Sharpe ratio: {expected_sr:.4f}")
print(f"\nTo achieve SR=2.0:")
print(f"  Need precision:      {prec_for_sr2:.2%}")
print(f"  Or need frequency:   {freq_for_sr2:.0f} trades/year")
print(f"\nFailure probability:   {fail_prob:.4%}")

# Simulate a realistic strategy obs_prec = 0.54 obs_freq = 150.0 # ~150 trades/year obs_wl = 1.2 # wins 20% larger than losses on average n_obs = 300 # 2 years of track record # Break-even precision be_prec = 1.0 / (1.0 + obs_wl) # Expected Sharpe expected_sr = pymlfinance.backtesting.sr_from_precision(obs_prec, obs_freq, obs_wl) # What precision would we need for SR=2? prec_for_sr2 = pymlfinance.backtesting.implied_precision(2.0, obs_freq, obs_wl) # What frequency would we need for SR=2? freq_for_sr2 = pymlfinance.backtesting.implied_frequency(2.0, obs_prec, obs_wl) # Failure probability fail_prob = pymlfinance.backtesting.strategy_failure_probability(obs_prec, n_obs, be_prec) print("Strategy Assessment") print("=" * 50) print(f"Observed precision: {obs_prec:.0%}") print(f"Trading frequency: {obs_freq:.0f} trades/year") print(f"Win/loss ratio: {obs_wl:.1f}") print(f"Track record: {n_obs} trades") print(f"\nBreak-even precision: {be_prec:.2%}") print(f"Expected Sharpe ratio: {expected_sr:.4f}") print(f"\nTo achieve SR=2.0:") print(f" Need precision: {prec_for_sr2:.2%}") print(f" Or need frequency: {freq_for_sr2:.0f} trades/year") print(f"\nFailure probability: {fail_prob:.4%}")

Strategy Assessment
==================================================
Observed precision:    54%
Trading frequency:     150 trades/year
Win/loss ratio:        1.2
Track record:          300 trades

Break-even precision:  45.45%
Expected Sharpe ratio: 2.0999

To achieve SR=2.0:
  Need precision:      53.60%
  Or need frequency:   136 trades/year

Failure probability:   0.1477%

Visualisation: Failure Probability Under Different Win/Loss Ratios¶

A higher win/loss ratio lowers the break-even precision, making failure less likely even at moderate hit rates.

In [12]:

wl_range = np.linspace(0.5, 3.0, 26)
prec_levels = [0.52, 0.55, 0.60]
n_trades = 500

fig, ax = plt.subplots(figsize=(12, 6))
for p in prec_levels:
    fail_probs = []
    for wl in wl_range:
        be = 1.0 / (1.0 + wl)
        fp = pymlfinance.backtesting.strategy_failure_probability(p, n_trades, be)
        fail_probs.append(fp)
    ax.plot(wl_range, fail_probs, linewidth=2, label=f'Precision = {p:.0%}')

ax.axhline(0.05, color='red', linestyle='--', linewidth=1.5, label='5% threshold')
ax.set_xlabel('Win/Loss Ratio')
ax.set_ylabel('Failure Probability')
ax.set_title(f'Failure Probability vs Win/Loss Ratio (n={n_trades})')
ax.legend()
ax.grid(True, alpha=0.3)
ax.set_ylim(-0.02, 0.55)
plt.tight_layout()
plt.show()

wl_range = np.linspace(0.5, 3.0, 26) prec_levels = [0.52, 0.55, 0.60] n_trades = 500 fig, ax = plt.subplots(figsize=(12, 6)) for p in prec_levels: fail_probs = [] for wl in wl_range: be = 1.0 / (1.0 + wl) fp = pymlfinance.backtesting.strategy_failure_probability(p, n_trades, be) fail_probs.append(fp) ax.plot(wl_range, fail_probs, linewidth=2, label=f'Precision = {p:.0%}') ax.axhline(0.05, color='red', linestyle='--', linewidth=1.5, label='5% threshold') ax.set_xlabel('Win/Loss Ratio') ax.set_ylabel('Failure Probability') ax.set_title(f'Failure Probability vs Win/Loss Ratio (n={n_trades})') ax.legend() ax.grid(True, alpha=0.3) ax.set_ylim(-0.02, 0.55) plt.tight_layout() plt.show()

Exercises¶

1. Find the minimum track record length needed for a 52% precision strategy to have <5% failure probability.
2. Compare the Sharpe surface for weekly (freq=52) vs daily (freq=252) strategies.
3. For a strategy with W/L=0.5, what precision is needed for SR=1?