Meyers

Analytics

Analytics

Advanced Mathematical Trading Strategies & Walk Forward Out-Of-Sample Analysis

applied to algorithmic trading of stocks, futures & forex

Info: (312) 280-1687 support@meyersanalytics.com

applied to algorithmic trading of stocks, futures & forex

Info: (312) 280-1687 support@meyersanalytics.com

Order Online
Power Walk Forward

Optimizer Walk Forward

Metric Explorer Walk Forward

Input Explorer Walk Forward

Surface Explorer Key Daily & Intraday

Trading Strategies Nth Order Fixed Memory

Polynomial Strategy Nth Order Fading Memory

Polynomial Strategy End Point Fast Fourier

Transform Strategy Goertzel DFT

Strategy Five Parameter

Parabolic Strategy Dennis Meyers

Working Papers

Optimizer Walk Forward

Metric Explorer Walk Forward

Input Explorer Walk Forward

Surface Explorer Key Daily & Intraday

Trading Strategies Nth Order Fixed Memory

Polynomial Strategy Nth Order Fading Memory

Polynomial Strategy End Point Fast Fourier

Transform Strategy Goertzel DFT

Strategy Five Parameter

Parabolic Strategy Dennis Meyers

Working Papers

Click here to see the WFME Input Form and a description of the inputs

Data mining or curve fitting a price series will always produce the best performance results. If you look hard enough using optimization you will always find patterns in the price data by mistaking chance fluctuations for genuine effects. But they are not real. Why use the PWFO walk forward technique? Why not just perform a TradeStation optimization on the whole price series and choose the input parameters that give the best total net profits or profit factor? Whenever you run a optimization (combinatorial search} over many different combinations of input parameters on noisy data on a fixed number of prices, no matter how many, the best performance parameters found are guaranteed to be due to "curve fitting" the noise and signal. When we run, say, 5000 different input parameter combinations , the best performance parameters will be from those system input variables that are able to produce profits from the price pattern and the random spurious price movements While the price patterns, if there, will repeat, the same spurious price movements will not. If the spurious movements that were captured by a certain set of input parameters were a large part of the total net profits, then choosing these input parameters will produce losses when traded on future data. These losses occur because the spurious movements will not be repeated in the same way. This is why system combinatorial searches with no out-of-sample testing cause loses when traded in real time from something that looked great in the test section.

In order to gain confidence that our system input parameter selection procedure on test data will produce profits "on average" in the future we must perform the walk forward out-of-sample analysis many times. Why not just do the analysis once? Well just as in poker, where there is considerable vagaries in hand to hand luck, walk forward out-of-sample analysis give considerable vagaries in week to week out-of-sample profit "luck". That is, by pure chance we may have chosen some input parameters that did well in the test section data **and** the out-of-sample section data. In order to minimize this type of "luck", statistically, we must repeat the walk forward out-of-sample (oos) analysis over many test/oos sections and take the average of our weekly results over all out-of-sample sections (we need at least 50 oos sections for statistically significant results). This average gives us an expected weekly return and a standard deviation of weekly returns which allows us to statistically estimate the expected equity and it's range for N weeks in the future.

Here is an example of just one of the many filters the WFME can generate in a run and the logic behind this particular filter. Very few strategies can sustain Profit Factors (PF) above 3 or 4 over time. High profit factors (PF) in the in-sample section usually mean poor performance in the out-of-sample-section. This is a kind of reversion to the mean. Therefore if we eliminate from the in-sample optimization results, all rows that have profit factors greater than 3, we will likely eliminate many of the cases that are due to curve fitting the noise. Also In addition, because in real time it is difficult to sustain more than seven losses in a row and still keep trading, we will eliminate all those in-sample rows that have more than 7 losses in a row(LR). After using a PF and LR filter as above there can still be 100's of rows left in the in-sample section of the PWFO file. The PWFO generates the metric **eqR2** which is the Trade Equity Regression Trend Line Coefficient of Correlation in the test section. As an example, let us choose the rows that contained the **Top 50 eqR2** values from the rows that are left from the **PF-LR** screen. That is the WFME sorts eqR2 from high to low, eliminates the rows that have PF>3 and LR>7 and then chooses the Top 50 Rows of whatever is left. This particular filter will now leave 50 cases or rows in the PWFO file that satisfy these filter conditions. We call this filter **t50eqR2|p<3|lr7** where t50eqR2 means the Top 50 eqR2 Rows left ** after** the PF-LR filter.

Using modern "Bootstrap" techniques, we can calculate the probability of obtaining each filter's total out-of-sample net profits by chance. By net we mean subtracting the cost and slippage of all round trip trades from the total out-of-sample profits. Here is how the bootstrap technique is applied. Suppose we have 300 out-of-sample PWFO files with 8000 different strategy input combination rows. Note each of the 8000 different strategy inputs rows have an in-sample section and an out-of-sample section in each of the 300 PWFO files. A given filter on the in-sample section chooses a set of strategy inputs based upon the filter parameters and those strategy inputs chosen generate net profits or losses when applied to the out-of-sample section. Applying this filter to each of the 300 in-sample sections gives in-sample strategy inputs that generate net profits or losses in each of the 300 out-of-sample sections. The sum of all the net profits and losses in the 300 out-of-sample sections generate the total net profits for that particular filter. Each of the different filters total net profits are calculated in the same way. Let us create a mirror random filter. However with the random filter, the random filter picks a random row's out-of-sample net profits in each of the 300 PWFO files (there are 8000 different rows from the 8000 different strategy inputs and the random filter chooses one of those 8000 rows at random). Suppose we repeat this random row section 5000 times. Each of the 5000 mirror filters will choose a random row's out-of-sample net profit(OSNP) of their own in each of the 300 PWFO files. At the end, each of the 5000 mirror filters will have 300 random OSNP's picked from the rows of the 300 PWFO files. The sum of the 300 random OSNP picks for each random filter will generate a random total out-of-sample net profit(tOnpNet). The average and standard deviation of the 5000 mirror filter's different random tOnpNets will allow us to calculate the chance probability for each our filter's tOnpNet. Thus given the randon filter's bootstrap random tOnpNet average and standard deviation, we can calculate the probability of obtaining our filter's tOnpNet by pure chance alone.

After the WFME has completed it's run click in the Excel icon above ** Ave File** on the WFME form and the

The WFME Columns are defined as follows

**Row 1, Columns: A**=The PWFO Stub,**B**=File Start Date,**C**=File End Date,**D**= Number of oos periods(in this example weeks),**M**= Bootstrap average,**N**= Bootstrap Standard Deviation,**O**=Number of filters run,**U**= Cost/trade.**Row 2, Columns:****Filter-Metric**= The filter that was run on the 378 PWFO in-sample sections. For example on Row 4, t50mWTr|p<1.5|l3-tWb is: top 50 median of winning trades in the in-sample sections after eliminating the in-sample rows that don't meet the following criteria: Profit Factor <=1.5. and consecutive losers in a row <=3. Out of the 50 rows left the WFME chooses the in-sample row with the highest tWb(total winning bars) metric.**toGP**= Total out-of-sample(oos) gross profit for these 378 oos periods(for this run periods = weeks).**aoGP**= Average oss gross profit for the 378 oos periods**aoTr**= Average oos trade gross profit**aoW**= Average of the out-of-sample profits of winning trades**aoL**= Average of the out-of-sample profits of losing trades**oW|oL**= Ratio of average oos winning trades(aoW) divided by average oos losing trades(aoL) .**PFtr**= Ratio of total winning trades divided by total losing trades**%Wtr**= The percentage if oos winning trades**t**= The student t statistic for the 378 oos periods. The higher the t statistic the higher the probability that this result was not due to pure chance**std**= The standard deviation of the 378 oos periods**LLtr**= The largest losing oos trade in all oos periods**LLp**= The largest losing oos period**eqDD**= The oos equity drawdown**LRp**= The largest number of losing oos periods in a row**#**= The number of oos periods this filter produced any profit or loss. Note for some oos periods there can be no strategy inputs that satisfy a given filters criteria and no trades will be made during that period.**eqTrn**= The straight line trend of the oos equity curve in $/oos period.**eqV^2**= The velocity of a 2nd order polynomial that is fit to the equity curve.**eqR2**= The correlation coefficient(R^2) of a straight line fit to the equity curve.**Dev^2**= A measure of equity curve smoothness. The square root of the average (equity curve minus a straight line)^2)**Blw**= The maximum number of oos periods the oos equity curve failed to make a new high.**BE**= Break even in oos periods. Assuming the average and standard deviation are from a normal distribution, this is the number of oos periods you would have to trade to have a 98% probability that your oos equity is above zero.**tOnpNet**= Total out-of-sample gross profit(toGP) minus the total trade cost. tOnpNet=toGP**-**(#)*ao#T*Cost.**Prob**= The probability that the filters tOnpNet was due to pure chance. For example, for row 4 where tOnpNet=69,180 there is a 1 in 2079 (4.81E-04) chance that the tOnpNet obtained by this filter was due to chance. Note for our filter, in row 4, the expected number of cases that we could obtain by pure chance that would match or exceed the $69,180 is 14415x 4.81x10-4 = 6.9 where 14415 is the total number of different filters we looked at in this run. This number basically says this if we repeated the random selection 14415 times we would find approximately 7 cases on average that would produce better profits than our filter. I usually like to see this number much less than 1. However the 14415 bootstrap random average was $21,351 which meant that 50% of the random picks were at or above $21,351. This indicates that this strategy is in general a profitable strategy no matter what the inputs. Thus in my opinion the 7 random cases that could beat our filter does not negate the validity of the out-of-sample filter chosen.

Shown below again is a snippet of the WFME Ave File discussed above. To generate an Equity plot of any given **Filter** we would select the **Filter** from Column A as shown in the snippet and then click on the Excel Add-Ins and then click on the black circled Meyers Analytics Macro Icon shown in the snippet. The Excel Equity plot and Table would then immediately appear in Excel as shown below. Note the equity plot shows the equity with costs **(brown line, new equity highs are shown as green dots)** and without costs**(blue line, new equity highs are shown as red dots)**.Any number of **Filters** could be selected, one at a time by highlighting the column A filter and clicking on the Add-In macro shown , and an Equity chart plus Table would be produced. This Meyers Analytics Excel Equity Chart Add-In macro is supplied with the Walk Forward Metric Explorer v7 and all the other explorers.

The Excel Table Columns are defined as follows

**Column A**The Filter-Metric. Example t50mWTr|p<1.5|l3-tWb**Column B**The PWFO File dates**Column C**= The out-of-sample(oos) gross profit for each date using the in-sample strategy inputs.**Column D**= The out-of-sample(oos) number of trades for each date using the in-sample strategy inputs.**Column E**= The out-of-sample(oos) Equity running total.**Column C**= The out-of-sample(oos) net profit (Gross Profit-OnT*Cost) for each date using the in-sample strategy inputs.**Column G**= The out-of-sample(oos) Net Equity (equity minus costs) running total.**Columns H-L**= Strategy Inputs found by the Filter in the in-sample section that gave the out-of-sample results on this same line.

**Please see video for a visual explanation. **
**Click Here For Video: How To Use The Walk Forward PWFO Metric Explorer v3>**

- How to setup, install and run the Walk Forward Performance Metric Explorer. The Walk Forward PWFO Metric Explorer is a stand alone exe file that can be executed directly from your desktop icon or from the Windows Start Program menu.
- How to use the WFME with your PWFO files.
- An explanation of each of the performance statistics columns.
- How to select the best WFME filter to use on real time runs (Past performance is no guarantee of future results).

The WFME Input Finder Excel Add-In

Supplied with the WFME is an Excel Add-In Filter. For the Excel Add-In, as shown below, you just click on a special Add-In icon on the Excel Toolbar and a pop up window displays. Fill in the WFME parameters and click on the **Run** button. The Excel Add-In macro will filter any number of PWFO files loaded in Excel, and will display the PWFO Excel file row that satisfies the typed in criteria.

The ** The Walk Forward PWFO Metric Explorer** package consisting of Manual, WFME EXE file, and WFME Excel Add-Ins is being offered, for

How To Order

To order online

Thank you for your Interest....Dennis Meyers