Data mining or curve fitting a price series will always produce the best performance results. If you look hard enough using optimization you will always find patterns in the price data by mistaking chance fluctuations for genuine effects. But they are not real. Why use the PWFO walk forward technique? Why not just perform a TradeStation optimization on the whole price series and choose the input parameters that give the best total net profits or profit factor? Whenever you run a optimization (combinatorial search} over many different combinations of input parameters on noisy data on a fixed number of prices, no matter how many, the best performance parameters found are guaranteed to be due to "curve fitting" the noise and signal. When we run, say, 5000 different input parameter combinations , the best performance parameters will be from those system input variables that are able to produce profits from the price pattern and the random spurious price movements While the price patterns, if there, will repeat, the same spurious price movements will not. If the spurious movements that were captured by a certain set of input parameters were a large part of the total net profits, then choosing these input parameters will produce losses when traded on future data. These losses occur because the spurious movements will not be repeated in the same way. This is why system combinatorial searches with no out-of-sample testing cause loses when traded in real time from something that looked great in the test section.
In order to gain confidence that our system input parameter selection procedure on test data will produce profits "on average" in the future we must perform the walk forward out-of-sample analysis many times. Why not just do the analysis once? Well just as in poker, where there is considerable vagaries in hand to hand luck, walk forward out-of-sample analysis give considerable vagaries in week to week out-of-sample profit "luck". That is, by pure chance we may have chosen some input parameters that did well in the test section data and the out-of-sample section data. In order to minimize this type of "luck", statistically, we must repeat the walk forward out-of-sample (oos) analysis over many test/oos sections and take the average of our weekly results over all out-of-sample sections (we need at least 50 oos sections for statistically significant results). This average gives us an expected weekly return and a standard deviation of weekly returns which allows us to statistically estimate the expected equity and it's range for N weeks in the future.
Here is an example of just one of the many filters the WFME can generate in a run and the logic behind this particular filter. Very few strategies can sustain Profit Factors (PF) above 3 or 4 over time. High profit factors (PF) in the in-sample section usually mean poor performance in the out-of-sample-section. This is a kind of reversion to the mean. Therefore if we eliminate from the in-sample optimization results, all rows that have profit factors greater than 3, we will likely eliminate many of the cases that are due to curve fitting the noise. Also In addition, because in real time it is difficult to sustain more than seven losses in a row and still keep trading, we will eliminate all those in-sample rows that have more than 7 losses in a row(LR). After using a PF and LR filter as above there can still be 100's of rows left in the in-sample section of the PWFO file. The PWFO generates the metric eqR2 which is the Trade Equity Regression Trend Line Coefficient of Correlation in the test section. As an example, let us choose the rows that contained the Top 50 eqR2 values from the rows that are left from the PF-LR screen. That is the WFME sorts eqR2 from high to low, eliminates the rows that have PF>3 and LR>7 and then chooses the Top 50 Rows of whatever is left. This particular filter will now leave 50 cases or rows in the PWFO file that satisfy these filter conditions. We call this filter t50eqR2|p<3|lr7 where t50eqR2 means the Top 50 eqR2 Rows left after the PF-LR filter. p<3 means PF<=3, "lr7" means LR<=7. Suppose for this filter, within the 50 PWFO rows that are left, we want the row that has the maximum PWFO mTrd metric (the maximum median trade profit) in the in-sample section. This would produce a filter named t50eqR2|p<3|lr7-mTrd. This filter leaves only one row in the PWFO in-sample section with it's associated strategy inputs and out-of-sample net profit in the out-of-sample section. This particular t50eqR2|p<3|lr7-mTrd filter is then calculated for the in-sample section of each of the PWFO files that are run by the WFME and the average out-of-sample performance from this in-sample filter, plus many other important statistics ( as shown here) is summarized along with the hundreds of other filter combinations that are constructed in a similar manner. This filter summary is sorted by total out-of-sample net profits after costs for all the various filters and written to a comma delimited Excel file by the WFME. When The WFME run is done just click on the Excel Icon next to the Run button and the WFME output file will appear in Excel or your spreadsheet
Using modern "Bootstrap" techniques, we can calculate the probability of obtaining each filter's total out-of-sample net profits by chance. By net we mean subtracting the cost and slippage of all round trip trades from the total out-of-sample profits. Here is how the bootstrap technique is applied. Suppose we have 300 out-of-sample PWFO files with 8000 different strategy input combination rows. Note each of the 8000 different strategy inputs rows have an in-sample section and an out-of-sample section in each of the 300 PWFO files. A given filter on the in-sample section chooses a set of strategy inputs based upon the filter parameters and those strategy inputs chosen generate net profits or losses when applied to the out-of-sample section. Applying this filter to each of the 300 in-sample sections gives in-sample strategy inputs that generate net profits or losses in each of the 300 out-of-sample sections. The sum of all the net profits and losses in the 300 out-of-sample sections generate the total net profits for that particular filter. Each of the different filters total net profits are calculated in the same way. Let us create a mirror random filter. However with the random filter, the random filter picks a random row's out-of-sample net profits in each of the 300 PWFO files (there are 8000 different rows from the 8000 different strategy inputs and the random filter chooses one of those 8000 rows at random). Suppose we repeat this random row section 5000 times. Each of the 5000 mirror filters will choose a random row's out-of-sample net profit(OSNP) of their own in each of the 300 PWFO files. At the end, each of the 5000 mirror filters will have 300 random OSNP's picked from the rows of the 300 PWFO files. The sum of the 300 random OSNP picks for each random filter will generate a random total out-of-sample net profit(tOnpNet). The average and standard deviation of the 5000 mirror filter's different random tOnpNets will allow us to calculate the chance probability for each our filter's tOnpNet. Thus given the randon filter's bootstrap random tOnpNet average and standard deviation, we can calculate the probability of obtaining our filter's tOnpNet by pure chance alone.
After the WFME has completed it's run click in the Excel icon above Ave File on the WFME form and the Ave File automatically loads into Excel. Below is a snippet of the Ave File output from a WFME run sorted by the total out-of-sample net profit statistic (tOnpNet). This example shows the partial output file from a WFMEx64v7 run on the PWFOv5t files generated with the End Point Fast Fourier Transform Strategy that was run on 1 contract of the Crude Light(CL) 5 minute bar futures for the 378 days from 8/11/2014 to 02/12/2016. The in-sample(IS) period is 5 trading weekdays and the out-of-sample(OOS) period is the next trading weekday following. The days are weekdays only. Weekdays where the OOS falls on a exchange holiday or partial days are eliminated. Holidays that fall on a weekday create a 4day IS. All other IS periods consist of 5 trading weekdays. Sundays were skipped because this strategy only traded between 8am to 14:30pm CST
The WFME Columns are defined as follows
Shown below again is a snippet of the WFME Ave File discussed above. To generate an Equity plot of any given Filter we would select the Filter from Column A as shown in the snippet and then click on the Excel Add-Ins and then click on the black circled Meyers Analytics Macro Icon shown in the snippet. The Excel Equity plot and Table would then immediately appear in Excel as shown below. Note the equity plot shows the equity with costs (brown line, new equity highs are shown as green dots) and without costs(blue line, new equity highs are shown as red dots).Any number of Filters could be selected, one at a time by highlighting the column A filter and clicking on the Add-In macro shown , and an Equity chart plus Table would be produced. This Meyers Analytics Excel Equity Chart Add-In macro is supplied with the Walk Forward Metric Explorer v7 and all the other explorers.
The Excel Table Columns are defined as follows
Please see video for a visual explanation.
Click Here For Video: How To Use The Walk Forward PWFO Metric Explorer v3>
The WFME Input Finder Excel Add-In
Supplied with the WFME is an Excel Add-In Filter. For the Excel Add-In, as shown below, you just click on a special Add-In icon on the Excel Toolbar and a pop up window displays. Fill in the WFME parameters and click on the Run button. The Excel Add-In macro will filter any number of PWFO files loaded in Excel, and will display the PWFO Excel file row that satisfies the typed in criteria.
The The Walk Forward PWFO Metric Explorer package consisting of Manual, WFME EXE file, and WFME Excel Add-Ins is being offered, for $395. Please note that the WFME will only read files generated by the PWFO product. The WFME has a "Key Licence" that only allows it to be installed on three computers.