Dutch Algotrading
patreon
NASOS v4 - Results I cannot explain...
🕑 Added 2024-09-20 14:00:01 +0000 UTCThe first strategy of which I cannot explain its trading results.
Introduction
Hello my dear Patrons and welcome to another analysis of a trading algorithm in this Patreon exclusive post.
is time I tested multiple strategies that are called NASOS. There can be many versions found on Github and during my initial backtests the version 4 is the number performed the best on my backtesting setup.
However there is something strange happening with this algoritm that I simply cannot explain for the first time I do all these tests.
Let’s first show you the source of this file and how it’s build…
Where to find it
As many other trading algorithms, this strategy is available on GitHub. If you search for the NASOSv4 file, then you will probably find lots of repositories that contain this file.
https://github.com/search?q=nasosv4&type=code
Let’s open the file and admire its contents…
The strategy
At the top of the file we can already find the first warning. You should set the trailing stop and custom stop loss parameters to true when live trading. And set the custom stop loss function to false when backtesting.
I will backtest this original file as it is here initially and then do a second test with the TSL disabled.
In this file the custom stop loss is already set to false here.
A coder called Rallipanos made this algorithm and someone called pluxury also dit something with the code. It’s not my code, im just testing it to see if it is worthwhile for further investigation and maybe live trading. The trading idea and workings come from these great minds and I am just reverse engineering what they had in mind with this algorithm.
So let’s continue…
The first section shows us the hyperspace parameters, so we already know we can set multiple indicators to optimize this strategy. However, I will not do this since my last experiences are that further optimization only worsens the performance.
Then we have the well-know Elliot oscillator, that seems to be present and used in most of the well performing strategies.
And after that the ROI table and Stop loss. Which has currently been set to sell at 15 percent loss if a trade goes bad.
Then we have some more hyperparameters to optimize for the indicators that are used in this strategy
And then there are the trailing stop loss settings and the time frame that indicates that this algo has been built for the 5 minute timeframe.
After this we discover some interesting methods.
The first one is the custom stop loss, that has been disabled by default. But Lets run over it anyway…
This custom stop loss method in the strategy dynamically adjusts the stop loss level based on the current profit of the trade. If the profit exceeds a second, higher threshold, the method uses a fixed stop loss plus any additional profit beyond this threshold.
Between two profit thresholds, it linearly interpolates the stop loss value. And below the lower threshold, it enforces a hard stop loss level. So this method aims to protect gains while allowing some room for price fluctuation, especially when profits are higher.
This confirm_trade_exit function in the strategy evaluates additional conditions to confirm or prevent a trade’s exit based on the last candle's data. It checks for specific price conditions against moving averages, like the hma and ema and tries to manages slippage by limiting the number of retries if the exit price is worse than expected.
Next we have the informative_1h_indicators function . This function is designed to fetch and potentially calculate additional indicators from a higher, 1-hour timeframe to inform decisions made on the primary trading timeframe. But as far as I can tell it seems to be a placeholder code, because almost all these calculations are commented out.
Now this normal_tf_indicators function computes various technical indicators for the primary trading timeframe. It calculates exponential moving averages (EMAs) for different numbers of candles specified in the buy and sell parameters, a Hull moving average, and multiple RSI values to track short and long-term momentum and trend directions within the trading strategy.
And the populate_indicators method orchestrates the incorporation of technical indicators into the trading strategy by merging the higher timeframe (1-hour) indicators from informative_1h_indicators with the primary timeframe indicators computed in normal_tf_indicators.
So that I think the purpose here is that it first gathers any relevant higher timeframe data to enrich the strategy's context. And after that it applies the primary timeframe calculations to provide a comprehensive set of indicators. And this all is used to drive buy and sell decisions within the strategy.
The populate_buy_trend method in the strategy defines the conditions under which a buy signal is generated.
You can see that it evaluates multiple technical indicators like RSI and EWO values, alongside specific price conditions relative to moving averages, to determine suitable entry points.
There are different conditions defined for different trading scenarios, and each of these scenarios is tagged distinctly (e.g., 'ewo1', 'ewo2', 'ewolow'). If you take a look at the output of the backtest, then you will see these buy reasons back. So you also have information which buy conditions occur most or have the highest profits.
Additionally, there's a dynamic check to avoid buying if the potential profit, as defined by a threshold relative to recent price highs, is not met. That method ensures that all specified conditions are met before signaling a buy, with additional provisions to cancel signals under certain situations through a series of "don't buy" conditions.
Now finally there is the populate_sell_trend method .
This method outlines the conditions under which a sell signal is triggered in the strategy. It uses a combination of price conditions relative to specific moving averages, RSI values, and volume to identify potential exit points.
Also in this case, conditions are grouped to capture scenarios where the price exceeds certain thresholds or when fast and slow RSI readings indicate a change in momentum, that might suggest a sell. Only when one or more predefined scenarios are satisfied the open position in question will be closed.
Lookahead bias
Now before I continue I also checked this algorithm on possible lookahead bias. But the built in function could not detect any as you can see.
Also I will show you later the difference in performance with and without the trailing stoploss set. So you can also take this bias into consideration when you decide to use this algo.
Backtest results TSL version
At the end of last year I discovered, what I thought, was the best trading algorithm. And I could not imagine that I would find even better performing trading strategies for my trading bot.
But as with many things in life, you can get pleasantly surprised.
And with the results of this trading strategy, the limits are again stretched beyond my expectations.
This trading strategy shows extremely good hypothetical trading results.
Just like the MacheteV8b, this algo could trade over 1.1 million hypothetical dollars together.
It has an over 90 percent win rate and max drawdown of 11 percent.
87 percent of the pairs I use seem to get positive results with this algorithm.
Only a Calmar ratio of 84 looks to be suspicious to me. But let’s not let that spoil the fun.
Let’s see how the equity curve looks like…
The over 1.1 million USDT has been reached by means of this journey of the equity curve.
What’s so special is that after the previous bull market top, there is almost no decline. The curve and the wins keep on going.
The drawdown profile over the plotted timeperiod looks like this:
Some peaks around 6 percent drawdown and an average drawdown of a very admirable 2 percent. Note that the 11 percent drawdown in the table has occurred before the plots timeperiod.
Also some weeks show very good profits. Like the week where almost 150.000 dollar has been traded. Now ofcourse there are also negative weeks, but these are manageable, although losses of over 25000 dollar would be a lot of pain. No matter how what happened before.
Furthermore, the win rate distribution and profit distribution box plots are as follows.
The box plot on the left shows a high median winrate, just below 1.0, suggesting that the strategy frequently wins a majority of its trades.
The interquartile range (IQR) is very compact, spanning from around 0.9 to just below 1.0. And this indicates a consistent performance in terms of winrate.
Now, there are several outliers below the box. And these indicate weeks where the winrate significantly dropped below the typical range. That is also what we saw in the previous plot.
The profit distribution boxplot shows a wider range of values compared to the winrate distribution. And these profits vary significantly from week to week.
The IQR is centered around a median close to zero, suggesting that in many weeks the strategy neither makes nor loses a significant amount of money.
The outliers in this plot indicate that there are weeks where the profit is exceptionally high, as much as around 150,000, as well as weeks where the strategy incurs losses, around -25,000.
All of this ends up in an indicator overview that looks like this:
The colum is the highest or almost the highest where it should be high. Like win percentage sharpe, calmar and cagr.
And the lowest or one of the lowest where it should be low. Like the drawdown.
This all results in a profit comparison plot between the top 10 best performing strategies that looks like this:
Where the NASOS performs the best of all. Even better than the MacheteV8b results.
And you already guessed it ofcourse. With these results, this trading algorithm ends up at the top spot of the Strategy League.
A worthy position of this trading algorithm with its spectacular results.
Backtest results NoTSL version
But here comes the strange and unexpected part.
Sometimes life can also throw you a curveball and you cannot explain why this happens...
:-\
As you might already know, I’m not a great fan of the Trailing stop loss function in backtesting. Since it can add a positive bias to the trading results. Using TSL can make a strategy more favorable because of it’s biased positive results.
It could very well be that these positive results will not occur in real trading.
So I also tested this trading strategy without the TSL enabled and it got the following SHOCKING results back…
With a devastating equity curve.
To verify this difference I did a second test on another dataset and other pairs.
On two machines with each their own setup.
Machine 1
Machine 2
Now what on earth is going on here????
Again the backtest results between the TSL and NoTSL versions are these:
How can the disablement of a trailing stop loss setting give such a large difference in this particular trading algorithm?
Also, look at the amount of trades here…
8383 trades with TSL enabled and 198 with TSL disabled.
How come?
Well one thing that shows in the other individual tests is that with the TSL enabled, the average duration of winners is 3 hours and losers have duration of around 17:25 hours.
But with TSL disabled these numbers suddenly change significantly.
214 days duration for winners and 45 days for losers. Why this sudden difference here??
If you see this colordiff here, then only these lines are different between the files:
The secret may be in the first two lines of this code:
Here Backtesting should have trailing stop = True and custom stop loss = False
But in live trading it should be the reverse…
Unfortunately the Freqtrade documentation (https://www.freqtrade.io/en/latest/stoploss/#trailing-stop-loss) does not warn about the usage of trailing stops in backtesting.
Also there’s nothing that the custom stop loss says (https://www.freqtrade.io/en/2021.2/strategy-advanced/#trailing-stoploss-with-positive-offset) about the difference in results between backtesting and live (forward) testing.
There is also no real explanation found in the ‘Assumptions’ documentation section(https://www.freqtrade.io/en/2021.2/strategy-advanced/#trailing-stoploss-with-positive-offset).
Except that:
Trailing stoploss
High happens first - adjusting stoploss
Low uses the adjusted stoploss (so sells with large high-low difference are backtested correctly)
ROI applies before trailing-stop, ensuring profits are "top-capped" at ROI if both ROI and trailing stop applies
But this still does not explain to me why there are such big differences here. So this remains a total mystery to me. Maybe somebody else can explain what’s going on here..?
For me now the only way to check to see if this a valid strategy or not, is to add it to my forward testing trading bot to see what the actual performances will be under ‘live’ conditions.
For now I will leave the results in my strategy league for what they are and if forward testing shows that this strategy is really bad, then I’ll remove the results. So not to misinform other people in the future.
Ending
So with this exceptinal conclusion I am at the end of this post.
Again, maybe somebody can explain this large difference if he/she tests this on their own setup. And if you did, then please add your findings to the comments section of this post.
This is it for me now, and I will see you in the next post.
Goodbye!
Comments
Dutch Algotrading
I understand your response. Personally I mostly test out a strategy and if I cannot understand it or is very complex - and in this case cannot understand why it does what it does - I keep things in the back of my mind an will stop further investigation. There are so many other algorithms to discover and limited time. So i'd rather spend time testing out these new ones instead of breaking my head on something I probably would not use anyway. But that does not stop me from presenting my results to others so that they have the benefits of my discoveries.
Ruben Cancho
With two static pairs (BTC and ETH) NASOSv4 only makes 3 trades during 2023, too conservative. Also difference between custom trailing stop and freqtrade trailing stop has some differences in backtesting (slightly better freqtrade trailing stop): Backtesting with custom trailing stop: Backtested 2023-01-01 16:40:00 -> 2024-01-01 00:00:00 | Max open trades : 2 ┃ Strategy ┃ Trades ┃ Avg Profit % ┃ Tot Profit USDT ┃ Tot Profit % ┃ Avg Duration ┃ Win Draw Loss Win% ┃ Drawdown ┃ │ NASOSv4 │ 3 │ 0.47 │ 4.594 │ 0.46 │ 1:03:00 │ 1 0 2 33.3 │ 2.257 USDT 0.22% │ Backtesting with freqtrade trailing stop: Backtested 2023-01-01 16:40:00 -> 2024-01-01 00:00:00 | Max open trades : 2 ┃ Strategy ┃ Trades ┃ Avg Profit % ┃ Tot Profit USDT ┃ Tot Profit % ┃ Avg Duration ┃ Win Draw Loss Win% ┃ Drawdown ┃ │ NASOSv4 │ 3 │ 1.59 │ 15.820 │ 1.58 │ 1:03:00 │ 3 0 0 100 │ 0 USDT 0.00% │ I would go with same setup in backtest and real, I don't understand why should the code should be different. But I need to understand why in my setup trades are so low...
Mihai Dobrescu
I think JasonFlash has found the problem in your test, and I can confirm. You don't follow correctly the instructions for the 2 variants: a) TSL enabled & custom_SL disabled b) TSL disabled & custom_SL enabled You have both disabled in your NoTSL variant. So you wait for a roi of 1000% (that will never come in most cases) to close the trade. The correct version's a) or b) will close a trade after a small +1.6% profit.
Mihai Dobrescu
Yes, same experience, yesterday first 2 trades on SUI. Stoploss setup makes no difference in entry signal.
Mihai Dobrescu
So no +200 Days for a trade, this is nonsense
Mihai Dobrescu
P.S. ... I'm running live the version with TSL disabled since 2 weeks, and today I had first 2 trades: Trade ID: 1 Current Pair: SUI/USDC Direction: Long (1.0x) Amount: 138.5 (207.972 USDC) Enter Tag: ewo1 Exit Reason: trailing_stop_loss Open Rate: 1.5016 Close Rate: 1.52449006 Open Date: 2024-09-20 13:15:03 Close Date: 2024-09-20 14:28:51 Close Profit: 1.52% (3.17 USDC) Trade ID: 2 Current Pair: SUI/USDC Direction: Long (1.0x) Amount: 142.44 (211.139 USDC) Enter Tag: ewo1 Exit Reason: trailing_stop_loss Open Rate: 1.4823 Close Rate: 1.5123928 Open Date: 2024-09-20 14:40:02 Close Date: 2024-09-20 16:01:21 Close Profit: 2.03% (4.286 USDC)
Mihai Dobrescu
Idea is that both versions use trailing SL. One version via trailing_stop = True, and the other via custom_stoploss that have identical trailing effect. And very similar values.
Mihai Dobrescu
trailing_stop_positive = 0.001 trailing_stop_positive_offset = 0.016 This trailing_stop_positive value is way too small, it will be triggered instantly. So this is like having roi at +1.6%. This has no trailing effect with this value of 0.001. Did you see all your winning trades with +1.6% ? We don't have a big difference between the trailing logic and the custom_stoploss logic. Both start at +1.6% and have extrem tight trailing. Trailing-enabled version is at -0.1% and custom_stoploss is starting with -0.2% trailing but going also to -0.1%. For crypto this is almost every 1 min candle. A big problem is in my opinion, that you test with this ancient FQ versions, full of bug's. I have translated NASOSv4 for the actual version with migrated functions (from V2 to V3). If we test with the actual version and get this nonsense between the 2 variants (that are very similar, and results should be close) we can open an issue on github.
Paris Urquhart
I thought you had forward tested this strategy with positive results. Is this not the case?
JasonFlash
I have been running this strategy in dry run for the past weeks and it did not make a single trade... I have the use_custom_stoploss on and trailing stop off. Also I noticed the NoTSL file you uploaded has Use_custom_stoploss False.
David Woodlock
Timely as I was looking at this strategy myself! I don't know why there is such a difference but will look into it more. Note ChatGPT 4o and O1-preview advise that the custom stoploss here is more difficult to Backtest accurately. "Difficult to Backtest: The dynamic nature makes it harder to simulate accurately in backtesting environments."
