+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ It can be helpful to understand how a strategy behaves according to the buy/entry tags used to +mark up different buy conditions. You might want to see more complex statistics about each buy and +sell condition above those provided by the default backtesting output. You may also want to +determine indicator values on the signal candle that resulted in a trade opening.
+Note
+The following buy reason analysis is only available for backtesting, not hyperopt.
+We need to run backtesting with the --export option set to signals to enable the exporting of
+signals and trades:
freqtrade backtesting -c <config.json> --timeframe <tf> --strategy <strategy_name> --timerange=<timerange> --export=signals
+This will tell freqtrade to output a pickled dictionary of strategy, pairs and corresponding
+DataFrame of the candles that resulted in buy signals. Depending on how many buys your strategy
+makes, this file may get quite large, so periodically check your user_data/backtest_results
+folder to delete old exports.
Before running your next backtest, make sure you either delete your old backtest results or run
+backtesting with the --cache none option to make sure no cached results are used.
If all goes well, you should now see a backtest-result-{timestamp}_signals.pkl file in the
+user_data/backtest_results folder.
To analyze the entry/exit tags, we now need to use the freqtrade backtesting-analysis command
+with --analysis-groups option provided with space-separated arguments:
freqtrade backtesting-analysis -c <config.json> --analysis-groups 0 1 2 3 4 5
+This command will read from the last backtesting results. The --analysis-groups option is
+used to specify the various tabular outputs showing the profit of each group or trade,
+ranging from the simplest (0) to the most detailed per pair, per buy and per sell tag (4):
More options are available by running with the -h option.
Normally, backtesting-analysis uses the latest backtest results, but if you wanted to go
+back to a previous backtest output, you need to supply the --export-filename option.
+You can supply the same parameter to backtest-analysis with the name of the final backtest
+output file. This allows you to keep historical versions of backtest results and re-analyse
+them at a later date:
freqtrade backtesting -c <config.json> --timeframe <tf> --strategy <strategy_name> --timerange=<timerange> --export=signals --export-filename=/tmp/mystrat_backtest.json
+You should see some output similar to below in the logs with the name of the timestamped +filename that was exported:
+2022-06-14 16:28:32,698 - freqtrade.misc - INFO - dumping json to "/tmp/mystrat_backtest-2022-06-14_16-28-32.json"
+You can then use that filename in backtesting-analysis:
freqtrade backtesting-analysis -c <config.json> --export-filename=/tmp/mystrat_backtest-2022-06-14_16-28-32.json
+To show only certain buy and sell tags in the displayed output, use the following two options:
+--enter-reason-list : Space-separated list of enter signals to analyse. Default: "all"
+--exit-reason-list : Space-separated list of exit signals to analyse. Default: "all"
+For example:
+freqtrade backtesting-analysis -c <config.json> --analysis-groups 0 2 --enter-reason-list enter_tag_a enter_tag_b --exit-reason-list roi custom_exit_tag_a stop_loss
+The real power of freqtrade backtesting-analysis comes from the ability to print out the indicator
+values present on signal candles to allow fine-grained investigation and tuning of buy signal
+indicators. To print out a column for a given set of indicators, use the --indicator-list
+option:
freqtrade backtesting-analysis -c <config.json> --analysis-groups 0 2 --enter-reason-list enter_tag_a enter_tag_b --exit-reason-list roi custom_exit_tag_a stop_loss --indicator-list rsi rsi_1h bb_lowerband ema_9 macd macdsignal
+The indicators have to be present in your strategy's main DataFrame (either for your main +timeframe or for informative timeframes) otherwise they will simply be ignored in the script +output.
+There are a range of candle and trade-related fields that are included in the analysis so are +automatically accessible by including them on the indicator-list, and these include:
+To show only trades between dates within your backtested timerange, supply the usual timerange option in YYYYMMDD-[YYYYMMDD] format:
--timerange : Timerange to filter output trades, start date inclusive, end date exclusive. e.g. 20220101-20221231
+For example, if your backtest timerange was 20220101-20221231 but you only want to output trades in January:
freqtrade backtesting-analysis -c <config.json> --timerange 20220101-20220201
+Use the --rejected-signals option to print out rejected signals.
freqtrade backtesting-analysis -c <config.json> --rejected-signals
+Some of the tabular outputs can become large, so printing them out to the terminal is not preferable.
+Use the --analysis-to-csv option to disable printing out of tables to standard out and write them to CSV files.
freqtrade backtesting-analysis -c <config.json> --analysis-to-csv
+By default this will write one file per output table you specified in the backtesting-analysis command, e.g.
freqtrade backtesting-analysis -c <config.json> --analysis-to-csv --rejected-signals --analysis-groups 0 1
+This will write to user_data/backtest_results:
To override where the files will be written, also specify the --analysis-csv-path option.
freqtrade backtesting-analysis -c <config.json> --analysis-to-csv --analysis-csv-path another/data/path/
+This page explains some advanced Hyperopt topics that may require higher +coding skills and Python knowledge than creation of an ordinal hyperoptimization +class.
+To use a custom loss function class, make sure that the function hyperopt_loss_function is defined in your custom hyperopt loss class.
+For the sample below, you then need to add the command line parameter --hyperopt-loss SuperDuperHyperOptLoss to your hyperopt call so this function is being used.
A sample of this can be found below, which is identical to the Default Hyperopt loss implementation. A full sample can be found in userdata/hyperopts.
+from datetime import datetime
+from typing import Any, Dict
+
+from pandas import DataFrame
+
+from freqtrade.constants import Config
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+TARGET_TRADES = 600
+EXPECTED_MAX_PROFIT = 3.0
+MAX_ACCEPTED_TRADE_DURATION = 300
+
+class SuperDuperHyperOptLoss(IHyperOptLoss):
+ """
+ Defines the default loss function for hyperopt
+ """
+
+ @staticmethod
+ def hyperopt_loss_function(results: DataFrame, trade_count: int,
+ min_date: datetime, max_date: datetime,
+ config: Config, processed: Dict[str, DataFrame],
+ backtest_stats: Dict[str, Any],
+ *args, **kwargs) -> float:
+ """
+ Objective function, returns smaller number for better results
+ This is the legacy algorithm (used until now in freqtrade).
+ Weights are distributed as follows:
+ * 0.4 to trade duration
+ * 0.25: Avoiding trade loss
+ * 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
+ """
+ total_profit = results['profit_ratio'].sum()
+ trade_duration = results['trade_duration'].mean()
+
+ trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
+ profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
+ duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
+ result = trade_loss + profit_loss + duration_loss
+ return result
+Currently, the arguments are:
+results: DataFrame containing the resulting trades.
+ The following columns are available in results (corresponds to the output-file of backtesting when used with --export trades):pair, profit_ratio, profit_abs, open_date, open_rate, fee_open, close_date, close_rate, fee_close, amount, trade_duration, is_open, exit_reason, stake_amount, min_rate, max_rate, stop_loss_ratio, stop_loss_abstrade_count: Amount of trades (identical to len(results))min_date: Start date of the timerange usedmin_date: End date of the timerange usedconfig: Config object used (Note: Not all strategy-related parameters will be updated here if they are part of a hyperopt space).processed: Dict of Dataframes with the pair as keys containing the data used for backtesting.backtest_stats: Backtesting statistics using the same format as the backtesting file "strategy" substructure. Available fields can be seen in generate_strategy_stats() in optimize_reports.py.This function needs to return a floating point number (float). Smaller numbers will be interpreted as better results. The parameters and balancing for this is up to you.
Note
+This function is called once per epoch - so please make sure to have this as optimized as possible to not slow hyperopt down unnecessarily.
+*args and **kwargs
Please keep the arguments *args and **kwargs in the interface to allow us to extend this interface in the future.
To override a pre-defined space (roi_space, generate_roi_table, stoploss_space, trailing_space, max_open_trades_space), define a nested class called Hyperopt and define the required spaces as follows:
from freqtrade.optimize.space import Categorical, Dimension, Integer, SKDecimal
+
+class MyAwesomeStrategy(IStrategy):
+ class HyperOpt:
+ # Define a custom stoploss space.
+ def stoploss_space():
+ return [SKDecimal(-0.05, -0.01, decimals=3, name='stoploss')]
+
+ # Define custom ROI space
+ def roi_space() -> List[Dimension]:
+ return [
+ Integer(10, 120, name='roi_t1'),
+ Integer(10, 60, name='roi_t2'),
+ Integer(10, 40, name='roi_t3'),
+ SKDecimal(0.01, 0.04, decimals=3, name='roi_p1'),
+ SKDecimal(0.01, 0.07, decimals=3, name='roi_p2'),
+ SKDecimal(0.01, 0.20, decimals=3, name='roi_p3'),
+ ]
+
+ def generate_roi_table(params: Dict) -> Dict[int, float]:
+
+ roi_table = {}
+ roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
+ roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
+ roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
+ roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
+
+ return roi_table
+
+ def trailing_space() -> List[Dimension]:
+ # All parameters here are mandatory, you can only modify their type or the range.
+ return [
+ # Fixed to true, if optimizing trailing_stop we assume to use trailing stop at all times.
+ Categorical([True], name='trailing_stop'),
+
+ SKDecimal(0.01, 0.35, decimals=3, name='trailing_stop_positive'),
+ # 'trailing_stop_positive_offset' should be greater than 'trailing_stop_positive',
+ # so this intermediate parameter is used as the value of the difference between
+ # them. The value of the 'trailing_stop_positive_offset' is constructed in the
+ # generate_trailing_params() method.
+ # This is similar to the hyperspace dimensions used for constructing the ROI tables.
+ SKDecimal(0.001, 0.1, decimals=3, name='trailing_stop_positive_offset_p1'),
+
+ Categorical([True, False], name='trailing_only_offset_is_reached'),
+ ]
+
+ # Define a custom max_open_trades space
+ def max_open_trades_space(self) -> List[Dimension]:
+ return [
+ Integer(-1, 10, name='max_open_trades'),
+ ]
+Note
+All overrides are optional and can be mixed/matched as necessary.
+Parameters can also be defined dynamically, but must be available to the instance once the bot_start() callback has been called.
class MyAwesomeStrategy(IStrategy):
+
+ def bot_start(self, **kwargs) -> None:
+ self.buy_adx = IntParameter(20, 30, default=30, optimize=True)
+
+ # ...
+Warning
+Parameters created this way will not show up in the list-strategies parameter count.
You can define your own estimator for Hyperopt by implementing generate_estimator() in the Hyperopt subclass.
class MyAwesomeStrategy(IStrategy):
+ class HyperOpt:
+ def generate_estimator(dimensions: List['Dimension'], **kwargs):
+ return "RF"
+Possible values are either one of "GP", "RF", "ET", "GBRT" (Details can be found in the scikit-optimize documentation), or "an instance of a class that inherits from RegressorMixin (from sklearn) and where the predict method has an optional return_std argument, which returns std(Y | x) along with E[Y | x]".
Some research will be necessary to find additional Regressors.
+Example for ExtraTreesRegressor ("ET") with additional parameters:
class MyAwesomeStrategy(IStrategy):
+ class HyperOpt:
+ def generate_estimator(dimensions: List['Dimension'], **kwargs):
+ from skopt.learning import ExtraTreesRegressor
+ # Corresponds to "ET" - but allows additional parameters.
+ return ExtraTreesRegressor(n_estimators=100)
+The dimensions parameter is the list of skopt.space.Dimension objects corresponding to the parameters to be optimized. It can be used to create isotropic kernels for the skopt.learning.GaussianProcessRegressor estimator. Here's an example:
class MyAwesomeStrategy(IStrategy):
+ class HyperOpt:
+ def generate_estimator(dimensions: List['Dimension'], **kwargs):
+ from skopt.utils import cook_estimator
+ from skopt.learning.gaussian_process.kernels import (Matern, ConstantKernel)
+ kernel_bounds = (0.0001, 10000)
+ kernel = (
+ ConstantKernel(1.0, kernel_bounds) *
+ Matern(length_scale=np.ones(len(dimensions)), length_scale_bounds=[kernel_bounds for d in dimensions], nu=2.5)
+ )
+ kernel += (
+ ConstantKernel(1.0, kernel_bounds) *
+ Matern(length_scale=np.ones(len(dimensions)), length_scale_bounds=[kernel_bounds for d in dimensions], nu=1.5)
+ )
+
+ return cook_estimator("GP", space=dimensions, kernel=kernel, n_restarts_optimizer=2)
+Note
+While custom estimators can be provided, it's up to you as User to do research on possible parameters and analyze / understand which ones should be used.
+If you're unsure about this, best use one of the Defaults ("ET" has proven to be the most versatile) without further parameters.
For the additional spaces, scikit-optimize (in combination with Freqtrade) provides the following space types:
+Categorical - Pick from a list of categories (e.g. Categorical(['a', 'b', 'c'], name="cat"))Integer - Pick from a range of whole numbers (e.g. Integer(1, 10, name='rsi'))SKDecimal - Pick from a range of decimal numbers with limited precision (e.g. SKDecimal(0.1, 0.5, decimals=3, name='adx')). Available only with freqtrade.Real - Pick from a range of decimal numbers with full precision (e.g. Real(0.1, 0.5, name='adx')You can import all of these from freqtrade.optimize.space, although Categorical, Integer and Real are only aliases for their corresponding scikit-optimize Spaces. SKDecimal is provided by freqtrade for faster optimizations.
from freqtrade.optimize.space import Categorical, Dimension, Integer, SKDecimal, Real # noqa
+SKDecimal vs. Real
+We recommend to use SKDecimal instead of the Real space in almost all cases. While the Real space provides full accuracy (up to ~16 decimal places) - this precision is rarely needed, and leads to unnecessary long hyperopt times.
Assuming the definition of a rather small space (SKDecimal(0.10, 0.15, decimals=2, name='xxx')) - SKDecimal will have 5 possibilities ([0.10, 0.11, 0.12, 0.13, 0.14, 0.15]).
A corresponding real space Real(0.10, 0.15 name='xxx') on the other hand has an almost unlimited number of possibilities ([0.10, 0.010000000001, 0.010000000002, ... 0.014999999999, 0.01500000000]).
This page explains some advanced tasks and configuration options that can be performed after the bot installation and may be uselful in some environments.
+If you do not know what things mentioned here mean, you probably do not need it.
+This section will show you how to run multiple bots at the same time, on the same machine.
+In order to keep track of your trades, profits, etc., freqtrade is using a SQLite database where it stores various types of information such as the trades you performed in the past and the current position(s) you are holding at any time. This allows you to keep track of your profits, but most importantly, keep track of ongoing activity if the bot process would be restarted or would be terminated unexpectedly.
+Freqtrade will, by default, use separate database files for dry-run and live bots (this assumes no database-url is given in either configuration nor via command line argument).
+For live trading mode, the default database will be tradesv3.sqlite and for dry-run it will be tradesv3.dryrun.sqlite.
The optional argument to the trade command used to specify the path of these files is --db-url, which requires a valid SQLAlchemy url.
+So when you are starting a bot with only the config and strategy arguments in dry-run mode, the following 2 commands would have the same outcome.
freqtrade trade -c MyConfig.json -s MyStrategy
+# is equivalent to
+freqtrade trade -c MyConfig.json -s MyStrategy --db-url sqlite:///tradesv3.dryrun.sqlite
+It means that if you are running the trade command in two different terminals, for example to test your strategy both for trades in USDT and in another instance for trades in BTC, you will have to run them with different databases.
+If you specify the URL of a database which does not exist, freqtrade will create one with the name you specified. So to test your custom strategy with BTC and USDT stake currencies, you could use the following commands (in 2 separate terminals):
+# Terminal 1:
+freqtrade trade -c MyConfigBTC.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesBTC.dryrun.sqlite
+# Terminal 2:
+freqtrade trade -c MyConfigUSDT.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesUSDT.dryrun.sqlite
+Conversely, if you wish to do the same thing in production mode, you will also have to create at least one new database (in addition to the default one) and specify the path to the "live" databases, for example:
+# Terminal 1:
+freqtrade trade -c MyConfigBTC.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesBTC.live.sqlite
+# Terminal 2:
+freqtrade trade -c MyConfigUSDT.json -s MyCustomStrategy --db-url sqlite:///user_data/tradesUSDT.live.sqlite
+For more information regarding usage of the sqlite databases, for example to manually enter or remove trades, please refer to the SQL Cheatsheet.
+To run multiple instances of freqtrade using docker you will need to edit the docker-compose.yml file and add all the instances you want as separate services. Remember, you can separate your configuration into multiple files, so it's a good idea to think about making them modular, then if you need to edit something common to all bots, you can do that in a single config file. +
---
+version: '3'
+services:
+ freqtrade1:
+ image: freqtradeorg/freqtrade:stable
+ # image: freqtradeorg/freqtrade:develop
+ # Use plotting image
+ # image: freqtradeorg/freqtrade:develop_plot
+ # Build step - only needed when additional dependencies are needed
+ # build:
+ # context: .
+ # dockerfile: "./docker/Dockerfile.custom"
+ restart: always
+ container_name: freqtrade1
+ volumes:
+ - "./user_data:/freqtrade/user_data"
+ # Expose api on port 8080 (localhost only)
+ # Please read the https://www.freqtrade.io/en/latest/rest-api/ documentation
+ # before enabling this.
+ ports:
+ - "127.0.0.1:8080:8080"
+ # Default command used when running `docker compose up`
+ command: >
+ trade
+ --logfile /freqtrade/user_data/logs/freqtrade1.log
+ --db-url sqlite:////freqtrade/user_data/tradesv3_freqtrade1.sqlite
+ --config /freqtrade/user_data/config.json
+ --config /freqtrade/user_data/config.freqtrade1.json
+ --strategy SampleStrategy
+
+ freqtrade2:
+ image: freqtradeorg/freqtrade:stable
+ # image: freqtradeorg/freqtrade:develop
+ # Use plotting image
+ # image: freqtradeorg/freqtrade:develop_plot
+ # Build step - only needed when additional dependencies are needed
+ # build:
+ # context: .
+ # dockerfile: "./docker/Dockerfile.custom"
+ restart: always
+ container_name: freqtrade2
+ volumes:
+ - "./user_data:/freqtrade/user_data"
+ # Expose api on port 8080 (localhost only)
+ # Please read the https://www.freqtrade.io/en/latest/rest-api/ documentation
+ # before enabling this.
+ ports:
+ - "127.0.0.1:8081:8080"
+ # Default command used when running `docker compose up`
+ command: >
+ trade
+ --logfile /freqtrade/user_data/logs/freqtrade2.log
+ --db-url sqlite:////freqtrade/user_data/tradesv3_freqtrade2.sqlite
+ --config /freqtrade/user_data/config.json
+ --config /freqtrade/user_data/config.freqtrade2.json
+ --strategy SampleStrategy
+Copy the freqtrade.service file to your systemd user directory (usually ~/.config/systemd/user) and update WorkingDirectory and ExecStart to match your setup.
Note
+Certain systems (like Raspbian) don't load service unit files from the user directory. In this case, copy freqtrade.service into /etc/systemd/user/ (requires superuser permissions).
After that you can start the daemon with:
+systemctl --user start freqtrade
+For this to be persistent (run when user is logged out) you'll need to enable linger for your freqtrade user.
sudo loginctl enable-linger "$USER"
+If you run the bot as a service, you can use systemd service manager as a software watchdog monitoring freqtrade bot
+state and restarting it in the case of failures. If the internals.sd_notify parameter is set to true in the
+configuration or the --sd-notify command line option is used, the bot will send keep-alive ping messages to systemd
+using the sd_notify (systemd notifications) protocol and will also tell systemd its current state (Running or Stopped)
+when it changes.
The freqtrade.service.watchdog file contains an example of the service unit configuration file which uses systemd
+as the watchdog.
Note
+The sd_notify communication between the bot and the systemd service manager will not work if the bot runs in a Docker container.
+On many Linux systems the bot can be configured to send its log messages to syslog or journald system services. Logging to a remote syslog server is also available on Windows. The special values for the --logfile command line option can be used for this.
To send Freqtrade log messages to a local or remote syslog service use the --logfile command line option with the value in the following format:
--logfile syslog:<syslog_address> -- send log messages to syslog service using the <syslog_address> as the syslog address.The syslog address can be either a Unix domain socket (socket filename) or a UDP socket specification, consisting of IP address and UDP port, separated by the : character.
So, the following are the examples of possible usages:
+--logfile syslog:/dev/log -- log to syslog (rsyslog) using the /dev/log socket, suitable for most systems.--logfile syslog -- same as above, the shortcut for /dev/log.--logfile syslog:/var/run/syslog -- log to syslog (rsyslog) using the /var/run/syslog socket. Use this on MacOS.--logfile syslog:localhost:514 -- log to local syslog using UDP socket, if it listens on port 514.--logfile syslog:<ip>:514 -- log to remote syslog at IP address and port 514. This may be used on Windows for remote logging to an external syslog server.Log messages are send to syslog with the user facility. So you can see them with the following commands:
tail -f /var/log/user, or On many systems syslog (rsyslog) fetches data from journald (and vice versa), so both --logfile syslog or --logfile journald can be used and the messages be viewed with both journalctl and a syslog viewer utility. You can combine this in any way which suites you better.
For rsyslog the messages from the bot can be redirected into a separate dedicated log file. To achieve this, add
if $programname startswith "freqtrade" then -/var/log/freqtrade.log
+to one of the rsyslog configuration files, for example at the end of the /etc/rsyslog.d/50-default.conf.
For syslog (rsyslog), the reduction mode can be switched on. This will reduce the number of repeating messages. For instance, multiple bot Heartbeat messages will be reduced to a single message when nothing else happens with the bot. To achieve this, set in /etc/rsyslog.conf:
# Filter duplicated messages
+$RepeatedMsgReduction on
+This needs the cysystemd python package installed as dependency (pip install cysystemd), which is not available on Windows. Hence, the whole journald logging functionality is not available for a bot running on Windows.
To send Freqtrade log messages to journald system service use the --logfile command line option with the value in the following format:
--logfile journald -- send log messages to journald.Log messages are send to journald with the user facility. So you can see them with the following commands:
journalctl -f -- shows Freqtrade log messages sent to journald along with other log messages fetched by journald.journalctl -f -u freqtrade.service -- this command can be used when the bot is run as a systemd service.There are many other options in the journalctl utility to filter the messages, see manual pages for this utility.
On many systems syslog (rsyslog) fetches data from journald (and vice versa), so both --logfile syslog or --logfile journald can be used and the messages be viewed with both journalctl and a syslog viewer utility. You can combine this in any way which suites you better.