lychiyu/freqtrade_origin
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2024-09-20 09:31:12 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #2329 from freqtrade/release_2019.9

Browse Source 
Release 2019.9
This commit is contained in:
Matthias 2019-10-01 20:04:18 +02:00 committed by GitHub
commit 6bbc0eefed
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
152 changed files with 2895 additions and 2122 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.coveragerc
View File

@ -1,6 +1,6 @@
[run]
omit =
scripts/*
freqtrade/tests/*
freqtrade/vendor/*
freqtrade/__main__.py
tests/*

5
.gitignore vendored
View File

@ -1,12 +1,9 @@
# Freqtrade rules
freqtrade/tests/testdata/*.json
hyperopt_conf.py
config*.json
*.sqlite
.hyperopt
logfile.txt
hyperopt_trials.pickle
user_data/*
!user_data/strategy/sample_strategy.py
!user_data/notebooks
user_data/notebooks/*
!user_data/notebooks/*example.ipynb

8
.travis.yml
View File

@ -22,19 +22,19 @@ jobs:
include:
- stage: tests
script:
- pytest --random-order --cov=freqtrade --cov-config=.coveragerc freqtrade/tests/
- pytest --random-order --cov=freqtrade --cov-config=.coveragerc
# Allow failure for coveralls
- coveralls || true
name: pytest
- script:
- cp config.json.example config.json
- freqtrade --datadir freqtrade/tests/testdata backtesting
- freqtrade --datadir tests/testdata backtesting
name: backtest
- script:
- cp config.json.example config.json
- freqtrade --datadir freqtrade/tests/testdata hyperopt -e 5
- freqtrade --datadir tests/testdata hyperopt -e 5
name: hyperopt
- script: flake8 freqtrade scripts
- script: flake8
name: flake8
- script:
# Test Documentation boxes -

8
CONTRIBUTING.md
View File

@ -11,7 +11,7 @@ Few pointers for contributions:
- Create your PR against the `develop` branch, not `master`.
- New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100).
If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg)
If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE)
or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR.
## Getting started
@ -28,19 +28,19 @@ make it pass. It means you have introduced a regression.
#### Test the whole project
```bash
pytest freqtrade
pytest
```
#### Test only one file
```bash
pytest freqtrade/tests/test_<file_name>.py
pytest tests/test_<file_name>.py
```
#### Test only one method from one file
```bash
pytest freqtrade/tests/test_<file_name>.py::test_<method_name>
pytest tests/test_<file_name>.py::test_<method_name>
```
### 2. Test if your code is PEP8 compliant

4
Dockerfile.pi
View File

@ -22,13 +22,13 @@ RUN tar -xzf /freqtrade/ta-lib-0.4.0-src.tar.gz \
ENV LD_LIBRARY_PATH /usr/local/lib
# Install berryconda
RUN wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh \
RUN wget -q https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh \
&& bash ./Berryconda3-2.0.0-Linux-armv7l.sh -b \
&& rm Berryconda3-2.0.0-Linux-armv7l.sh
# Install dependencies
COPY requirements-common.txt /freqtrade/
RUN ~/berryconda3/bin/conda install -y numpy pandas scipy \
RUN ~/berryconda3/bin/conda install -y numpy pandas \
&& ~/berryconda3/bin/pip install -r requirements-common.txt --no-cache-dir
# Install and execute

1
MANIFEST.in
View File

@ -2,4 +2,3 @@ include LICENSE
include README.md
include config.json.example
recursive-include freqtrade *.py
include freqtrade/tests/testdata/*.json

4
README.md
View File

@ -141,7 +141,7 @@ Accounts having BNB accounts use this to pay for fees - if your first trade happ
For any questions not covered by the documentation or for further
information about the bot, we encourage you to join our slack channel.
- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg).
- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE).
### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue)
@ -172,7 +172,7 @@ to understand the requirements before sending your pull-requests.
Coding is not a neccessity to contribute - maybe start with improving our documentation?
Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
**Important:** Always create your PR against the `develop` branch, not `master`.

2
build_helpers/publish_docker.sh
View File

@ -23,7 +23,7 @@ if [ $? -ne 0 ]; then
fi
# Run backtest
docker run --rm -it -v $(pwd)/config.json.example:/freqtrade/config.json:ro freqtrade:${TAG} --datadir freqtrade/tests/testdata backtesting
docker run --rm -it -v $(pwd)/config.json.example:/freqtrade/config.json:ro -v $(pwd)/tests:/tests freqtrade:${TAG} --datadir /tests/testdata backtesting
if [ $? -ne 0 ]; then
echo "failed running backtest"

1
config_full.json.example
View File

@ -38,6 +38,7 @@
"order_types": {
"buy": "limit",
"sell": "limit",
"emergencysell": "market",
"stoploss": "market",
"stoploss_on_exchange": false,
"stoploss_on_exchange_interval": 60

BIN
docs/assets/plot-dataframe.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 173 KiB

BIN
docs/assets/plot-profit.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 121 KiB

139
docs/backtesting.md
View File

@ -1,41 +1,9 @@
# Backtesting
This page explains how to validate your strategy performance by using
Backtesting.
This page explains how to validate your strategy performance by using Backtesting.
## Getting data for backtesting and hyperopt
To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the `freqtrade download-data` command.
If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes.
Exchange and pairs will come from `config.json` (if specified using `-c/--config`). Otherwise `--exchange` becomes mandatory.
Alternatively, a `pairs.json` file can be used.
If you are using Binance for example:
- create a directory `user_data/data/binance` and copy `pairs.json` in that directory.
- update the `pairs.json` to contain the currency pairs you are interested in.
```bash
mkdir -p user_data/data/binance
cp freqtrade/tests/testdata/pairs.json user_data/data/binance
```
Then run:
```bash
freqtrade download-data --exchange binance
```
This will download ticker data for all the currency pairs you defined in `pairs.json`.
- To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
- To change the exchange used to download the tickers, please use a different configuration file (you'll probably need to adjust ratelimits etc.)
- To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
- To download ticker data for only 10 days, use `--days 10` (defaults to 30 days).
- Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
Backtesting requires historic data to be available.
To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation.
## Test your strategy with Backtesting
@ -43,18 +11,16 @@ Now you have good Buy and Sell strategies and some historic data, you want to te
real data. This is what we call
[backtesting](https://en.wikipedia.org/wiki/Backtesting).
Backtesting will use the crypto-currencies (pair) from your config file
and load static tickers located in
[/freqtrade/tests/testdata](https://github.com/freqtrade/freqtrade/tree/develop/freqtrade/tests/testdata).
If the 5 min and 1 min ticker for the crypto-currencies to test is not
already in the `testdata` directory, backtesting will download them
automatically. Testdata files will not be updated until you specify it.
Backtesting will use the crypto-currencies (pairs) from your config file
and load ticker data from `user_data/data/<exchange>` by default.
If no data is available for the exchange / pair / ticker interval combination, backtesting will
ask you to download them first using `freqtrade download-data`.
For details on downloading, please refer to the [Data Downloading](data-download.md) section in the documentation.
The result of backtesting will confirm you if your bot has better odds of making a profit than a loss.
The backtesting is very easy with freqtrade.
The result of backtesting will confirm if your bot has better odds of making a profit than a loss.
### Run a backtesting against the currencies listed in your config file
#### With 5 min tickers (Per default)
```bash
@ -79,18 +45,18 @@ freqtrade backtesting --datadir user_data/data/bittrex-20180101
#### With a (custom) strategy file
```bash
freqtrade -s TestStrategy backtesting
freqtrade -s SampleStrategy backtesting
```
Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory.
Where `-s SampleStrategy` refers to the class name within the strategy file `sample_strategy.py` found in the `freqtrade/user_data/strategies` directory.
#### Comparing multiple Strategies
```bash
freqtrade backtesting --strategy-list TestStrategy1 AwesomeStrategy --ticker-interval 5m
freqtrade backtesting --strategy-list SampleStrategy1 AwesomeStrategy --ticker-interval 5m
```
Where `TestStrategy1` and `AwesomeStrategy` refer to class names of strategies.
Where `SampleStrategy1` and `AwesomeStrategy` refer to class names of strategies.
#### Exporting trades to file
@ -103,36 +69,35 @@ The exported trades can be used for [further analysis](#further-backtest-result-
#### Exporting trades to file specifying a custom filename
```bash
freqtrade backtesting --export trades --export-filename=backtest_teststrategy.json
freqtrade backtesting --export trades --export-filename=backtest_samplestrategy.json
```
#### Running backtest with smaller testset
#### Running backtest with smaller testset by using timerange
Use the `--timerange` argument to change how much of the testset
you want to use. The last N ticks/timeframes will be used.
Use the `--timerange` argument to change how much of the testset you want to use.
Example:
For example, running backtesting with the `--timerange=20190501-` option will use all available data starting with May 1st, 2019 from your inputdata.
```bash
freqtrade backtesting --timerange=-200
freqtrade backtesting --timerange=20190501-
```
#### Advanced use of timerange
Doing `--timerange=-200` will get the last 200 timeframes
from your inputdata. You can also specify specific dates,
or a range span indexed by start and stop.
You can also specify particular dates or a range span indexed by start and stop.
The full timerange specification:
- Use last 123 tickframes of data: `--timerange=-123`
- Use first 123 tickframes of data: `--timerange=123-`
- Use tickframes from line 123 through 456: `--timerange=123-456`
- Use tickframes till 2018/01/31: `--timerange=-20180131`
- Use tickframes since 2018/01/31: `--timerange=20180131-`
- Use tickframes since 2018/01/31 till 2018/03/01 : `--timerange=20180131-20180301`
- Use tickframes between POSIX timestamps 1527595200 1527618600:
`--timerange=1527595200-1527618600`
- Use last 123 tickframes of data: `--timerange=-123`
- Use first 123 tickframes of data: `--timerange=123-`
- Use tickframes from line 123 through 456: `--timerange=123-456`
!!! warning
Be carefull when using non-date functions - these do not allow you to specify precise dates, so if you updated the test-data it will probably use a different dataset.
## Understand the backtesting result
@ -178,11 +143,12 @@ A backtesting result will look like that:
| TOTAL | 2 | 0.78 | 1.57 | 0.00007855 | 0.78 | 4:00:00 | 2 | 0 |
```
The 1st table will contain all trades the bot made.
The 1st table contains all trades the bot made, including "left open trades".
The 2nd table will contain a recap of sell reasons.
The 2nd table contains a recap of sell reasons.
The 3rd table will contain all trades the bot had to `forcesell` at the end of the backtest period to present a full picture.
The 3rd table contains all trades the bot had to `forcesell` at the end of the backtest period to present a full picture.
This is necessary to simulate realistic behaviour, since the backtest period has to end at some point, while realistically, you could leave the bot running forever.
These trades are also included in the first table, but are extracted separately for clarity.
The last line will give you the overall performance of your strategy,
@ -192,22 +158,16 @@ here:
| TOTAL | 429 | 0.36 | 152.41 | 0.00762792 | 76.20 | 4:12:00 | 186 | 243 |
```
We understand the bot has made `429` trades for an average duration of
`4:12:00`, with a performance of `76.20%` (profit), that means it has
The bot has made `429` trades for an average duration of `4:12:00`, with a performance of `76.20%` (profit), that means it has
earned a total of `0.00762792 BTC` starting with a capital of 0.01 BTC.
The column `avg profit %` shows the average profit for all trades made while the column `cum profit %` sums all the profits/losses.
The column `tot profit %` shows instead the total profit % in relation to allocated capital
(`max_open_trades * stake_amount`). In the above results we have `max_open_trades=2 stake_amount=0.005` in config
so `(76.20/100) * (0.005 * 2) =~ 0.00762792 BTC`.
The column `avg profit %` shows the average profit for all trades made while the column `cum profit %` sums up all the profits/losses.
The column `tot profit %` shows instead the total profit % in relation to allocated capital (`max_open_trades * stake_amount`).
In the above results we have `max_open_trades=2` and `stake_amount=0.005` in config so `tot_profit %` will be `(76.20/100) * (0.005 * 2) =~ 0.00762792 BTC`.
As you will see your strategy performance will be influenced by your buy
strategy, your sell strategy, and also by the `minimal_roi` and
`stop_loss` you have set.
Your strategy performance is influenced by your buy strategy, your sell strategy, and also by the `minimal_roi` and `stop_loss` you have set.
As for an example if your minimal_roi is only `"0": 0.01`. You cannot
expect the bot to make more profit than 1% (because it will sell every
time a trade will reach 1%).
For example, if your `minimal_roi` is only `"0": 0.01` you cannot expect the bot to make more profit than 1% (because it will sell every time a trade reaches 1%).
```json
"minimal_roi": {
@ -216,22 +176,33 @@ time a trade will reach 1%).
```
On the other hand, if you set a too high `minimal_roi` like `"0": 0.55`
(55%), there is a lot of chance that the bot will never reach this
profit. Hence, keep in mind that your performance is a mix of your
strategies, your configuration, and the crypto-currency you have set up.
(55%), there is almost no chance that the bot will ever reach this profit.
Hence, keep in mind that your performance is an integral mix of all different elements of the strategy, your configuration, and the crypto-currency pairs you have set up.
### Assumptions made by backtesting
Since backtesting lacks some detailed information about what happens within a candle, it needs to take a few assumptions:
- Buys happen at open-price
- Low happens before high for stoploss, protecting capital first.
- ROI sells are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the sell will be at 2%)
- Stoploss sells happen exactly at stoploss price, even if low was lower
- Trailing stoploss
- High happens first - adjusting stoploss
- Low uses the adjusted stoploss (so sells with large high-low difference are backtested correctly)
- Sell-reason does not explain if a trade was positive or negative, just what triggered the sell (this can look odd if negative ROI values are used)
### Further backtest-result analysis
To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis as shown in our [data analysis](data-analysis.md#backtesting) backtesting section.
## Backtesting multiple strategies
To backtest multiple strategies, a list of Strategies can be provided.
To compare multiple strategies, a list of Strategies can be provided to backtesting.
This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this should give a nice runtime boost.
strategies you'd like to compare, this will give a nice runtime boost.
All listed Strategies need to be in the same directory.
@ -241,7 +212,7 @@ freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strat
This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up.
Detailed output for all strategies one after the other will be available, so make sure to scroll up to see the details per strategy.
```
=========================================================== Strategy Summary ===========================================================

12
docs/bot-usage.md
View File

@ -184,10 +184,6 @@ optional arguments:
Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT
Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--eps, --enable-position-stacking
Allow buying the same pair multiple times (position
stacking).
@ -245,10 +241,6 @@ optional arguments:
Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT
Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--customhyperopt NAME
Specify hyperopt class name (default:
`DefaultHyperOpts`).
@ -310,10 +302,6 @@ optional arguments:
Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT
Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--stoplosses STOPLOSS_RANGE
Defines a range of stoploss against which edge will
assess the strategy the format is "min,max,step"

68
docs/configuration.md
View File

@ -1,14 +1,15 @@
# Configure the bot
This page explains how to configure the bot.
Freqtrade has many configurable features and possibilities.
By default, these settings are configured via the configuration file (see below).
## The Freqtrade configuration file
The bot uses a set of configuration parameters during its operation that all together conform the bot configuration. It normally reads its configuration from a file (Freqtrade configuration file).
Per default, the bot loads configuration from the `config.json` file located in the current working directory.
Per default, the bot loads the configuration from the `config.json` file, located in the current working directory.
You can change the name of the configuration file used by the bot with the `-c/--config` command line option.
You can specify a different configuration file used by the bot with the `-c/--config` command line option.
In some advanced use cases, multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
@ -22,19 +23,26 @@ The Freqtrade configuration file is to be written in the JSON format.
Additionally to the standard JSON syntax, you may use one-line `// ...` and multi-line `/* ... */` comments in your configuration files and trailing commas in the lists of parameters.
Do not worry if you are not familiar with JSON format -- simply open the configuration file with an editor of your choice, make some changes to the parameters you need, save your changes and, finally, restart the bot or, if it was previously stopped, run it again with the changes you made to the configuration. The bot validates syntax of the configuration file at startup and will warn you if you made any errors editing it.
Do not worry if you are not familiar with JSON format -- simply open the configuration file with an editor of your choice, make some changes to the parameters you need, save your changes and, finally, restart the bot or, if it was previously stopped, run it again with the changes you made to the configuration. The bot validates syntax of the configuration file at startup and will warn you if you made any errors editing it, pointing out problematic lines.
## Configuration parameters
The table below will list all configuration parameters available.
Mandatory parameters are marked as **Required**.
Freqtrade can also load many options via command line (CLI) arguments (check out the commands `--help` output for details).
The prevelance for all Options is as follows:
- CLI arguments override any other option
- Configuration files are used in sequence (last file wins), and override Strategy configurations.
- Strategy configurations are only used if they are not set via configuration or via command line arguments. These options are market with [Strategy Override](#parameters-in-the-strategy) in the below table.
Mandatory parameters are marked as **Required**, which means that they are required to be set in one of the possible ways.
| Command | Default | Description |
|----------|---------|-------------|
| `max_open_trades` | 3 | **Required.** Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades)
| `stake_currency` | BTC | **Required.** Crypto-currency used for trading. [Strategy Override](#parameters-in-the-strategy).
| `stake_amount` | 0.05 | **Required.** Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to `"unlimited"` to allow the bot to use all available balance. [Strategy Override](#parameters-in-the-strategy).
| `stake_currency` | BTC | **Required.** Crypto-currency used for trading.
| `stake_amount` | 0.05 | **Required.** Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to `"unlimited"` to allow the bot to use all available balance.
| `amount_reserve_percent` | 0.05 | Reserve some amount in min pair stake amount. Default is 5%. The bot will reserve `amount_reserve_percent` + stop-loss value when calculating min pair stake amount in order to avoid possible trade refusals.
| `ticker_interval` | [1m, 5m, 15m, 30m, 1h, 1d, ...] | The ticker interval to use (1min, 5 min, 15 min, 30 min, 1 hour or 1 day). Default is 5 minutes. [Strategy Override](#parameters-in-the-strategy).
| `fiat_display_currency` | USD | **Required.** Fiat currency used to show your profits. More information below.
@ -61,8 +69,9 @@ Mandatory parameters are marked as **Required**.
| `order_time_in_force` | None | Configure time in force for buy and sell orders. [More information below](#understand-order_time_in_force). [Strategy Override](#parameters-in-the-strategy).
| `exchange.name` | | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename).
| `exchange.sandbox` | false | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.
| `exchange.key` | '' | API key to use for the exchange. Only required when you are in production mode.
| `exchange.secret` | '' | API secret to use for the exchange. Only required when you are in production mode.
| `exchange.key` | '' | API key to use for the exchange. Only required when you are in production mode. ***Keep it in secrete, do not disclose publicly.***
| `exchange.secret` | '' | API secret to use for the exchange. Only required when you are in production mode. ***Keep it in secrete, do not disclose publicly.***
| `exchange.password` | '' | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests. ***Keep it in secrete, do not disclose publicly.***
| `exchange.pair_whitelist` | [] | List of pairs to use by the bot for trading and to check for potential trades during backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)).
| `exchange.pair_blacklist` | [] | List of pairs the bot must absolutely avoid for trading and backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)).
| `exchange.ccxt_config` | None | Additional CCXT parameters passed to the regular ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation)
@ -76,8 +85,8 @@ Mandatory parameters are marked as **Required**.
| `pairlist.method` | StaticPairList | Use static or dynamic volume-based pairlist. [More information below](#dynamic-pairlists).
| `pairlist.config` | None | Additional configuration for dynamic pairlists. [More information below](#dynamic-pairlists).
| `telegram.enabled` | true | **Required.** Enable or not the usage of Telegram.
| `telegram.token` | token | Your Telegram bot token. Only required if `telegram.enabled` is `true`.
| `telegram.chat_id` | chat_id | Your personal Telegram account id. Only required if `telegram.enabled` is `true`.
| `telegram.token` | token | Your Telegram bot token. Only required if `telegram.enabled` is `true`. ***Keep it in secrete, do not disclose publicly.***
| `telegram.chat_id` | chat_id | Your personal Telegram account id. Only required if `telegram.enabled` is `true`. ***Keep it in secrete, do not disclose publicly.***
| `webhook.enabled` | false | Enable usage of Webhook notifications
| `webhook.url` | false | URL for the webhook. Only required if `webhook.enabled` is `true`. See the [webhook documentation](webhook-config.md) for more details.
| `webhook.webhookbuy` | false | Payload to send on buy. Only required if `webhook.enabled` is `true`. See the [webhook documentationV](webhook-config.md) for more details.
@ -98,8 +107,6 @@ Mandatory parameters are marked as **Required**.
The following parameters can be set in either configuration file or strategy.
Values set in the configuration file always overwrite values set in the strategy.
* `stake_currency`
* `stake_amount`
* `ticker_interval`
* `minimal_roi`
* `stoploss`
@ -191,19 +198,20 @@ end up paying more then would probably have been necessary.
### Understand order_types
The `order_types` configuration parameter contains a dict mapping order-types to
market-types as well as stoploss on or off exchange type and stoploss on exchange
update interval in seconds. This allows to buy using limit orders, sell using
limit-orders, and create stoploss orders using market. It also allows to set the
stoploss "on exchange" which means stoploss order would be placed immediately once
the buy order is fulfilled. In case stoploss on exchange and `trailing_stop` are
both set, then the bot will use `stoploss_on_exchange_interval` to check it periodically
and update it if necessary (e.x. in case of trailing stoploss).
This can be set in the configuration file or in the strategy.
Values set in the configuration file overwrites values set in the strategy.
The `order_types` configuration parameter maps actions (`buy`, `sell`, `stoploss`) to order-types (`market`, `limit`, ...) as well as configures stoploss to be on the exchange and defines stoploss on exchange update interval in seconds.
If this is configured, all 4 values (`buy`, `sell`, `stoploss` and
`stoploss_on_exchange`) need to be present, otherwise the bot will warn about it and fail to start.
This allows to buy using limit orders, sell using
limit-orders, and create stoplosses using using market orders. It also allows to set the
stoploss "on exchange" which means stoploss order would be placed immediately once
the buy order is fulfilled.
If `stoploss_on_exchange` and `trailing_stop` are both set, then the bot will use `stoploss_on_exchange_interval` to check and update the stoploss on exchange periodically.
`order_types` can be set in the configuration file or in the strategy.
`order_types` set in the configuration file overwrites values set in the strategy as a whole, so you need to configure the whole `order_types` dictionary in one place.
If this is configured, the following 4 values (`buy`, `sell`, `stoploss` and
`stoploss_on_exchange`) need to be present, otherwise the bot will fail to start.
`emergencysell` is an optional value, which defaults to `market` and is used when creating stoploss on exchange orders fails.
The below is the default which is used if this is not configured in either strategy or configuration file.
Syntax for Strategy:
@ -212,6 +220,7 @@ Syntax for Strategy:
order_types = {
"buy": "limit",
"sell": "limit",
"emergencysell": "market",
"stoploss": "market",
"stoploss_on_exchange": False,
"stoploss_on_exchange_interval": 60
@ -224,6 +233,7 @@ Configuration:
"order_types": {
"buy": "limit",
"sell": "limit",
"emergencysell": "market",
"stoploss": "market",
"stoploss_on_exchange": false,
"stoploss_on_exchange_interval": 60
@ -238,11 +248,13 @@ Configuration:
!!! Note
Stoploss on exchange interval is not mandatory. Do not change its value if you are
unsure of what you are doing. For more information about how stoploss works please
read [the stoploss documentation](stoploss.md).
refer to [the stoploss documentation](stoploss.md).
!!! Note
In case of stoploss on exchange if the stoploss is cancelled manually then
the bot would recreate one.
If `stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new order.
!!! Warning stoploss_on_exchange failures
If stoploss on exchange creation fails for some reason, then an "emergency sell" is initiated. By default, this will sell the asset using a market order. The order-type for the emergency-sell can be changed by setting the `emergencysell` value in the `order_types` dictionary - however this is not advised.
### Understand order_time_in_force

16
docs/data-analysis.md
View File

@ -91,7 +91,8 @@ df.groupby("pair")["sell_reason"].value_counts()
### Load multiple configuration files
This option can be useful to inspect the results of passing in multiple configs
This option can be useful to inspect the results of passing in multiple configs.
This will also run through the whole Configuration initialization, so the configuration is completely initialized to be passed to other methods.
``` python
import json
@ -101,7 +102,16 @@ from freqtrade.configuration import Configuration
config = Configuration.from_files(["config1.json", "config2.json"])
# Show the config in memory
print(json.dumps(config, indent=1))
print(json.dumps(config['original_config'], indent=2))
```
For Interactive environments, have an additional configuration specifying `user_data_dir` and pass this in last, so you don't have to change directories while running the bot.
Best avoid relative paths, since this starts at the storage location of the jupyter notebook, unless the directory is changed.
``` json
{
"user_data_dir": "~/.freqtrade/"
}
```
### Load exchange data to a pandas dataframe
@ -139,7 +149,7 @@ You can override strategy settings as demonstrated below.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'TestStrategy'
strategy_name = 'SampleStrategy'
# Path to user data
user_data_dir = 'user_data'
# Location of the strategy

62
docs/data-download.md Normal file
View File

@ -0,0 +1,62 @@
# Data Downloading
## Getting data for backtesting and hyperopt
To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the `freqtrade download-data` command.
If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes for the last 30 days.
Exchange and pairs will come from `config.json` (if specified using `-c/--config`).
Otherwise `--exchange` becomes mandatory.
!!! Tip Updating existing data
If you already have backtesting data available in your data-directory and would like to refresh this data up to today, use `--days xx` with a number slightly higher than the missing number of days. Freqtrade will keep the available data and only download the missing data.
Be carefull though: If the number is too small (which would result in a few missing days), the whole dataset will be removed and only xx days will be downloaded.
### Pairs file
In alternative to the whitelist from `config.json`, a `pairs.json` file can be used.
If you are using Binance for example:
- create a directory `user_data/data/binance` and copy or create the `pairs.json` file in that directory.
- update the `pairs.json` file to contain the currency pairs you are interested in.
```bash
mkdir -p user_data/data/binance
cp freqtrade/tests/testdata/pairs.json user_data/data/binance
```
The format of the `pairs.json` file is a simple json list.
Mixing different stake-currencies is allowed for this file, since it's only used for downloading.
``` json
[
"ETH/BTC",
"ETH/USDT",
"BTC/USDT",
"XRP/ETH"
]
```
### start download
Then run:
```bash
freqtrade download-data --exchange binance
```
This will download ticker data for all the currency pairs you defined in `pairs.json`.
### Other Notes
- To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
- To change the exchange used to download the tickers, please use a different configuration file (you'll probably need to adjust ratelimits etc.)
- To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
- To download ticker data for only 10 days, use `--days 10` (defaults to 30 days).
- Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
## Next step
Great, you now have backtest data downloaded, so you can now start [backtesting](backtesting.md) your strategy.

6
docs/deprecated.md
View File

@ -4,7 +4,7 @@ This page contains description of the command line arguments, configuration para
and the bot features that were declared as DEPRECATED by the bot development team
and are no longer supported. Please avoid their usage in your configuration.
## Deprecated
## Removed features
### the `--refresh-pairs-cached` command line option
@ -12,9 +12,7 @@ and are no longer supported. Please avoid their usage in your configuration.
Since this leads to much confusion, and slows down backtesting (while not being part of backtesting) this has been singled out
as a seperate freqtrade subcommand `freqtrade download-data`.
This command line option was deprecated in `2019.7-dev` and will be removed after the next release.
## Removed features
This command line option was deprecated in 2019.7-dev (develop branch) and removed in 2019.9 (master branch).
### The **--dynamic-whitelist** command line option

4
docs/developer.md
View File

@ -2,7 +2,7 @@
This page is intended for developers of FreqTrade, people who want to contribute to the FreqTrade codebase or documentation, or people who want to understand the source code of the application they're running.
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) where you can ask questions.
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) where you can ask questions.
## Documentation
@ -30,7 +30,7 @@ These are available from `conftest.py` and can be imported in any test module.
A sample check looks as follows:
``` python
from freqtrade.tests.conftest import log_has, log_has_re
from tests.conftest import log_has, log_has_re
def test_method_to_test(caplog):
method_to_test()

67
docs/hyperopt.md
View File

@ -6,6 +6,9 @@ algorithms included in the `scikit-optimize` package to accomplish this. The
search will burn all your CPU cores, make your laptop sound like a fighter jet
and still take a long time.
Hyperopt requires historic data to be available, just as backtesting does.
To learn how to get data for the pairs and exchange you're interrested in, head over to the [Data Downloading](data-download.md) section of the documentation.
!!! Bug
Hyperopt will crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133)
@ -372,35 +375,42 @@ Buy hyperspace params:
'rsi-enabled': True,
'trigger': 'bb_lower'}
ROI table:
{ 0: 0.10674752302642071,
21: 0.09158372701087236,
78: 0.03634636907306948,
{ 0: 0.10674,
21: 0.09158,
78: 0.03634,
118: 0}
```
This would translate to the following ROI table:
In order to use this best ROI table found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `minimal_roi` attribute of your custom strategy:
``` python
minimal_roi = {
"118": 0,
"78": 0.0363,
"21": 0.0915,
"0": 0.106
```
# Minimal ROI designed for the strategy.
# This attribute will be overridden if the config file contains "minimal_roi"
minimal_roi = {
0: 0.10674,
21: 0.09158,
78: 0.03634,
118: 0
}
```
As stated in the comment, you can also use it as the value of the `minimal_roi` setting in the configuration file.
If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps) with the values that can vary in the following ranges:
#### Default ROI Search Space
| # | minutes | ROI percentage |
|---|---|---|
| 1 | always 0 | 0.03...0.31 |
| 2 | 10...40 | 0.02...0.11 |
| 3 | 20...100 | 0.01...0.04 |
| 4 | 30...220 | always 0 |
If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps). Hyperopt implements adaptive ranges for ROI tables with ranges for values in the ROI steps that depend on the ticker_interval used. By default the values can vary in the following ranges (for some of the most used ticker intervals, values are rounded to 5 digits after the decimal point):
This structure of the ROI table is sufficient in most cases. Override the `roi_space()` method defining the ranges desired if you need components of the ROI tables to vary in other ranges.
| # step | 1m | | 5m | | 1h | | 1d | |
|---|---|---|---|---|---|---|---|---|
| 1 | 0 | 0.01161...0.11992 | 0 | 0.03...0.31 | 0 | 0.06883...0.71124 | 0 | 0.12178...1.25835 |
| 2 | 2...8 | 0.00774...0.04255 | 10...40 | 0.02...0.11 | 120...480 | 0.04589...0.25238 | 2880...11520 | 0.08118...0.44651 |
| 3 | 4...20 | 0.00387...0.01547 | 20...100 | 0.01...0.04 | 240...1200 | 0.02294...0.09177 | 5760...28800 | 0.04059...0.16237 |
| 4 | 6...44 | 0.0 | 30...220 | 0.0 | 360...2640 | 0.0 | 8640...63360 | 0.0 |
Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization in these methods if you need a different structure of the ROI table or other amount of rows (steps) in the ROI tables.
These ranges should be sufficient in most cases. The minutes in the steps (ROI dict keys) are scaled linearly depending on the ticker interval used. The ROI values in the steps (ROI dict values) are scaled logarithmically depending on the ticker interval used.
If you have the `generate_roi_table()` and `roi_space()` methods in your custom hyperopt file, remove them in order to utilize these adaptive ROI tables and the ROI hyperoptimization space generated by Freqtrade by default.
Override the `roi_space()` method if you need components of the ROI tables to vary in other ranges. Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization if you need a different structure of the ROI tables or other amount of rows (steps). A sample for these methods can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_advanced.py).
### Understand Hyperopt Stoploss results
@ -417,12 +427,25 @@ Buy hyperspace params:
'adx-enabled': False,
'rsi-enabled': True,
'trigger': 'bb_lower'}
Stoploss: -0.37996664668703606
Stoploss: -0.27996
```
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.5...-0.02, which is sufficient in most cases.
In order to use this best stoploss value found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `stoploss` attribute of your custom strategy:
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization.
```
# Optimal stoploss designed for the strategy
# This attribute will be overridden if the config file contains "stoploss"
stoploss = -0.27996
```
As stated in the comment, you can also use it as the value of the `stoploss` setting in the configuration file.
#### Default Stoploss Search Space
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.35...-0.02, which is sufficient in most cases.
If you have the `stoploss_space()` method in your custom hyperopt file, remove it in order to utilize Stoploss hyperoptimization space generated by Freqtrade by default.
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_advanced.py).
### Validate backtesting results

2
docs/index.md
View File

@ -64,7 +64,7 @@ To run this bot we recommend you a cloud instance with a minimum of:
Help / Slack
For any questions not covered by the documentation or for further information about the bot, we encourage you to join our Slack channel.
Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) to join Slack channel.
Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) to join Slack channel.
## Ready to try?

11
docs/installation.md
View File

@ -99,8 +99,8 @@ sudo apt-get install build-essential git
Before installing FreqTrade on a Raspberry Pi running the official Raspbian Image, make sure you have at least Python 3.6 installed. The default image only provides Python 3.5. Probably the easiest way to get a recent version of python is [miniconda](https://repo.continuum.io/miniconda/).
The following assumes that miniconda3 is installed and available in your environment. Last miniconda3 installation file use python 3.4, we will update to python 3.6 on this installation.
It's recommended to use (mini)conda for this as installation/compilation of `numpy`, `scipy` and `pandas` takes a long time.
The following assumes that miniconda3 is installed and available in your environment. Since the last miniconda3 installation file uses python 3.4, we will update to python 3.6 on this installation.
It's recommended to use (mini)conda for this as installation/compilation of `numpy` and `pandas` takes a long time.
Additional package to install on your Raspbian, `libffi-dev` required by cryptography (from python-telegram-bot).
@ -109,13 +109,17 @@ conda config --add channels rpi
conda install python=3.6
conda create -n freqtrade python=3.6
conda activate freqtrade
conda install scipy pandas numpy
conda install pandas numpy
sudo apt install libffi-dev
python3 -m pip install -r requirements-common.txt
python3 -m pip install -e .
```
!!! Note
This does not install hyperopt dependencies. To install these, please use `python3 -m pip install -e .[hyperopt]`.
We do not advise to run hyperopt on a Raspberry Pi, since this is a very resource-heavy operation, which should be done on powerful machine.
### Common
#### 1. Install TA-Lib
@ -175,7 +179,6 @@ cp config.json.example config.json
``` bash
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
python3 -m pip install -e .
```

164
docs/plotting.md
View File

@ -2,9 +2,9 @@
This page explains how to plot prices, indicators and profits.
## Installation
## Installation / Setup
Plotting scripts use Plotly library. Install/upgrade it with:
Plotting modules use the Plotly library. You can install / upgrade this by running the following command:
``` bash
pip install -U -r requirements-plot.txt
@ -12,90 +12,172 @@ pip install -U -r requirements-plot.txt
## Plot price and indicators
Usage for the price plotter:
The `freqtrade plot-dataframe` subcommand shows an interactive graph with three subplots:
* Main plot with candlestics and indicators following price (sma/ema)
* Volume bars
* Additional indicators as specified by `--indicators2`
![plot-dataframe](assets/plot-dataframe.png)
Possible arguments:
```
usage: freqtrade plot-dataframe [-h] [-p PAIRS [PAIRS ...]]
[--indicators1 INDICATORS1 [INDICATORS1 ...]]
[--indicators2 INDICATORS2 [INDICATORS2 ...]]
[--plot-limit INT] [--db-url PATH]
[--trade-source {DB,file}] [--export EXPORT]
[--export-filename PATH]
[--timerange TIMERANGE]
optional arguments:
-h, --help show this help message and exit
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Show profits for only these pairs. Pairs are space-
separated.
--indicators1 INDICATORS1 [INDICATORS1 ...]
Set indicators from your strategy you want in the
first row of the graph. Space-separated list. Example:
`ema3 ema5`. Default: `['sma', 'ema3', 'ema5']`.
--indicators2 INDICATORS2 [INDICATORS2 ...]
Set indicators from your strategy you want in the
third row of the graph. Space-separated list. Example:
`fastd fastk`. Default: `['macd', 'macdsignal']`.
--plot-limit INT Specify tick limit for plotting. Notice: too high
values cause huge files. Default: 750.
--db-url PATH Override trades database URL, this is useful in custom
deployments (default: `sqlite:///tradesv3.sqlite` for
Live Run mode, `sqlite://` for Dry Run).
--trade-source {DB,file}
Specify the source for trades (Can be DB or file
(backtest file)) Default: file
--export EXPORT Export backtest results, argument are: trades.
Example: `--export=trades`
--export-filename PATH
Save backtest results to the file with this filename
(default: `user_data/backtest_results/backtest-
result.json`). Requires `--export` to be set as well.
Example: `--export-filename=user_data/backtest_results
/backtest_today.json`
--timerange TIMERANGE
Specify what timerange of data to use.
``` bash
python3 script/plot_dataframe.py [-h] [-p pairs]
```
Example
Example:
``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH
freqtrade plot-dataframe -p BTC/ETH
```
The `-p` pairs argument can be used to specify pairs you would like to plot.
The `-p/--pairs` argument can be used to specify pairs you would like to plot.
!!! Note
The `freqtrade plot-dataframe` subcommand generates one plot-file per pair.
Specify custom indicators.
Use `--indicators1` for the main plot and `--indicators2` for the subplot below (if values are in a different range than prices).
!!! tip
You will almost certainly want to specify a custom strategy! This can be done by adding `-s Classname` / `--strategy ClassName` to the command.
``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH --indicators1 sma,ema --indicators2 macd
freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --indicators1 sma ema --indicators2 macd
```
### Advanced use
### Further usage examples
To plot multiple pairs, separate them with a comma:
To plot multiple pairs, separate them with a space:
``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH,XRP/ETH
freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH XRP/ETH
```
To plot a timerange (to zoom in):
To plot a timerange (to zoom in)
``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH --timerange=20180801-20180805
freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --timerange=20180801-20180805
```
To plot trades stored in a database use `--db-url` argument:
To plot trades stored in a database use `--db-url` in combination with `--trade-source DB`:
``` bash
python3 scripts/plot_dataframe.py --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB
freqtrade --strategy AwesomeStrategy plot-dataframe --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB
```
To plot trades from a backtesting result, use `--export-filename <filename>`
``` bash
python3 scripts/plot_dataframe.py --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
```
To plot a custom strategy the strategy should have first be backtested.
The results may then be plotted with the -s argument:
``` bash
python3 scripts/plot_dataframe.py -s Strategy_Name -p BTC/ETH --datadir user_data/data/<exchange_name>/
freqtrade --strategy AwesomeStrategy plot-dataframe --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
```
## Plot profit
The profit plotter shows a picture with three plots:
![plot-profit](assets/plot-profit.png)
The `freqtrade plot-profit` subcommand shows an interactive graph with three plots:
1) Average closing price for all pairs
2) The summarized profit made by backtesting.
Note that this is not the real-world profit, but
more of an estimate.
3) Each pair individually profit
Note that this is not the real-world profit, but more of an estimate.
3) Profit for each individual pair
The first graph is good to get a grip of how the overall market
progresses.
The first graph is good to get a grip of how the overall market progresses.
The second graph will show how your algorithm works or doesn't.
Perhaps you want an algorithm that steadily makes small profits,
or one that acts less seldom, but makes big swings.
The second graph will show if your algorithm works or doesn't.
Perhaps you want an algorithm that steadily makes small profits, or one that acts less often, but makes big swings.
The third graph can be useful to spot outliers, events in pairs
that makes profit spikes.
The third graph can be useful to spot outliers, events in pairs that cause profit spikes.
Usage for the profit plotter:
Possible options for the `freqtrade plot-profit` subcommand:
```
usage: freqtrade plot-profit [-h] [-p PAIRS [PAIRS ...]]
[--timerange TIMERANGE] [--export EXPORT]
[--export-filename PATH] [--db-url PATH]
[--trade-source {DB,file}]
optional arguments:
-h, --help show this help message and exit
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Show profits for only these pairs. Pairs are space-
separated.
--timerange TIMERANGE
Specify what timerange of data to use.
--export EXPORT Export backtest results, argument are: trades.
Example: `--export=trades`
--export-filename PATH
Save backtest results to the file with this filename
(default: `user_data/backtest_results/backtest-
result.json`). Requires `--export` to be set as well.
Example: `--export-filename=user_data/backtest_results
/backtest_today.json`
--db-url PATH Override trades database URL, this is useful in custom
deployments (default: `sqlite:///tradesv3.sqlite` for
Live Run mode, `sqlite://` for Dry Run).
--trade-source {DB,file}
Specify the source for trades (Can be DB or file
(backtest file)) Default: file
``` bash
python3 script/plot_profit.py [-h] [-p pair] [--datadir directory] [--ticker_interval num]
```
The `-p` pair argument, can be used to plot a single pair
The `-p/--pairs` argument, can be used to limit the pairs that are considered for this calculation.
Example
Examples:
Use custom backtest-export file
``` bash
python3 scripts/plot_profit.py --datadir ../freqtrade/freqtrade/tests/testdata-20171221/ -p LTC/BTC
freqtrade plot-profit -p LTC/BTC --export-filename user_data/backtest_results/backtest-result-Strategy005.json
```
Use custom database
``` bash
freqtrade plot-profit -p LTC/BTC --db-url sqlite:///tradesv3.sqlite --trade-source DB
```
``` bash
freqtrade plot-profit --datadir user_data/data/binance_save/ -p LTC/BTC
```

2
docs/requirements-docs.txt
View File

@ -1 +1 @@
mkdocs-material==4.4.0
mkdocs-material==4.4.2

24
docs/strategy-customization.md
View File

@ -24,7 +24,7 @@ strategy file will be updated on Github. Put your custom strategy file
into the directory `user_data/strategies`.
Best copy the test-strategy and modify this copy to avoid having bot-updates override your changes.
`cp user_data/strategies/test_strategy.py user_data/strategies/awesome-strategy.py`
`cp user_data/strategies/sample_strategy.py user_data/strategies/awesome-strategy.py`
### Anatomy of a strategy
@ -36,14 +36,19 @@ A strategy file contains all the information needed to build a good strategy:
- Minimal ROI recommended
- Stoploss strongly recommended
The bot also include a sample strategy called `TestStrategy` you can update: `user_data/strategies/test_strategy.py`.
You can test it with the parameter: `--strategy TestStrategy`
The bot also include a sample strategy called `SampleStrategy` you can update: `user_data/strategies/sample_strategy.py`.
You can test it with the parameter: `--strategy SampleStrategy`
Additionally, there is an attribute called `INTERFACE_VERSION`, which defines the version of the strategy interface the bot should use.
The current version is 2 - which is also the default when it's not set explicitly in the strategy.
Future versions will require this to be set.
```bash
freqtrade --strategy AwesomeStrategy
```
**For the following section we will use the [user_data/strategies/test_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py)
**For the following section we will use the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py)
file as reference.**
!!! Note Strategies and Backtesting
@ -109,9 +114,8 @@ def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame
return dataframe
```
!!! Note "Want more indicator examples?"
Look into the [user_data/strategies/test_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py).<br/>
Look into the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py).
Then uncomment indicators you need.
### Buy signal rules
@ -122,7 +126,7 @@ It's important to always return the dataframe without removing/modifying the col
This will method will also define a new column, `"buy"`, which needs to contain 1 for buys, and 0 for "no action".
Sample from `user_data/strategies/test_strategy.py`:
Sample from `user_data/strategies/sample_strategy.py`:
```python
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -152,7 +156,7 @@ It's important to always return the dataframe without removing/modifying the col
This will method will also define a new column, `"sell"`, which needs to contain 1 for sells, and 0 for "no action".
Sample from `user_data/strategies/test_strategy.py`:
Sample from `user_data/strategies/sample_strategy.py`:
```python
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -220,7 +224,7 @@ This would signify a stoploss of -10%.
For the full documentation on stoploss features, look at the dedicated [stoploss page](stoploss.md).
If your exchange supports it, it's recommended to also set `"stoploss_on_exchange"` in the order dict, so your stoploss is on the exchange and cannot be missed for network-problems (or other problems).
If your exchange supports it, it's recommended to also set `"stoploss_on_exchange"` in the order_types dictionary, so your stoploss is on the exchange and cannot be missed due to network problems, high load or other reasons.
For more information on order_types please look [here](configuration.md#understand-order_types).
@ -407,7 +411,7 @@ To get additional Ideas for strategies, head over to our [strategy repository](h
Feel free to use any of them as inspiration for your own strategies.
We're happy to accept Pull Requests containing new Strategies to that repo.
We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) which is a great place to get and/or share ideas.
We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) which is a great place to get and/or share ideas.
## Next step

11
environment.yml
View File

@ -9,25 +9,26 @@ dependencies:
- wheel
- numpy
- pandas
- scipy
- SQLAlchemy
- scikit-learn
- arrow
- requests
- urllib3
- wrapt
- joblib
- jsonschema
- tabulate
- python-rapidjson
- filelock
- flask
- python-dotenv
- cachetools
- scikit-optimize
- python-telegram-bot
# Optional for plotting
- plotly
# Optional for hyperopt
- scipy
- scikit-optimize
- scikit-learn
- filelock
- joblib
# Optional for development
- flake8
- pytest

15
freqtrade/__init__.py
View File

@ -1,5 +1,16 @@
""" FreqTrade bot """
__version__ = '2019.8-1'
__version__ = '2019.9'
if __version__ == 'develop':
try:
import subprocess
__version__ = 'develop-' + subprocess.check_output(
['git', 'log', '--format="%h"', '-n 1'],
stderr=subprocess.DEVNULL).decode("utf-8").rstrip().strip('"')
except Exception:
# git not available, ignore
pass
class DependencyException(Exception):
@ -11,7 +22,7 @@ class DependencyException(Exception):
class OperationalException(Exception):
"""
Requires manual intervention.
Requires manual intervention and will usually stop the bot.
This happens when an exchange returns an unexpected error during runtime
or given configuration is invalid.
"""

56
freqtrade/configuration/arguments.py
View File

@ -2,10 +2,11 @@
This module contains the argument manager class
"""
import argparse
from typing import List, Optional
from pathlib import Path
from typing import Any, Dict, List, Optional
from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
from freqtrade import constants
from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir", "user_data_dir"]
@ -14,7 +15,7 @@ ARGS_STRATEGY = ["strategy", "strategy_path"]
ARGS_MAIN = ARGS_COMMON + ARGS_STRATEGY + ["db_url", "sd_notify"]
ARGS_COMMON_OPTIMIZE = ["ticker_interval", "timerange",
"max_open_trades", "stake_amount", "refresh_pairs"]
"max_open_trades", "stake_amount"]
ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_positions",
"strategy_list", "export", "exportfilename"]
@ -34,33 +35,29 @@ ARGS_CREATE_USERDIR = ["user_data_dir"]
ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "exchange", "timeframes", "erase"]
ARGS_PLOT_DATAFRAME = (ARGS_COMMON + ARGS_STRATEGY +
["pairs", "indicators1", "indicators2", "plot_limit", "db_url",
"trade_source", "export", "exportfilename", "timerange",
"refresh_pairs"])
ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit", "db_url",
"trade_source", "export", "exportfilename", "timerange", "ticker_interval"]
ARGS_PLOT_PROFIT = (ARGS_COMMON + ARGS_STRATEGY +
["pairs", "timerange", "export", "exportfilename", "db_url", "trade_source"])
ARGS_PLOT_PROFIT = ["pairs", "timerange", "export", "exportfilename", "db_url",
"trade_source", "ticker_interval"]
NO_CONF_REQURIED = ["start_download_data"]
NO_CONF_REQURIED = ["create-userdir", "download-data", "plot-dataframe", "plot-profit"]
class Arguments(object):
class Arguments:
"""
Arguments Class. Manage the arguments received by the cli
"""
def __init__(self, args: Optional[List[str]], description: str,
no_default_config: bool = False) -> None:
def __init__(self, args: Optional[List[str]]) -> None:
self.args = args
self._parsed_arg: Optional[argparse.Namespace] = None
self.parser = argparse.ArgumentParser(description=description)
self._no_default_config = no_default_config
self.parser = argparse.ArgumentParser(description='Free, open source crypto trading bot')
def _load_args(self) -> None:
self._build_args(optionlist=ARGS_MAIN)
self._build_subcommands()
def get_parsed_arg(self) -> argparse.Namespace:
def get_parsed_arg(self) -> Dict[str, Any]:
"""
Return the list of arguments
:return: List[str] List of arguments
@ -69,7 +66,7 @@ class Arguments(object):
self._load_args()
self._parsed_arg = self._parse_args()
return self._parsed_arg
return vars(self._parsed_arg)
def _parse_args(self) -> argparse.Namespace:
"""
@ -77,12 +74,13 @@ class Arguments(object):
"""
parsed_arg = self.parser.parse_args(self.args)
# When no config is provided, but a config exists, use that configuration!
# Workaround issue in argparse with action='append' and default value
# (see https://bugs.python.org/issue16399)
# Allow no-config for certain commands (like downloading / plotting)
if (not self._no_default_config and parsed_arg.config is None
and not (hasattr(parsed_arg, 'func')
and parsed_arg.func.__name__ in NO_CONF_REQURIED)):
if (parsed_arg.config is None and ((Path.cwd() / constants.DEFAULT_CONFIG).is_file() or
not ('subparser' in parsed_arg and parsed_arg.subparser in NO_CONF_REQURIED))):
parsed_arg.config = [constants.DEFAULT_CONFIG]
return parsed_arg
@ -119,6 +117,7 @@ class Arguments(object):
hyperopt_cmd.set_defaults(func=start_hyperopt)
self._build_args(optionlist=ARGS_HYPEROPT, parser=hyperopt_cmd)
# add create-userdir subcommand
create_userdir_cmd = subparsers.add_parser('create-userdir',
help="Create user-data directory.")
create_userdir_cmd.set_defaults(func=start_create_userdir)
@ -139,3 +138,20 @@ class Arguments(object):
)
download_data_cmd.set_defaults(func=start_download_data)
self._build_args(optionlist=ARGS_DOWNLOAD_DATA, parser=download_data_cmd)
# Add Plotting subcommand
from freqtrade.plot.plot_utils import start_plot_dataframe, start_plot_profit
plot_dataframe_cmd = subparsers.add_parser(
'plot-dataframe',
help='Plot candles with indicators.'
)
plot_dataframe_cmd.set_defaults(func=start_plot_dataframe)
self._build_args(optionlist=ARGS_PLOT_DATAFRAME, parser=plot_dataframe_cmd)
# Plot profit
plot_profit_cmd = subparsers.add_parser(
'plot-profit',
help='Generate plot showing profits.'
)
plot_profit_cmd.set_defaults(func=start_plot_profit)
self._build_args(optionlist=ARGS_PLOT_PROFIT, parser=plot_profit_cmd)

13
freqtrade/configuration/check_exchange.py
View File

@ -5,6 +5,7 @@ from freqtrade import OperationalException
from freqtrade.exchange import (available_exchanges, get_exchange_bad_reason,
is_exchange_available, is_exchange_bad,
is_exchange_officially_supported)
from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
@ -19,9 +20,21 @@ def check_exchange(config: Dict[str, Any], check_for_bad: bool = True) -> bool:
raises an exception if the exchange if not supported by ccxt
and thus is not known for the Freqtrade at all.
"""
if config['runmode'] in [RunMode.PLOT] and not config.get('exchange', {}).get('name'):
# Skip checking exchange in plot mode, since it requires no exchange
return True
logger.info("Checking exchange...")
exchange = config.get('exchange', {}).get('name').lower()
if not exchange:
raise OperationalException(
f'This command requires a configured exchange. You should either use '
f'`--exchange <exchange_name>` or specify a configuration file via `--config`.\n'
f'The following exchanges are supported by ccxt: '
f'{", ".join(available_exchanges())}'
)
if not is_exchange_available(exchange):
raise OperationalException(
f'Exchange "{exchange}" is not supported by ccxt '

17
freqtrade/configuration/cli_options.py
View File

@ -107,13 +107,6 @@ AVAILABLE_CLI_OPTIONS = {
help='Specify stake_amount.',
type=float,
),
"refresh_pairs": Arg(
'-r', '--refresh-pairs-cached',
help='Refresh the pairs files in tests/testdata with the latest data from the '
'exchange. Use it if you want to run your optimization commands with '
'up-to-date data.',
action='store_true',
),
# Backtesting
"position_stacking": Arg(
'--eps', '--enable-position-stacking',
@ -292,14 +285,16 @@ AVAILABLE_CLI_OPTIONS = {
"indicators1": Arg(
'--indicators1',
help='Set indicators from your strategy you want in the first row of the graph. '
'Comma-separated list. Example: `ema3,ema5`. Default: `%(default)s`.',
default='sma,ema3,ema5',
'Space-separated list. Example: `ema3 ema5`. Default: `%(default)s`.',
default=['sma', 'ema3', 'ema5'],
nargs='+',
),
"indicators2": Arg(
'--indicators2',
help='Set indicators from your strategy you want in the third row of the graph. '
'Comma-separated list. Example: `fastd,fastk`. Default: `%(default)s`.',
default='macd,macdsignal',
'Space-separated list. Example: `fastd fastk`. Default: `%(default)s`.',
default=['macd', 'macdsignal'],
nargs='+',
),
"plot_limit": Arg(
'--plot-limit',

85
freqtrade/configuration/configuration.py
View File

@ -3,7 +3,6 @@ This module contains the configuration class
"""
import logging
import warnings
from argparse import Namespace
from copy import deepcopy
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
@ -22,13 +21,13 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
class Configuration(object):
class Configuration:
"""
Class to read and init the bot configuration
Reuse this class for the bot, backtesting, hyperopt and every script that required configuration
"""
def __init__(self, args: Namespace, runmode: RunMode = None) -> None:
def __init__(self, args: Dict[str, Any], runmode: RunMode = None) -> None:
self.args = args
self.config: Optional[Dict[str, Any]] = None
self.runmode = runmode
@ -50,9 +49,16 @@ class Configuration(object):
and merging their contents.
Files are loaded in sequence, parameters in later configuration files
override the same parameter from an earlier file (last definition wins).
Runs through the whole Configuration initialization, so all expected config entries
are available to interactive environments.
:param files: List of file paths
:return: configuration dictionary
"""
c = Configuration({"config": files}, RunMode.OTHER)
return c.get_config()
def load_from_files(self, files: List[str]) -> Dict[str, Any]:
# Keep this method as staticmethod, so it can be used from interactive environments
config: Dict[str, Any] = {}
@ -82,7 +88,10 @@ class Configuration(object):
:return: Configuration dictionary
"""
# Load all configs
config: Dict[str, Any] = Configuration.from_files(self.args.config)
config: Dict[str, Any] = self.load_from_files(self.args["config"])
# Keep a copy of the original configuration file
config['original_config'] = deepcopy(config)
self._process_common_options(config)
@ -107,13 +116,10 @@ class Configuration(object):
the -v/--verbose, --logfile options
"""
# Log level
if 'verbosity' in self.args and self.args.verbosity:
config.update({'verbosity': self.args.verbosity})
else:
config.update({'verbosity': 0})
config.update({'verbosity': self.args.get("verbosity", 0)})
if 'logfile' in self.args and self.args.logfile:
config.update({'logfile': self.args.logfile})
if 'logfile' in self.args and self.args["logfile"]:
config.update({'logfile': self.args["logfile"]})
setup_logging(config)
@ -122,15 +128,15 @@ class Configuration(object):
self._process_logging_options(config)
# Set strategy if not specified in config and or if it's non default
if self.args.strategy != constants.DEFAULT_STRATEGY or not config.get('strategy'):
config.update({'strategy': self.args.strategy})
if self.args.get("strategy") != constants.DEFAULT_STRATEGY or not config.get('strategy'):
config.update({'strategy': self.args.get("strategy")})
self._args_to_config(config, argname='strategy_path',
logstring='Using additional Strategy lookup path: {}')
if ('db_url' in self.args and self.args.db_url and
self.args.db_url != constants.DEFAULT_DB_PROD_URL):
config.update({'db_url': self.args.db_url})
if ('db_url' in self.args and self.args["db_url"] and
self.args["db_url"] != constants.DEFAULT_DB_PROD_URL):
config.update({'db_url': self.args["db_url"]})
logger.info('Parameter --db-url detected ...')
if config.get('dry_run', False):
@ -153,7 +159,7 @@ class Configuration(object):
config['max_open_trades'] = float('inf')
# Support for sd_notify
if 'sd_notify' in self.args and self.args.sd_notify:
if 'sd_notify' in self.args and self.args["sd_notify"]:
config['internals'].update({'sd_notify': True})
def _process_datadir_options(self, config: Dict[str, Any]) -> None:
@ -162,12 +168,12 @@ class Configuration(object):
--user-data, --datadir
"""
# Check exchange parameter here - otherwise `datadir` might be wrong.
if "exchange" in self.args and self.args.exchange:
config['exchange']['name'] = self.args.exchange
if "exchange" in self.args and self.args["exchange"]:
config['exchange']['name'] = self.args["exchange"]
logger.info(f"Using exchange {config['exchange']['name']}")
if 'user_data_dir' in self.args and self.args.user_data_dir:
config.update({'user_data_dir': self.args.user_data_dir})
if 'user_data_dir' in self.args and self.args["user_data_dir"]:
config.update({'user_data_dir': self.args["user_data_dir"]})
elif 'user_data_dir' not in config:
# Default to cwd/user_data (legacy option ...)
config.update({'user_data_dir': str(Path.cwd() / "user_data")})
@ -176,10 +182,7 @@ class Configuration(object):
config['user_data_dir'] = create_userdata_dir(config['user_data_dir'], create_dir=False)
logger.info('Using user-data directory: %s ...', config['user_data_dir'])
if 'datadir' in self.args and self.args.datadir:
config.update({'datadir': create_datadir(config, self.args.datadir)})
else:
config.update({'datadir': create_datadir(config, None)})
config.update({'datadir': create_datadir(config, self.args.get("datadir", None))})
logger.info('Using data directory: %s ...', config.get('datadir'))
def _process_optimize_options(self, config: Dict[str, Any]) -> None:
@ -192,12 +195,12 @@ class Configuration(object):
self._args_to_config(config, argname='position_stacking',
logstring='Parameter --enable-position-stacking detected ...')
if 'use_max_market_positions' in self.args and not self.args.use_max_market_positions:
if 'use_max_market_positions' in self.args and not self.args["use_max_market_positions"]:
config.update({'use_max_market_positions': False})
logger.info('Parameter --disable-max-market-positions detected ...')
logger.info('max_open_trades set to unlimited ...')
elif 'max_open_trades' in self.args and self.args.max_open_trades:
config.update({'max_open_trades': self.args.max_open_trades})
elif 'max_open_trades' in self.args and self.args["max_open_trades"]:
config.update({'max_open_trades': self.args["max_open_trades"]})
logger.info('Parameter --max_open_trades detected, '
'overriding max_open_trades to: %s ...', config.get('max_open_trades'))
else:
@ -212,12 +215,8 @@ class Configuration(object):
self._process_datadir_options(config)
self._args_to_config(config, argname='refresh_pairs',
logstring='Parameter -r/--refresh-pairs-cached detected ...',
deprecated_msg='-r/--refresh-pairs-cached will be removed soon.')
self._args_to_config(config, argname='strategy_list',
logstring='Using strategy list of {} Strategies', logfun=len)
logstring='Using strategy list of {} strategies', logfun=len)
self._args_to_config(config, argname='ticker_interval',
logstring='Overriding ticker interval with Command line argument')
@ -229,16 +228,16 @@ class Configuration(object):
logstring='Storing backtest results to {} ...')
# Edge section:
if 'stoploss_range' in self.args and self.args.stoploss_range:
txt_range = eval(self.args.stoploss_range)
if 'stoploss_range' in self.args and self.args["stoploss_range"]:
txt_range = eval(self.args["stoploss_range"])
config['edge'].update({'stoploss_range_min': txt_range[0]})
config['edge'].update({'stoploss_range_max': txt_range[1]})
config['edge'].update({'stoploss_range_step': txt_range[2]})
logger.info('Parameter --stoplosses detected: %s ...', self.args.stoploss_range)
logger.info('Parameter --stoplosses detected: %s ...', self.args["stoploss_range"])
# Hyperopt section
self._args_to_config(config, argname='hyperopt',
logstring='Using Hyperopt file {}')
logstring='Using Hyperopt class name: {}')
self._args_to_config(config, argname='hyperopt_path',
logstring='Using additional Hyperopt lookup path: {}')
@ -254,7 +253,7 @@ class Configuration(object):
self._args_to_config(config, argname='print_all',
logstring='Parameter --print-all detected ...')
if 'print_colorized' in self.args and not self.args.print_colorized:
if 'print_colorized' in self.args and not self.args["print_colorized"]:
logger.info('Parameter --no-color detected ...')
config.update({'print_colorized': False})
else:
@ -276,7 +275,7 @@ class Configuration(object):
logstring='Hyperopt continue: {}')
self._args_to_config(config, argname='hyperopt_loss',
logstring='Using loss function: {}')
logstring='Using Hyperopt loss class name: {}')
def _process_plot_options(self, config: Dict[str, Any]) -> None:
@ -324,9 +323,9 @@ class Configuration(object):
sample: logfun=len (prints the length of the found
configuration instead of the content)
"""
if argname in self.args and getattr(self.args, argname):
if argname in self.args and self.args[argname]:
config.update({argname: getattr(self.args, argname)})
config.update({argname: self.args[argname]})
if logfun:
logger.info(logstring.format(logfun(config[argname])))
else:
@ -346,8 +345,8 @@ class Configuration(object):
if "pairs" in config:
return
if "pairs_file" in self.args and self.args.pairs_file:
pairs_file = Path(self.args.pairs_file)
if "pairs_file" in self.args and self.args["pairs_file"]:
pairs_file = Path(self.args["pairs_file"])
logger.info(f'Reading pairs file "{pairs_file}".')
# Download pairs from the pairs file if no config is specified
# or if pairs file is specified explicitely
@ -358,7 +357,7 @@ class Configuration(object):
config['pairs'].sort()
return
if "config" in self.args and self.args.config:
if "config" in self.args and self.args["config"]:
logger.info("Using pairlist from configuration.")
config['pairs'] = config.get('exchange', {}).get('pair_whitelist')
else:

2
freqtrade/configuration/timerange.py
View File

@ -7,7 +7,7 @@ from typing import Optional
import arrow
class TimeRange():
class TimeRange:
"""
object defining timerange inputs.
[start/stop]type defines if [start/stop]ts shall be used.

2
freqtrade/constants.py
View File

@ -22,6 +22,7 @@ ORDERTYPE_POSSIBILITIES = ['limit', 'market']
ORDERTIF_POSSIBILITIES = ['gtc', 'fok', 'ioc']
AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList']
DRY_RUN_WALLET = 999.9
MATH_CLOSE_PREC = 1e-14 # Precision used for float comparisons
TICKER_INTERVALS = [
'1m', '3m', '5m', '15m', '30m',
@ -121,6 +122,7 @@ CONF_SCHEMA = {
'properties': {
'buy': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'sell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'emergencysell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'stoploss': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'stoploss_on_exchange': {'type': 'boolean'},
'stoploss_on_exchange_interval': {'type': 'number'}

2
freqtrade/data/__init__.py
View File

@ -2,7 +2,7 @@
Module to handle data operations for freqtrade
"""
# limit what's imported when using `from freqtrad.data import *``
# limit what's imported when using `from freqtrade.data import *`
__all__ = [
'converter'
]

13
freqtrade/data/btanalysis.py
View File

@ -112,16 +112,16 @@ def load_trades_from_db(db_url: str) -> pd.DataFrame:
return trades
def load_trades(config) -> pd.DataFrame:
def load_trades(source: str, db_url: str, exportfilename: str) -> pd.DataFrame:
"""
Based on configuration option "trade_source":
* loads data from DB (using `db_url`)
* loads data from backtestfile (using `exportfilename`)
"""
if config["trade_source"] == "DB":
return load_trades_from_db(config["db_url"])
elif config["trade_source"] == "file":
return load_backtest_data(Path(config["exportfilename"]))
if source == "DB":
return load_trades_from_db(db_url)
elif source == "file":
return load_backtest_data(Path(exportfilename))
def extract_trades_of_period(dataframe: pd.DataFrame, trades: pd.DataFrame) -> pd.DataFrame:
@ -157,7 +157,8 @@ def create_cum_profit(df: pd.DataFrame, trades: pd.DataFrame, col_name: str) ->
:param trades: DataFrame containing trades (requires columns close_time and profitperc)
:return: Returns df with one additional column, col_name, containing the cumulative profit.
"""
df[col_name] = trades.set_index('close_time')['profitperc'].cumsum()
# Use groupby/sum().cumsum() to avoid errors when multiple trades sold at the same candle.
df[col_name] = trades.groupby('close_time')['profitperc'].sum().cumsum()
# Set first value to 0
df.loc[df.iloc[0].name, col_name] = 0
# FFill to get continuous

6
freqtrade/data/dataprovider.py
View File

@ -17,7 +17,7 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
class DataProvider():
class DataProvider:
def __init__(self, config: dict, exchange: Exchange) -> None:
self._config = config
@ -65,9 +65,7 @@ class DataProvider():
"""
return load_pair_history(pair=pair,
ticker_interval=ticker_interval or self._config['ticker_interval'],
refresh_pairs=False,
datadir=Path(self._config['datadir']) if self._config.get(
'datadir') else None
datadir=Path(self._config['datadir'])
)
def get_pair_dataframe(self, pair: str, ticker_interval: str = None) -> DataFrame:

74
freqtrade/data/history.py
View File

@ -57,11 +57,11 @@ def trim_tickerlist(tickerlist: List[Dict], timerange: TimeRange) -> List[Dict]:
return tickerlist[start_index:stop_index]
def load_tickerdata_file(datadir: Optional[Path], pair: str, ticker_interval: str,
def load_tickerdata_file(datadir: Path, pair: str, ticker_interval: str,
timerange: Optional[TimeRange] = None) -> Optional[list]:
"""
Load a pair from file, either .json.gz or .json
:return: tickerlist or None if unsuccesful
:return: tickerlist or None if unsuccessful
"""
filename = pair_data_filename(datadir, pair, ticker_interval)
pairdata = misc.file_load_json(filename)
@ -73,7 +73,7 @@ def load_tickerdata_file(datadir: Optional[Path], pair: str, ticker_interval: st
return pairdata
def store_tickerdata_file(datadir: Optional[Path], pair: str,
def store_tickerdata_file(datadir: Path, pair: str,
ticker_interval: str, data: list, is_zip: bool = False):
"""
Stores tickerdata to file
@ -84,7 +84,7 @@ def store_tickerdata_file(datadir: Optional[Path], pair: str,
def load_pair_history(pair: str,
ticker_interval: str,
datadir: Optional[Path],
datadir: Path,
timerange: TimeRange = TimeRange(None, None, 0, 0),
refresh_pairs: bool = False,
exchange: Optional[Exchange] = None,
@ -129,37 +129,28 @@ def load_pair_history(pair: str,
else:
logger.warning(
f'No history data for pair: "{pair}", interval: {ticker_interval}. '
'Use --refresh-pairs-cached option or `freqtrade download-data` '
'script to download the data'
'Use `freqtrade download-data` to download the data'
)
return None
def load_data(datadir: Optional[Path],
def load_data(datadir: Path,
ticker_interval: str,
pairs: List[str],
refresh_pairs: bool = False,
exchange: Optional[Exchange] = None,
timerange: TimeRange = TimeRange(None, None, 0, 0),
fill_up_missing: bool = True,
live: bool = False
) -> Dict[str, DataFrame]:
"""
Loads ticker history data for a list of pairs the given parameters
Loads ticker history data for a list of pairs
:return: dict(<pair>:<tickerlist>)
TODO: refresh_pairs is still used by edge to keep the data uptodate.
This should be replaced in the future. Instead, writing the current candles to disk
from dataprovider should be implemented, as this would avoid loading ohlcv data twice.
exchange and refresh_pairs are then not needed here nor in load_pair_history.
"""
result: Dict[str, DataFrame] = {}
if live:
if exchange:
logger.info('Live: Downloading data for all defined pairs ...')
exchange.refresh_latest_ohlcv([(pair, ticker_interval) for pair in pairs])
result = {key[0]: value for key, value in exchange._klines.items() if value is not None}
else:
raise OperationalException(
"Exchange needs to be initialized when using live data."
)
else:
logger.info('Using local backtesting data ...')
for pair in pairs:
hist = load_pair_history(pair=pair, ticker_interval=ticker_interval,
@ -172,19 +163,13 @@ def load_data(datadir: Optional[Path],
return result
def make_testdata_path(datadir: Optional[Path]) -> Path:
"""Return the path where testdata files are stored"""
return datadir or (Path(__file__).parent.parent / "tests" / "testdata").resolve()
def pair_data_filename(datadir: Optional[Path], pair: str, ticker_interval: str) -> Path:
path = make_testdata_path(datadir)
def pair_data_filename(datadir: Path, pair: str, ticker_interval: str) -> Path:
pair_s = pair.replace("/", "_")
filename = path.joinpath(f'{pair_s}-{ticker_interval}.json')
filename = datadir.joinpath(f'{pair_s}-{ticker_interval}.json')
return filename
def load_cached_data_for_updating(datadir: Optional[Path], pair: str, ticker_interval: str,
def load_cached_data_for_updating(datadir: Path, pair: str, ticker_interval: str,
timerange: Optional[TimeRange]) -> Tuple[List[Any],
Optional[int]]:
"""
@ -224,7 +209,7 @@ def load_cached_data_for_updating(datadir: Optional[Path], pair: str, ticker_int
return (data, since_ms)
def download_pair_history(datadir: Optional[Path],
def download_pair_history(datadir: Path,
exchange: Optional[Exchange],
pair: str,
ticker_interval: str = '5m',
@ -280,6 +265,35 @@ def download_pair_history(datadir: Optional[Path],
return False
def refresh_backtest_ohlcv_data(exchange: Exchange, pairs: List[str], timeframes: List[str],
dl_path: Path, timerange: TimeRange,
erase=False) -> List[str]:
"""
Refresh stored ohlcv data for backtesting and hyperopt operations.
Used by freqtrade download-data
:return: Pairs not available
"""
pairs_not_available = []
for pair in pairs:
if pair not in exchange.markets:
pairs_not_available.append(pair)
logger.info(f"Skipping pair {pair}...")
continue
for ticker_interval in timeframes:
dl_file = pair_data_filename(dl_path, pair, ticker_interval)
if erase and dl_file.exists():
logger.info(
f'Deleting existing data for pair {pair}, interval {ticker_interval}.')
dl_file.unlink()
logger.info(f'Downloading pair {pair}, interval {ticker_interval}.')
download_pair_history(datadir=dl_path, exchange=exchange,
pair=pair, ticker_interval=str(ticker_interval),
timerange=timerange)
return pairs_not_available
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
"""
Get the maximum timeframe for the given backtest data

4
freqtrade/edge/__init__.py
View File

@ -28,7 +28,7 @@ class PairInfo(NamedTuple):
avg_trade_duration: float
class Edge():
class Edge:
"""
Calculates Win Rate, Risk Reward Ratio, Expectancy
against historical data for a give set of markets and a strategy
@ -93,7 +93,7 @@ class Edge():
logger.info('Using local backtesting data (using whitelist in given config) ...')
data = history.load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None,
datadir=Path(self.config['datadir']),
pairs=pairs,
ticker_interval=self.strategy.ticker_interval,
refresh_pairs=self._refresh_pairs,

56
freqtrade/exchange/binance.py
View File

@ -2,6 +2,10 @@
import logging
from typing import Dict
import ccxt
from freqtrade import (DependencyException, InvalidOrderException,
OperationalException, TemporaryError)
from freqtrade.exchange import Exchange
logger = logging.getLogger(__name__)
@ -25,3 +29,55 @@ class Binance(Exchange):
limit = min(list(filter(lambda x: limit <= x, limit_range)))
return super().get_order_book(pair, limit)
def stoploss_limit(self, pair: str, amount: float, stop_price: float, rate: float) -> Dict:
"""
creates a stoploss limit order.
this stoploss-limit is binance-specific.
It may work with a limited number of other exchanges, but this has not been tested yet.
"""
ordertype = "stop_loss_limit"
stop_price = self.symbol_price_prec(pair, stop_price)
# Ensure rate is less than stop price
if stop_price <= rate:
raise OperationalException(
'In stoploss limit order, stop price should be more than limit price')
if self._config['dry_run']:
dry_order = self.dry_run_order(
pair, ordertype, "sell", amount, stop_price)
return dry_order
try:
params = self._params.copy()
params.update({'stopPrice': stop_price})
amount = self.symbol_amount_prec(pair, amount)
rate = self.symbol_price_prec(pair, rate)
order = self._api.create_order(pair, ordertype, 'sell',
amount, rate, params)
logger.info('stoploss limit order added for %s. '
'stop price: %s. limit: %s', pair, stop_price, rate)
return order
except ccxt.InsufficientFunds as e:
raise DependencyException(
f'Insufficient funds to create {ordertype} sell order on market {pair}.'
f'Tried to sell amount {amount} at rate {rate}. '
f'Message: {e}') from e
except ccxt.InvalidOrder as e:
# Errors:
# `binance Order would trigger immediately.`
raise InvalidOrderException(
f'Could not create {ordertype} sell order on market {pair}. '
f'Tried to sell amount {amount} at rate {rate}. '
f'Message: {e}') from e
except (ccxt.NetworkError, ccxt.ExchangeError) as e:
raise TemporaryError(
f'Could not place sell order due to {e.__class__.__name__}. Message: {e}') from e
except ccxt.BaseError as e:
raise OperationalException(e) from e

55
freqtrade/exchange/exchange.py
View File

@ -14,6 +14,7 @@ from typing import Any, Dict, List, Optional, Tuple
import arrow
import ccxt
import ccxt.async_support as ccxt_async
from ccxt.base.decimal_to_precision import ROUND_UP, ROUND_DOWN
from pandas import DataFrame
from freqtrade import (DependencyException, InvalidOrderException,
@ -68,7 +69,7 @@ def retrier(f):
return wrapper
class Exchange(object):
class Exchange:
_config: Dict = {}
_params: Dict = {}
@ -320,7 +321,7 @@ class Exchange(object):
if (order_types.get("stoploss_on_exchange")
and not self._ft_has.get("stoploss_on_exchange", False)):
raise OperationalException(
'On exchange stoploss is not supported for %s.' % self.name
f'On exchange stoploss is not supported for {self.name}.'
)
def validate_order_time_in_force(self, order_time_in_force: Dict) -> None:
@ -366,7 +367,7 @@ class Exchange(object):
def dry_run_order(self, pair: str, ordertype: str, side: str, amount: float,
rate: float, params: Dict = {}) -> Dict[str, Any]:
order_id = f'dry_run_{side}_{randint(0, 10**6)}'
dry_order = { # TODO: additional entry should be added for stoploss limit
dry_order = {
"id": order_id,
'pair': pair,
'price': rate,
@ -381,6 +382,7 @@ class Exchange(object):
"info": {}
}
self._store_dry_order(dry_order)
# Copy order and close it - so the returned order is open unless it's a market order
return dry_order
def _store_dry_order(self, dry_order: Dict) -> None:
@ -391,6 +393,8 @@ class Exchange(object):
"filled": closed_order["amount"],
"remaining": 0
})
if closed_order["type"] in ["stop_loss_limit"]:
closed_order["info"].update({"stopPrice": closed_order["price"]})
self._dry_run_open_orders[closed_order["id"]] = closed_order
def create_order(self, pair: str, ordertype: str, side: str, amount: float,
@ -450,30 +454,14 @@ class Exchange(object):
def stoploss_limit(self, pair: str, amount: float, stop_price: float, rate: float) -> Dict:
"""
creates a stoploss limit order.
NOTICE: it is not supported by all exchanges. only binance is tested for now.
TODO: implementation maybe needs to be moved to the binance subclass
Since ccxt does not unify stoploss-limit orders yet, this needs to be implemented in each
exchange's subclass.
The exception below should never raise, since we disallow
starting the bot in validate_ordertypes()
Note: Changes to this interface need to be applied to all sub-classes too.
"""
ordertype = "stop_loss_limit"
stop_price = self.symbol_price_prec(pair, stop_price)
# Ensure rate is less than stop price
if stop_price <= rate:
raise OperationalException(
'In stoploss limit order, stop price should be more than limit price')
if self._config['dry_run']:
dry_order = self.dry_run_order(
pair, ordertype, "sell", amount, stop_price)
return dry_order
params = self._params.copy()
params.update({'stopPrice': stop_price})
order = self.create_order(pair, ordertype, 'sell', amount, rate, params)
logger.info('stoploss limit order added for %s. '
'stop price: %s. limit: %s', pair, stop_price, rate)
return order
raise OperationalException(f"stoploss_limit is not implemented for {self.name}.")
@retrier
def get_balance(self, currency: str) -> float:
@ -824,11 +812,9 @@ def timeframe_to_prev_date(timeframe: str, date: datetime = None) -> datetime:
"""
if not date:
date = datetime.now(timezone.utc)
timeframe_secs = timeframe_to_seconds(timeframe)
# Get offset based on timerame_secs
offset = date.timestamp() % timeframe_secs
# Subtract seconds passed since last offset
new_timestamp = date.timestamp() - offset
new_timestamp = ccxt.Exchange.round_timeframe(timeframe, date.timestamp() * 1000,
ROUND_DOWN) // 1000
return datetime.fromtimestamp(new_timestamp, tz=timezone.utc)
@ -839,9 +825,8 @@ def timeframe_to_next_date(timeframe: str, date: datetime = None) -> datetime:
:param date: date to use. Defaults to utcnow()
:returns: date of next candle (with utc timezone)
"""
prevdate = timeframe_to_prev_date(timeframe, date)
timeframe_secs = timeframe_to_seconds(timeframe)
# Add one interval to previous candle
new_timestamp = prevdate.timestamp() + timeframe_secs
if not date:
date = datetime.now(timezone.utc)
new_timestamp = ccxt.Exchange.round_timeframe(timeframe, date.timestamp() * 1000,
ROUND_UP) // 1000
return datetime.fromtimestamp(new_timestamp, tz=timezone.utc)

34
freqtrade/exchange/kraken.py
View File

@ -2,7 +2,11 @@
import logging
from typing import Dict
import ccxt
from freqtrade import OperationalException, TemporaryError
from freqtrade.exchange import Exchange
from freqtrade.exchange.exchange import retrier
logger = logging.getLogger(__name__)
@ -10,3 +14,33 @@ logger = logging.getLogger(__name__)
class Kraken(Exchange):
_params: Dict = {"trading_agreement": "agree"}
@retrier
def get_balances(self) -> dict:
if self._config['dry_run']:
return {}
try:
balances = self._api.fetch_balance()
# Remove additional info from ccxt results
balances.pop("info", None)
balances.pop("free", None)
balances.pop("total", None)
balances.pop("used", None)
orders = self._api.fetch_open_orders()
order_list = [(x["symbol"].split("/")[0 if x["side"] == "sell" else 1],
x["remaining"],
# Don't remove the below comment, this can be important for debuggung
# x["side"], x["amount"],
) for x in orders]
for bal in balances:
balances[bal]['used'] = sum(order[1] for order in order_list if order[0] == bal)
balances[bal]['free'] = balances[bal]['total'] - balances[bal]['used']
return balances
except (ccxt.NetworkError, ccxt.ExchangeError) as e:
raise TemporaryError(
f'Could not get balance due to {e.__class__.__name__}. Message: {e}') from e
except ccxt.BaseError as e:
raise OperationalException(e) from e

119
freqtrade/freqtradebot.py
View File

@ -6,6 +6,7 @@ import copy
import logging
import traceback
from datetime import datetime
from math import isclose
from typing import Any, Dict, List, Optional, Tuple
import arrow
@ -21,7 +22,7 @@ from freqtrade.exchange import timeframe_to_minutes, timeframe_to_next_date
from freqtrade.persistence import Trade
from freqtrade.rpc import RPCManager, RPCMessageType
from freqtrade.resolvers import ExchangeResolver, StrategyResolver, PairListResolver
from freqtrade.state import State, RunMode
from freqtrade.state import State
from freqtrade.strategy.interface import SellType, IStrategy
from freqtrade.wallets import Wallets
@ -29,7 +30,7 @@ from freqtrade.wallets import Wallets
logger = logging.getLogger(__name__)
class FreqtradeBot(object):
class FreqtradeBot:
"""
Freqtrade is the main class of the bot.
This is from here the bot start its logic.
@ -79,12 +80,6 @@ class FreqtradeBot(object):
persistence.init(self.config.get('db_url', None),
clean_open_orders=self.config.get('dry_run', False))
# Stoploss on exchange does not make sense, therefore we need to disable that.
if (self.dataprovider.runmode == RunMode.DRY_RUN and
self.strategy.order_types.get('stoploss_on_exchange', False)):
logger.info("Disabling stoploss_on_exchange during dry-run.")
self.strategy.order_types['stoploss_on_exchange'] = False
config['order_types']['stoploss_on_exchange'] = False
# Set initial bot state from config
initial_state = self.config.get('initial_state')
self.state = State[initial_state.upper()] if initial_state else State.STOPPED
@ -216,7 +211,7 @@ class FreqtradeBot(object):
if stake_amount == constants.UNLIMITED_STAKE_AMOUNT:
open_trades = len(Trade.get_open_trades())
if open_trades >= self.config['max_open_trades']:
logger.warning('Can\'t open a new trade: max number of trades is reached')
logger.warning("Can't open a new trade: max number of trades is reached")
return None
return available_amount / (self.config['max_open_trades'] - open_trades)
@ -351,8 +346,8 @@ class FreqtradeBot(object):
min_stake_amount = self._get_min_pair_stake_amount(pair_s, buy_limit_requested)
if min_stake_amount is not None and min_stake_amount > stake_amount:
logger.warning(
f'Can\'t open a new trade for {pair_s}: stake amount '
f'is too small ({stake_amount} < {min_stake_amount})'
f"Can't open a new trade for {pair_s}: stake amount "
f"is too small ({stake_amount} < {min_stake_amount})"
)
return False
@ -516,7 +511,7 @@ class FreqtradeBot(object):
trade.pair.startswith(exectrade['fee']['currency'])):
fee_abs += exectrade['fee']['cost']
if amount != order_amount:
if not isclose(amount, order_amount, abs_tol=constants.MATH_CLOSE_PREC):
logger.warning(f"Amount {amount} does not match amount {trade.amount}")
raise OperationalException("Half bought? Amounts don't match")
real_amount = amount - fee_abs
@ -541,7 +536,7 @@ class FreqtradeBot(object):
# Try update amount (binance-fix)
try:
new_amount = self.get_real_amount(trade, order)
if order['amount'] != new_amount:
if not isclose(order['amount'], new_amount, abs_tol=constants.MATH_CLOSE_PREC):
order['amount'] = new_amount
# Fee was applied, so set to 0
trade.fee_open = 0
@ -617,6 +612,33 @@ class FreqtradeBot(object):
logger.debug('Found no sell signal for %s.', trade)
return False
def create_stoploss_order(self, trade: Trade, stop_price: float, rate: float) -> bool:
"""
Abstracts creating stoploss orders from the logic.
Handles errors and updates the trade database object.
Force-sells the pair (using EmergencySell reason) in case of Problems creating the order.
:return: True if the order succeeded, and False in case of problems.
"""
# Limit price threshold: As limit price should always be below price
LIMIT_PRICE_PCT = 0.99
try:
stoploss_order = self.exchange.stoploss_limit(pair=trade.pair, amount=trade.amount,
stop_price=stop_price,
rate=rate * LIMIT_PRICE_PCT)
trade.stoploss_order_id = str(stoploss_order['id'])
return True
except InvalidOrderException as e:
trade.stoploss_order_id = None
logger.error(f'Unable to place a stoploss order on exchange. {e}')
logger.warning('Selling the trade forcefully')
self.execute_sell(trade, trade.stop_loss, sell_reason=SellType.EMERGENCY_SELL)
except DependencyException:
trade.stoploss_order_id = None
logger.exception('Unable to place a stoploss order on exchange.')
return False
def handle_stoploss_on_exchange(self, trade: Trade) -> bool:
"""
Check if trade is fulfilled in which case the stoploss
@ -635,49 +657,25 @@ class FreqtradeBot(object):
except InvalidOrderException as exception:
logger.warning('Unable to fetch stoploss order: %s', exception)
# If trade open order id does not exist: buy order is fulfilled
buy_order_fulfilled = not trade.open_order_id
# Limit price threshold: As limit price should always be below price
limit_price_pct = 0.99
# If buy order is fulfilled but there is no stoploss, we add a stoploss on exchange
if (buy_order_fulfilled and not stoploss_order):
if self.edge:
stoploss = self.edge.stoploss(pair=trade.pair)
else:
stoploss = self.strategy.stoploss
if (not trade.open_order_id and not stoploss_order):
stoploss = self.edge.stoploss(pair=trade.pair) if self.edge else self.strategy.stoploss
stop_price = trade.open_rate * (1 + stoploss)
# limit price should be less than stop price.
limit_price = stop_price * limit_price_pct
try:
stoploss_order_id = self.exchange.stoploss_limit(
pair=trade.pair, amount=trade.amount, stop_price=stop_price, rate=limit_price
)['id']
trade.stoploss_order_id = str(stoploss_order_id)
if self.create_stoploss_order(trade=trade, stop_price=stop_price, rate=stop_price):
trade.stoploss_last_update = datetime.now()
return False
except DependencyException as exception:
trade.stoploss_order_id = None
logger.warning('Unable to place a stoploss order on exchange: %s', exception)
# If stoploss order is canceled for some reason we add it
if stoploss_order and stoploss_order['status'] == 'canceled':
try:
stoploss_order_id = self.exchange.stoploss_limit(
pair=trade.pair, amount=trade.amount,
stop_price=trade.stop_loss, rate=trade.stop_loss * limit_price_pct
)['id']
trade.stoploss_order_id = str(stoploss_order_id)
if self.create_stoploss_order(trade=trade, stop_price=trade.stop_loss,
rate=trade.stop_loss):
return False
except DependencyException as exception:
else:
trade.stoploss_order_id = None
logger.warning('Stoploss order was cancelled, '
'but unable to recreate one: %s', exception)
logger.warning('Stoploss order was cancelled, but unable to recreate one.')
# We check if stoploss order is fulfilled
if stoploss_order and stoploss_order['status'] == 'closed':
@ -686,7 +684,7 @@ class FreqtradeBot(object):
# Lock pair for one candle to prevent immediate rebuys
self.strategy.lock_pair(trade.pair,
timeframe_to_next_date(self.config['ticker_interval']))
self._notify_sell(trade)
self._notify_sell(trade, "stoploss")
return True
# Finally we check if stoploss on exchange should be moved up because of trailing.
@ -720,16 +718,12 @@ class FreqtradeBot(object):
logger.exception(f"Could not cancel stoploss order {order['id']} "
f"for pair {trade.pair}")
try:
# creating the new one
stoploss_order_id = self.exchange.stoploss_limit(
pair=trade.pair, amount=trade.amount,
stop_price=trade.stop_loss, rate=trade.stop_loss * 0.99
)['id']
trade.stoploss_order_id = str(stoploss_order_id)
except DependencyException:
trade.stoploss_order_id = None
logger.exception(f"Could not create trailing stoploss order "
# Create new stoploss order
if self.create_stoploss_order(trade=trade, stop_price=trade.stop_loss,
rate=trade.stop_loss):
return False
else:
logger.warning(f"Could not create trailing stoploss order "
f"for pair {trade.pair}.")
def _check_and_execute_sell(self, trade: Trade, sell_rate: float,
@ -883,9 +877,14 @@ class FreqtradeBot(object):
except InvalidOrderException:
logger.exception(f"Could not cancel stoploss order {trade.stoploss_order_id}")
ordertype = self.strategy.order_types[sell_type]
if sell_reason == SellType.EMERGENCY_SELL:
# Emergencysells (default to market!)
ordertype = self.strategy.order_types.get("emergencysell", "market")
# Execute sell and update trade record
order = self.exchange.sell(pair=str(trade.pair),
ordertype=self.strategy.order_types[sell_type],
ordertype=ordertype,
amount=trade.amount, rate=limit,
time_in_force=self.strategy.order_time_in_force['sell']
)
@ -901,9 +900,9 @@ class FreqtradeBot(object):
# Lock pair for one candle to prevent immediate rebuys
self.strategy.lock_pair(trade.pair, timeframe_to_next_date(self.config['ticker_interval']))
self._notify_sell(trade)
self._notify_sell(trade, ordertype)
def _notify_sell(self, trade: Trade):
def _notify_sell(self, trade: Trade, order_type: str):
"""
Sends rpc notification when a sell occured.
"""
@ -920,7 +919,7 @@ class FreqtradeBot(object):
'pair': trade.pair,
'gain': gain,
'limit': trade.close_rate_requested,
'order_type': self.strategy.order_types['sell'],
'order_type': order_type,
'amount': trade.amount,
'open_rate': trade.open_rate,
'current_rate': current_rate,

12
freqtrade/main.py
View File

@ -11,7 +11,6 @@ if sys.version_info < (3, 6):
# flake8: noqa E402
import logging
from argparse import Namespace
from typing import Any, List
from freqtrade import OperationalException
@ -31,16 +30,13 @@ def main(sysargv: List[str] = None) -> None:
return_code: Any = 1
worker = None
try:
arguments = Arguments(
sysargv,
'Free, open source crypto trading bot'
)
args: Namespace = arguments.get_parsed_arg()
arguments = Arguments(sysargv)
args = arguments.get_parsed_arg()
# A subcommand has been issued.
# Means if Backtesting or Hyperopt have been called we exit the bot
if hasattr(args, 'func'):
args.func(args)
if 'func' in args:
args['func'](args)
# TODO: fetch return_code as returned by the command function here
return_code = 0
else:

7
freqtrade/misc.py
View File

@ -114,3 +114,10 @@ def deep_merge_dicts(source, destination):
destination[key] = value
return destination
def round_dict(d, n):
"""
Rounds float values in the dict to n digits after the decimal point.
"""
return {k: (round(v, n) if isinstance(v, float) else v) for k, v in d.items()}

29
freqtrade/optimize/__init__.py
View File

@ -1,10 +1,7 @@
import logging
from argparse import Namespace
from typing import Any, Dict
from filelock import FileLock, Timeout
from freqtrade import DependencyException, constants
from freqtrade import DependencyException, constants, OperationalException
from freqtrade.state import RunMode
from freqtrade.utils import setup_utils_configuration
@ -12,7 +9,7 @@ from freqtrade.utils import setup_utils_configuration
logger = logging.getLogger(__name__)
def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
def setup_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
"""
Prepare the configuration for the Hyperopt module
:param args: Cli args from Arguments()
@ -25,20 +22,10 @@ def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
raise DependencyException('stake amount could not be "%s" for backtesting' %
constants.UNLIMITED_STAKE_AMOUNT)
if method == RunMode.HYPEROPT:
# Special cases for Hyperopt
if config.get('strategy') and config.get('strategy') != 'DefaultStrategy':
logger.error("Please don't use --strategy for hyperopt.")
logger.error(
"Read the documentation at "
"https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md "
"to understand how to configure hyperopt.")
raise DependencyException("--strategy configured but not supported for hyperopt")
return config
def start_backtesting(args: Namespace) -> None:
def start_backtesting(args: Dict[str, Any]) -> None:
"""
Start Backtesting script
:param args: Cli args from Arguments()
@ -57,15 +44,19 @@ def start_backtesting(args: Namespace) -> None:
backtesting.start()
def start_hyperopt(args: Namespace) -> None:
def start_hyperopt(args: Dict[str, Any]) -> None:
"""
Start hyperopt script
:param args: Cli args from Arguments()
:return: None
"""
# Import here to avoid loading hyperopt module when it's not used
try:
from filelock import FileLock, Timeout
from freqtrade.optimize.hyperopt import Hyperopt
except ImportError as e:
raise OperationalException(
f"{e}. Please ensure that the hyperopt dependencies are installed.") from e
# Initialize configuration
config = setup_configuration(args, RunMode.HYPEROPT)
@ -95,7 +86,7 @@ def start_hyperopt(args: Namespace) -> None:
# Same in Edge and Backtesting start() functions.
def start_edge(args: Namespace) -> None:
def start_edge(args: Dict[str, Any]) -> None:
"""
Start Edge script
:param args: Cli args from Arguments()

42
freqtrade/optimize/backtesting.py
View File

@ -44,7 +44,7 @@ class BacktestResult(NamedTuple):
sell_reason: SellType
class Backtesting(object):
class Backtesting:
"""
Backtesting class, this class contains all the logic to run a backtest
@ -81,6 +81,12 @@ class Backtesting(object):
# No strategy list specified, only one strategy
self.strategylist.append(StrategyResolver(self.config).strategy)
if "ticker_interval" not in self.config:
raise OperationalException("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`")
self.ticker_interval = str(self.config.get('ticker_interval'))
self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
# Load one (first) strategy
self._set_strategy(self.strategylist[0])
@ -89,14 +95,6 @@ class Backtesting(object):
Load strategy into backtesting
"""
self.strategy = strategy
if "ticker_interval" not in self.config:
raise OperationalException("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`")
self.ticker_interval = self.config.get('ticker_interval')
self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
self.advise_buy = strategy.advise_buy
self.advise_sell = strategy.advise_sell
# Set stoploss_on_exchange to false for backtesting,
# since a "perfect" stoploss-sell is assumed anyway
# And the regular "stoploss" function would not apply to that case
@ -219,8 +217,8 @@ class Backtesting(object):
for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
ticker_data = self.strategy.advise_sell(
self.strategy.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
# to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
@ -239,14 +237,16 @@ class Backtesting(object):
stake_amount: float, max_open_trades: int) -> Optional[BacktestResult]:
trade = Trade(
pair=pair,
open_rate=buy_row.open,
open_date=buy_row.date,
stake_amount=stake_amount,
amount=stake_amount / buy_row.open,
fee_open=self.fee,
fee_close=self.fee
fee_close=self.fee,
is_open=True,
)
logger.debug(f"{pair} - Backtesting emulates creation of new trade: {trade}.")
# calculate win/lose forwards from buy point
for sell_row in partial_ticker:
if max_open_trades > 0:
@ -289,7 +289,7 @@ class Backtesting(object):
if partial_ticker:
# no sell condition found - trade stil open at end of backtest period
sell_row = partial_ticker[-1]
btr = BacktestResult(pair=pair,
bt_res = BacktestResult(pair=pair,
profit_percent=trade.calc_profit_percent(rate=sell_row.open),
profit_abs=trade.calc_profit(rate=sell_row.open),
open_time=buy_row.date,
@ -303,9 +303,11 @@ class Backtesting(object):
close_rate=sell_row.open,
sell_reason=SellType.FORCE_SELL
)
logger.debug('Force_selling still open trade %s with %s perc - %s', btr.pair,
btr.profit_percent, btr.profit_abs)
return btr
logger.debug(f"{pair} - Force selling still open trade, "
f"profit percent: {bt_res.profit_percent}, "
f"profit abs: {bt_res.profit_abs}")
return bt_res
return None
def backtest(self, args: Dict) -> DataFrame:
@ -384,6 +386,8 @@ class Backtesting(object):
max_open_trades)
if trade_entry:
logger.debug(f"{pair} - Locking pair till "
f"close_time={trade_entry.close_time}")
lock_pair_until[pair] = trade_entry.close_time
trades.append(trade_entry)
else:
@ -407,11 +411,9 @@ class Backtesting(object):
timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange')))
data = history.load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None,
datadir=Path(self.config['datadir']),
pairs=pairs,
ticker_interval=self.ticker_interval,
refresh_pairs=self.config.get('refresh_pairs', False),
exchange=self.exchange,
timerange=timerange,
)

5
freqtrade/optimize/edge_cli.py
View File

@ -16,7 +16,7 @@ from freqtrade.resolvers import StrategyResolver
logger = logging.getLogger(__name__)
class EdgeCli(object):
class EdgeCli:
"""
EdgeCli class, this class contains all the logic to run edge backtesting
@ -39,7 +39,8 @@ class EdgeCli(object):
self.strategy = StrategyResolver(self.config).strategy
self.edge = Edge(config, self.exchange, self.strategy)
self.edge._refresh_pairs = self.config.get('refresh_pairs', False)
# Set refresh_pairs to false for edge-cli (it must be true for edge)
self.edge._refresh_pairs = False
self.timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange')))

94
freqtrade/optimize/hyperopt.py
View File

@ -24,8 +24,10 @@ from skopt.space import Dimension
from freqtrade.configuration import TimeRange
from freqtrade.data.history import load_data, get_timeframe
from freqtrade.misc import round_dict
from freqtrade.optimize.backtesting import Backtesting
# Import IHyperOptLoss to allow users import from this file
# Import IHyperOpt and IHyperOptLoss to allow unpickling classes from these modules
from freqtrade.optimize.hyperopt_interface import IHyperOpt # noqa: F4
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F4
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
@ -34,6 +36,11 @@ logger = logging.getLogger(__name__)
INITIAL_POINTS = 30
# Keep no more than 2*SKOPT_MODELS_MAX_NUM models
# in the skopt models list
SKOPT_MODELS_MAX_NUM = 10
MAX_LOSS = 100000 # just a big enough number to be bad result in loss optimization
@ -47,10 +54,11 @@ class Hyperopt:
"""
def __init__(self, config: Dict[str, Any]) -> None:
self.config = config
self.backtesting = Backtesting(self.config)
self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
self.backtesting = Backtesting(self.config)
self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
@ -71,11 +79,15 @@ class Hyperopt:
self.trials: List = []
# Populate functions here (hasattr is slow so should not be run during "regular" operations)
if hasattr(self.custom_hyperopt, 'populate_indicators'):
self.backtesting.strategy.advise_indicators = \
self.custom_hyperopt.populate_indicators # type: ignore
if hasattr(self.custom_hyperopt, 'populate_buy_trend'):
self.backtesting.advise_buy = self.custom_hyperopt.populate_buy_trend # type: ignore
self.backtesting.strategy.advise_buy = \
self.custom_hyperopt.populate_buy_trend # type: ignore
if hasattr(self.custom_hyperopt, 'populate_sell_trend'):
self.backtesting.advise_sell = self.custom_hyperopt.populate_sell_trend # type: ignore
self.backtesting.strategy.advise_sell = \
self.custom_hyperopt.populate_sell_trend # type: ignore
# Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
if self.config.get('use_max_market_positions', True):
@ -83,7 +95,7 @@ class Hyperopt:
else:
logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
self.max_open_trades = 0
self.position_stacking = self.config.get('position_stacking', False),
self.position_stacking = self.config.get('position_stacking', False)
if self.has_space('sell'):
# Make sure experimental is enabled
@ -107,7 +119,9 @@ class Hyperopt:
p.unlink()
def get_args(self, params):
dimensions = self.hyperopt_space()
dimensions = self.dimensions
# Ensure the number of dimensions match
# the number of parameters in the list x.
if len(params) != len(dimensions):
@ -124,14 +138,14 @@ class Hyperopt:
Save hyperopt trials to file
"""
if self.trials:
logger.info('Saving %d evaluations to \'%s\'', len(self.trials), self.trials_file)
logger.info("Saving %d evaluations to '%s'", len(self.trials), self.trials_file)
dump(self.trials, self.trials_file)
def read_trials(self) -> List:
"""
Read hyperopt trials file
"""
logger.info('Reading Trials from \'%s\'', self.trials_file)
logger.info("Reading Trials from '%s'", self.trials_file)
trials = load(self.trials_file)
self.trials_file.unlink()
return trials
@ -178,9 +192,11 @@ class Hyperopt:
indent=4)
if self.has_space('roi'):
print("ROI table:")
pprint(self.custom_hyperopt.generate_roi_table(params), indent=4)
# Round printed values to 5 digits after the decimal point
pprint(round_dict(self.custom_hyperopt.generate_roi_table(params), 5), indent=4)
if self.has_space('stoploss'):
print(f"Stoploss: {params.get('stoploss')}")
# Also round to 5 digits after the decimal point
print(f"Stoploss: {round(params.get('stoploss'), 5)}")
def log_results(self, results) -> None:
"""
@ -244,20 +260,24 @@ class Hyperopt:
spaces += self.custom_hyperopt.stoploss_space()
return spaces
def generate_optimizer(self, _params: Dict) -> Dict:
def generate_optimizer(self, _params: Dict, iteration=None) -> Dict:
"""
Used Optimize function. Called once per epoch to optimize whatever is configured.
Keep this function as optimized as possible!
"""
params = self.get_args(_params)
if self.has_space('roi'):
self.backtesting.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params)
self.backtesting.strategy.minimal_roi = \
self.custom_hyperopt.generate_roi_table(params)
if self.has_space('buy'):
self.backtesting.advise_buy = self.custom_hyperopt.buy_strategy_generator(params)
self.backtesting.strategy.advise_buy = \
self.custom_hyperopt.buy_strategy_generator(params)
if self.has_space('sell'):
self.backtesting.advise_sell = self.custom_hyperopt.sell_strategy_generator(params)
self.backtesting.strategy.advise_sell = \
self.custom_hyperopt.sell_strategy_generator(params)
if self.has_space('stoploss'):
self.backtesting.strategy.stoploss = params['stoploss']
@ -318,9 +338,9 @@ class Hyperopt:
f'Total profit {total_profit: 11.8f} {stake_cur} '
f'({profit: 7.2f}Σ%). Avg duration {duration:5.1f} mins.')
def get_optimizer(self, cpu_count) -> Optimizer:
def get_optimizer(self, dimensions, cpu_count) -> Optimizer:
return Optimizer(
self.hyperopt_space(),
dimensions,
base_estimator="ET",
acq_optimizer="auto",
n_initial_points=INITIAL_POINTS,
@ -328,9 +348,26 @@ class Hyperopt:
random_state=self.config.get('hyperopt_random_state', None)
)
def run_optimizer_parallel(self, parallel, asked) -> List:
def fix_optimizer_models_list(self):
"""
WORKAROUND: Since skopt is not actively supported, this resolves problems with skopt
memory usage, see also: https://github.com/scikit-optimize/scikit-optimize/pull/746
This may cease working when skopt updates if implementation of this intrinsic
part changes.
"""
n = len(self.opt.models) - SKOPT_MODELS_MAX_NUM
# Keep no more than 2*SKOPT_MODELS_MAX_NUM models in the skopt models list,
# remove the old ones. These are actually of no use, the current model
# from the estimator is the only one used in the skopt optimizer.
# Freqtrade code also does not inspect details of the models.
if n >= SKOPT_MODELS_MAX_NUM:
logger.debug(f"Fixing skopt models list, removing {n} old items...")
del self.opt.models[0:n]
def run_optimizer_parallel(self, parallel, asked, i) -> List:
return parallel(delayed(
wrap_non_picklable_objects(self.generate_optimizer))(v) for v in asked)
wrap_non_picklable_objects(self.generate_optimizer))(v, i) for v in asked)
def load_previous_results(self):
""" read trials file if we have one """
@ -345,11 +382,9 @@ class Hyperopt:
timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange')))
data = load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None,
datadir=Path(self.config['datadir']),
pairs=self.config['exchange']['pair_whitelist'],
ticker_interval=self.backtesting.ticker_interval,
refresh_pairs=self.config.get('refresh_pairs', False),
exchange=self.backtesting.exchange,
timerange=timerange
)
@ -366,9 +401,6 @@ class Hyperopt:
(max_date - min_date).days
)
self.backtesting.strategy.advise_indicators = \
self.custom_hyperopt.populate_indicators # type: ignore
preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data)
dump(preprocessed, self.tickerdata_pickle)
@ -379,11 +411,12 @@ class Hyperopt:
self.load_previous_results()
cpus = cpu_count()
logger.info(f'Found {cpus} CPU cores. Let\'s make them scream!')
logger.info(f"Found {cpus} CPU cores. Let's make them scream!")
config_jobs = self.config.get('hyperopt_jobs', -1)
logger.info(f'Number of parallel jobs set as: {config_jobs}')
opt = self.get_optimizer(config_jobs)
self.dimensions = self.hyperopt_space()
self.opt = self.get_optimizer(self.dimensions, config_jobs)
if self.config.get('print_colorized', False):
colorama_init(autoreset=True)
@ -394,9 +427,10 @@ class Hyperopt:
logger.info(f'Effective number of parallel workers used: {jobs}')
EVALS = max(self.total_epochs // jobs, 1)
for i in range(EVALS):
asked = opt.ask(n_points=jobs)
f_val = self.run_optimizer_parallel(parallel, asked)
opt.tell(asked, [v['loss'] for v in f_val])
asked = self.opt.ask(n_points=jobs)
f_val = self.run_optimizer_parallel(parallel, asked, i)
self.opt.tell(asked, [v['loss'] for v in f_val])
self.fix_optimizer_models_list()
for j in range(jobs):
current = i * jobs + j
val = f_val[j]

140
freqtrade/optimize/hyperopt_interface.py
View File

@ -2,6 +2,8 @@
IHyperOpt interface
This module defines the interface to apply for hyperopts
"""
import logging
import math
from abc import ABC, abstractmethod
from typing import Dict, Any, Callable, List
@ -9,19 +11,37 @@ from typing import Dict, Any, Callable, List
from pandas import DataFrame
from skopt.space import Dimension, Integer, Real
from freqtrade import OperationalException
from freqtrade.exchange import timeframe_to_minutes
from freqtrade.misc import round_dict
logger = logging.getLogger(__name__)
def _format_exception_message(method: str, space: str) -> str:
return (f"The '{space}' space is included into the hyperoptimization "
f"but {method}() method is not found in your "
f"custom Hyperopt class. You should either implement this "
f"method or remove the '{space}' space from hyperoptimization.")
class IHyperOpt(ABC):
"""
Interface for freqtrade hyperopts
Defines the mandatory structure must follow any custom strategies
Defines the mandatory structure must follow any custom hyperopts
Attributes you can use:
minimal_roi -> Dict: Minimal ROI designed for the strategy
stoploss -> float: optimal stoploss designed for the strategy
Class attributes you can use:
ticker_interval -> int: value of the ticker interval to use for the strategy
"""
ticker_interval: str
def __init__(self, config: dict) -> None:
self.config = config
# Assign ticker_interval to be used in hyperopt
IHyperOpt.ticker_interval = str(config['ticker_interval'])
@staticmethod
@abstractmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -32,32 +52,32 @@ class IHyperOpt(ABC):
"""
@staticmethod
@abstractmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Create a buy strategy generator.
"""
raise OperationalException(_format_exception_message('buy_strategy_generator', 'buy'))
@staticmethod
@abstractmethod
def sell_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Create a sell strategy generator.
"""
raise OperationalException(_format_exception_message('sell_strategy_generator', 'sell'))
@staticmethod
@abstractmethod
def indicator_space() -> List[Dimension]:
"""
Create an indicator space.
"""
raise OperationalException(_format_exception_message('indicator_space', 'buy'))
@staticmethod
@abstractmethod
def sell_indicator_space() -> List[Dimension]:
"""
Create a sell indicator space.
"""
raise OperationalException(_format_exception_message('sell_indicator_space', 'sell'))
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
@ -75,6 +95,83 @@ class IHyperOpt(ABC):
return roi_table
@staticmethod
def roi_space() -> List[Dimension]:
"""
Create a ROI space.
Defines values to search for each ROI steps.
This method implements adaptive roi hyperspace with varied
ranges for parameters which automatically adapts to the
ticker interval used.
It's used by Freqtrade by default, if no custom roi_space method is defined.
"""
# Default scaling coefficients for the roi hyperspace. Can be changed
# to adjust resulting ranges of the ROI tables.
# Increase if you need wider ranges in the roi hyperspace, decrease if shorter
# ranges are needed.
roi_t_alpha = 1.0
roi_p_alpha = 1.0
ticker_interval_mins = timeframe_to_minutes(IHyperOpt.ticker_interval)
# We define here limits for the ROI space parameters automagically adapted to the
# ticker_interval used by the bot:
#
# * 'roi_t' (limits for the time intervals in the ROI tables) components
# are scaled linearly.
# * 'roi_p' (limits for the ROI value steps) components are scaled logarithmically.
#
# The scaling is designed so that it maps exactly to the legacy Freqtrade roi_space()
# method for the 5m ticker interval.
roi_t_scale = ticker_interval_mins / 5
roi_p_scale = math.log1p(ticker_interval_mins) / math.log1p(5)
roi_limits = {
'roi_t1_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t1_max': int(120 * roi_t_scale * roi_t_alpha),
'roi_t2_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t2_max': int(60 * roi_t_scale * roi_t_alpha),
'roi_t3_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t3_max': int(40 * roi_t_scale * roi_t_alpha),
'roi_p1_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p1_max': 0.04 * roi_p_scale * roi_p_alpha,
'roi_p2_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p2_max': 0.07 * roi_p_scale * roi_p_alpha,
'roi_p3_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p3_max': 0.20 * roi_p_scale * roi_p_alpha,
}
logger.debug(f"Using roi space limits: {roi_limits}")
p = {
'roi_t1': roi_limits['roi_t1_min'],
'roi_t2': roi_limits['roi_t2_min'],
'roi_t3': roi_limits['roi_t3_min'],
'roi_p1': roi_limits['roi_p1_min'],
'roi_p2': roi_limits['roi_p2_min'],
'roi_p3': roi_limits['roi_p3_min'],
}
logger.info(f"Min roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}")
p = {
'roi_t1': roi_limits['roi_t1_max'],
'roi_t2': roi_limits['roi_t2_max'],
'roi_t3': roi_limits['roi_t3_max'],
'roi_p1': roi_limits['roi_p1_max'],
'roi_p2': roi_limits['roi_p2_max'],
'roi_p3': roi_limits['roi_p3_max'],
}
logger.info(f"Max roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}")
return [
Integer(roi_limits['roi_t1_min'], roi_limits['roi_t1_max'], name='roi_t1'),
Integer(roi_limits['roi_t2_min'], roi_limits['roi_t2_max'], name='roi_t2'),
Integer(roi_limits['roi_t3_min'], roi_limits['roi_t3_max'], name='roi_t3'),
Real(roi_limits['roi_p1_min'], roi_limits['roi_p1_max'], name='roi_p1'),
Real(roi_limits['roi_p2_min'], roi_limits['roi_p2_max'], name='roi_p2'),
Real(roi_limits['roi_p3_min'], roi_limits['roi_p3_max'], name='roi_p3'),
]
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
@ -84,22 +181,17 @@ class IHyperOpt(ABC):
You may override it in your custom Hyperopt class.
"""
return [
Real(-0.5, -0.02, name='stoploss'),
Real(-0.35, -0.02, name='stoploss'),
]
@staticmethod
def roi_space() -> List[Dimension]:
"""
Create a ROI space.
# This is needed for proper unpickling the class attribute ticker_interval
# which is set to the actual value by the resolver.
# Why do I still need such shamanic mantras in modern python?
def __getstate__(self):
state = self.__dict__.copy()
state['ticker_interval'] = self.ticker_interval
return state
Defines values to search for each ROI steps.
You may override it in your custom Hyperopt class.
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]
def __setstate__(self, state):
self.__dict__.update(state)
IHyperOpt.ticker_interval = state['ticker_interval']

52
freqtrade/persistence.py
View File

@ -1,7 +1,6 @@
"""
This module contains the class to persist trades into SQLite
"""
import logging
from datetime import datetime
from decimal import Decimal
@ -19,8 +18,10 @@ from sqlalchemy.pool import StaticPool
from freqtrade import OperationalException
logger = logging.getLogger(__name__)
_DECL_BASE: Any = declarative_base()
_SQL_DOCS_URL = 'http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls'
@ -48,8 +49,8 @@ def init(db_url: str, clean_open_orders: bool = False) -> None:
try:
engine = create_engine(db_url, **kwargs)
except NoSuchModuleError:
raise OperationalException(f'Given value for db_url: \'{db_url}\' '
f'is no valid database URL! (See {_SQL_DOCS_URL})')
raise OperationalException(f"Given value for db_url: '{db_url}' "
f"is no valid database URL! (See {_SQL_DOCS_URL})")
session = scoped_session(sessionmaker(bind=engine, autoflush=True, autocommit=True))
Trade.session = session()
@ -209,7 +210,7 @@ class Trade(_DECL_BASE):
ticker_interval = Column(Integer, nullable=True)
def __repr__(self):
open_since = arrow.get(self.open_date).humanize() if self.is_open else 'closed'
open_since = self.open_date.strftime('%Y-%m-%d %H:%M:%S') if self.is_open else 'closed'
return (f'Trade(id={self.id}, pair={self.pair}, amount={self.amount:.8f}, '
f'open_rate={self.open_rate:.8f}, open_since={open_since})')
@ -250,7 +251,6 @@ class Trade(_DECL_BASE):
:param initial: Called to initiate stop_loss.
Skips everything if self.stop_loss is already set.
"""
if initial and not (self.stop_loss is None or self.stop_loss == 0):
# Don't modify if called with initial and nothing to do
return
@ -259,7 +259,7 @@ class Trade(_DECL_BASE):
# no stop loss assigned yet
if not self.stop_loss:
logger.debug("assigning new stop loss")
logger.debug(f"{self.pair} - Assigning new stoploss...")
self.stop_loss = new_loss
self.stop_loss_pct = -1 * abs(stoploss)
self.initial_stop_loss = new_loss
@ -269,21 +269,20 @@ class Trade(_DECL_BASE):
# evaluate if the stop loss needs to be updated
else:
if new_loss > self.stop_loss: # stop losses only walk up, never down!
logger.debug(f"{self.pair} - Adjusting stoploss...")
self.stop_loss = new_loss
self.stop_loss_pct = -1 * abs(stoploss)
self.stoploss_last_update = datetime.utcnow()
logger.debug("adjusted stop loss")
else:
logger.debug("keeping current stop loss")
logger.debug(f"{self.pair} - Keeping current stoploss...")
logger.debug(
f"{self.pair} - current price {current_price:.8f}, "
f"bought at {self.open_rate:.8f} and calculated "
f"stop loss is at: {self.initial_stop_loss:.8f} initial "
f"stop at {self.stop_loss:.8f}. "
f"trailing stop loss saved us: "
f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f} "
f"and max observed rate was {self.max_rate:.8f}")
f"{self.pair} - Stoploss adjusted. current_price={current_price:.8f}, "
f"open_rate={self.open_rate:.8f}, max_rate={self.max_rate:.8f}, "
f"initial_stop_loss={self.initial_stop_loss:.8f}, "
f"stop_loss={self.stop_loss:.8f}. "
f"Trailing stoploss saved us: "
f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f}.")
def update(self, order: Dict) -> None:
"""
@ -331,23 +330,18 @@ class Trade(_DECL_BASE):
self
)
def calc_open_trade_price(
self,
fee: Optional[float] = None) -> float:
def calc_open_trade_price(self, fee: Optional[float] = None) -> float:
"""
Calculate the open_rate including fee.
:param fee: fee to use on the open rate (optional).
If rate is not set self.fee will be used
:return: Price in of the open trade incl. Fees
"""
buy_trade = (Decimal(self.amount) * Decimal(self.open_rate))
fees = buy_trade * Decimal(fee or self.fee_open)
return float(buy_trade + fees)
def calc_close_trade_price(
self,
rate: Optional[float] = None,
def calc_close_trade_price(self, rate: Optional[float] = None,
fee: Optional[float] = None) -> float:
"""
Calculate the close_rate including fee
@ -357,7 +351,6 @@ class Trade(_DECL_BASE):
If rate is not set self.close_rate will be used
:return: Price in BTC of the open trade
"""
if rate is None and not self.close_rate:
return 0.0
@ -365,9 +358,7 @@ class Trade(_DECL_BASE):
fees = sell_trade * Decimal(fee or self.fee_close)
return float(sell_trade - fees)
def calc_profit(
self,
rate: Optional[float] = None,
def calc_profit(self, rate: Optional[float] = None,
fee: Optional[float] = None) -> float:
"""
Calculate the absolute profit in stake currency between Close and Open trade
@ -385,9 +376,7 @@ class Trade(_DECL_BASE):
profit = close_trade_price - open_trade_price
return float(f"{profit:.8f}")
def calc_profit_percent(
self,
rate: Optional[float] = None,
def calc_profit_percent(self, rate: Optional[float] = None,
fee: Optional[float] = None) -> float:
"""
Calculates the profit in percentage (including fee).
@ -396,7 +385,6 @@ class Trade(_DECL_BASE):
:param fee: fee to use on the close rate (optional).
:return: profit in percentage as float
"""
open_trade_price = self.calc_open_trade_price()
close_trade_price = self.calc_close_trade_price(
rate=(rate or self.close_rate),
@ -436,8 +424,8 @@ class Trade(_DECL_BASE):
and trade.initial_stop_loss_pct != desired_stoploss):
# Stoploss value got changed
logger.info(f"Stoploss for {trade} needs adjustment.")
logger.info(f"Stoploss for {trade} needs adjustment...")
# Force reset of stoploss
trade.stop_loss = None
trade.adjust_stop_loss(trade.open_rate, desired_stoploss)
logger.info(f"new stoploss: {trade.stop_loss}, ")
logger.info(f"New stoploss: {trade.stop_loss}.")

36
freqtrade/plot/plot_utils.py Normal file
View File

@ -0,0 +1,36 @@
from typing import Any, Dict
from freqtrade import OperationalException
from freqtrade.state import RunMode
from freqtrade.utils import setup_utils_configuration
def validate_plot_args(args: Dict[str, Any]):
if not args.get('datadir') and not args.get('config'):
raise OperationalException(
"You need to specify either `--datadir` or `--config` "
"for plot-profit and plot-dataframe.")
def start_plot_dataframe(args: Dict[str, Any]) -> None:
"""
Entrypoint for dataframe plotting
"""
# Import here to avoid errors if plot-dependencies are not installed.
from freqtrade.plot.plotting import load_and_plot_trades
validate_plot_args(args)
config = setup_utils_configuration(args, RunMode.PLOT)
load_and_plot_trades(config)
def start_plot_profit(args: Dict[str, Any]) -> None:
"""
Entrypoint for plot_profit
"""
# Import here to avoid errors if plot-dependencies are not installed.
from freqtrade.plot.plotting import plot_profit
validate_plot_args(args)
config = setup_utils_configuration(args, RunMode.PLOT)
plot_profit(config)

104
freqtrade/plot/plotting.py
View File

@ -1,15 +1,14 @@
import logging
from pathlib import Path
from typing import Dict, List, Optional
from typing import Any, Dict, List
import pandas as pd
from freqtrade.configuration import TimeRange
from freqtrade.data import history
from freqtrade.data.btanalysis import (combine_tickers_with_mean,
create_cum_profit, load_trades)
from freqtrade.exchange import Exchange
from freqtrade.resolvers import ExchangeResolver, StrategyResolver
create_cum_profit,
extract_trades_of_period, load_trades)
from freqtrade.resolvers import StrategyResolver
logger = logging.getLogger(__name__)
@ -19,23 +18,16 @@ try:
from plotly.offline import plot
import plotly.graph_objects as go
except ImportError:
logger.exception("Module plotly not found \n Please install using `pip install plotly`")
logger.exception("Module plotly not found \n Please install using `pip3 install plotly`")
exit(1)
def init_plotscript(config):
"""
Initialize objects needed for plotting
:return: Dict with tickers, trades, pairs and strategy
:return: Dict with tickers, trades and pairs
"""
exchange: Optional[Exchange] = None
# Exchange is only needed when downloading data!
if config.get("refresh_pairs", False):
exchange = ExchangeResolver(config.get('exchange', {}).get('name'),
config).exchange
strategy = StrategyResolver(config).strategy
if "pairs" in config:
pairs = config["pairs"]
else:
@ -47,17 +39,18 @@ def init_plotscript(config):
tickers = history.load_data(
datadir=Path(str(config.get("datadir"))),
pairs=pairs,
ticker_interval=config['ticker_interval'],
refresh_pairs=config.get('refresh_pairs', False),
ticker_interval=config.get('ticker_interval', '5m'),
timerange=timerange,
exchange=exchange,
)
trades = load_trades(config)
trades = load_trades(config['trade_source'],
db_url=config.get('db_url'),
exportfilename=config.get('exportfilename'),
)
return {"tickers": tickers,
"trades": trades,
"pairs": pairs,
"strategy": strategy,
}
@ -280,8 +273,15 @@ def generate_profit_graph(pairs: str, tickers: Dict[str, pd.DataFrame],
name='Avg close price',
)
fig = make_subplots(rows=3, cols=1, shared_xaxes=True, row_width=[1, 1, 1])
fig['layout'].update(title="Profit plot")
fig = make_subplots(rows=3, cols=1, shared_xaxes=True,
row_width=[1, 1, 1],
vertical_spacing=0.05,
subplot_titles=["AVG Close Price", "Combined Profit", "Profit per pair"])
fig['layout'].update(title="Freqtrade Profit plot")
fig['layout']['yaxis1'].update(title='Price')
fig['layout']['yaxis2'].update(title='Profit')
fig['layout']['yaxis3'].update(title='Profit')
fig['layout']['xaxis']['rangeslider'].update(visible=False)
fig.add_trace(avgclose, 1, 1)
fig = add_profit(fig, 2, df_comb, 'cum_profit', 'Profit')
@ -321,3 +321,65 @@ def store_plot_file(fig, filename: str, directory: Path, auto_open: bool = False
plot(fig, filename=str(_filename),
auto_open=auto_open)
logger.info(f"Stored plot as {_filename}")
def load_and_plot_trades(config: Dict[str, Any]):
"""
From configuration provided
- Initializes plot-script
- Get tickers data
- Generate Dafaframes populated with indicators and signals based on configured strategy
- Load trades excecuted during the selected period
- Generate Plotly plot objects
- Generate plot files
:return: None
"""
strategy = StrategyResolver(config).strategy
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
pair_counter = 0
for pair, data in plot_elements["tickers"].items():
pair_counter += 1
logger.info("analyse pair %s", pair)
tickers = {}
tickers[pair] = data
dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
trades_pair = trades.loc[trades['pair'] == pair]
trades_pair = extract_trades_of_period(dataframe, trades_pair)
fig = generate_candlestick_graph(
pair=pair,
data=dataframe,
trades=trades_pair,
indicators1=config["indicators1"],
indicators2=config["indicators2"],
)
store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
directory=config['user_data_dir'] / "plot")
logger.info('End of plotting process. %s plots generated', pair_counter)
def plot_profit(config: Dict[str, Any]) -> None:
"""
Plots the total profit for all pairs.
Note, the profit calculation isn't realistic.
But should be somewhat proportional, and therefor useful
in helping out to find a good algorithm.
"""
plot_elements = init_plotscript(config)
trades = load_trades(config['trade_source'],
db_url=str(config.get('db_url')),
exportfilename=str(config.get('exportfilename')),
)
# Filter trades to relevant pairs
trades = trades[trades['pair'].isin(plot_elements["pairs"])]
# Create an average close price of all the pairs that were involved.
# this could be useful to gauge the overall market trend
fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
store_plot_file(fig, filename='freqtrade-profit-plot.html',
directory=config['user_data_dir'] / "plot", auto_open=True)

13
freqtrade/resolvers/hyperopt_resolver.py
View File

@ -34,15 +34,12 @@ class HyperOptResolver(IResolver):
self.hyperopt = self._load_hyperopt(hyperopt_name, config,
extra_dir=config.get('hyperopt_path'))
# Assign ticker_interval to be used in hyperopt
self.hyperopt.__class__.ticker_interval = str(config['ticker_interval'])
if not hasattr(self.hyperopt, 'populate_buy_trend'):
logger.warning("Custom Hyperopt does not provide populate_buy_trend. "
"Using populate_buy_trend from DefaultStrategy.")
logger.warning("Hyperopt class does not provide populate_buy_trend() method. "
"Using populate_buy_trend from the strategy.")
if not hasattr(self.hyperopt, 'populate_sell_trend'):
logger.warning("Custom Hyperopt does not provide populate_sell_trend. "
"Using populate_sell_trend from DefaultStrategy.")
logger.warning("Hyperopt class does not provide populate_sell_trend() method. "
"Using populate_sell_trend from the strategy.")
def _load_hyperopt(
self, hyperopt_name: str, config: Dict, extra_dir: Optional[str] = None) -> IHyperOpt:
@ -65,7 +62,7 @@ class HyperOptResolver(IResolver):
abs_paths.insert(0, Path(extra_dir).resolve())
hyperopt = self._load_object(paths=abs_paths, object_type=IHyperOpt,
object_name=hyperopt_name)
object_name=hyperopt_name, kwargs={'config': config})
if hyperopt:
return hyperopt
raise OperationalException(

2
freqtrade/resolvers/iresolver.py
View File

@ -12,7 +12,7 @@ from typing import Any, List, Optional, Tuple, Type, Union
logger = logging.getLogger(__name__)
class IResolver(object):
class IResolver:
"""
This class contains all the logic to load custom classes
"""

12
freqtrade/resolvers/strategy_resolver.py
View File

@ -13,7 +13,6 @@ from typing import Dict, Optional
from freqtrade import constants, OperationalException
from freqtrade.resolvers import IResolver
from freqtrade.strategy import import_strategy
from freqtrade.strategy.interface import IStrategy
logger = logging.getLogger(__name__)
@ -153,13 +152,12 @@ class StrategyResolver(IResolver):
strategy._populate_fun_len = len(getfullargspec(strategy.populate_indicators).args)
strategy._buy_fun_len = len(getfullargspec(strategy.populate_buy_trend).args)
strategy._sell_fun_len = len(getfullargspec(strategy.populate_sell_trend).args)
if any([x == 2 for x in [strategy._populate_fun_len,
strategy._buy_fun_len,
strategy._sell_fun_len]]):
strategy.INTERFACE_VERSION = 1
try:
return import_strategy(strategy, config=config)
except TypeError as e:
logger.warning(
f"Impossible to load strategy '{strategy_name}'. "
f"Error: {e}")
return strategy
raise OperationalException(
f"Impossible to load Strategy '{strategy_name}'. This class does not exist "

6
freqtrade/rpc/fiat_convert.py
View File

@ -15,7 +15,7 @@ from freqtrade.constants import SUPPORTED_FIAT
logger = logging.getLogger(__name__)
class CryptoFiat(object):
class CryptoFiat:
"""
Object to describe what is the price of Crypto-currency in a FIAT
"""
@ -60,7 +60,7 @@ class CryptoFiat(object):
return self._expiration - time.time() <= 0
class CryptoToFiatConverter(object):
class CryptoToFiatConverter:
"""
Main class to initiate Crypto to FIAT.
This object contains a list of pair Crypto, FIAT
@ -104,7 +104,7 @@ class CryptoToFiatConverter(object):
:return: float, value in fiat of the crypto-currency amount
"""
if crypto_symbol == fiat_symbol:
return crypto_amount
return float(crypto_amount)
price = self.get_price(crypto_symbol=crypto_symbol, fiat_symbol=fiat_symbol)
return float(crypto_amount) * float(price)

8
freqtrade/rpc/rpc.py
View File

@ -54,7 +54,7 @@ class RPCException(Exception):
}
class RPC(object):
class RPC:
"""
RPC class can be used to have extra feature, like bot data, and access to DB data
"""
@ -294,9 +294,9 @@ class RPC(object):
total = total + est_btc
output.append({
'currency': coin,
'available': balance['free'],
'balance': balance['total'],
'pending': balance['used'],
'free': balance['free'] if balance['free'] is not None else 0,
'balance': balance['total'] if balance['total'] is not None else 0,
'used': balance['used'] if balance['used'] is not None else 0,
'est_btc': est_btc,
})
if total == 0.0:

5
freqtrade/rpc/rpc_manager.py
View File

@ -9,7 +9,7 @@ from freqtrade.rpc import RPC, RPCMessageType
logger = logging.getLogger(__name__)
class RPCManager(object):
class RPCManager:
"""
Class to manage RPC objects (Telegram, Slack, ...)
"""
@ -56,7 +56,10 @@ class RPCManager(object):
logger.info('Sending rpc message: %s', msg)
for mod in self.registered_modules:
logger.debug('Forwarding message to rpc.%s', mod.name)
try:
mod.send_msg(msg)
except NotImplementedError:
logger.error(f"Message type {msg['type']} not implemented by handler {mod.name}.")
def startup_messages(self, config, pairlist) -> None:
if config.get('dry_run', False):

131
freqtrade/rpc/telegram.py
View File

@ -4,12 +4,12 @@
This module manage Telegram communication
"""
import logging
from typing import Any, Callable, Dict, List
from typing import Any, Callable, Dict
from tabulate import tabulate
from telegram import Bot, ParseMode, ReplyKeyboardMarkup, Update
from telegram import ParseMode, ReplyKeyboardMarkup, Update
from telegram.error import NetworkError, TelegramError
from telegram.ext import CommandHandler, Updater
from telegram.ext import CommandHandler, Updater, CallbackContext
from freqtrade.__init__ import __version__
from freqtrade.rpc import RPC, RPCException, RPCMessageType
@ -31,7 +31,7 @@ def authorized_only(command_handler: Callable[..., None]) -> Callable[..., Any]:
"""
def wrapper(self, *args, **kwargs):
""" Decorator logic """
update = kwargs.get('update') or args[1]
update = kwargs.get('update') or args[0]
# Reject unauthorized messages
chat_id = int(self._config['telegram']['chat_id'])
@ -79,7 +79,8 @@ class Telegram(RPC):
registers all known command handlers
and starts polling for message updates
"""
self._updater = Updater(token=self._config['telegram']['token'], workers=0)
self._updater = Updater(token=self._config['telegram']['token'], workers=0,
use_context=True)
# Register command handler and start telegram message polling
handles = [
@ -96,7 +97,7 @@ class Telegram(RPC):
CommandHandler('reload_conf', self._reload_conf),
CommandHandler('stopbuy', self._stopbuy),
CommandHandler('whitelist', self._whitelist),
CommandHandler('blacklist', self._blacklist, pass_args=True),
CommandHandler('blacklist', self._blacklist),
CommandHandler('edge', self._edge),
CommandHandler('help', self._help),
CommandHandler('version', self._version),
@ -175,7 +176,7 @@ class Telegram(RPC):
self._send_msg(message)
@authorized_only
def _status(self, bot: Bot, update: Update) -> None:
def _status(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /status.
Returns the current TradeThread status
@ -184,11 +185,8 @@ class Telegram(RPC):
:return: None
"""
# Check if additional parameters are passed
params = update.message.text.replace('/status', '').split(' ') \
if update.message.text else []
if 'table' in params:
self._status_table(bot, update)
if 'table' in context.args:
self._status_table(update, context)
return
try:
@ -221,13 +219,13 @@ class Telegram(RPC):
messages.append("\n".join([l for l in lines if l]).format(**r))
for msg in messages:
self._send_msg(msg, bot=bot)
self._send_msg(msg)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _status_table(self, bot: Bot, update: Update) -> None:
def _status_table(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /status table.
Returns the current TradeThread status in table format
@ -240,10 +238,10 @@ class Telegram(RPC):
message = tabulate(df_statuses, headers='keys', tablefmt='simple')
self._send_msg(f"<pre>{message}</pre>", parse_mode=ParseMode.HTML)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _daily(self, bot: Bot, update: Update) -> None:
def _daily(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /daily <n>
Returns a daily profit (in BTC) over the last n days.
@ -254,8 +252,8 @@ class Telegram(RPC):
stake_cur = self._config['stake_currency']
fiat_disp_cur = self._config.get('fiat_display_currency', '')
try:
timescale = int(update.message.text.replace('/daily', '').strip())
except (TypeError, ValueError):
timescale = int(context.args[0])
except (TypeError, ValueError, IndexError):
timescale = 7
try:
stats = self._rpc_daily_profit(
@ -272,12 +270,12 @@ class Telegram(RPC):
],
tablefmt='simple')
message = f'<b>Daily Profit over the last {timescale} days</b>:\n<pre>{stats_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML)
self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _profit(self, bot: Bot, update: Update) -> None:
def _profit(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /profit.
Returns a cumulative profit statistics.
@ -317,12 +315,12 @@ class Telegram(RPC):
f"*Latest Trade opened:* `{latest_trade_date}`\n" \
f"*Avg. Duration:* `{avg_duration}`\n" \
f"*Best Performing:* `{best_pair}: {best_rate:.2f}%`"
self._send_msg(markdown_msg, bot=bot)
self._send_msg(markdown_msg)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _balance(self, bot: Bot, update: Update) -> None:
def _balance(self, update: Update, context: CallbackContext) -> None:
""" Handler for /balance """
try:
result = self._rpc_balance(self._config.get('fiat_display_currency', ''))
@ -330,16 +328,16 @@ class Telegram(RPC):
for currency in result['currencies']:
if currency['est_btc'] > 0.0001:
curr_output = "*{currency}:*\n" \
"\t`Available: {available: .8f}`\n" \
"\t`Available: {free: .8f}`\n" \
"\t`Balance: {balance: .8f}`\n" \
"\t`Pending: {pending: .8f}`\n" \
"\t`Pending: {used: .8f}`\n" \
"\t`Est. BTC: {est_btc: .8f}`\n".format(**currency)
else:
curr_output = "*{currency}:* not showing <1$ amount \n".format(**currency)
# Handle overflowing messsage length
if len(output + curr_output) >= MAX_TELEGRAM_MESSAGE_LENGTH:
self._send_msg(output, bot=bot)
self._send_msg(output)
output = curr_output
else:
output += curr_output
@ -347,12 +345,12 @@ class Telegram(RPC):
output += "\n*Estimated Value*:\n" \
"\t`BTC: {total: .8f}`\n" \
"\t`{symbol}: {value: .2f}`\n".format(**result)
self._send_msg(output, bot=bot)
self._send_msg(output)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _start(self, bot: Bot, update: Update) -> None:
def _start(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /start.
Starts TradeThread
@ -361,10 +359,10 @@ class Telegram(RPC):
:return: None
"""
msg = self._rpc_start()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot)
self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only
def _stop(self, bot: Bot, update: Update) -> None:
def _stop(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /stop.
Stops TradeThread
@ -373,10 +371,10 @@ class Telegram(RPC):
:return: None
"""
msg = self._rpc_stop()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot)
self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only
def _reload_conf(self, bot: Bot, update: Update) -> None:
def _reload_conf(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /reload_conf.
Triggers a config file reload
@ -385,10 +383,10 @@ class Telegram(RPC):
:return: None
"""
msg = self._rpc_reload_conf()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot)
self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only
def _stopbuy(self, bot: Bot, update: Update) -> None:
def _stopbuy(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /stop_buy.
Sets max_open_trades to 0 and gracefully sells all open trades
@ -397,10 +395,10 @@ class Telegram(RPC):
:return: None
"""
msg = self._rpc_stopbuy()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot)
self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only
def _forcesell(self, bot: Bot, update: Update) -> None:
def _forcesell(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /forcesell <id>.
Sells the given trade at current price
@ -409,16 +407,16 @@ class Telegram(RPC):
:return: None
"""
trade_id = update.message.text.replace('/forcesell', '').strip()
trade_id = context.args[0] if len(context.args) > 0 else None
try:
msg = self._rpc_forcesell(trade_id)
self._send_msg('Forcesell Result: `{result}`'.format(**msg), bot=bot)
self._send_msg('Forcesell Result: `{result}`'.format(**msg))
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _forcebuy(self, bot: Bot, update: Update) -> None:
def _forcebuy(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /forcebuy <asset> <price>.
Buys a pair trade at the given or current price
@ -427,16 +425,15 @@ class Telegram(RPC):
:return: None
"""
message = update.message.text.replace('/forcebuy', '').strip().split()
pair = message[0]
price = float(message[1]) if len(message) > 1 else None
pair = context.args[0]
price = float(context.args[1]) if len(context.args) > 1 else None
try:
self._rpc_forcebuy(pair, price)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _performance(self, bot: Bot, update: Update) -> None:
def _performance(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /performance.
Shows a performance statistic from finished trades
@ -455,10 +452,10 @@ class Telegram(RPC):
message = '<b>Performance:</b>\n{}'.format(stats)
self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _count(self, bot: Bot, update: Update) -> None:
def _count(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /count.
Returns the number of trades running
@ -475,10 +472,10 @@ class Telegram(RPC):
logger.debug(message)
self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _whitelist(self, bot: Bot, update: Update) -> None:
def _whitelist(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /whitelist
Shows the currently active whitelist
@ -492,17 +489,17 @@ class Telegram(RPC):
logger.debug(message)
self._send_msg(message)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _blacklist(self, bot: Bot, update: Update, args: List[str]) -> None:
def _blacklist(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /blacklist
Shows the currently active blacklist
"""
try:
blacklist = self._rpc_blacklist(args)
blacklist = self._rpc_blacklist(context.args)
message = f"Blacklist contains {blacklist['length']} pairs\n"
message += f"`{', '.join(blacklist['blacklist'])}`"
@ -510,10 +507,10 @@ class Telegram(RPC):
logger.debug(message)
self._send_msg(message)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _edge(self, bot: Bot, update: Update) -> None:
def _edge(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /edge
Shows information related to Edge
@ -522,12 +519,12 @@ class Telegram(RPC):
edge_pairs = self._rpc_edge()
edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple')
message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML)
self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e:
self._send_msg(str(e), bot=bot)
self._send_msg(str(e))
@authorized_only
def _help(self, bot: Bot, update: Update) -> None:
def _help(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /help.
Show commands of the bot
@ -559,10 +556,10 @@ class Telegram(RPC):
"*/help:* `This help message`\n" \
"*/version:* `Show version`"
self._send_msg(message, bot=bot)
self._send_msg(message)
@authorized_only
def _version(self, bot: Bot, update: Update) -> None:
def _version(self, update: Update, context: CallbackContext) -> None:
"""
Handler for /version.
Show version information
@ -570,10 +567,9 @@ class Telegram(RPC):
:param update: message update
:return: None
"""
self._send_msg('*Version:* `{}`'.format(__version__), bot=bot)
self._send_msg('*Version:* `{}`'.format(__version__))
def _send_msg(self, msg: str, bot: Bot = None,
parse_mode: ParseMode = ParseMode.MARKDOWN) -> None:
def _send_msg(self, msg: str, parse_mode: ParseMode = ParseMode.MARKDOWN) -> None:
"""
Send given markdown message
:param msg: message
@ -581,7 +577,6 @@ class Telegram(RPC):
:param parse_mode: telegram parse mode
:return: None
"""
bot = bot or self._updater.bot
keyboard = [['/daily', '/profit', '/balance'],
['/status', '/status table', '/performance'],
@ -591,7 +586,7 @@ class Telegram(RPC):
try:
try:
bot.send_message(
self._updater.bot.send_message(
self._config['telegram']['chat_id'],
text=msg,
parse_mode=parse_mode,
@ -604,7 +599,7 @@ class Telegram(RPC):
'Telegram NetworkError: %s! Trying one more time.',
network_err.message
)
bot.send_message(
self._updater.bot.send_message(
self._config['telegram']['chat_id'],
text=msg,
parse_mode=parse_mode,

4
freqtrade/rpc/webhook.py
View File

@ -43,7 +43,9 @@ class Webhook(RPC):
valuedict = self._config['webhook'].get('webhookbuy', None)
elif msg['type'] == RPCMessageType.SELL_NOTIFICATION:
valuedict = self._config['webhook'].get('webhooksell', None)
elif msg['type'] == RPCMessageType.STATUS_NOTIFICATION:
elif msg['type'] in(RPCMessageType.STATUS_NOTIFICATION,
RPCMessageType.CUSTOM_NOTIFICATION,
RPCMessageType.WARNING_NOTIFICATION):
valuedict = self._config['webhook'].get('webhookstatus', None)
else:
raise NotImplementedError('Unknown message type: {}'.format(msg['type']))

1
freqtrade/state.py
View File

@ -25,4 +25,5 @@ class RunMode(Enum):
BACKTEST = "backtest"
EDGE = "edge"
HYPEROPT = "hyperopt"
PLOT = "plot"
OTHER = "other" # Used for plotting scripts and test

46
freqtrade/strategy/__init__.py
View File

@ -1,45 +1 @@
import logging
import sys
from copy import deepcopy
from freqtrade.strategy.interface import IStrategy
# Import Default-Strategy to have hyperopt correctly resolve
from freqtrade.strategy.default_strategy import DefaultStrategy # noqa: F401
logger = logging.getLogger(__name__)
def import_strategy(strategy: IStrategy, config: dict) -> IStrategy:
"""
Imports given Strategy instance to global scope
of freqtrade.strategy and returns an instance of it
"""
# Copy all attributes from base class and class
comb = {**strategy.__class__.__dict__, **strategy.__dict__}
# Delete '_abc_impl' from dict as deepcopy fails on 3.7 with
# `TypeError: can't pickle _abc_data objects``
# This will only apply to python 3.7
if sys.version_info.major == 3 and sys.version_info.minor == 7 and '_abc_impl' in comb:
del comb['_abc_impl']
attr = deepcopy(comb)
# Adjust module name
attr['__module__'] = 'freqtrade.strategy'
name = strategy.__class__.__name__
clazz = type(name, (IStrategy,), attr)
logger.debug(
'Imported strategy %s.%s as %s.%s',
strategy.__module__, strategy.__class__.__name__,
clazz.__module__, strategy.__class__.__name__,
)
# Modify global scope to declare class
globals()[name] = clazz
return clazz(config)
from freqtrade.strategy.interface import IStrategy # noqa: F401

126
freqtrade/strategy/default_strategy.py
View File

@ -4,15 +4,18 @@ import talib.abstract as ta
from pandas import DataFrame
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.indicator_helpers import fishers_inverse
from freqtrade.strategy.interface import IStrategy
class DefaultStrategy(IStrategy):
"""
Default Strategy provided by freqtrade bot.
You can override it with your own strategy
Please do not modify this strategy, it's intended for internal use only.
Please look at the SampleStrategy in the user_data/strategy directory
or strategy repository https://github.com/freqtrade/freqtrade-strategies
for samples and inspiration.
"""
INTERFACE_VERSION = 2
# Minimal ROI designed for the strategy
minimal_roi = {
@ -73,67 +76,25 @@ class DefaultStrategy(IStrategy):
# ADX
dataframe['adx'] = ta.ADX(dataframe)
# Awesome oscillator
dataframe['ao'] = qtpylib.awesome_oscillator(dataframe)
"""
# Commodity Channel Index: values Oversold:<-100, Overbought:>100
dataframe['cci'] = ta.CCI(dataframe)
"""
# MACD
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['macdhist'] = macd['macdhist']
# MFI
dataframe['mfi'] = ta.MFI(dataframe)
# Minus Directional Indicator / Movement
dataframe['minus_dm'] = ta.MINUS_DM(dataframe)
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Plus Directional Indicator / Movement
dataframe['plus_dm'] = ta.PLUS_DM(dataframe)
dataframe['plus_di'] = ta.PLUS_DI(dataframe)
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
"""
# ROC
dataframe['roc'] = ta.ROC(dataframe)
"""
# RSI
dataframe['rsi'] = ta.RSI(dataframe)
# Inverse Fisher transform on RSI, values [-1.0, 1.0] (https://goo.gl/2JGGoy)
dataframe['fisher_rsi'] = fishers_inverse(dataframe['rsi'])
# Inverse Fisher transform on RSI normalized, value [0.0, 100.0] (https://goo.gl/2JGGoy)
dataframe['fisher_rsi_norma'] = 50 * (dataframe['fisher_rsi'] + 1)
# Stoch
stoch = ta.STOCH(dataframe)
dataframe['slowd'] = stoch['slowd']
dataframe['slowk'] = stoch['slowk']
# Stoch fast
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['fastk'] = stoch_fast['fastk']
"""
# Stoch RSI
stoch_rsi = ta.STOCHRSI(dataframe)
dataframe['fastd_rsi'] = stoch_rsi['fastd']
dataframe['fastk_rsi'] = stoch_rsi['fastk']
"""
# Overlap Studies
# ------------------------------------
# Previous Bollinger bands
# Because ta.BBANDS implementation is broken with small numbers, it actually
# returns middle band for all the three bands. Switch to qtpylib.bollinger_bands
# and use middle band instead.
dataframe['blower'] = ta.BBANDS(dataframe, nbdevup=2, nbdevdn=2)['lowerband']
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
@ -142,88 +103,11 @@ class DefaultStrategy(IStrategy):
dataframe['bb_upperband'] = bollinger['upper']
# EMA - Exponential Moving Average
dataframe['ema3'] = ta.EMA(dataframe, timeperiod=3)
dataframe['ema5'] = ta.EMA(dataframe, timeperiod=5)
dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10)
dataframe['ema50'] = ta.EMA(dataframe, timeperiod=50)
dataframe['ema100'] = ta.EMA(dataframe, timeperiod=100)
# SAR Parabol
dataframe['sar'] = ta.SAR(dataframe)
# SMA - Simple Moving Average
dataframe['sma'] = ta.SMA(dataframe, timeperiod=40)
# TEMA - Triple Exponential Moving Average
dataframe['tema'] = ta.TEMA(dataframe, timeperiod=9)
# Cycle Indicator
# ------------------------------------
# Hilbert Transform Indicator - SineWave
hilbert = ta.HT_SINE(dataframe)
dataframe['htsine'] = hilbert['sine']
dataframe['htleadsine'] = hilbert['leadsine']
# Pattern Recognition - Bullish candlestick patterns
# ------------------------------------
"""
# Hammer: values [0, 100]
dataframe['CDLHAMMER'] = ta.CDLHAMMER(dataframe)
# Inverted Hammer: values [0, 100]
dataframe['CDLINVERTEDHAMMER'] = ta.CDLINVERTEDHAMMER(dataframe)
# Dragonfly Doji: values [0, 100]
dataframe['CDLDRAGONFLYDOJI'] = ta.CDLDRAGONFLYDOJI(dataframe)
# Piercing Line: values [0, 100]
dataframe['CDLPIERCING'] = ta.CDLPIERCING(dataframe) # values [0, 100]
# Morningstar: values [0, 100]
dataframe['CDLMORNINGSTAR'] = ta.CDLMORNINGSTAR(dataframe) # values [0, 100]
# Three White Soldiers: values [0, 100]
dataframe['CDL3WHITESOLDIERS'] = ta.CDL3WHITESOLDIERS(dataframe) # values [0, 100]
"""
# Pattern Recognition - Bearish candlestick patterns
# ------------------------------------
"""
# Hanging Man: values [0, 100]
dataframe['CDLHANGINGMAN'] = ta.CDLHANGINGMAN(dataframe)
# Shooting Star: values [0, 100]
dataframe['CDLSHOOTINGSTAR'] = ta.CDLSHOOTINGSTAR(dataframe)
# Gravestone Doji: values [0, 100]
dataframe['CDLGRAVESTONEDOJI'] = ta.CDLGRAVESTONEDOJI(dataframe)
# Dark Cloud Cover: values [0, 100]
dataframe['CDLDARKCLOUDCOVER'] = ta.CDLDARKCLOUDCOVER(dataframe)
# Evening Doji Star: values [0, 100]
dataframe['CDLEVENINGDOJISTAR'] = ta.CDLEVENINGDOJISTAR(dataframe)
# Evening Star: values [0, 100]
dataframe['CDLEVENINGSTAR'] = ta.CDLEVENINGSTAR(dataframe)
"""
# Pattern Recognition - Bullish/Bearish candlestick patterns
# ------------------------------------
"""
# Three Line Strike: values [0, -100, 100]
dataframe['CDL3LINESTRIKE'] = ta.CDL3LINESTRIKE(dataframe)
# Spinning Top: values [0, -100, 100]
dataframe['CDLSPINNINGTOP'] = ta.CDLSPINNINGTOP(dataframe) # values [0, -100, 100]
# Engulfing: values [0, -100, 100]
dataframe['CDLENGULFING'] = ta.CDLENGULFING(dataframe) # values [0, -100, 100]
# Harami: values [0, -100, 100]
dataframe['CDLHARAMI'] = ta.CDLHARAMI(dataframe) # values [0, -100, 100]
# Three Outside Up/Down: values [0, -100, 100]
dataframe['CDL3OUTSIDE'] = ta.CDL3OUTSIDE(dataframe) # values [0, -100, 100]
# Three Inside Up/Down: values [0, -100, 100]
dataframe['CDL3INSIDE'] = ta.CDL3INSIDE(dataframe) # values [0, -100, 100]
"""
# Chart type
# ------------------------------------
# Heikinashi stategy
heikinashi = qtpylib.heikinashi(dataframe)
dataframe['ha_open'] = heikinashi['open']
dataframe['ha_close'] = heikinashi['close']
dataframe['ha_high'] = heikinashi['high']
dataframe['ha_low'] = heikinashi['low']
return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

46
freqtrade/strategy/interface.py
View File

@ -39,6 +39,7 @@ class SellType(Enum):
TRAILING_STOP_LOSS = "trailing_stop_loss"
SELL_SIGNAL = "sell_signal"
FORCE_SELL = "force_sell"
EMERGENCY_SELL = "emergency_sell"
NONE = ""
@ -60,6 +61,11 @@ class IStrategy(ABC):
stoploss -> float: optimal stoploss designed for the strategy
ticker_interval -> str: value of the ticker interval to use for the strategy
"""
# Strategy interface version
# Default to version 2
# Version 1 is the initial interface without metadata dict
# Version 2 populate_* include metadata dict
INTERFACE_VERSION: int = 2
_populate_fun_len: int = 0
_buy_fun_len: int = 0
@ -196,7 +202,6 @@ class IStrategy(ABC):
:param metadata: Metadata dictionary with additional data (e.g. 'pair')
:return: DataFrame with ticker data and indicator data
"""
pair = str(metadata.get('pair'))
# Test if seen this pair and last candle before.
@ -286,7 +291,6 @@ class IStrategy(ABC):
:param force_stoploss: Externally provided stoploss
:return: True if trade should be sold, False otherwise
"""
# Set current rate to low for backtesting sell
current_rate = low or rate
current_profit = trade.calc_profit_percent(current_rate)
@ -298,6 +302,8 @@ class IStrategy(ABC):
force_stoploss=force_stoploss, high=high)
if stoplossflag.sell_flag:
logger.debug(f"{trade.pair} - Stoploss hit. sell_flag=True, "
f"sell_type={stoplossflag.sell_type}")
return stoplossflag
# Set current rate to high for backtesting sell
@ -306,22 +312,31 @@ class IStrategy(ABC):
experimental = self.config.get('experimental', {})
if buy and experimental.get('ignore_roi_if_buy_signal', False):
logger.debug('Buy signal still active - not selling.')
# This one is noisy, commented out
# logger.debug(f"{trade.pair} - Buy signal still active. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
# Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee)
if self.min_roi_reached(trade=trade, current_profit=current_profit, current_time=date):
logger.debug('Required profit reached. Selling..')
logger.debug(f"{trade.pair} - Required profit reached. sell_flag=True, "
f"sell_type=SellType.ROI")
return SellCheckTuple(sell_flag=True, sell_type=SellType.ROI)
if experimental.get('sell_profit_only', False):
logger.debug('Checking if trade is profitable..')
# This one is noisy, commented out
# logger.debug(f"{trade.pair} - Checking if trade is profitable...")
if trade.calc_profit(rate=rate) <= 0:
# This one is noisy, commented out
# logger.debug(f"{trade.pair} - Trade is not profitable. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
if sell and not buy and experimental.get('use_sell_signal', False):
logger.debug('Sell signal received. Selling..')
logger.debug(f"{trade.pair} - Sell signal received. sell_flag=True, "
f"sell_type=SellType.SELL_SIGNAL")
return SellCheckTuple(sell_flag=True, sell_type=SellType.SELL_SIGNAL)
# This one is noisy, commented out...
# logger.debug(f"{trade.pair} - No sell signal. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
def stop_loss_reached(self, current_rate: float, trade: Trade,
@ -332,7 +347,6 @@ class IStrategy(ABC):
decides to sell or not
:param current_profit: current profit in percent
"""
trailing_stop = self.config.get('trailing_stop', False)
stop_loss_value = force_stoploss if force_stoploss else self.stoploss
@ -353,7 +367,7 @@ class IStrategy(ABC):
if 'trailing_stop_positive' in self.config and high_profit > sl_offset:
# Ignore mypy error check in configuration that this is a float
stop_loss_value = self.config.get('trailing_stop_positive') # type: ignore
logger.debug(f"using positive stop loss: {stop_loss_value} "
logger.debug(f"{trade.pair} - Using positive stoploss: {stop_loss_value} "
f"offset: {sl_offset:.4g} profit: {current_profit:.4f}%")
trade.adjust_stop_loss(high or current_rate, stop_loss_value)
@ -363,20 +377,20 @@ class IStrategy(ABC):
(trade.stop_loss >= current_rate) and
(not self.order_types.get('stoploss_on_exchange'))):
selltype = SellType.STOP_LOSS
sell_type = SellType.STOP_LOSS
# If initial stoploss is not the same as current one then it is trailing.
if trade.initial_stop_loss != trade.stop_loss:
selltype = SellType.TRAILING_STOP_LOSS
sell_type = SellType.TRAILING_STOP_LOSS
logger.debug(
f"HIT STOP: current price at {current_rate:.6f}, "
f"stop loss is {trade.stop_loss:.6f}, "
f"initial stop loss was at {trade.initial_stop_loss:.6f}, "
f"{trade.pair} - HIT STOP: current price at {current_rate:.6f}, "
f"stoploss is {trade.stop_loss:.6f}, "
f"initial stoploss was at {trade.initial_stop_loss:.6f}, "
f"trade opened at {trade.open_rate:.6f}")
logger.debug(f"trailing stop saved {trade.stop_loss - trade.initial_stop_loss:.6f}")
logger.debug(f"{trade.pair} - Trailing stop saved "
f"{trade.stop_loss - trade.initial_stop_loss:.6f}")
logger.debug('Stop loss hit.')
return SellCheckTuple(sell_flag=True, sell_type=selltype)
return SellCheckTuple(sell_flag=True, sell_type=sell_type)
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)

69
freqtrade/tests/exchange/test_kraken.py
View File

@ -1,69 +0,0 @@
# pragma pylint: disable=missing-docstring, C0103, bad-continuation, global-statement
# pragma pylint: disable=protected-access
from random import randint
from unittest.mock import MagicMock
from freqtrade.tests.conftest import get_patched_exchange
def test_buy_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'limit'
time_in_force = 'ioc'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.buy(pair='ETH/BTC', ordertype=order_type,
amount=1, rate=200, time_in_force=time_in_force)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'timeInForce': 'ioc',
'trading_agreement': 'agree'}
def test_sell_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
assert api_mock.create_order.call_args[0][5] == {'trading_agreement': 'agree'}

181
freqtrade/tests/test_arguments.py
View File

@ -1,181 +0,0 @@
# pragma pylint: disable=missing-docstring, C0103
import argparse
import pytest
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
from freqtrade.configuration.cli_options import check_int_positive
# Parse common command-line-arguments. Used for all tools
def test_parse_args_none() -> None:
arguments = Arguments([], '')
assert isinstance(arguments, Arguments)
assert isinstance(arguments.parser, argparse.ArgumentParser)
def test_parse_args_defaults() -> None:
args = Arguments([], '').get_parsed_arg()
assert args.config == ['config.json']
assert args.strategy_path is None
assert args.datadir is None
assert args.verbosity == 0
def test_parse_args_config() -> None:
args = Arguments(['-c', '/dev/null'], '').get_parsed_arg()
assert args.config == ['/dev/null']
args = Arguments(['--config', '/dev/null'], '').get_parsed_arg()
assert args.config == ['/dev/null']
args = Arguments(['--config', '/dev/null',
'--config', '/dev/zero'],
'').get_parsed_arg()
assert args.config == ['/dev/null', '/dev/zero']
def test_parse_args_db_url() -> None:
args = Arguments(['--db-url', 'sqlite:///test.sqlite'], '').get_parsed_arg()
assert args.db_url == 'sqlite:///test.sqlite'
def test_parse_args_verbose() -> None:
args = Arguments(['-v'], '').get_parsed_arg()
assert args.verbosity == 1
args = Arguments(['--verbose'], '').get_parsed_arg()
assert args.verbosity == 1
def test_common_scripts_options() -> None:
args = Arguments(['download-data', '-p', 'ETH/BTC', 'XRP/BTC'], '').get_parsed_arg()
assert args.pairs == ['ETH/BTC', 'XRP/BTC']
assert hasattr(args, "func")
def test_parse_args_version() -> None:
with pytest.raises(SystemExit, match=r'0'):
Arguments(['--version'], '').get_parsed_arg()
def test_parse_args_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['-c'], '').get_parsed_arg()
def test_parse_args_strategy() -> None:
args = Arguments(['--strategy', 'SomeStrategy'], '').get_parsed_arg()
assert args.strategy == 'SomeStrategy'
def test_parse_args_strategy_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['--strategy'], '').get_parsed_arg()
def test_parse_args_strategy_path() -> None:
args = Arguments(['--strategy-path', '/some/path'], '').get_parsed_arg()
assert args.strategy_path == '/some/path'
def test_parse_args_strategy_path_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['--strategy-path'], '').get_parsed_arg()
def test_parse_args_backtesting_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['backtesting --ticker-interval'], '').get_parsed_arg()
with pytest.raises(SystemExit, match=r'2'):
Arguments(['backtesting --ticker-interval', 'abc'], '').get_parsed_arg()
def test_parse_args_backtesting_custom() -> None:
args = [
'-c', 'test_conf.json',
'backtesting',
'--ticker-interval', '1m',
'--refresh-pairs-cached',
'--strategy-list',
'DefaultStrategy',
'TestStrategy'
]
call_args = Arguments(args, '').get_parsed_arg()
assert call_args.config == ['test_conf.json']
assert call_args.verbosity == 0
assert call_args.subparser == 'backtesting'
assert call_args.func is not None
assert call_args.ticker_interval == '1m'
assert call_args.refresh_pairs is True
assert type(call_args.strategy_list) is list
assert len(call_args.strategy_list) == 2
def test_parse_args_hyperopt_custom() -> None:
args = [
'-c', 'test_conf.json',
'hyperopt',
'--epochs', '20',
'--spaces', 'buy'
]
call_args = Arguments(args, '').get_parsed_arg()
assert call_args.config == ['test_conf.json']
assert call_args.epochs == 20
assert call_args.verbosity == 0
assert call_args.subparser == 'hyperopt'
assert call_args.spaces == ['buy']
assert call_args.func is not None
def test_download_data_options() -> None:
args = [
'--datadir', 'datadir/directory',
'download-data',
'--pairs-file', 'file_with_pairs',
'--days', '30',
'--exchange', 'binance'
]
args = Arguments(args, '').get_parsed_arg()
assert args.pairs_file == 'file_with_pairs'
assert args.datadir == 'datadir/directory'
assert args.days == 30
assert args.exchange == 'binance'
def test_plot_dataframe_options() -> None:
args = [
'--indicators1', 'sma10,sma100',
'--indicators2', 'macd,fastd,fastk',
'--plot-limit', '30',
'-p', 'UNITTEST/BTC',
]
arguments = Arguments(args, '')
arguments._build_args(ARGS_PLOT_DATAFRAME)
pargs = arguments._parse_args()
assert pargs.indicators1 == "sma10,sma100"
assert pargs.indicators2 == "macd,fastd,fastk"
assert pargs.plot_limit == 30
assert pargs.pairs == ["UNITTEST/BTC"]
def test_check_int_positive() -> None:
assert check_int_positive("3") == 3
assert check_int_positive("1") == 1
assert check_int_positive("100") == 100
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("-2")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("0")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("3.5")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("DeadBeef")

54
freqtrade/utils.py
View File

@ -1,14 +1,14 @@
import logging
import sys
from argparse import Namespace
from pathlib import Path
from typing import Any, Dict
from typing import Any, Dict, List
import arrow
from freqtrade import OperationalException
from freqtrade.configuration import Configuration, TimeRange
from freqtrade.configuration.directory_operations import create_userdata_dir
from freqtrade.data.history import download_pair_history
from freqtrade.data.history import refresh_backtest_ohlcv_data
from freqtrade.exchange import available_exchanges
from freqtrade.resolvers import ExchangeResolver
from freqtrade.state import RunMode
@ -16,7 +16,7 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
def setup_utils_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
"""
Prepare the configuration for utils subcommands
:param args: Cli args from Arguments()
@ -33,34 +33,34 @@ def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any
return config
def start_list_exchanges(args: Namespace) -> None:
def start_list_exchanges(args: Dict[str, Any]) -> None:
"""
Print available exchanges
:param args: Cli args from Arguments()
:return: None
"""
if args.print_one_column:
if args['print_one_column']:
print('\n'.join(available_exchanges()))
else:
print(f"Exchanges supported by ccxt and available for Freqtrade: "
f"{', '.join(available_exchanges())}")
def start_create_userdir(args: Namespace) -> None:
def start_create_userdir(args: Dict[str, Any]) -> None:
"""
Create "user_data" directory to contain user data strategies, hyperopts, ...)
:param args: Cli args from Arguments()
:return: None
"""
if "user_data_dir" in args and args.user_data_dir:
create_userdata_dir(args.user_data_dir, create_dir=True)
if "user_data_dir" in args and args["user_data_dir"]:
create_userdata_dir(args["user_data_dir"], create_dir=True)
else:
logger.warning("`create-userdir` requires --userdir to be set.")
sys.exit(1)
def start_download_data(args: Namespace) -> None:
def start_download_data(args: Dict[str, Any]) -> None:
"""
Download data (former download_backtest_data.py script)
"""
@ -71,43 +71,29 @@ def start_download_data(args: Namespace) -> None:
time_since = arrow.utcnow().shift(days=-config['days']).strftime("%Y%m%d")
timerange = TimeRange.parse_timerange(f'{time_since}-')
if 'pairs' not in config:
raise OperationalException(
"Downloading data requires a list of pairs. "
"Please check the documentation on how to configure this.")
dl_path = Path(config['datadir'])
logger.info(f'About to download pairs: {config["pairs"]}, '
f'intervals: {config["timeframes"]} to {dl_path}')
pairs_not_available = []
pairs_not_available: List[str] = []
try:
# Init exchange
exchange = ExchangeResolver(config['exchange']['name'], config).exchange
for pair in config["pairs"]:
if pair not in exchange.markets:
pairs_not_available.append(pair)
logger.info(f"Skipping pair {pair}...")
continue
for ticker_interval in config["timeframes"]:
pair_print = pair.replace('/', '_')
filename = f'{pair_print}-{ticker_interval}.json'
dl_file = dl_path.joinpath(filename)
if config.get("erase") and dl_file.exists():
logger.info(
f'Deleting existing data for pair {pair}, interval {ticker_interval}.')
dl_file.unlink()
logger.info(f'Downloading pair {pair}, interval {ticker_interval}.')
download_pair_history(datadir=dl_path, exchange=exchange,
pair=pair, ticker_interval=str(ticker_interval),
timerange=timerange)
pairs_not_available = refresh_backtest_ohlcv_data(
exchange, pairs=config["pairs"], timeframes=config["timeframes"],
dl_path=Path(config['datadir']), timerange=timerange, erase=config.get("erase"))
except KeyboardInterrupt:
sys.exit("SIGINT received, aborting ...")
finally:
if pairs_not_available:
logger.info(
f"Pairs [{','.join(pairs_not_available)}] not available "
logger.info(f"Pairs [{','.join(pairs_not_available)}] not available "
f"on exchange {config['exchange']['name']}.")
# configuration.resolve_pairs_list()
print(config)

2
freqtrade/wallets.py
View File

@ -17,7 +17,7 @@ class Wallet(NamedTuple):
total: float = 0
class Wallets(object):
class Wallets:
def __init__(self, config: dict, exchange: Exchange) -> None:
self._config = config

15
freqtrade/worker.py
View File

@ -4,27 +4,26 @@ Main Freqtrade worker class.
import logging
import time
import traceback
from argparse import Namespace
from typing import Any, Callable, Optional
from typing import Any, Callable, Dict, Optional
import sdnotify
from freqtrade import (constants, OperationalException, TemporaryError,
__version__)
from freqtrade import (OperationalException, TemporaryError, __version__,
constants)
from freqtrade.configuration import Configuration
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType
from freqtrade.state import State
logger = logging.getLogger(__name__)
class Worker(object):
class Worker:
"""
Freqtradebot worker class
"""
def __init__(self, args: Namespace, config=None) -> None:
def __init__(self, args: Dict[str, Any], config=None) -> None:
"""
Init all variables and objects the bot needs to work
"""

5
mkdocs.yml
View File

@ -11,15 +11,16 @@ nav:
- Telegram: telegram-usage.md
- Web Hook: webhook-config.md
- REST API: rest-api.md
- Data Downloading: data-download.md
- Backtesting: backtesting.md
- Hyperopt: hyperopt.md
- Edge positioning: edge.md
- Plotting: plotting.md
- Deprecated features: deprecated.md
- FAQ: faq.md
- Data Analysis: data-analysis.md
- Plotting: plotting.md
- SQL Cheatsheet: sql_cheatsheet.md
- Sandbox testing: sandbox-testing.md
- Deprecated features: deprecated.md
- Contributors guide: developer.md
theme:
name: material

18
requirements-common.txt
View File

@ -1,28 +1,22 @@
# requirements without requirements installable via conda
# mainly used for Raspberry pi installs
ccxt==1.18.1068
SQLAlchemy==1.3.7
python-telegram-bot==11.1.0
arrow==0.14.5
ccxt==1.18.1180
SQLAlchemy==1.3.8
python-telegram-bot==12.1.1
arrow==0.15.2
cachetools==3.1.1
requests==2.22.0
urllib3==1.25.3
urllib3==1.25.5
wrapt==1.11.2
scikit-learn==0.21.3
joblib==0.13.2
jsonschema==3.0.2
TA-Lib==0.4.17
tabulate==0.8.3
coinmarketcap==5.0.3
# Required for hyperopt
scikit-optimize==0.5.2
filelock==3.0.12
# find first, C search in arrays
py_find_1st==1.1.4
#Load ticker files 30% faster
# Load ticker files 30% faster
python-rapidjson==0.8.0
# Notify systemd

3
requirements-dev.txt
View File

@ -1,13 +1,14 @@
# Include all requirements to run the bot.
-r requirements.txt
-r requirements-plot.txt
-r requirements-hyperopt.txt
coveralls==1.8.2
flake8==3.7.8
flake8-type-annotations==0.1.0
flake8-tidy-imports==2.0.0
mypy==0.720
pytest==5.1.0
pytest==5.1.3
pytest-asyncio==0.10.0
pytest-cov==2.7.1
pytest-mock==1.10.4

9
requirements-hyperopt.txt Normal file
View File

@ -0,0 +1,9 @@
# Include all requirements to run the bot.
# -r requirements.txt
# Required for hyperopt
scipy==1.3.1
scikit-learn==0.21.3
scikit-optimize==0.5.2
filelock==3.0.12
joblib==0.13.2

2
requirements-plot.txt
View File

@ -1,5 +1,5 @@
# Include all requirements to run the bot.
-r requirements.txt
plotly==4.1.0
plotly==4.1.1

5
requirements.txt
View File

@ -1,6 +1,5 @@
# Load common requirements
-r requirements-common.txt
numpy==1.17.0
pandas==0.25.0
scipy==1.3.1
numpy==1.17.2
pandas==0.25.1

99
scripts/plot_dataframe.py
View File

@ -1,100 +1,11 @@
#!/usr/bin/env python3
"""
Script to display when the bot will buy on specific pair(s)
Use `python plot_dataframe.py --help` to display the command line arguments
Indicators recommended
Row 1: sma, ema3, ema5, ema10, ema50
Row 3: macd, rsi, fisher_rsi, mfi, slowd, slowk, fastd, fastk
Example of usage:
> python3 scripts/plot_dataframe.py --pairs BTC/EUR,XRP/BTC -d user_data/data/
--indicators1 sma,ema3 --indicators2 fastk,fastd
"""
import logging
import sys
from typing import Any, Dict, List
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
from freqtrade.data.btanalysis import extract_trades_of_period
from freqtrade.optimize import setup_configuration
from freqtrade.plot.plotting import (init_plotscript, generate_candlestick_graph,
store_plot_file,
generate_plot_filename)
from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
def analyse_and_plot_pairs(config: Dict[str, Any]):
"""
From arguments provided in cli:
-Initialise backtest env
-Get tickers data
-Generate Dafaframes populated with indicators and signals
-Load trades excecuted on same periods
-Generate Plotly plot objects
-Generate plot files
:return: None
"""
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
strategy = plot_elements["strategy"]
print("This script has been integrated into freqtrade "
"and its functionality is available by calling `freqtrade plot-dataframe`.")
print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
"for details.")
pair_counter = 0
for pair, data in plot_elements["tickers"].items():
pair_counter += 1
logger.info("analyse pair %s", pair)
tickers = {}
tickers[pair] = data
dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
trades_pair = trades.loc[trades['pair'] == pair]
trades_pair = extract_trades_of_period(dataframe, trades_pair)
fig = generate_candlestick_graph(
pair=pair,
data=dataframe,
trades=trades_pair,
indicators1=config["indicators1"].split(","),
indicators2=config["indicators2"].split(",")
)
store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
directory=config['user_data_dir'] / "plot")
logger.info('End of ploting process %s plots generated', pair_counter)
def plot_parse_args(args: List[str]) -> Dict[str, Any]:
"""
Parse args passed to the script
:param args: Cli arguments
:return: args: Array with all arguments
"""
arguments = Arguments(args, 'Graph dataframe')
arguments._build_args(optionlist=ARGS_PLOT_DATAFRAME)
parsed_args = arguments._parse_args()
# Load the configuration
config = setup_configuration(parsed_args, RunMode.OTHER)
return config
def main(sysargv: List[str]) -> None:
"""
This function will initiate the bot and start the trading loop.
:return: None
"""
logger.info('Starting Plot Dataframe')
analyse_and_plot_pairs(
plot_parse_args(sysargv)
)
exit()
if __name__ == '__main__':
main(sys.argv[1:])
sys.exit(1)

65
scripts/plot_profit.py
View File

@ -1,66 +1,11 @@
#!/usr/bin/env python3
"""
Script to display profits
Use `python plot_profit.py --help` to display the command line arguments
"""
import logging
import sys
from typing import Any, Dict, List
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_PROFIT
from freqtrade.optimize import setup_configuration
from freqtrade.plot.plotting import init_plotscript, generate_profit_graph, store_plot_file
from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
def plot_profit(config: Dict[str, Any]) -> None:
"""
Plots the total profit for all pairs.
Note, the profit calculation isn't realistic.
But should be somewhat proportional, and therefor useful
in helping out to find a good algorithm.
"""
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
# Filter trades to relevant pairs
trades = trades[trades['pair'].isin(plot_elements["pairs"])]
print("This script has been integrated into freqtrade "
"and its functionality is available by calling `freqtrade plot-profit`.")
print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
"for details.")
# Create an average close price of all the pairs that were involved.
# this could be useful to gauge the overall market trend
fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
store_plot_file(fig, filename='freqtrade-profit-plot.html',
directory=config['user_data_dir'] / "plot", auto_open=True)
def plot_parse_args(args: List[str]) -> Dict[str, Any]:
"""
Parse args passed to the script
:param args: Cli arguments
:return: args: Array with all arguments
"""
arguments = Arguments(args, 'Graph profits')
arguments._build_args(optionlist=ARGS_PLOT_PROFIT)
parsed_args = arguments._parse_args()
# Load the configuration
config = setup_configuration(parsed_args, RunMode.OTHER)
return config
def main(sysargv: List[str]) -> None:
"""
This function will initiate the bot and start the trading loop.
:return: None
"""
logger.info('Starting Plot Dataframe')
plot_profit(
plot_parse_args(sysargv)
)
if __name__ == '__main__':
main(sys.argv[1:])
sys.exit(1)

7
setup.cfg
View File

@ -2,9 +2,14 @@
#ignore =
max-line-length = 100
max-complexity = 12
exclude =
.git,
__pycache__,
.eggs,
user_data,
[mypy]
ignore_missing_imports = True
[mypy-freqtrade.tests.*]
[mypy-tests.*]
ignore_errors = True

33
setup.py
View File

@ -6,11 +6,25 @@ if version_info.major == 3 and version_info.minor < 6 or \
print('Your Python interpreter must be 3.6 or greater!')
exit(1)
from freqtrade import __version__
from pathlib import Path # noqa: E402
from freqtrade import __version__ # noqa: E402
readme_file = Path(__file__).parent / "README.md"
readme_long = "Crypto Trading Bot"
if readme_file.is_file():
readme_long = (Path(__file__).parent / "README.md").read_text()
# Requirements used for submodules
api = ['flask']
plot = ['plotly>=4.0']
hyperopt = [
'scipy',
'scikit-learn',
'scikit-optimize',
'filelock',
'joblib',
]
develop = [
'coveralls',
@ -31,13 +45,15 @@ jupyter = [
'ipykernel',
]
all_extra = api + plot + develop + jupyter
all_extra = api + plot + develop + jupyter + hyperopt
setup(name='freqtrade',
version=__version__,
description='Crypto Trading Bot',
long_description=readme_long,
long_description_content_type="text/markdown",
url='https://github.com/freqtrade/freqtrade',
author='gcarq and contributors',
author='Freqtrade Team',
author_email='michael.egger@tsn.at',
license='GPLv3',
packages=['freqtrade'],
@ -45,7 +61,7 @@ setup(name='freqtrade',
tests_require=['pytest', 'pytest-mock', 'pytest-cov'],
install_requires=[
# from requirements-common.txt
'ccxt>=1.18',
'ccxt>=1.18.1080',
'SQLAlchemy',
'python-telegram-bot',
'arrow',
@ -53,14 +69,10 @@ setup(name='freqtrade',
'requests',
'urllib3',
'wrapt',
'scikit-learn',
'joblib',
'jsonschema',
'TA-Lib',
'tabulate',
'coinmarketcap',
'scikit-optimize',
'filelock',
'py_find_1st',
'python-rapidjson',
'sdnotify',
@ -68,15 +80,14 @@ setup(name='freqtrade',
# from requirements.txt
'numpy',
'pandas',
'scipy',
],
extras_require={
'api': api,
'dev': all_extra,
'plot': plot,
'all': all_extra,
'jupyter': jupyter,
'hyperopt': hyperopt,
'all': all_extra,
},
include_package_data=True,
zip_safe=False,

0
freqtrade/tests/__init__.py → tests/__init__.py
View File

0
freqtrade/tests/config_test_comments.json → tests/config_test_comments.json
View File

38
freqtrade/tests/conftest.py → tests/conftest.py
View File

@ -45,7 +45,7 @@ def log_has_re(line, logs):
def get_args(args):
return Arguments(args, '').get_parsed_arg()
return Arguments(args).get_parsed_arg()
def patched_configuration_load_config_file(mocker, config) -> None:
@ -117,7 +117,7 @@ def patch_freqtradebot(mocker, config) -> None:
"""
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
persistence.init(config['db_url'])
patch_exchange(mocker, None)
patch_exchange(mocker)
mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
@ -182,7 +182,7 @@ def init_persistence(default_conf):
@pytest.fixture(scope="function")
def default_conf():
def default_conf(testdatadir):
""" Returns validated configuration suitable for most tests """
configuration = {
"max_open_trades": 1,
@ -237,6 +237,7 @@ def default_conf():
"token": "token",
"chat_id": "0"
},
"datadir": str(testdatadir),
"initial_state": "running",
"db_url": "sqlite://",
"user_data_dir": Path("user_data"),
@ -890,8 +891,8 @@ def tickers():
@pytest.fixture
def result():
with Path('freqtrade/tests/testdata/UNITTEST_BTC-1m.json').open('r') as data_file:
def result(testdatadir):
with (testdatadir / 'UNITTEST_BTC-1m.json').open('r') as data_file:
return parse_ticker_dataframe(json.load(data_file), '1m', pair="UNITTEST/BTC",
fill_missing=True)
@ -1047,3 +1048,30 @@ def rpc_balance():
'used': 0.0
},
}
@pytest.fixture
def testdatadir() -> Path:
"""Return the path where testdata files are stored"""
return (Path(__file__).parent / "testdata").resolve()
@pytest.fixture(scope="function")
def import_fails() -> None:
# Source of this test-method:
# https://stackoverflow.com/questions/2481511/mocking-importerror-in-python
import builtins
realimport = builtins.__import__
def mockedimport(name, *args, **kwargs):
if name in ["filelock"]:
raise ImportError(f"No module named '{name}'")
return realimport(name, *args, **kwargs)
builtins.__import__ = mockedimport
# Run test - then cleanup
yield
# restore previous importfunction
builtins.__import__ = realimport

0
freqtrade/tests/data/__init__.py → tests/data/__init__.py
View File

34
freqtrade/tests/data/test_btanalysis.py → tests/data/test_btanalysis.py
View File

@ -11,14 +11,13 @@ from freqtrade.data.btanalysis import (BT_DATA_COLUMNS,
extract_trades_of_period,
load_backtest_data, load_trades,
load_trades_from_db)
from freqtrade.data.history import (load_data, load_pair_history,
make_testdata_path)
from freqtrade.tests.test_persistence import create_mock_trades
from freqtrade.data.history import load_data, load_pair_history
from tests.test_persistence import create_mock_trades
def test_load_backtest_data():
def test_load_backtest_data(testdatadir):
filename = make_testdata_path(None) / "backtest-result_test.json"
filename = testdatadir / "backtest-result_test.json"
bt_data = load_backtest_data(filename)
assert isinstance(bt_data, DataFrame)
assert list(bt_data.columns) == BT_DATA_COLUMNS + ["profitabs"]
@ -52,12 +51,12 @@ def test_load_trades_db(default_conf, fee, mocker):
assert col in trades.columns
def test_extract_trades_of_period():
def test_extract_trades_of_period(testdatadir):
pair = "UNITTEST/BTC"
timerange = TimeRange(None, 'line', 0, -1000)
data = load_pair_history(pair=pair, ticker_interval='1m',
datadir=None, timerange=timerange)
datadir=testdatadir, timerange=timerange)
# timerange = 2017-11-14 06:07 - 2017-11-14 22:58:00
trades = DataFrame(
@ -89,25 +88,28 @@ def test_load_trades(default_conf, mocker):
db_mock = mocker.patch("freqtrade.data.btanalysis.load_trades_from_db", MagicMock())
bt_mock = mocker.patch("freqtrade.data.btanalysis.load_backtest_data", MagicMock())
default_conf['trade_source'] = "DB"
load_trades(default_conf)
load_trades("DB",
db_url=default_conf.get('db_url'),
exportfilename=default_conf.get('exportfilename'),
)
assert db_mock.call_count == 1
assert bt_mock.call_count == 0
db_mock.reset_mock()
bt_mock.reset_mock()
default_conf['trade_source'] = "file"
default_conf['exportfilename'] = "testfile.json"
load_trades(default_conf)
load_trades("file",
db_url=default_conf.get('db_url'),
exportfilename=default_conf.get('exportfilename'),)
assert db_mock.call_count == 0
assert bt_mock.call_count == 1
def test_combine_tickers_with_mean():
def test_combine_tickers_with_mean(testdatadir):
pairs = ["ETH/BTC", "XLM/BTC"]
tickers = load_data(datadir=None,
tickers = load_data(datadir=testdatadir,
pairs=pairs,
ticker_interval='5m'
)
@ -118,13 +120,13 @@ def test_combine_tickers_with_mean():
assert "mean" in df.columns
def test_create_cum_profit():
filename = make_testdata_path(None) / "backtest-result_test.json"
def test_create_cum_profit(testdatadir):
filename = testdatadir / "backtest-result_test.json"
bt_data = load_backtest_data(filename)
timerange = TimeRange.parse_timerange("20180110-20180112")
df = load_pair_history(pair="POWR/BTC", ticker_interval='5m',
datadir=None, timerange=timerange)
datadir=testdatadir, timerange=timerange)
cum_profits = create_cum_profit(df.set_index('date'),
bt_data[bt_data["pair"] == 'POWR/BTC'],

7
freqtrade/tests/data/test_converter.py → tests/data/test_converter.py
View File

@ -3,7 +3,7 @@ import logging
from freqtrade.data.converter import parse_ticker_dataframe, ohlcv_fill_up_missing_data
from freqtrade.data.history import load_pair_history, validate_backtest_data, get_timeframe
from freqtrade.tests.conftest import log_has
from tests.conftest import log_has
def test_dataframe_correct_columns(result):
@ -21,10 +21,9 @@ def test_parse_ticker_dataframe(ticker_history_list, caplog):
assert log_has('Parsing tickerlist to dataframe', caplog)
def test_ohlcv_fill_up_missing_data(caplog):
data = load_pair_history(datadir=None,
def test_ohlcv_fill_up_missing_data(testdatadir, caplog):
data = load_pair_history(datadir=testdatadir,
ticker_interval='1m',
refresh_pairs=False,
pair='UNITTEST/BTC',
fill_up_missing=False)
caplog.set_level(logging.DEBUG)

4
freqtrade/tests/data/test_dataprovider.py → tests/data/test_dataprovider.py
View File

@ -4,7 +4,7 @@ from pandas import DataFrame
from freqtrade.data.dataprovider import DataProvider
from freqtrade.state import RunMode
from freqtrade.tests.conftest import get_patched_exchange
from tests.conftest import get_patched_exchange
def test_ohlcv(mocker, default_conf, ticker_history):
@ -45,8 +45,6 @@ def test_historic_ohlcv(mocker, default_conf, ticker_history):
data = dp.historic_ohlcv("UNITTEST/BTC", "5m")
assert isinstance(data, DataFrame)
assert historymock.call_count == 1
assert historymock.call_args_list[0][1]["datadir"] is None
assert historymock.call_args_list[0][1]["refresh_pairs"] is False
assert historymock.call_args_list[0][1]["ticker_interval"] == "5m"

159
freqtrade/tests/data/test_history.py → tests/data/test_history.py
View File

@ -5,7 +5,7 @@ import os
import uuid
from pathlib import Path
from shutil import copyfile
from unittest.mock import MagicMock
from unittest.mock import MagicMock, PropertyMock
import arrow
import pytest
@ -16,13 +16,13 @@ from freqtrade.configuration import TimeRange
from freqtrade.data import history
from freqtrade.data.history import (download_pair_history,
load_cached_data_for_updating,
load_tickerdata_file, make_testdata_path,
load_tickerdata_file,
refresh_backtest_ohlcv_data,
trim_tickerlist)
from freqtrade.exchange import timeframe_to_minutes
from freqtrade.misc import file_dump_json
from freqtrade.strategy.default_strategy import DefaultStrategy
from freqtrade.tests.conftest import (get_patched_exchange, log_has,
patch_exchange)
from tests.conftest import get_patched_exchange, log_has, log_has_re, patch_exchange
# Change this if modifying UNITTEST/BTC testdatafile
_BTC_UNITTEST_LENGTH = 13681
@ -59,8 +59,8 @@ def _clean_test_file(file: str) -> None:
os.rename(file_swp, file)
def test_load_data_30min_ticker(mocker, caplog, default_conf) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='30m', datadir=None)
def test_load_data_30min_ticker(mocker, caplog, default_conf, testdatadir) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='30m', datadir=testdatadir)
assert isinstance(ld, DataFrame)
assert not log_has(
'Download history data for pair: "UNITTEST/BTC", interval: 30m '
@ -68,22 +68,21 @@ def test_load_data_30min_ticker(mocker, caplog, default_conf) -> None:
)
def test_load_data_7min_ticker(mocker, caplog, default_conf) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='7m', datadir=None)
def test_load_data_7min_ticker(mocker, caplog, default_conf, testdatadir) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='7m', datadir=testdatadir)
assert not isinstance(ld, DataFrame)
assert ld is None
assert log_has(
'No history data for pair: "UNITTEST/BTC", interval: 7m. '
'Use --refresh-pairs-cached option or `freqtrade download-data` '
'script to download the data', caplog
'Use `freqtrade download-data` to download the data', caplog
)
def test_load_data_1min_ticker(ticker_history, mocker, caplog) -> None:
def test_load_data_1min_ticker(ticker_history, mocker, caplog, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-1m.json')
_backup_file(file, copy_file=True)
history.load_data(datadir=None, ticker_interval='1m', pairs=['UNITTEST/BTC'])
history.load_data(datadir=testdatadir, ticker_interval='1m', pairs=['UNITTEST/BTC'])
assert os.path.isfile(file) is True
assert not log_has(
'Download history data for pair: "UNITTEST/BTC", interval: 1m '
@ -92,7 +91,8 @@ def test_load_data_1min_ticker(ticker_history, mocker, caplog) -> None:
_clean_test_file(file)
def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, default_conf) -> None:
def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog,
default_conf, testdatadir) -> None:
"""
Test load_pair_history() with 1 min ticker
"""
@ -102,30 +102,28 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, defau
_backup_file(file)
# do not download a new pair if refresh_pairs isn't set
history.load_pair_history(datadir=None,
history.load_pair_history(datadir=testdatadir,
ticker_interval='1m',
refresh_pairs=False,
pair='MEME/BTC')
assert os.path.isfile(file) is False
assert log_has(
'No history data for pair: "MEME/BTC", interval: 1m. '
'Use --refresh-pairs-cached option or `freqtrade download-data` '
'script to download the data', caplog
'Use `freqtrade download-data` to download the data', caplog
)
# download a new pair if refresh_pairs is set
history.load_pair_history(datadir=None,
history.load_pair_history(datadir=testdatadir,
ticker_interval='1m',
refresh_pairs=True,
exchange=exchange,
pair='MEME/BTC')
assert os.path.isfile(file) is True
assert log_has(
assert log_has_re(
'Download history data for pair: "MEME/BTC", interval: 1m '
'and store in None.', caplog
'and store in .*', caplog
)
with pytest.raises(OperationalException, match=r'Exchange needs to be initialized when.*'):
history.load_pair_history(datadir=None,
history.load_pair_history(datadir=testdatadir,
ticker_interval='1m',
refresh_pairs=True,
exchange=None,
@ -133,33 +131,8 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, defau
_clean_test_file(file)
def test_load_data_live(default_conf, mocker, caplog) -> None:
refresh_mock = MagicMock()
mocker.patch("freqtrade.exchange.Exchange.refresh_latest_ohlcv", refresh_mock)
exchange = get_patched_exchange(mocker, default_conf)
history.load_data(datadir=None, ticker_interval='5m',
pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
live=True,
exchange=exchange)
assert refresh_mock.call_count == 1
assert len(refresh_mock.call_args_list[0][0][0]) == 2
assert log_has('Live: Downloading data for all defined pairs ...', caplog)
def test_load_data_live_noexchange(default_conf, mocker, caplog) -> None:
with pytest.raises(OperationalException,
match=r'Exchange needs to be initialized when using live data.'):
history.load_data(datadir=None, ticker_interval='5m',
pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
exchange=None,
live=True,
)
def test_testdata_path() -> None:
assert str(Path('freqtrade') / 'tests' / 'testdata') in str(make_testdata_path(None))
def test_testdata_path(testdatadir) -> None:
assert str(Path('tests') / 'testdata') in str(testdatadir)
def test_load_cached_data_for_updating(mocker) -> None:
@ -247,7 +220,7 @@ def test_load_cached_data_for_updating(mocker) -> None:
assert start_ts is None
def test_download_pair_history(ticker_history_list, mocker, default_conf) -> None:
def test_download_pair_history(ticker_history_list, mocker, default_conf, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history_list)
exchange = get_patched_exchange(mocker, default_conf)
file1_1 = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-1m.json')
@ -263,10 +236,10 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
assert os.path.isfile(file1_1) is False
assert os.path.isfile(file2_1) is False
assert download_pair_history(datadir=None, exchange=exchange,
assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC',
ticker_interval='1m')
assert download_pair_history(datadir=None, exchange=exchange,
assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='CFI/BTC',
ticker_interval='1m')
assert not exchange._pairs_last_refresh_time
@ -280,10 +253,10 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
assert os.path.isfile(file1_5) is False
assert os.path.isfile(file2_5) is False
assert download_pair_history(datadir=None, exchange=exchange,
assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC',
ticker_interval='5m')
assert download_pair_history(datadir=None, exchange=exchange,
assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='CFI/BTC',
ticker_interval='5m')
assert not exchange._pairs_last_refresh_time
@ -295,7 +268,7 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
_clean_test_file(file2_5)
def test_download_pair_history2(mocker, default_conf) -> None:
def test_download_pair_history2(mocker, default_conf, testdatadir) -> None:
tick = [
[1509836520000, 0.00162008, 0.00162008, 0.00162008, 0.00162008, 108.14853839],
[1509836580000, 0.00161, 0.00161, 0.00161, 0.00161, 82.390199]
@ -303,12 +276,13 @@ def test_download_pair_history2(mocker, default_conf) -> None:
json_dump_mock = mocker.patch('freqtrade.misc.file_dump_json', return_value=None)
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=tick)
exchange = get_patched_exchange(mocker, default_conf)
download_pair_history(None, exchange, pair="UNITTEST/BTC", ticker_interval='1m')
download_pair_history(None, exchange, pair="UNITTEST/BTC", ticker_interval='3m')
download_pair_history(testdatadir, exchange, pair="UNITTEST/BTC", ticker_interval='1m')
download_pair_history(testdatadir, exchange, pair="UNITTEST/BTC", ticker_interval='3m')
assert json_dump_mock.call_count == 2
def test_download_backtesting_data_exception(ticker_history, mocker, caplog, default_conf) -> None:
def test_download_backtesting_data_exception(ticker_history, mocker, caplog,
default_conf, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv',
side_effect=Exception('File Error'))
@ -319,7 +293,7 @@ def test_download_backtesting_data_exception(ticker_history, mocker, caplog, def
_backup_file(file1_1)
_backup_file(file1_5)
assert not download_pair_history(datadir=None, exchange=exchange,
assert not download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC',
ticker_interval='1m')
# clean files freshly downloaded
@ -331,23 +305,22 @@ def test_download_backtesting_data_exception(ticker_history, mocker, caplog, def
)
def test_load_tickerdata_file() -> None:
def test_load_tickerdata_file(testdatadir) -> None:
# 7 does not exist in either format.
assert not load_tickerdata_file(None, 'UNITTEST/BTC', '7m')
assert not load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '7m')
# 1 exists only as a .json
tickerdata = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
tickerdata = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
assert _BTC_UNITTEST_LENGTH == len(tickerdata)
# 8 .json is empty and will fail if it's loaded. .json.gz is a copy of 1.json
tickerdata = load_tickerdata_file(None, 'UNITTEST/BTC', '8m')
tickerdata = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '8m')
assert _BTC_UNITTEST_LENGTH == len(tickerdata)
def test_load_partial_missing(caplog) -> None:
def test_load_partial_missing(testdatadir, caplog) -> None:
# Make sure we start fresh - test missing data at start
start = arrow.get('2018-01-01T00:00:00')
end = arrow.get('2018-01-11T00:00:00')
tickerdata = history.load_data(None, '5m', ['UNITTEST/BTC'],
refresh_pairs=False,
tickerdata = history.load_data(testdatadir, '5m', ['UNITTEST/BTC'],
timerange=TimeRange('date', 'date',
start.timestamp, end.timestamp))
# timedifference in 5 minutes
@ -361,8 +334,8 @@ def test_load_partial_missing(caplog) -> None:
caplog.clear()
start = arrow.get('2018-01-10T00:00:00')
end = arrow.get('2018-02-20T00:00:00')
tickerdata = history.load_data(datadir=None, ticker_interval='5m',
pairs=['UNITTEST/BTC'], refresh_pairs=False,
tickerdata = history.load_data(datadir=testdatadir, ticker_interval='5m',
pairs=['UNITTEST/BTC'],
timerange=TimeRange('date', 'date',
start.timestamp, end.timestamp))
# timedifference in 5 minutes
@ -501,13 +474,13 @@ def test_file_dump_json_tofile() -> None:
_clean_test_file(file)
def test_get_timeframe(default_conf, mocker) -> None:
def test_get_timeframe(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker)
strategy = DefaultStrategy(default_conf)
data = strategy.tickerdata_to_dataframe(
history.load_data(
datadir=None,
datadir=testdatadir,
ticker_interval='1m',
pairs=['UNITTEST/BTC']
)
@ -517,13 +490,13 @@ def test_get_timeframe(default_conf, mocker) -> None:
assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None:
def test_validate_backtest_data_warn(default_conf, mocker, caplog, testdatadir) -> None:
patch_exchange(mocker)
strategy = DefaultStrategy(default_conf)
data = strategy.tickerdata_to_dataframe(
history.load_data(
datadir=None,
datadir=testdatadir,
ticker_interval='1m',
pairs=['UNITTEST/BTC'],
fill_up_missing=False
@ -539,14 +512,14 @@ def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None:
caplog)
def test_validate_backtest_data(default_conf, mocker, caplog) -> None:
def test_validate_backtest_data(default_conf, mocker, caplog, testdatadir) -> None:
patch_exchange(mocker)
strategy = DefaultStrategy(default_conf)
timerange = TimeRange('index', 'index', 200, 250)
data = strategy.tickerdata_to_dataframe(
history.load_data(
datadir=None,
datadir=testdatadir,
ticker_interval='5m',
pairs=['UNITTEST/BTC'],
timerange=timerange
@ -558,3 +531,43 @@ def test_validate_backtest_data(default_conf, mocker, caplog) -> None:
assert not history.validate_backtest_data(data['UNITTEST/BTC'], 'UNITTEST/BTC',
min_date, max_date, timeframe_to_minutes('5m'))
assert len(caplog.record_tuples) == 0
def test_refresh_backtest_ohlcv_data(mocker, default_conf, markets, caplog, testdatadir):
dl_mock = mocker.patch('freqtrade.data.history.download_pair_history', MagicMock())
mocker.patch(
'freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets)
)
mocker.patch.object(Path, "exists", MagicMock(return_value=True))
mocker.patch.object(Path, "unlink", MagicMock())
ex = get_patched_exchange(mocker, default_conf)
timerange = TimeRange.parse_timerange("20190101-20190102")
refresh_backtest_ohlcv_data(exchange=ex, pairs=["ETH/BTC", "XRP/BTC"],
timeframes=["1m", "5m"], dl_path=testdatadir,
timerange=timerange, erase=True
)
assert dl_mock.call_count == 4
assert dl_mock.call_args[1]['timerange'].starttype == 'date'
assert log_has("Downloading pair ETH/BTC, interval 1m.", caplog)
def test_download_data_no_markets(mocker, default_conf, caplog, testdatadir):
dl_mock = mocker.patch('freqtrade.data.history.download_pair_history', MagicMock())
mocker.patch(
'freqtrade.exchange.Exchange.markets', PropertyMock(return_value={})
)
ex = get_patched_exchange(mocker, default_conf)
timerange = TimeRange.parse_timerange("20190101-20190102")
unav_pairs = refresh_backtest_ohlcv_data(exchange=ex, pairs=["ETH/BTC", "XRP/BTC"],
timeframes=["1m", "5m"],
dl_path=testdatadir,
timerange=timerange, erase=False
)
assert dl_mock.call_count == 0
assert "ETH/BTC" in unav_pairs
assert "XRP/BTC" in unav_pairs
assert log_has("Skipping pair ETH/BTC...", caplog)

0
freqtrade/tests/edge/__init__.py → tests/edge/__init__.py
View File

10
freqtrade/tests/edge/test_edge.py → tests/edge/test_edge.py
View File

@ -14,9 +14,8 @@ from freqtrade import OperationalException
from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.edge import Edge, PairInfo
from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import get_patched_freqtradebot, log_has
from freqtrade.tests.optimize import (BTContainer, BTrade,
_build_backtest_dataframe,
from tests.conftest import get_patched_freqtradebot, log_has
from tests.optimize import (BTContainer, BTrade, _build_backtest_dataframe,
_get_frame_time_from_offset)
# Cases to be tested:
@ -156,8 +155,6 @@ def test_edge_results(edge_conf, mocker, caplog, data) -> None:
trades = edge._find_trades_for_stoploss_range(frame, 'TEST/BTC', [data.stop_loss])
results = edge._fill_calculable_fields(DataFrame(trades)) if trades else DataFrame()
print(results)
assert len(trades) == len(data.trades)
if not results.empty:
@ -291,7 +288,6 @@ def mocked_load_data(datadir, pairs=[], ticker_interval='0m', refresh_pairs=Fals
def test_edge_process_downloaded_data(mocker, edge_conf):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', mocked_load_data)
@ -303,7 +299,6 @@ def test_edge_process_downloaded_data(mocker, edge_conf):
def test_edge_process_no_data(mocker, edge_conf, caplog):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', MagicMock(return_value={}))
@ -316,7 +311,6 @@ def test_edge_process_no_data(mocker, edge_conf, caplog):
def test_edge_process_no_trades(mocker, edge_conf, caplog):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', mocked_load_data)

0
freqtrade/tests/exchange/__init__.py → tests/exchange/__init__.py
View File

92
tests/exchange/test_binance.py Normal file
View File

@ -0,0 +1,92 @@
from random import randint
from unittest.mock import MagicMock
import ccxt
import pytest
from freqtrade import (DependencyException, InvalidOrderException,
OperationalException, TemporaryError)
from tests.conftest import get_patched_exchange
def test_stoploss_limit_order(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'stop_loss_limit'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'stopPrice': 220}
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds("0 balance"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(InvalidOrderException):
api_mock.create_order = MagicMock(
side_effect=ccxt.InvalidOrder("binance Order would trigger immediately."))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError("No connection"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(OperationalException, match=r".*DeadBeef.*"):
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError("DeadBeef"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
def test_stoploss_limit_order_dry_run(default_conf, mocker):
api_mock = MagicMock()
order_type = 'stop_loss_limit'
default_conf['dry_run'] = True
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert 'type' in order
assert order['type'] == order_type
assert order['price'] == 220
assert order['amount'] == 1

112
freqtrade/tests/exchange/test_exchange.py → tests/exchange/test_exchange.py
View File

@ -20,7 +20,7 @@ from freqtrade.exchange.exchange import (API_RETRY_COUNT, timeframe_to_minutes,
timeframe_to_prev_date,
timeframe_to_seconds)
from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from freqtrade.tests.conftest import get_patched_exchange, log_has, log_has_re
from tests.conftest import get_patched_exchange, log_has, log_has_re
# Make sure to always keep one exchange here which is NOT subclassed!!
EXCHANGES = ['bittrex', 'binance', 'kraken', ]
@ -101,18 +101,21 @@ def test_destroy(default_conf, mocker, caplog):
def test_init_exception(default_conf, mocker):
default_conf['exchange']['name'] = 'wrong_exchange_name'
with pytest.raises(
OperationalException,
match='Exchange {} is not supported'.format(default_conf['exchange']['name'])):
with pytest.raises(OperationalException,
match=f"Exchange {default_conf['exchange']['name']} is not supported"):
Exchange(default_conf)
default_conf['exchange']['name'] = 'binance'
with pytest.raises(
OperationalException,
match='Exchange {} is not supported'.format(default_conf['exchange']['name'])):
with pytest.raises(OperationalException,
match=f"Exchange {default_conf['exchange']['name']} is not supported"):
mocker.patch("ccxt.binance", MagicMock(side_effect=AttributeError))
Exchange(default_conf)
with pytest.raises(OperationalException,
match=r"Initialization of ccxt failed. Reason: DeadBeef"):
mocker.patch("ccxt.binance", MagicMock(side_effect=ccxt.BaseError("DeadBeef")))
Exchange(default_conf)
def test_exchange_resolver(default_conf, mocker, caplog):
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=MagicMock()))
@ -864,7 +867,7 @@ def test_get_balance_dry_run(default_conf, mocker):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_get_balance_prod(default_conf, mocker, exchange_name):
api_mock = MagicMock()
api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4}})
api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4, 'total': 123.4}})
default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
@ -880,6 +883,7 @@ def test_get_balance_prod(default_conf, mocker, exchange_name):
with pytest.raises(TemporaryError, match=r'.*balance due to malformed exchange response:.*'):
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
mocker.patch('freqtrade.exchange.Exchange.get_balances', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Kraken.get_balances', MagicMock(return_value={}))
exchange.get_balance(currency='BTC')
@ -1032,7 +1036,6 @@ def test_get_historic_ohlcv(default_conf, mocker, caplog, exchange_name):
exchange._async_get_candle_history = Mock(wraps=mock_candle_hist)
# one_call calculation * 1.8 should do 2 calls
since = 5 * 60 * 500 * 1.8
print(f"since = {since}")
ret = exchange.get_historic_ohlcv(pair, "5m", int((arrow.utcnow().timestamp - since) * 1000))
assert exchange._async_get_candle_history.call_count == 2
@ -1337,7 +1340,6 @@ def test_get_order(default_conf, mocker, exchange_name):
order.myid = 123
exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
exchange._dry_run_open_orders['X'] = order
print(exchange.get_order('X', 'TKN/BTC'))
assert exchange.get_order('X', 'TKN/BTC').myid == 123
with pytest.raises(InvalidOrderException, match=r'Tried to get an invalid dry-run-order.*'):
@ -1362,9 +1364,7 @@ def test_get_order(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_name(default_conf, mocker, exchange_name):
mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
default_conf['exchange']['name'] = exchange_name
exchange = Exchange(default_conf)
exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
assert exchange.name == exchange_name.title()
assert exchange.id == exchange_name
@ -1436,87 +1436,11 @@ def test_get_fee(default_conf, mocker, exchange_name):
'get_fee', 'calculate_fee')
def test_stoploss_limit_order(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'stop_loss_limit'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'stopPrice': 220}
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds("0 balance"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
def test_stoploss_limit_order_unsupported_exchange(default_conf, mocker):
exchange = get_patched_exchange(mocker, default_conf, 'bittrex')
with pytest.raises(OperationalException, match=r"stoploss_limit is not implemented .*"):
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InvalidOrder("Order not found"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError("No connection"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(OperationalException):
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError("DeadBeef"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
def test_stoploss_limit_order_dry_run(default_conf, mocker):
api_mock = MagicMock()
order_type = 'stop_loss_limit'
default_conf['dry_run'] = True
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert 'type' in order
assert order['type'] == order_type
assert order['price'] == 220
assert order['amount'] == 1
def test_merge_ft_has_dict(default_conf, mocker):
mocker.patch.multiple('freqtrade.exchange.Exchange',
@ -1604,7 +1528,7 @@ def test_timeframe_to_prev_date():
assert timeframe_to_prev_date(interval, date) == result
date = datetime.now(tz=timezone.utc)
assert timeframe_to_prev_date("5m", date) < date
assert timeframe_to_prev_date("5m") < date
def test_timeframe_to_next_date():
@ -1629,4 +1553,4 @@ def test_timeframe_to_next_date():
assert timeframe_to_next_date(interval, date) == result
date = datetime.now(tz=timezone.utc)
assert timeframe_to_next_date("5m", date) > date
assert timeframe_to_next_date("5m") > date

151
tests/exchange/test_kraken.py Normal file
View File

@ -0,0 +1,151 @@
# pragma pylint: disable=missing-docstring, C0103, bad-continuation, global-statement
# pragma pylint: disable=protected-access
from random import randint
from unittest.mock import MagicMock
from tests.conftest import get_patched_exchange
from tests.exchange.test_exchange import ccxt_exceptionhandlers
def test_buy_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'limit'
time_in_force = 'ioc'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.buy(pair='ETH/BTC', ordertype=order_type,
amount=1, rate=200, time_in_force=time_in_force)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'timeInForce': 'ioc',
'trading_agreement': 'agree'}
def test_sell_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
assert api_mock.create_order.call_args[0][5] == {'trading_agreement': 'agree'}
def test_get_balances_prod(default_conf, mocker):
balance_item = {
'free': None,
'total': 10.0,
'used': 0.0
}
api_mock = MagicMock()
api_mock.fetch_balance = MagicMock(return_value={
'1ST': balance_item.copy(),
'2ST': balance_item.copy(),
'3ST': balance_item.copy(),
'4ST': balance_item.copy(),
})
kraken_open_orders = [{'symbol': '1ST/EUR',
'type': 'limit',
'side': 'sell',
'price': 20,
'cost': 0.0,
'amount': 1.0,
'filled': 0.0,
'average': 0.0,
'remaining': 1.0,
},
{'status': 'open',
'symbol': '2ST/EUR',
'type': 'limit',
'side': 'sell',
'price': 20.0,
'cost': 0.0,
'amount': 2.0,
'filled': 0.0,
'average': 0.0,
'remaining': 2.0,
},
{'status': 'open',
'symbol': '2ST/USD',
'type': 'limit',
'side': 'sell',
'price': 20.0,
'cost': 0.0,
'amount': 2.0,
'filled': 0.0,
'average': 0.0,
'remaining': 2.0,
},
{'status': 'open',
'symbol': 'BTC/3ST',
'type': 'limit',
'side': 'buy',
'price': 20,
'cost': 0.0,
'amount': 3.0,
'filled': 0.0,
'average': 0.0,
'remaining': 3.0,
}]
api_mock.fetch_open_orders = MagicMock(return_value=kraken_open_orders)
default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
balances = exchange.get_balances()
assert len(balances) == 4
assert balances['1ST']['free'] == 9.0
assert balances['1ST']['total'] == 10.0
assert balances['1ST']['used'] == 1.0
assert balances['2ST']['free'] == 6.0
assert balances['2ST']['total'] == 10.0
assert balances['2ST']['used'] == 4.0
assert balances['3ST']['free'] == 7.0
assert balances['3ST']['total'] == 10.0
assert balances['3ST']['used'] == 3.0
assert balances['4ST']['free'] == 10.0
assert balances['4ST']['total'] == 10.0
assert balances['4ST']['used'] == 0.0
ccxt_exceptionhandlers(mocker, default_conf, api_mock, "kraken",
"get_balances", "fetch_balance")

0
freqtrade/tests/optimize/__init__.py → tests/optimize/__init__.py
View File

13
freqtrade/tests/optimize/test_backtest_detail.py → tests/optimize/test_backtest_detail.py
View File

@ -8,11 +8,9 @@ from pandas import DataFrame
from freqtrade.data.history import get_timeframe
from freqtrade.optimize.backtesting import Backtesting
from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import patch_exchange
from freqtrade.tests.optimize import (BTContainer, BTrade,
_build_backtest_dataframe,
_get_frame_time_from_offset,
tests_ticker_interval)
from tests.conftest import patch_exchange
from tests.optimize import (BTContainer, BTrade, _build_backtest_dataframe,
_get_frame_time_from_offset, tests_ticker_interval)
# Test 0: Sell with signal sell in candle 3
# Test with Stop-loss at 1%
@ -293,8 +291,8 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
patch_exchange(mocker)
frame = _build_backtest_dataframe(data.data)
backtesting = Backtesting(default_conf)
backtesting.advise_buy = lambda a, m: frame
backtesting.advise_sell = lambda a, m: frame
backtesting.strategy.advise_buy = lambda a, m: frame
backtesting.strategy.advise_sell = lambda a, m: frame
caplog.set_level(logging.DEBUG)
pair = "UNITTEST/BTC"
@ -310,7 +308,6 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
'end_date': max_date,
}
)
print(results.T)
assert len(results) == len(data.trades)
assert round(results["profit_percent"].sum(), 3) == round(data.profit_perc, 3)

109
freqtrade/tests/optimize/test_backtesting.py → tests/optimize/test_backtesting.py
View File

@ -22,8 +22,7 @@ from freqtrade.optimize.backtesting import Backtesting
from freqtrade.state import RunMode
from freqtrade.strategy.default_strategy import DefaultStrategy
from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
patch_exchange,
from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patched_configuration_load_config_file)
@ -34,9 +33,9 @@ def trim_dictlist(dict_list, num):
return new
def load_data_test(what):
def load_data_test(what, testdatadir):
timerange = TimeRange(None, 'line', 0, -101)
pair = history.load_tickerdata_file(None, ticker_interval='1m',
pair = history.load_tickerdata_file(testdatadir, ticker_interval='1m',
pair='UNITTEST/BTC', timerange=timerange)
datalen = len(pair)
@ -79,12 +78,12 @@ def load_data_test(what):
fill_missing=True)}
def simple_backtest(config, contour, num_results, mocker) -> None:
def simple_backtest(config, contour, num_results, mocker, testdatadir) -> None:
patch_exchange(mocker)
config['ticker_interval'] = '1m'
backtesting = Backtesting(config)
data = load_data_test(contour)
data = load_data_test(contour, testdatadir)
processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(processed)
assert isinstance(processed, dict)
@ -118,8 +117,8 @@ def _load_pair_as_ticks(pair, tickfreq):
# FIX: fixturize this?
def _make_backtest_conf(mocker, conf=None, pair='UNITTEST/BTC', record=None):
data = history.load_data(datadir=None, ticker_interval='1m', pairs=[pair])
def _make_backtest_conf(mocker, datadir, conf=None, pair='UNITTEST/BTC', record=None):
data = history.load_data(datadir=datadir, ticker_interval='1m', pairs=[pair])
data = trim_dictlist(data, -201)
patch_exchange(mocker)
backtesting = Backtesting(conf)
@ -189,16 +188,12 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert 'position_stacking' not in config
assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config
assert 'export' not in config
assert 'runmode' in config
assert config['runmode'] == RunMode.BACKTEST
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch(
@ -214,7 +209,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
'--ticker-interval', '1m',
'--enable-position-stacking',
'--disable-max-market-positions',
'--refresh-pairs-cached',
'--timerange', ':100',
'--export', '/bar/foo',
'--export-filename', 'foo_bar.json'
@ -241,9 +235,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
assert log_has('max_open_trades set to unlimited ...', caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
@ -314,8 +305,8 @@ def test_backtesting_init(mocker, default_conf, order_types) -> None:
assert backtesting.config == default_conf
assert backtesting.ticker_interval == '5m'
assert callable(backtesting.strategy.tickerdata_to_dataframe)
assert callable(backtesting.advise_buy)
assert callable(backtesting.advise_sell)
assert callable(backtesting.strategy.advise_buy)
assert callable(backtesting.strategy.advise_sell)
assert isinstance(backtesting.strategy.dp, DataProvider)
get_fee.assert_called()
assert backtesting.fee == 0.5
@ -330,7 +321,7 @@ def test_backtesting_init_no_ticker_interval(mocker, default_conf, caplog) -> No
patch_exchange(mocker)
del default_conf['ticker_interval']
default_conf['strategy_list'] = ['DefaultStrategy',
'TestStrategy']
'SampleStrategy']
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.5))
with pytest.raises(OperationalException):
@ -339,10 +330,10 @@ def test_backtesting_init_no_ticker_interval(mocker, default_conf, caplog) -> No
"or as cli argument `--ticker-interval 5m`", caplog)
def test_tickerdata_to_dataframe_bt(default_conf, mocker) -> None:
def test_tickerdata_to_dataframe_bt(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker)
timerange = TimeRange(None, 'line', 0, -100)
tick = history.load_tickerdata_file(None, 'UNITTEST/BTC', '1m', timerange=timerange)
tick = history.load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m', timerange=timerange)
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)}
@ -452,11 +443,10 @@ def test_generate_text_table_strategyn(default_conf, mocker):
'| LTC/BTC | 3 | 30.00 | 90.00 '
'| 1.30000000 | 45.00 | 0:20:00 | 3 | 0 |'
)
print(backtesting._generate_text_table_strategy(all_results=results))
assert backtesting._generate_text_table_strategy(all_results=results) == result_str
def test_backtesting_start(default_conf, mocker, caplog) -> None:
def test_backtesting_start(default_conf, mocker, testdatadir, caplog) -> None:
def get_timeframe(input1):
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
@ -472,7 +462,7 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
default_conf['ticker_interval'] = '1m'
default_conf['datadir'] = None
default_conf['datadir'] = testdatadir
default_conf['export'] = None
default_conf['timerange'] = '-100'
@ -489,7 +479,7 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
assert log_has(line, caplog)
def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
def test_backtesting_start_no_data(default_conf, mocker, caplog, testdatadir) -> None:
def get_timeframe(input1):
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
@ -505,7 +495,7 @@ def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
default_conf['ticker_interval'] = "1m"
default_conf['datadir'] = None
default_conf['datadir'] = testdatadir
default_conf['export'] = None
default_conf['timerange'] = '20180101-20180102'
@ -516,13 +506,13 @@ def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
assert log_has('No data found. Terminating.', caplog)
def test_backtest(default_conf, fee, mocker) -> None:
def test_backtest(default_conf, fee, mocker, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker)
backtesting = Backtesting(default_conf)
pair = 'UNITTEST/BTC'
timerange = TimeRange(None, 'line', 0, -201)
data = history.load_data(datadir=None, ticker_interval='5m', pairs=['UNITTEST/BTC'],
data = history.load_data(datadir=testdatadir, ticker_interval='5m', pairs=['UNITTEST/BTC'],
timerange=timerange)
data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(data_processed)
@ -570,14 +560,14 @@ def test_backtest(default_conf, fee, mocker) -> None:
t["close_rate"], 6) < round(ln.iloc[0]["high"], 6))
def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None:
def test_backtest_1min_ticker_interval(default_conf, fee, mocker, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker)
backtesting = Backtesting(default_conf)
# Run a backtesting for an exiting 1min ticker_interval
timerange = TimeRange(None, 'line', 0, -200)
data = history.load_data(datadir=None, ticker_interval='1m', pairs=['UNITTEST/BTC'],
data = history.load_data(datadir=testdatadir, ticker_interval='1m', pairs=['UNITTEST/BTC'],
timerange=timerange)
processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(processed)
@ -595,21 +585,21 @@ def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None:
assert len(results) == 1
def test_processed(default_conf, mocker) -> None:
def test_processed(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker)
backtesting = Backtesting(default_conf)
dict_of_tickerrows = load_data_test('raise')
dict_of_tickerrows = load_data_test('raise', testdatadir)
dataframes = backtesting.strategy.tickerdata_to_dataframe(dict_of_tickerrows)
dataframe = dataframes['UNITTEST/BTC']
cols = dataframe.columns
# assert the dataframe got some of the indicator columns
for col in ['close', 'high', 'low', 'open', 'date',
'ema50', 'ao', 'macd', 'plus_dm']:
'ema10', 'rsi', 'fastd', 'plus_di']:
assert col in cols
def test_backtest_pricecontours(default_conf, fee, mocker) -> None:
def test_backtest_pricecontours(default_conf, fee, mocker, testdatadir) -> None:
# TODO: Evaluate usefullness of this, the patterns and buy-signls are unrealistic
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
tests = [['raise', 19], ['lower', 0], ['sine', 35]]
@ -617,49 +607,50 @@ def test_backtest_pricecontours(default_conf, fee, mocker) -> None:
default_conf['experimental'] = {"use_sell_signal": True}
for [contour, numres] in tests:
simple_backtest(default_conf, contour, numres, mocker)
simple_backtest(default_conf, contour, numres, mocker, testdatadir)
def test_backtest_clash_buy_sell(mocker, default_conf):
def test_backtest_clash_buy_sell(mocker, default_conf, testdatadir):
# Override the default buy trend function in our default_strategy
def fun(dataframe=None, pair=None):
buy_value = 1
sell_value = 1
return _trend(dataframe, buy_value, sell_value)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
backtesting = Backtesting(default_conf)
backtesting.advise_buy = fun # Override
backtesting.advise_sell = fun # Override
backtesting.strategy.advise_buy = fun # Override
backtesting.strategy.advise_sell = fun # Override
results = backtesting.backtest(backtest_conf)
assert results.empty
def test_backtest_only_sell(mocker, default_conf):
def test_backtest_only_sell(mocker, default_conf, testdatadir):
# Override the default buy trend function in our default_strategy
def fun(dataframe=None, pair=None):
buy_value = 0
sell_value = 1
return _trend(dataframe, buy_value, sell_value)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
backtesting = Backtesting(default_conf)
backtesting.advise_buy = fun # Override
backtesting.advise_sell = fun # Override
backtesting.strategy.advise_buy = fun # Override
backtesting.strategy.advise_sell = fun # Override
results = backtesting.backtest(backtest_conf)
assert results.empty
def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
def test_backtest_alternate_buy_sell(default_conf, fee, mocker, testdatadir):
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
mocker.patch('freqtrade.optimize.backtesting.file_dump_json', MagicMock())
backtest_conf = _make_backtest_conf(mocker, conf=default_conf, pair='UNITTEST/BTC')
backtest_conf = _make_backtest_conf(mocker, conf=default_conf,
pair='UNITTEST/BTC', datadir=testdatadir)
# We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True}
default_conf['ticker_interval'] = '1m'
backtesting = Backtesting(default_conf)
backtesting.advise_buy = _trend_alternate # Override
backtesting.advise_sell = _trend_alternate # Override
backtesting.strategy.advise_buy = _trend_alternate # Override
backtesting.strategy.advise_sell = _trend_alternate # Override
results = backtesting.backtest(backtest_conf)
backtesting._store_backtest_result("test_.json", results)
# 200 candles in backtest data
@ -672,7 +663,7 @@ def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
@pytest.mark.parametrize("pair", ['ADA/BTC', 'LTC/BTC'])
@pytest.mark.parametrize("tres", [0, 20, 30])
def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair, testdatadir):
def _trend_alternate_hold(dataframe=None, metadata=None):
"""
@ -690,7 +681,7 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
patch_exchange(mocker)
pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC']
data = history.load_data(datadir=None, ticker_interval='5m', pairs=pairs)
data = history.load_data(datadir=testdatadir, ticker_interval='5m', pairs=pairs)
# Only use 500 lines to increase performance
data = trim_dictlist(data, -500)
@ -701,8 +692,8 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
default_conf['ticker_interval'] = '5m'
backtesting = Backtesting(default_conf)
backtesting.advise_buy = _trend_alternate_hold # Override
backtesting.advise_sell = _trend_alternate_hold # Override
backtesting.strategy.advise_buy = _trend_alternate_hold # Override
backtesting.strategy.advise_sell = _trend_alternate_hold # Override
data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(data_processed)
@ -806,7 +797,7 @@ def test_backtest_record(default_conf, fee, mocker):
assert dur > 0
def test_backtest_start_timerange(default_conf, mocker, caplog):
def test_backtest_start_timerange(default_conf, mocker, caplog, testdatadir):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
async def load_pairs(pair, timeframe, since):
@ -823,7 +814,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
args = [
'--config', 'config.json',
'--strategy', 'DefaultStrategy',
'--datadir', 'freqtrade/tests/testdata',
'--datadir', str(testdatadir),
'backtesting',
'--ticker-interval', '1m',
'--timerange', '-100',
@ -837,7 +828,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
'Parameter --timerange detected: -100 ...',
'Using data directory: freqtrade/tests/testdata ...',
f'Using data directory: {testdatadir} ...',
'Using stake_currency: BTC ...',
'Using stake_amount: 0.001 ...',
'Backtesting with data from 2017-11-14T21:17:00+00:00 '
@ -849,7 +840,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
assert log_has(line, caplog)
def test_backtest_start_multi_strat(default_conf, mocker, caplog):
def test_backtest_start_multi_strat(default_conf, mocker, caplog, testdatadir):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
async def load_pairs(pair, timeframe, since):
@ -869,7 +860,7 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
args = [
'--config', 'config.json',
'--datadir', 'freqtrade/tests/testdata',
'--datadir', str(testdatadir),
'backtesting',
'--ticker-interval', '1m',
'--timerange', '-100',
@ -877,7 +868,7 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
'--disable-max-market-positions',
'--strategy-list',
'DefaultStrategy',
'TestStrategy',
'SampleStrategy',
]
args = get_args(args)
start_backtesting(args)
@ -891,14 +882,14 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
'Parameter --timerange detected: -100 ...',
'Using data directory: freqtrade/tests/testdata ...',
f'Using data directory: {testdatadir} ...',
'Using stake_currency: BTC ...',
'Using stake_amount: 0.001 ...',
'Backtesting with data from 2017-11-14T21:17:00+00:00 '
'up to 2017-11-14T22:58:00+00:00 (0 days)..',
'Parameter --enable-position-stacking detected ...',
'Running backtesting for Strategy DefaultStrategy',
'Running backtesting for Strategy TestStrategy',
'Running backtesting for Strategy SampleStrategy',
]
for line in exists:

12
freqtrade/tests/optimize/test_edge_cli.py → tests/optimize/test_edge_cli.py
View File

@ -3,14 +3,11 @@
from unittest.mock import MagicMock
import pytest
from freqtrade.edge import PairInfo
from freqtrade.optimize import setup_configuration, start_edge
from freqtrade.optimize.edge_cli import EdgeCli
from freqtrade.state import RunMode
from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
patch_exchange,
from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patched_configuration_load_config_file)
@ -36,14 +33,10 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert 'ticker_interval' in config
assert not log_has_re('Parameter -i/--ticker-interval detected .*', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config
assert 'stoploss_range' not in config
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, edge_conf)
mocker.patch(
@ -57,7 +50,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
'--datadir', '/foo/bar',
'edge',
'--ticker-interval', '1m',
'--refresh-pairs-cached',
'--timerange', ':100',
'--stoplosses=-0.01,-0.10,-0.001'
]
@ -75,8 +67,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
assert log_has('Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)

312
freqtrade/tests/optimize/test_hyperopt.py → tests/optimize/test_hyperopt.py
View File

@ -1,26 +1,25 @@
# pragma pylint: disable=missing-docstring,W0212,C0103
import os
from datetime import datetime
from pathlib import Path
from unittest.mock import MagicMock, PropertyMock
import pandas as pd
import pytest
from arrow import Arrow
from filelock import Timeout
from pathlib import Path
from freqtrade import DependencyException, OperationalException
from freqtrade import OperationalException
from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.data.history import load_tickerdata_file
from freqtrade.optimize import setup_configuration, start_hyperopt
from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss
from freqtrade.optimize.hyperopt import Hyperopt
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
from freqtrade.resolvers.hyperopt_resolver import (HyperOptLossResolver,
HyperOptResolver)
from freqtrade.state import RunMode
from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
patch_exchange,
from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patched_configuration_load_config_file)
@ -36,25 +35,23 @@ def hyperopt_results():
return pd.DataFrame(
{
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
'profit_percent': [0.1, 0.2, 0.3],
'profit_abs': [0.2, 0.4, 0.5],
'profit_percent': [-0.1, 0.2, 0.3],
'profit_abs': [-0.2, 0.4, 0.6],
'trade_duration': [10, 30, 10],
'profit': [2, 0, 0],
'loss': [0, 0, 1],
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI]
}
)
# Functions for recurrent object patching
def create_trials(mocker, hyperopt) -> None:
def create_trials(mocker, hyperopt, testdatadir) -> None:
"""
When creating trials, mock the hyperopt Trials so that *by default*
- we don't create any pickle'd files in the filesystem
- we might have a pickle'd file so make sure that we return
false when looking for it
"""
hyperopt.trials_file = Path('freqtrade/tests/optimize/ut_trials.pickle')
hyperopt.trials_file = testdatadir / 'optimize/ut_trials.pickle'
mocker.patch.object(Path, "is_file", MagicMock(return_value=False))
stat_mock = MagicMock()
@ -89,15 +86,11 @@ def test_setup_hyperopt_configuration_without_arguments(mocker, default_conf, ca
assert 'position_stacking' not in config
assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config
assert 'runmode' in config
assert config['runmode'] == RunMode.HYPEROPT
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch(
@ -111,7 +104,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
'hyperopt',
'--ticker-interval', '1m',
'--timerange', ':100',
'--refresh-pairs-cached',
'--enable-position-stacking',
'--disable-max-market-positions',
'--epochs', '1000',
@ -140,9 +132,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
assert log_has('max_open_trades set to unlimited ...', caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
@ -164,15 +153,15 @@ def test_hyperoptresolver(mocker, default_conf, caplog) -> None:
delattr(hyperopts, 'populate_sell_trend')
mocker.patch(
'freqtrade.resolvers.hyperopt_resolver.HyperOptResolver._load_hyperopt',
MagicMock(return_value=hyperopts)
MagicMock(return_value=hyperopts(default_conf))
)
x = HyperOptResolver(default_conf, ).hyperopt
assert not hasattr(x, 'populate_buy_trend')
assert not hasattr(x, 'populate_sell_trend')
assert log_has("Custom Hyperopt does not provide populate_sell_trend. "
"Using populate_sell_trend from DefaultStrategy.", caplog)
assert log_has("Custom Hyperopt does not provide populate_buy_trend. "
"Using populate_buy_trend from DefaultStrategy.", caplog)
assert log_has("Hyperopt class does not provide populate_sell_trend() method. "
"Using populate_sell_trend from the strategy.", caplog)
assert log_has("Hyperopt class does not provide populate_buy_trend() method. "
"Using populate_buy_trend from the strategy.", caplog)
assert hasattr(x, "ticker_interval")
@ -201,6 +190,24 @@ def test_hyperoptlossresolver_wrongname(mocker, default_conf, caplog) -> None:
HyperOptLossResolver(default_conf, ).hyperopt
def test_start_not_installed(mocker, default_conf, caplog, import_fails) -> None:
start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch('freqtrade.optimize.hyperopt.Hyperopt.start', start_mock)
patch_exchange(mocker)
args = [
'--config', 'config.json',
'hyperopt',
'--epochs', '5'
]
args = get_args(args)
with pytest.raises(OperationalException, match=r"Please ensure that the hyperopt dependencies"):
start_hyperopt(args)
def test_start(mocker, default_conf, caplog) -> None:
start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf)
@ -215,9 +222,6 @@ def test_start(mocker, default_conf, caplog) -> None:
args = get_args(args)
start_hyperopt(args)
import pprint
pprint.pprint(caplog.record_tuples)
assert log_has('Starting freqtrade in Hyperopt mode', caplog)
assert start_mock.call_count == 1
@ -240,30 +244,9 @@ def test_start_no_data(mocker, default_conf, caplog) -> None:
args = get_args(args)
start_hyperopt(args)
import pprint
pprint.pprint(caplog.record_tuples)
assert log_has('No data found. Terminating.', caplog)
def test_start_failure(mocker, default_conf, caplog) -> None:
start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch('freqtrade.optimize.hyperopt.Hyperopt.start', start_mock)
patch_exchange(mocker)
args = [
'--config', 'config.json',
'--strategy', 'TestStrategy',
'hyperopt',
'--epochs', '5'
]
args = get_args(args)
with pytest.raises(DependencyException):
start_hyperopt(args)
assert log_has("Please don't use --strategy for hyperopt.", caplog)
def test_start_filelock(mocker, default_conf, caplog) -> None:
start_mock = MagicMock(side_effect=Timeout(Hyperopt.get_lock_filename(default_conf)))
patched_configuration_load_config_file(mocker, default_conf)
@ -374,23 +357,23 @@ def test_no_log_if_loss_does_not_improve(hyperopt, caplog) -> None:
assert caplog.record_tuples == []
def test_save_trials_saves_trials(mocker, hyperopt, caplog) -> None:
trials = create_trials(mocker, hyperopt)
def test_save_trials_saves_trials(mocker, hyperopt, testdatadir, caplog) -> None:
trials = create_trials(mocker, hyperopt, testdatadir)
mock_dump = mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None)
hyperopt.trials = trials
hyperopt.save_trials()
trials_file = os.path.join('freqtrade', 'tests', 'optimize', 'ut_trials.pickle')
assert log_has('Saving 1 evaluations to \'{}\''.format(trials_file), caplog)
trials_file = testdatadir / 'optimize' / 'ut_trials.pickle'
assert log_has(f"Saving 1 evaluations to '{trials_file}'", caplog)
mock_dump.assert_called_once()
def test_read_trials_returns_trials_file(mocker, hyperopt, caplog) -> None:
trials = create_trials(mocker, hyperopt)
def test_read_trials_returns_trials_file(mocker, hyperopt, testdatadir, caplog) -> None:
trials = create_trials(mocker, hyperopt, testdatadir)
mock_load = mocker.patch('freqtrade.optimize.hyperopt.load', return_value=trials)
hyperopt_trial = hyperopt.read_trials()
trials_file = os.path.join('freqtrade', 'tests', 'optimize', 'ut_trials.pickle')
assert log_has('Reading Trials from \'{}\''.format(trials_file), caplog)
trials_file = testdatadir / 'optimize' / 'ut_trials.pickle'
assert log_has(f"Reading Trials from '{trials_file}'", caplog)
assert hyperopt_trial == trials
mock_load.assert_called_once()
@ -418,7 +401,8 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}])
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result',
'params': {'buy': {}, 'sell': {}, 'roi': {}, 'stoploss': 0.0}}])
)
patch_exchange(mocker)
@ -441,8 +425,8 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting, "advise_sell")
assert hasattr(hyperopt.backtesting, "advise_buy")
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
@ -484,8 +468,8 @@ def test_has_space(hyperopt):
assert hyperopt.has_space('buy')
def test_populate_indicators(hyperopt) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
def test_populate_indicators(hyperopt, testdatadir) -> None:
tick = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)}
dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist)
@ -498,8 +482,8 @@ def test_populate_indicators(hyperopt) -> None:
assert 'rsi' in dataframe
def test_buy_strategy_generator(hyperopt) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
def test_buy_strategy_generator(hyperopt, testdatadir) -> None:
tick = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)}
dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist)
@ -584,6 +568,7 @@ def test_generate_optimizer(mocker, default_conf) -> None:
}
hyperopt = Hyperopt(default_conf)
hyperopt.dimensions = hyperopt.hyperopt_space()
generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values()))
assert generate_optimizer_value == response_expected
@ -693,3 +678,200 @@ def test_print_json_spaces_roi_stoploss(mocker, default_conf, caplog, capsys) ->
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
def test_simplified_interface_roi_stoploss(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{
'loss': 1, 'results_explanation': 'foo result', 'params': {'stoploss': 0.0}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'roi stoploss',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
def test_simplified_interface_all_failed(mocker, default_conf, caplog, capsys) -> None:
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'all',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
with pytest.raises(OperationalException, match=r"The 'buy' space is included into *"):
hyperopt.start()
def test_simplified_interface_buy(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'buy',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
# TODO: sell_strategy_generator() is actually not called because
# run_optimizer_parallel() is mocked
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
def test_simplified_interface_sell(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'sell',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
# TODO: buy_strategy_generator() is actually not called because
# run_optimizer_parallel() is mocked
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
@pytest.mark.parametrize("method,space", [
('buy_strategy_generator', 'buy'),
('indicator_space', 'buy'),
('sell_strategy_generator', 'sell'),
('sell_indicator_space', 'sell'),
])
def test_simplified_interface_failed(mocker, default_conf, caplog, capsys, method, space) -> None:
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': space,
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
delattr(hyperopt.custom_hyperopt.__class__, method)
with pytest.raises(OperationalException, match=f"The '{space}' space is included into *"):
hyperopt.start()

0
freqtrade/tests/pairlist/__init__.py → tests/pairlist/__init__.py
View File

2
freqtrade/tests/pairlist/test_pairlist.py → tests/pairlist/test_pairlist.py
View File

@ -5,7 +5,7 @@ from unittest.mock import MagicMock, PropertyMock
from freqtrade import OperationalException
from freqtrade.constants import AVAILABLE_PAIRLISTS
from freqtrade.resolvers import PairListResolver
from freqtrade.tests.conftest import get_patched_freqtradebot
from tests.conftest import get_patched_freqtradebot
import pytest
# whitelist, blacklist

0
freqtrade/tests/rpc/__init__.py → tests/rpc/__init__.py
View File

Some files were not shown because too many files have changed in this diff Show More