Merge branch 'develop' into align_userdata

This commit is contained in:
Matthias 2019-08-05 06:55:51 +02:00
commit 383b24ab84
18 changed files with 267 additions and 80 deletions

View File

@ -5,6 +5,7 @@ If it hasn't been reported, please create a new issue.
## Step 2: Describe your environment ## Step 2: Describe your environment
* Operating system: ____
* Python Version: _____ (`python -V`) * Python Version: _____ (`python -V`)
* CCXT version: _____ (`pip freeze | grep ccxt`) * CCXT version: _____ (`pip freeze | grep ccxt`)
* Branch: Master | Develop * Branch: Master | Develop

View File

@ -6,6 +6,128 @@ A good way for this is using Jupyter (notebook or lab) - which provides an inter
The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results. The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results.
## Strategy development problem analysis
Debugging a strategy (are there no buy signals, ...) can be very time-consuming.
FreqTrade tries to help you by exposing a few helper-functions, which can be very handy.
It's recommended using Juptyer Notebooks for analysis, since it offers a dynamic way to rerun certain parts of the code.
The following is a full code-snippet, which will be explained by both comments, and step by step below.
```python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = '../xmatt/strategies'
# Location of the data
data_location = '../freqtrade/user_data/data/binance/'
# Only use one pair here
pair = "XRP_ETH"
### End constants
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
### Start strategy reload
# Load strategy - best done in a new cell
# Rerun each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
### Explanation
#### Imports and constant definition
``` python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = 'user_data/strategies'
# Location of the data
data_location = 'user_data/data/binance'
# Only use one pair here
pair = "XRP_ETH"
```
This first section imports necessary modules, and defines some constants you'll probably need to adjust for your case.
#### Load candles
``` python
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
```
This second section loads the historic data and prints the amount of candles in the DataFrame.
You can also inspect this dataframe by using `bt_data.head()` or `bt_data.tail()`.
#### Run strategy and analyze results
Now, it's time to load and run your strategy.
For this, I recommend using a new cell in your notebook, since you'll want to repeat this until you're satisfied with your strategy.
``` python
# Load strategy - best done in a new cell
# Needs to be ran each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
The code snippet loads and analyzes the strategy, calculates and prints the number of buy signals.
The last 2 lines serve to analyze the dataframe in detail.
This can be important if your strategy did not generate any buy signals.
Note that using `data.head()` would also work, however this is misleading since most indicators have some "startup" time at the start of a backtested dataframe.
There can be many things wrong, some signs to look for are:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
## Backtesting ## Backtesting
To analyze your backtest results, you can [export the trades](#exporting-trades-to-file). To analyze your backtest results, you can [export the trades](#exporting-trades-to-file).

View File

@ -45,6 +45,16 @@ the tutorial [here|Testing-new-strategies-with-Hyperopt](bot-usage.md#hyperopt-c
You can use the `/forcesell all` command from Telegram. You can use the `/forcesell all` command from Telegram.
### I get the message "RESTRICTED_MARKET"
Currently known to happen for US Bittrex users.
Bittrex split its exchange into US and International versions.
The International version has more pairs available, however the API always returns all pairs, so there is currently no automated way to detect if you're affected by the restriction.
If you have restricted pairs in your whitelist, you'll get a warning message in the log on FreqTrade startup for each restricted pair.
If you're an "International" Customer on the Bittrex exchange, then this warning will probably not impact you.
If you're a US customer, the bot will fail to create orders for these pairs, and you should remove them from your Whitelist.
## Hyperopt module ## Hyperopt module
### How many epoch do I need to get a good Hyperopt result? ### How many epoch do I need to get a good Hyperopt result?

View File

@ -303,8 +303,10 @@ Given the following result from hyperopt:
``` ```
Best result: Best result:
135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins.
with values: 44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params:
{ 'adx-value': 44, { 'adx-value': 44,
'rsi-value': 29, 'rsi-value': 29,
'adx-enabled': False, 'adx-enabled': False,
@ -347,21 +349,15 @@ If you are optimizing ROI, you're result will look as follows and include a ROI
``` ```
Best result: Best result:
135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins.
with values: 44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params:
{ 'adx-value': 44, { 'adx-value': 44,
'rsi-value': 29, 'rsi-value': 29,
'adx-enabled': false, 'adx-enabled': False,
'rsi-enabled': True, 'rsi-enabled': True,
'trigger': 'bb_lower', 'trigger': 'bb_lower'}
'roi_t1': 40,
'roi_t2': 57,
'roi_t3': 21,
'roi_p1': 0.03634636907306948,
'roi_p2': 0.055237357937802885,
'roi_p3': 0.015163796015548354,
'stoploss': -0.37996664668703606
}
ROI table: ROI table:
{ 0: 0.10674752302642071, { 0: 0.10674752302642071,
21: 0.09158372701087236, 21: 0.09158372701087236,
@ -372,9 +368,9 @@ ROI table:
This would translate to the following ROI table: This would translate to the following ROI table:
``` python ``` python
minimal_roi = { minimal_roi = {
"118": 0, "118": 0,
"78": 0.0363463, "78": 0.0363,
"21": 0.0915, "21": 0.0915,
"0": 0.106 "0": 0.106
} }

View File

@ -81,19 +81,30 @@ def load_trades_from_db(db_url: str) -> pd.DataFrame:
""" """
trades: pd.DataFrame = pd.DataFrame([], columns=BT_DATA_COLUMNS) trades: pd.DataFrame = pd.DataFrame([], columns=BT_DATA_COLUMNS)
persistence.init(db_url, clean_open_orders=False) persistence.init(db_url, clean_open_orders=False)
columns = ["pair", "profit", "open_time", "close_time",
"open_rate", "close_rate", "duration", "sell_reason",
"max_rate", "min_rate"]
trades = pd.DataFrame([(t.pair, t.calc_profit(), columns = ["pair", "open_time", "close_time", "profit", "profitperc",
"open_rate", "close_rate", "amount", "duration", "sell_reason",
"fee_open", "fee_close", "open_rate_requested", "close_rate_requested",
"stake_amount", "max_rate", "min_rate", "id", "exchange",
"stop_loss", "initial_stop_loss", "strategy", "ticker_interval"]
trades = pd.DataFrame([(t.pair,
t.open_date.replace(tzinfo=pytz.UTC), t.open_date.replace(tzinfo=pytz.UTC),
t.close_date.replace(tzinfo=pytz.UTC) if t.close_date else None, t.close_date.replace(tzinfo=pytz.UTC) if t.close_date else None,
t.open_rate, t.close_rate, t.calc_profit(), t.calc_profit_percent(),
t.close_date.timestamp() - t.open_date.timestamp() t.open_rate, t.close_rate, t.amount,
if t.close_date else None, (t.close_date.timestamp() - t.open_date.timestamp()
if t.close_date else None),
t.sell_reason, t.sell_reason,
t.fee_open, t.fee_close,
t.open_rate_requested,
t.close_rate_requested,
t.stake_amount,
t.max_rate, t.max_rate,
t.min_rate, t.min_rate,
t.id, t.exchange,
t.stop_loss, t.initial_stop_loss,
t.strategy, t.ticker_interval
) )
for t in Trade.query.all()], for t in Trade.query.all()],
columns=columns) columns=columns)

View File

@ -260,7 +260,7 @@ class Exchange(object):
if not self.markets: if not self.markets:
logger.warning('Unable to validate pairs (assuming they are correct).') logger.warning('Unable to validate pairs (assuming they are correct).')
# return return
for pair in pairs: for pair in pairs:
# Note: ccxt has BaseCurrency/QuoteCurrency format for pairs # Note: ccxt has BaseCurrency/QuoteCurrency format for pairs
@ -269,6 +269,12 @@ class Exchange(object):
raise OperationalException( raise OperationalException(
f'Pair {pair} is not available on {self.name}. ' f'Pair {pair} is not available on {self.name}. '
f'Please remove {pair} from your whitelist.') f'Please remove {pair} from your whitelist.')
elif self.markets[pair].get('info', {}).get('IsRestricted', False):
# Warn users about restricted pairs in whitelist.
# We cannot determine reliably if Users are affected.
logger.warning(f"Pair {pair} is restricted for some users on this exchange."
f"Please check if you are impacted by this restriction "
f"on the exchange and eventually remove {pair} from your whitelist.")
def get_valid_pair_combination(self, curr_1, curr_2) -> str: def get_valid_pair_combination(self, curr_1, curr_2) -> str:
""" """

View File

@ -10,7 +10,7 @@ import sys
from operator import itemgetter from operator import itemgetter
from pathlib import Path from pathlib import Path
from pprint import pprint from pprint import pprint
from typing import Any, Dict, List from typing import Any, Dict, List, Optional
from joblib import Parallel, delayed, dump, load, wrap_non_picklable_objects, cpu_count from joblib import Parallel, delayed, dump, load, wrap_non_picklable_objects, cpu_count
from pandas import DataFrame from pandas import DataFrame
@ -138,11 +138,20 @@ class Hyperopt(Backtesting):
params = best_result['params'] params = best_result['params']
log_str = self.format_results_logstring(best_result) log_str = self.format_results_logstring(best_result)
print(f"\nBest result:\n{log_str}\nwith values:") print(f"\nBest result:\n\n{log_str}\n")
pprint(params, indent=4) if self.has_space('buy'):
print('Buy hyperspace params:')
pprint({p.name: params.get(p.name) for p in self.hyperopt_space('buy')},
indent=4)
if self.has_space('sell'):
print('Sell hyperspace params:')
pprint({p.name: params.get(p.name) for p in self.hyperopt_space('sell')},
indent=4)
if self.has_space('roi'): if self.has_space('roi'):
print("ROI table:") print("ROI table:")
pprint(self.custom_hyperopt.generate_roi_table(params), indent=4) pprint(self.custom_hyperopt.generate_roi_table(params), indent=4)
if self.has_space('stoploss'):
print(f"Stoploss: {params.get('stoploss')}")
def log_results(self, results) -> None: def log_results(self, results) -> None:
""" """
@ -176,21 +185,24 @@ class Hyperopt(Backtesting):
""" """
return any(s in self.config['spaces'] for s in [space, 'all']) return any(s in self.config['spaces'] for s in [space, 'all'])
def hyperopt_space(self) -> List[Dimension]: def hyperopt_space(self, space: Optional[str] = None) -> List[Dimension]:
""" """
Return the space to use during Hyperopt Return the dimensions in the hyperoptimization space.
:param space: Defines hyperspace to return dimensions for.
If None, then the self.has_space() will be used to return dimensions
for all hyperspaces used.
""" """
spaces: List[Dimension] = [] spaces: List[Dimension] = []
if self.has_space('buy'): if space == 'buy' or (space is None and self.has_space('buy')):
logger.debug("Hyperopt has 'buy' space") logger.debug("Hyperopt has 'buy' space")
spaces += self.custom_hyperopt.indicator_space() spaces += self.custom_hyperopt.indicator_space()
if self.has_space('sell'): if space == 'sell' or (space is None and self.has_space('sell')):
logger.debug("Hyperopt has 'sell' space") logger.debug("Hyperopt has 'sell' space")
spaces += self.custom_hyperopt.sell_indicator_space() spaces += self.custom_hyperopt.sell_indicator_space()
if self.has_space('roi'): if space == 'roi' or (space is None and self.has_space('roi')):
logger.debug("Hyperopt has 'roi' space") logger.debug("Hyperopt has 'roi' space")
spaces += self.custom_hyperopt.roi_space() spaces += self.custom_hyperopt.roi_space()
if self.has_space('stoploss'): if space == 'stoploss' or (space is None and self.has_space('stoploss')):
logger.debug("Hyperopt has 'stoploss' space") logger.debug("Hyperopt has 'stoploss' space")
spaces += self.custom_hyperopt.stoploss_space() spaces += self.custom_hyperopt.stoploss_space()
return spaces return spaces

View File

@ -316,8 +316,9 @@ def store_plot_file(fig, filename: str, directory: Path, auto_open: bool = False
:param ticker_interval: Used as part of the filename :param ticker_interval: Used as part of the filename
:return: None :return: None
""" """
directory.mkdir(parents=True, exist_ok=True) directory.mkdir(parents=True, exist_ok=True)
plot(fig, filename=str(directory.joinpath(filename)), _filename = directory.joinpath(filename)
plot(fig, filename=str(_filename),
auto_open=auto_open) auto_open=auto_open)
logger.info(f"Stored plot as {_filename}")

View File

@ -158,6 +158,23 @@ class IStrategy(ABC):
""" """
Parses the given ticker history and returns a populated DataFrame Parses the given ticker history and returns a populated DataFrame
add several TA indicators and buy signal to it add several TA indicators and buy signal to it
:param dataframe: Dataframe containing ticker data
:param metadata: Metadata dictionary with additional data (e.g. 'pair')
:return: DataFrame with ticker data and indicator data
"""
logger.debug("TA Analysis Launched")
dataframe = self.advise_indicators(dataframe, metadata)
dataframe = self.advise_buy(dataframe, metadata)
dataframe = self.advise_sell(dataframe, metadata)
return dataframe
def _analyze_ticker_internal(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Parses the given ticker history and returns a populated DataFrame
add several TA indicators and buy signal to it
WARNING: Used internally only, may skip analysis if `process_only_new_candles` is set.
:param dataframe: Dataframe containing ticker data
:param metadata: Metadata dictionary with additional data (e.g. 'pair')
:return: DataFrame with ticker data and indicator data :return: DataFrame with ticker data and indicator data
""" """
@ -168,10 +185,7 @@ class IStrategy(ABC):
if (not self.process_only_new_candles or if (not self.process_only_new_candles or
self._last_candle_seen_per_pair.get(pair, None) != dataframe.iloc[-1]['date']): self._last_candle_seen_per_pair.get(pair, None) != dataframe.iloc[-1]['date']):
# Defs that only make change on new candle data. # Defs that only make change on new candle data.
logger.debug("TA Analysis Launched") dataframe = self.analyze_ticker(dataframe, metadata)
dataframe = self.advise_indicators(dataframe, metadata)
dataframe = self.advise_buy(dataframe, metadata)
dataframe = self.advise_sell(dataframe, metadata)
self._last_candle_seen_per_pair[pair] = dataframe.iloc[-1]['date'] self._last_candle_seen_per_pair[pair] = dataframe.iloc[-1]['date']
else: else:
logger.debug("Skipping TA Analysis for already analyzed candle") logger.debug("Skipping TA Analysis for already analyzed candle")
@ -198,7 +212,7 @@ class IStrategy(ABC):
return False, False return False, False
try: try:
dataframe = self.analyze_ticker(dataframe, {'pair': pair}) dataframe = self._analyze_ticker_internal(dataframe, {'pair': pair})
except ValueError as error: except ValueError as error:
logger.warning( logger.warning(
'Unable to analyze ticker for pair %s: %s', 'Unable to analyze ticker for pair %s: %s',

View File

@ -305,7 +305,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'TKN/BTC': { 'TKN/BTC': {
'id': 'tknbtc', 'id': 'tknbtc',
@ -330,7 +330,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'BLK/BTC': { 'BLK/BTC': {
'id': 'blkbtc', 'id': 'blkbtc',
@ -355,7 +355,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'LTC/BTC': { 'LTC/BTC': {
'id': 'ltcbtc', 'id': 'ltcbtc',
@ -380,7 +380,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'XRP/BTC': { 'XRP/BTC': {
'id': 'xrpbtc', 'id': 'xrpbtc',
@ -405,7 +405,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'NEO/BTC': { 'NEO/BTC': {
'id': 'neobtc', 'id': 'neobtc',
@ -430,7 +430,7 @@ def markets():
'max': 500000, 'max': 500000,
}, },
}, },
'info': '', 'info': {},
}, },
'BTT/BTC': { 'BTT/BTC': {
'id': 'BTTBTC', 'id': 'BTTBTC',
@ -458,7 +458,7 @@ def markets():
'max': None 'max': None
} }
}, },
'info': "", 'info': {},
}, },
'ETH/USDT': { 'ETH/USDT': {
'id': 'USDT-ETH', 'id': 'USDT-ETH',
@ -480,7 +480,7 @@ def markets():
} }
}, },
'active': True, 'active': True,
'info': "" 'info': {},
}, },
'LTC/USDT': { 'LTC/USDT': {
'id': 'USDT-LTC', 'id': 'USDT-LTC',
@ -502,7 +502,7 @@ def markets():
'max': None 'max': None
} }
}, },
'info': "" 'info': {},
} }
} }

View File

@ -45,6 +45,11 @@ def test_load_trades_db(default_conf, fee, mocker):
assert isinstance(trades, DataFrame) assert isinstance(trades, DataFrame)
assert "pair" in trades.columns assert "pair" in trades.columns
assert "open_time" in trades.columns assert "open_time" in trades.columns
assert "profitperc" in trades.columns
for col in BT_DATA_COLUMNS:
if col not in ['index', 'open_at_end']:
assert col in trades.columns
def test_extract_trades_of_period(): def test_extract_trades_of_period():

View File

@ -318,7 +318,7 @@ def test__reload_markets_exception(default_conf, mocker, caplog):
def test_validate_pairs(default_conf, mocker): # test exchange.validate_pairs directly def test_validate_pairs(default_conf, mocker): # test exchange.validate_pairs directly
api_mock = MagicMock() api_mock = MagicMock()
type(api_mock).markets = PropertyMock(return_value={ type(api_mock).markets = PropertyMock(return_value={
'ETH/BTC': '', 'LTC/BTC': '', 'XRP/BTC': '', 'NEO/BTC': '' 'ETH/BTC': {}, 'LTC/BTC': {}, 'XRP/BTC': {}, 'NEO/BTC': {}
}) })
id_mock = PropertyMock(return_value='test_exchange') id_mock = PropertyMock(return_value='test_exchange')
type(api_mock).id = id_mock type(api_mock).id = id_mock
@ -332,7 +332,7 @@ def test_validate_pairs(default_conf, mocker): # test exchange.validate_pairs d
def test_validate_pairs_not_available(default_conf, mocker): def test_validate_pairs_not_available(default_conf, mocker):
api_mock = MagicMock() api_mock = MagicMock()
type(api_mock).markets = PropertyMock(return_value={ type(api_mock).markets = PropertyMock(return_value={
'XRP/BTC': 'inactive' 'XRP/BTC': {'inactive': True}
}) })
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock)) mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock))
mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock()) mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
@ -361,6 +361,23 @@ def test_validate_pairs_exception(default_conf, mocker, caplog):
caplog.record_tuples) caplog.record_tuples)
def test_validate_pairs_restricted(default_conf, mocker, caplog):
api_mock = MagicMock()
type(api_mock).markets = PropertyMock(return_value={
'ETH/BTC': {}, 'LTC/BTC': {}, 'NEO/BTC': {},
'XRP/BTC': {'info': {'IsRestricted': True}}
})
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock))
mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
mocker.patch('freqtrade.exchange.Exchange._load_async_markets', MagicMock())
Exchange(default_conf)
assert log_has(f"Pair XRP/BTC is restricted for some users on this exchange."
f"Please check if you are impacted by this restriction "
f"on the exchange and eventually remove XRP/BTC from your whitelist.",
caplog.record_tuples)
def test_validate_timeframes(default_conf, mocker): def test_validate_timeframes(default_conf, mocker):
default_conf["ticker_interval"] = "5m" default_conf["ticker_interval"] = "5m"
api_mock = MagicMock() api_mock = MagicMock()

View File

@ -202,6 +202,7 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert config['runmode'] == RunMode.BACKTEST assert config['runmode'] == RunMode.BACKTEST
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None: def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
mocker.patch( mocker.patch(
@ -812,6 +813,7 @@ def test_backtest_record(default_conf, fee, mocker):
assert dur > 0 assert dur > 0
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_backtest_start_live(default_conf, mocker, caplog): def test_backtest_start_live(default_conf, mocker, caplog):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
@ -858,6 +860,7 @@ def test_backtest_start_live(default_conf, mocker, caplog):
assert log_has(line, caplog.record_tuples) assert log_has(line, caplog.record_tuples)
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_backtest_start_multi_strat(default_conf, mocker, caplog): def test_backtest_start_multi_strat(default_conf, mocker, caplog):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']

View File

@ -466,7 +466,7 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
parallel.assert_called_once() parallel.assert_called_once()
out, err = capsys.readouterr() out, err = capsys.readouterr()
assert 'Best result:\n* 1/1: foo result Objective: 1.00000\nwith values:\n' in out assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations # Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2 assert dumper.call_count == 2

View File

@ -19,13 +19,13 @@ _STRATEGY = DefaultStrategy(config={})
def test_returns_latest_buy_signal(mocker, default_conf, ticker_history): def test_returns_latest_buy_signal(mocker, default_conf, ticker_history):
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame([{'buy': 1, 'sell': 0, 'date': arrow.utcnow()}]) return_value=DataFrame([{'buy': 1, 'sell': 0, 'date': arrow.utcnow()}])
) )
assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (True, False) assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (True, False)
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame([{'buy': 0, 'sell': 1, 'date': arrow.utcnow()}]) return_value=DataFrame([{'buy': 0, 'sell': 1, 'date': arrow.utcnow()}])
) )
assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (False, True) assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (False, True)
@ -33,14 +33,14 @@ def test_returns_latest_buy_signal(mocker, default_conf, ticker_history):
def test_returns_latest_sell_signal(mocker, default_conf, ticker_history): def test_returns_latest_sell_signal(mocker, default_conf, ticker_history):
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame([{'sell': 1, 'buy': 0, 'date': arrow.utcnow()}]) return_value=DataFrame([{'sell': 1, 'buy': 0, 'date': arrow.utcnow()}])
) )
assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (False, True) assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (False, True)
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame([{'sell': 0, 'buy': 1, 'date': arrow.utcnow()}]) return_value=DataFrame([{'sell': 0, 'buy': 1, 'date': arrow.utcnow()}])
) )
assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (True, False) assert _STRATEGY.get_signal('ETH/BTC', '5m', ticker_history) == (True, False)
@ -60,7 +60,7 @@ def test_get_signal_empty(default_conf, mocker, caplog):
def test_get_signal_exception_valueerror(default_conf, mocker, caplog, ticker_history): def test_get_signal_exception_valueerror(default_conf, mocker, caplog, ticker_history):
caplog.set_level(logging.INFO) caplog.set_level(logging.INFO)
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
side_effect=ValueError('xyz') side_effect=ValueError('xyz')
) )
assert (False, False) == _STRATEGY.get_signal('foo', default_conf['ticker_interval'], assert (False, False) == _STRATEGY.get_signal('foo', default_conf['ticker_interval'],
@ -71,7 +71,7 @@ def test_get_signal_exception_valueerror(default_conf, mocker, caplog, ticker_hi
def test_get_signal_empty_dataframe(default_conf, mocker, caplog, ticker_history): def test_get_signal_empty_dataframe(default_conf, mocker, caplog, ticker_history):
caplog.set_level(logging.INFO) caplog.set_level(logging.INFO)
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame([]) return_value=DataFrame([])
) )
assert (False, False) == _STRATEGY.get_signal('xyz', default_conf['ticker_interval'], assert (False, False) == _STRATEGY.get_signal('xyz', default_conf['ticker_interval'],
@ -86,7 +86,7 @@ def test_get_signal_old_dataframe(default_conf, mocker, caplog, ticker_history):
oldtime = arrow.utcnow().shift(minutes=-16) oldtime = arrow.utcnow().shift(minutes=-16)
ticks = DataFrame([{'buy': 1, 'date': oldtime}]) ticks = DataFrame([{'buy': 1, 'date': oldtime}])
mocker.patch.object( mocker.patch.object(
_STRATEGY, 'analyze_ticker', _STRATEGY, '_analyze_ticker_internal',
return_value=DataFrame(ticks) return_value=DataFrame(ticks)
) )
assert (False, False) == _STRATEGY.get_signal('xyz', default_conf['ticker_interval'], assert (False, False) == _STRATEGY.get_signal('xyz', default_conf['ticker_interval'],
@ -252,7 +252,7 @@ def test_analyze_ticker_default(ticker_history, mocker, caplog) -> None:
caplog.record_tuples) caplog.record_tuples)
def test_analyze_ticker_skip_analyze(ticker_history, mocker, caplog) -> None: def test__analyze_ticker_internal_skip_analyze(ticker_history, mocker, caplog) -> None:
caplog.set_level(logging.DEBUG) caplog.set_level(logging.DEBUG)
ind_mock = MagicMock(side_effect=lambda x, meta: x) ind_mock = MagicMock(side_effect=lambda x, meta: x)
buy_mock = MagicMock(side_effect=lambda x, meta: x) buy_mock = MagicMock(side_effect=lambda x, meta: x)
@ -267,7 +267,7 @@ def test_analyze_ticker_skip_analyze(ticker_history, mocker, caplog) -> None:
strategy = DefaultStrategy({}) strategy = DefaultStrategy({})
strategy.process_only_new_candles = True strategy.process_only_new_candles = True
ret = strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'}) ret = strategy._analyze_ticker_internal(ticker_history, {'pair': 'ETH/BTC'})
assert 'high' in ret.columns assert 'high' in ret.columns
assert 'low' in ret.columns assert 'low' in ret.columns
assert 'close' in ret.columns assert 'close' in ret.columns
@ -280,7 +280,7 @@ def test_analyze_ticker_skip_analyze(ticker_history, mocker, caplog) -> None:
caplog.record_tuples) caplog.record_tuples)
caplog.clear() caplog.clear()
ret = strategy.analyze_ticker(ticker_history, {'pair': 'ETH/BTC'}) ret = strategy._analyze_ticker_internal(ticker_history, {'pair': 'ETH/BTC'})
# No analysis happens as process_only_new_candles is true # No analysis happens as process_only_new_candles is true
assert ind_mock.call_count == 1 assert ind_mock.call_count == 1
assert buy_mock.call_count == 1 assert buy_mock.call_count == 1

View File

@ -327,6 +327,7 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert 'export' not in config assert 'export' not in config
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> None: def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
mocker.patch( mocker.patch(

View File

@ -217,6 +217,8 @@ def test_generate_plot_file(mocker, caplog):
assert plot_mock.call_args[0][0] == fig assert plot_mock.call_args[0][0] == fig
assert (plot_mock.call_args_list[0][1]['filename'] assert (plot_mock.call_args_list[0][1]['filename']
== "user_data/plots/freqtrade-plot-UNITTEST_BTC-5m.html") == "user_data/plots/freqtrade-plot-UNITTEST_BTC-5m.html")
assert log_has("Stored plot as user_data/plots/freqtrade-plot-UNITTEST_BTC-5m.html",
caplog.record_tuples)
def test_add_profit(): def test_add_profit():

View File

@ -16,8 +16,6 @@ import logging
import sys import sys
from typing import Any, Dict, List from typing import Any, Dict, List
import pandas as pd
from freqtrade.configuration import Arguments from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
from freqtrade.data.btanalysis import extract_trades_of_period from freqtrade.data.btanalysis import extract_trades_of_period
@ -30,20 +28,6 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def generate_dataframe(strategy, tickers, pair) -> pd.DataFrame:
"""
Get tickers then Populate strategy indicators and signals, then return the full dataframe
:return: the DataFrame of a pair
"""
dataframes = strategy.tickerdata_to_dataframe(tickers)
dataframe = dataframes[pair]
dataframe = strategy.advise_buy(dataframe, {'pair': pair})
dataframe = strategy.advise_sell(dataframe, {'pair': pair})
return dataframe
def analyse_and_plot_pairs(config: Dict[str, Any]): def analyse_and_plot_pairs(config: Dict[str, Any]):
""" """
From arguments provided in cli: From arguments provided in cli:
@ -57,6 +41,7 @@ def analyse_and_plot_pairs(config: Dict[str, Any]):
""" """
plot_elements = init_plotscript(config) plot_elements = init_plotscript(config)
trades = plot_elements['trades'] trades = plot_elements['trades']
strategy = plot_elements["strategy"]
pair_counter = 0 pair_counter = 0
for pair, data in plot_elements["tickers"].items(): for pair, data in plot_elements["tickers"].items():
@ -64,7 +49,8 @@ def analyse_and_plot_pairs(config: Dict[str, Any]):
logger.info("analyse pair %s", pair) logger.info("analyse pair %s", pair)
tickers = {} tickers = {}
tickers[pair] = data tickers[pair] = data
dataframe = generate_dataframe(plot_elements["strategy"], tickers, pair)
dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
trades_pair = trades.loc[trades['pair'] == pair] trades_pair = trades.loc[trades['pair'] == pair]
trades_pair = extract_trades_of_period(dataframe, trades_pair) trades_pair = extract_trades_of_period(dataframe, trades_pair)