lychiyu/freqtrade_origin
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2024-09-20 09:31:12 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

conflict with develop resolved

Browse Source
This commit is contained in:
Misagh 2019-04-05 20:23:15 +02:00
commit 4b2eb22989
23 changed files with 546 additions and 403 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/bot-usage.md
View File

@ -79,7 +79,7 @@ prevent unintended disclosure of sensitive private data when you publish example
of your configuration in the project issues or in the Internet.
See more details on this technique with examples in the documentation page on
[configuration](bot-configuration.md).
[configuration](configuration.md).
### How to use **--strategy**?

4
docs/hyperopt.md
View File

@ -62,7 +62,7 @@ If you have updated the buy strategy, ie. changed the contents of
#### Sell optimization
Similar to the buy-signal above, sell-signals can also be optimized.
Similar to the buy-signal above, sell-signals can also be optimized.
Place the corresponding settings into the following methods
* Inside `sell_indicator_space()` - the parameters hyperopt shall be optimizing.
@ -163,7 +163,7 @@ running at least several thousand evaluations.
The `--spaces all` flag determines that all possible parameters should be optimized. Possibilities are listed below.
!!! Warning
When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file.
When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file.
### Execute Hyperopt with Different Ticker-Data Source

8
docs/sql_cheatsheet.md
View File

@ -1,5 +1,5 @@
# SQL Helper
This page constains some help if you want to edit your sqlite db.
This page contains some help if you want to edit your sqlite db.
## Install sqlite3
**Ubuntu/Debian installation**
@ -66,11 +66,11 @@ SELECT * FROM trades;
## Fix trade still open after a manual sell on the exchange
!!! Warning
Manually selling on the exchange should not be done by default, since the bot does not detect this and will try to sell anyway.
/foresell <tradeid> should accomplish the same thing.
Manually selling a pair on the exchange will not be detected by the bot and it will try to sell anyway. Whenever possible, forcesell <tradeid> should be used to accomplish the same thing.
It is strongly advised to backup your database file before making any manual changes.
!!! Note
This should not be necessary after /forcesell, as forcesell orders are closed automatically by the bot on the next iteration.
This should not be necessary after /forcesell, as forcesell orders are closed automatically by the bot on the next iteration.
```sql
UPDATE trades

74
docs/telegram-usage.md
View File

@ -1,13 +1,13 @@
# Telegram usage
This page explains how to command your bot with Telegram.
## Prerequisite
To control your bot with Telegram, you need first to
[set up a Telegram bot](installation.md)
and add your Telegram API keys into your config file.
## Telegram commands
Per default, the Telegram bot shows predefined commands. Some commands
are only available by sending them to the bot. The table below list the
official commands. You can ask at any moment for help with `/help`.
@ -65,16 +65,14 @@ Once all positions are sold, run `/stop` to completely stop the bot.
For each open trade, the bot will send you the following message.
> **Trade ID:** `123`
> **Current Pair:** CVC/BTC
> **Open Since:** `1 days ago`
> **Amount:** `26.64180098`
> **Open Rate:** `0.00007489`
> **Close Rate:** `None`
> **Current Rate:** `0.00007489`
> **Close Profit:** `None`
> **Current Profit:** `12.95%`
> **Open Order:** `None`
> **Trade ID:** `123` `(since 1 days ago)`
> **Current Pair:** CVC/BTC
> **Open Since:** `1 days ago`
> **Amount:** `26.64180098`
> **Open Rate:** `0.00007489`
> **Current Rate:** `0.00007489`
> **Current Profit:** `12.95%`
> **Stoploss:** `0.00007389 (-0.02%)`
### /status table
@ -99,18 +97,18 @@ current max
Return a summary of your profit/loss and performance.
> **ROI:** Close trades
> ∙ `0.00485701 BTC (258.45%)`
> ∙ `62.968 USD`
> **ROI:** All trades
> ∙ `0.00255280 BTC (143.43%)`
> ∙ `33.095 EUR`
>
> **Total Trade Count:** `138`
> **First Trade opened:** `3 days ago`
> **Latest Trade opened:** `2 minutes ago`
> **Avg. Duration:** `2:33:45`
> **Best Performing:** `PAY/BTC: 50.23%`
> **ROI:** Close trades
> ∙ `0.00485701 BTC (258.45%)`
> ∙ `62.968 USD`
> **ROI:** All trades
> ∙ `0.00255280 BTC (143.43%)`
> ∙ `33.095 EUR`
>
> **Total Trade Count:** `138`
> **First Trade opened:** `3 days ago`
> **Latest Trade opened:** `2 minutes ago`
> **Avg. Duration:** `2:33:45`
> **Best Performing:** `PAY/BTC: 50.23%`
### /forcesell <trade_id>
@ -128,26 +126,26 @@ Note that for this to work, `forcebuy_enable` needs to be set to true.
Return the performance of each crypto-currency the bot has sold.
> Performance:
> 1. `RCN/BTC 57.77%`
> 2. `PAY/BTC 56.91%`
> 3. `VIB/BTC 47.07%`
> 4. `SALT/BTC 30.24%`
> 5. `STORJ/BTC 27.24%`
> ...
> 1. `RCN/BTC 57.77%`
> 2. `PAY/BTC 56.91%`
> 3. `VIB/BTC 47.07%`
> 4. `SALT/BTC 30.24%`
> 5. `STORJ/BTC 27.24%`
> ...
### /balance
Return the balance of all crypto-currency your have on the exchange.
> **Currency:** BTC
> **Available:** 3.05890234
> **Balance:** 3.05890234
> **Pending:** 0.0
> **Currency:** BTC
> **Available:** 3.05890234
> **Balance:** 3.05890234
> **Pending:** 0.0
> **Currency:** CVC
> **Available:** 86.64180098
> **Balance:** 86.64180098
> **Pending:** 0.0
> **Currency:** CVC
> **Available:** 86.64180098
> **Balance:** 86.64180098
> **Pending:** 0.0
### /daily <n>

40
docs/webhook-config.md
View File

@ -1,7 +1,5 @@
# Webhook usage
This page explains how to configure your bot to talk to webhooks.
## Configuration
Enable webhooks by adding a webhook-section to your configuration file, and setting `webhook.enabled` to `true`.
@ -39,32 +37,30 @@ Different payloads can be configured for different events. Not all fields are ne
The fields in `webhook.webhookbuy` are filled when the bot executes a buy. Parameters are filled using string.format.
Possible parameters are:
* exchange
* pair
* limit
* stake_amount
* stake_amount_fiat
* stake_currency
* fiat_currency
* `exchange`
* `pair`
* `limit`
* `stake_amount`
* `stake_currency`
* `fiat_currency`
### Webhooksell
The fields in `webhook.webhooksell` are filled when the bot sells a trade. Parameters are filled using string.format.
Possible parameters are:
* exchange
* pair
* gain
* limit
* amount
* open_rate
* current_rate
* profit_amount
* profit_percent
* profit_fiat
* stake_currency
* fiat_currency
* sell_reason
* `exchange`
* `pair`
* `gain`
* `limit`
* `amount`
* `open_rate`
* `current_rate`
* `profit_amount`
* `profit_percent`
* `stake_currency`
* `fiat_currency`
* `sell_reason`
### Webhookstatus

16
freqtrade/edge/__init__.py
View File

@ -203,6 +203,22 @@ class Edge():
return self._final_pairs
def accepted_pairs(self) -> list:
"""
return a list of accepted pairs along with their winrate, expectancy and stoploss
"""
final = []
for pair, info in self._cached_pairs.items():
if info.expectancy > float(self.edge_config.get('minimum_expectancy', 0.2)) and \
info.winrate > float(self.edge_config.get('minimum_winrate', 0.60)):
final.append({
'Pair': pair,
'Winrate': info.winrate,
'Expectancy': info.expectancy,
'Stoploss': info.stoploss,
})
return final
def _fill_calculable_fields(self, result: DataFrame) -> DataFrame:
"""
The result frame contains a number of columns that are calculable

180
freqtrade/freqtradebot.py
View File

@ -4,14 +4,12 @@ Freqtrade is the main module of this bot. It contains the class Freqtrade()
import copy
import logging
import time
import traceback
from datetime import datetime
from typing import Any, Callable, Dict, List, Optional, Tuple
from typing import Any, Dict, List, Optional, Tuple
import arrow
from requests.exceptions import RequestException
import sdnotify
from freqtrade import (DependencyException, OperationalException, InvalidOrderException,
TemporaryError, __version__, constants, persistence)
@ -42,20 +40,14 @@ class FreqtradeBot(object):
to get the config dict.
"""
logger.info(
'Starting freqtrade %s',
__version__,
)
logger.info('Starting freqtrade %s', __version__)
# Init bot states
# Init bot state
self.state = State.STOPPED
# Init objects
self.config = config
self._sd_notify = sdnotify.SystemdNotifier() if \
self.config.get('internals', {}).get('sd_notify', False) else None
self.strategy: IStrategy = StrategyResolver(self.config).strategy
self.rpc: RPCManager = RPCManager(self)
@ -79,29 +71,12 @@ class FreqtradeBot(object):
self.config.get('edge', {}).get('enabled', False) else None
self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist']
self._init_modules()
# Tell the systemd that we completed initialization phase
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def _init_modules(self) -> None:
"""
Initializes all modules and updates the config
:return: None
"""
# Initialize all modules
persistence.init(self.config)
# Set initial application state
# Set initial bot state from config
initial_state = self.config.get('initial_state')
if initial_state:
self.state = State[initial_state.upper()]
else:
self.state = State.STOPPED
self.state = State[initial_state.upper()] if initial_state else State.STOPPED
def cleanup(self) -> None:
"""
@ -113,130 +88,50 @@ class FreqtradeBot(object):
self.rpc.cleanup()
persistence.cleanup()
def stopping(self) -> None:
# Tell systemd that we are exiting now
if self._sd_notify:
logger.debug("sd_notify: STOPPING=1")
self._sd_notify.notify("STOPPING=1")
def reconfigure(self) -> None:
# Tell systemd that we initiated reconfiguring
if self._sd_notify:
logger.debug("sd_notify: RELOADING=1")
self._sd_notify.notify("RELOADING=1")
def worker(self, old_state: State = None) -> State:
"""
Trading routine that must be run at each loop
:param old_state: the previous service state from the previous call
:return: current service state
"""
# Log state transition
state = self.state
if state != old_state:
self.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'{state.name.lower()}'
})
logger.info('Changing state to: %s', state.name)
if state == State.RUNNING:
self.rpc.startup_messages(self.config, self.pairlists)
throttle_secs = self.config.get('internals', {}).get(
'process_throttle_secs',
constants.PROCESS_THROTTLE_SECS
)
if state == State.STOPPED:
# Ping systemd watchdog before sleeping in the stopped state
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: STOPPED.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: STOPPED.")
time.sleep(throttle_secs)
elif state == State.RUNNING:
# Ping systemd watchdog before throttling
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: RUNNING.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: RUNNING.")
self._throttle(func=self._process, min_secs=throttle_secs)
return state
def _throttle(self, func: Callable[..., Any], min_secs: float, *args, **kwargs) -> Any:
"""
Throttles the given callable that it
takes at least `min_secs` to finish execution.
:param func: Any callable
:param min_secs: minimum execution time in seconds
:return: Any
"""
start = time.time()
result = func(*args, **kwargs)
end = time.time()
duration = max(min_secs - (end - start), 0.0)
logger.debug('Throttling %s for %.2f seconds', func.__name__, duration)
time.sleep(duration)
return result
def _process(self) -> bool:
def process(self) -> bool:
"""
Queries the persistence layer for open trades and handles them,
otherwise a new trade is created.
:return: True if one or more trades has been created or closed, False otherwise
"""
state_changed = False
try:
# Check whether markets have to be reloaded
self.exchange._reload_markets()
# Refresh whitelist
self.pairlists.refresh_pairlist()
self.active_pair_whitelist = self.pairlists.whitelist
# Check whether markets have to be reloaded
self.exchange._reload_markets()
# Calculating Edge positioning
if self.edge:
self.edge.calculate()
self.active_pair_whitelist = self.edge.adjust(self.active_pair_whitelist)
# Refresh whitelist
self.pairlists.refresh_pairlist()
self.active_pair_whitelist = self.pairlists.whitelist
# Query trades from persistence layer
trades = Trade.get_open_trades()
# Calculating Edge positioning
if self.edge:
self.edge.calculate()
self.active_pair_whitelist = self.edge.adjust(self.active_pair_whitelist)
# Extend active-pair whitelist with pairs from open trades
# It ensures that tickers are downloaded for open trades
self._extend_whitelist_with_trades(self.active_pair_whitelist, trades)
# Query trades from persistence layer
trades = Trade.get_open_trades()
# Refreshing candles
self.dataprovider.refresh(self._create_pair_whitelist(self.active_pair_whitelist),
self.strategy.informative_pairs())
# Extend active-pair whitelist with pairs from open trades
# It ensures that tickers are downloaded for open trades
self._extend_whitelist_with_trades(self.active_pair_whitelist, trades)
# First process current opened trades
for trade in trades:
state_changed |= self.process_maybe_execute_sell(trade)
# Refreshing candles
self.dataprovider.refresh(self._create_pair_whitelist(self.active_pair_whitelist),
self.strategy.informative_pairs())
# Then looking for buy opportunities
if len(trades) < self.config['max_open_trades']:
state_changed = self.process_maybe_execute_buy()
# First process current opened trades
for trade in trades:
state_changed |= self.process_maybe_execute_sell(trade)
if 'unfilledtimeout' in self.config:
# Check and handle any timed out open orders
self.check_handle_timedout()
Trade.session.flush()
# Then looking for buy opportunities
if len(trades) < self.config['max_open_trades']:
state_changed = self.process_maybe_execute_buy()
if 'unfilledtimeout' in self.config:
# Check and handle any timed out open orders
self.check_handle_timedout()
Trade.session.flush()
except TemporaryError as error:
logger.warning(f"Error: {error}, retrying in {constants.RETRY_TIMEOUT} seconds...")
time.sleep(constants.RETRY_TIMEOUT)
except OperationalException:
tb = traceback.format_exc()
hint = 'Issue `/start` if you think it is safe to restart.'
self.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'OperationalException:\n```\n{tb}```{hint}'
})
logger.exception('OperationalException. Stopping trader ...')
self.state = State.STOPPED
return state_changed
def _extend_whitelist_with_trades(self, whitelist: List[str], trades: List[Any]):
@ -356,6 +251,10 @@ class FreqtradeBot(object):
interval = self.strategy.ticker_interval
whitelist = copy.deepcopy(self.active_pair_whitelist)
if not whitelist:
logger.warning("Whitelist is empty.")
return False
# Remove currently opened and latest pairs from whitelist
for trade in Trade.get_open_trades():
if trade.pair in whitelist:
@ -363,7 +262,8 @@ class FreqtradeBot(object):
logger.debug('Ignoring %s in pair whitelist', trade.pair)
if not whitelist:
raise DependencyException('No currency pairs in whitelist')
logger.info("No currency pair in whitelist, but checking to sell open trades.")
return False
# running get_signal on historical data fetched
for _pair in whitelist:

52
freqtrade/main.py
View File

@ -10,10 +10,9 @@ from typing import List
from freqtrade import OperationalException
from freqtrade.arguments import Arguments
from freqtrade.configuration import Configuration, set_loggers
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType
from freqtrade.configuration import set_loggers
from freqtrade.worker import Worker
logger = logging.getLogger('freqtrade')
@ -27,7 +26,7 @@ def main(sysargv: List[str]) -> None:
sysargv,
'Free, open source crypto trading bot'
)
args = arguments.get_parsed_arg()
args: Namespace = arguments.get_parsed_arg()
# A subcommand has been issued.
# Means if Backtesting or Hyperopt have been called we exit the bot
@ -35,20 +34,12 @@ def main(sysargv: List[str]) -> None:
args.func(args)
return
freqtrade = None
worker = None
return_code = 1
try:
# Load and validate configuration
config = Configuration(args, None).get_config()
# Init the bot
freqtrade = FreqtradeBot(config)
state = None
while True:
state = freqtrade.worker(old_state=state)
if state == State.RELOAD_CONF:
freqtrade = reconfigure(freqtrade, args)
# Load and run worker
worker = Worker(args)
worker.run()
except KeyboardInterrupt:
logger.info('SIGINT received, aborting ...')
@ -59,34 +50,11 @@ def main(sysargv: List[str]) -> None:
except BaseException:
logger.exception('Fatal exception!')
finally:
if freqtrade:
freqtrade.stopping()
freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'process died'
})
freqtrade.cleanup()
if worker:
worker.exit()
sys.exit(return_code)
def reconfigure(freqtrade: FreqtradeBot, args: Namespace) -> FreqtradeBot:
"""
Cleans up current instance, reloads the configuration and returns the new instance
"""
freqtrade.reconfigure()
# Clean up current modules
freqtrade.cleanup()
# Create new instance
freqtrade = FreqtradeBot(Configuration(args, None).get_config())
freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'config reloaded'
})
return freqtrade
if __name__ == '__main__':
set_loggers()
main(sys.argv[1:])

84
freqtrade/optimize/backtesting.py
View File

@ -210,6 +210,32 @@ class Backtesting(object):
logger.info('Dumping backtest results to %s', recordfilename)
file_dump_json(recordfilename, records)
def _get_ticker_list(self, processed) -> Dict[str, DataFrame]:
"""
Helper function to convert a processed tickerlist into a list for performance reasons.
Used by backtest() - so keep this optimized for performance.
"""
headers = ['date', 'buy', 'open', 'close', 'sell', 'low', 'high']
ticker: Dict = {}
# Create ticker dict
for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
# to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
ticker_data.loc[:, 'sell'] = ticker_data['sell'].shift(1)
ticker_data.drop(ticker_data.head(1).index, inplace=True)
# Convert from Pandas to list for performance reasons
# (Looping Pandas is slow.)
ticker[pair] = [x for x in ticker_data.itertuples()]
return ticker
def _get_sell_trade_entry(
self, pair: str, buy_row: DataFrame,
partial_ticker: List, trade_count_lock: Dict, args: Dict) -> Optional[BacktestResult]:
@ -304,7 +330,6 @@ class Backtesting(object):
position_stacking: do we allow position stacking? (default: False)
:return: DataFrame
"""
headers = ['date', 'buy', 'open', 'close', 'sell', 'low', 'high']
processed = args['processed']
max_open_trades = args.get('max_open_trades', 0)
position_stacking = args.get('position_stacking', False)
@ -312,54 +337,50 @@ class Backtesting(object):
end_date = args['end_date']
trades = []
trade_count_lock: Dict = {}
ticker: Dict = {}
pairs = []
# Create ticker dict
for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
# to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
ticker_data.loc[:, 'sell'] = ticker_data['sell'].shift(1)
ticker_data.drop(ticker_data.head(1).index, inplace=True)
# Convert from Pandas to list for performance reasons
# (Looping Pandas is slow.)
ticker[pair] = [x for x in ticker_data.itertuples()]
pairs.append(pair)
# Dict of ticker-lists for performance (looping lists is a lot faster than dataframes)
ticker: Dict = self._get_ticker_list(processed)
lock_pair_until: Dict = {}
# Indexes per pair, so some pairs are allowed to have a missing start.
indexes: Dict = {}
tmp = start_date + timedelta(minutes=self.ticker_interval_mins)
index = 0
# Loop timerange and test per pair
# Loop timerange and get candle for each pair at that point in time
while tmp < end_date:
# print(f"time: {tmp}")
for i, pair in enumerate(ticker):
if pair not in indexes:
indexes[pair] = 0
try:
row = ticker[pair][index]
row = ticker[pair][indexes[pair]]
except IndexError:
# missing Data for one pair ...
# missing Data for one pair at the end.
# Warnings for this are shown by `validate_backtest_data`
continue
# Waits until the time-counter reaches the start of the data for this pair.
if row.date > tmp.datetime:
continue
indexes[pair] += 1
if row.buy == 0 or row.sell == 1:
continue # skip rows where no buy signal or that would immediately sell off
if not position_stacking:
if pair in lock_pair_until and row.date <= lock_pair_until[pair]:
continue
if (not position_stacking and pair in lock_pair_until
and row.date <= lock_pair_until[pair]):
# without positionstacking, we can only have one open trade per pair.
continue
if max_open_trades > 0:
# Check if max_open_trades has already been reached for the given date
if not trade_count_lock.get(row.date, 0) < max_open_trades:
continue
trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1
trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][index + 1:],
trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][indexes[pair]:],
trade_count_lock, args)
if trade_entry:
@ -367,11 +388,10 @@ class Backtesting(object):
trades.append(trade_entry)
else:
# Set lock_pair_until to end of testing period if trade could not be closed
# This happens only if the buy-signal was with the last candle
lock_pair_until[pair] = end_date
lock_pair_until[pair] = end_date.datetime
# Move time one configured time_interval ahead.
tmp += timedelta(minutes=self.ticker_interval_mins)
index += 1
return DataFrame.from_records(trades, columns=BacktestResult._fields)
def start(self) -> None:

20
freqtrade/rpc/rpc.py
View File

@ -103,16 +103,23 @@ class RPC(object):
results.append(dict(
trade_id=trade.id,
pair=trade.pair,
base_currency=self._freqtrade.config['stake_currency'],
date=arrow.get(trade.open_date),
open_rate=trade.open_rate,
close_rate=trade.close_rate,
current_rate=current_rate,
amount=round(trade.amount, 8),
stake_amount=round(trade.stake_amount, 8),
close_profit=fmt_close_profit,
current_profit=round(current_profit * 100, 2),
stop_loss=trade.stop_loss,
stop_loss_pct=(trade.stop_loss_pct * 100)
if trade.stop_loss_pct else None,
initial_stop_loss=trade.initial_stop_loss,
initial_stop_loss_pct=(trade.initial_stop_loss_pct * 100)
if trade.initial_stop_loss_pct else None,
open_order='({} {} rem={:.8f})'.format(
order['type'], order['side'], order['remaining']
order['type'], order['side'], order['remaining']
) if order else None,
))
return results
@ -481,13 +488,4 @@ class RPC(object):
""" Returns information related to Edge """
if not self._freqtrade.edge:
raise RPCException(f'Edge is not enabled.')
return [
{
'Pair': k,
'Winrate': v.winrate,
'Expectancy': v.expectancy,
'Stoploss': v.stoploss,
}
for k, v in self._freqtrade.edge._cached_pairs.items()
]
return self._freqtrade.edge.accepted_pairs()

18
freqtrade/rpc/telegram.py
View File

@ -197,16 +197,25 @@ class Telegram(RPC):
messages = []
for r in results:
lines = [
"*Trade ID:* `{trade_id}` (since `{date}`)",
"*Trade ID:* `{trade_id}` `(since {date})`",
"*Current Pair:* {pair}",
"*Amount:* `{amount}`",
"*Amount:* `{amount} ({stake_amount} {base_currency})`",
"*Open Rate:* `{open_rate:.8f}`",
"*Close Rate:* `{close_rate}`" if r['close_rate'] else "",
"*Current Rate:* `{current_rate:.8f}`",
"*Close Profit:* `{close_profit}`" if r['close_profit'] else "",
"*Current Profit:* `{current_profit:.2f}%`",
"*Stoploss:* `{stop_loss:.8f}`",
"*Open Order:* `{open_order}`" if r['open_order'] else "",
# Adding initial stoploss only if it is different from stoploss
"*Initial Stoploss:* `{initial_stop_loss:.8f}` " +
("`({initial_stop_loss_pct:.2f}%)`" if r['initial_stop_loss_pct'] else "")
if r['stop_loss'] != r['initial_stop_loss'] else "",
# Adding stoploss and stoploss percentage only if it is not None
"*Stoploss:* `{stop_loss:.8f}` " +
("`({stop_loss_pct:.2f}%)`" if r['stop_loss_pct'] else ""),
"*Open Order:* `{open_order}`" if r['open_order'] else ""
]
messages.append("\n".join(filter(None, lines)).format(**r))
@ -502,7 +511,6 @@ class Telegram(RPC):
"""
try:
edge_pairs = self._rpc_edge()
print(edge_pairs)
edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple')
message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML)

11
freqtrade/tests/conftest.py
View File

@ -16,6 +16,7 @@ from freqtrade.edge import Edge, PairInfo
from freqtrade.exchange import Exchange
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.resolvers import ExchangeResolver
from freqtrade.worker import Worker
logging.getLogger('').setLevel(logging.INFO)
@ -87,7 +88,7 @@ def get_patched_edge(mocker, config) -> Edge:
# Functions for recurrent object patching
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
def patch_freqtradebot(mocker, config) -> None:
"""
This function patch _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
@ -100,9 +101,17 @@ def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
patch_freqtradebot(mocker, config)
return FreqtradeBot(config)
def get_patched_worker(mocker, config) -> Worker:
patch_freqtradebot(mocker, config)
return Worker(args=None, config=config)
@pytest.fixture(autouse=True)
def patch_coinmarketcap(mocker) -> None:
"""

4
freqtrade/tests/edge/test_edge.py
View File

@ -122,8 +122,8 @@ def test_edge_results(edge_conf, mocker, caplog, data) -> None:
for c, trade in enumerate(data.trades):
res = results.iloc[c]
assert res.exit_type == trade.sell_reason
assert res.open_time == _get_frame_time_from_offset(trade.open_tick)
assert res.close_time == _get_frame_time_from_offset(trade.close_tick)
assert arrow.get(res.open_time) == _get_frame_time_from_offset(trade.open_tick)
assert arrow.get(res.close_time) == _get_frame_time_from_offset(trade.close_tick)
def test_adjust(mocker, edge_conf):

2
freqtrade/tests/optimize/__init__.py
View File

@ -33,7 +33,7 @@ class BTContainer(NamedTuple):
def _get_frame_time_from_offset(offset):
return ticker_start_time.shift(minutes=(offset * TICKER_INTERVAL_MINUTES[tests_ticker_interval])
).datetime.replace(tzinfo=None)
).datetime
def _build_backtest_dataframe(ticker_with_signals):

19
freqtrade/tests/optimize/test_backtesting.py
View File

@ -685,25 +685,32 @@ def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
assert len(results.loc[results.open_at_end]) == 0
def test_backtest_multi_pair(default_conf, fee, mocker):
@pytest.mark.parametrize("pair", ['ADA/BTC', 'LTC/BTC'])
@pytest.mark.parametrize("tres", [0, 20, 30])
def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
def _trend_alternate_hold(dataframe=None, metadata=None):
"""
Buy every 8th candle - sell every other 8th -2 (hold on to pairs a bit)
Buy every xth candle - sell every other xth -2 (hold on to pairs a bit)
"""
multi = 8
if metadata['pair'] in('ETH/BTC', 'LTC/BTC'):
multi = 20
else:
multi = 18
dataframe['buy'] = np.where(dataframe.index % multi == 0, 1, 0)
dataframe['sell'] = np.where((dataframe.index + multi - 2) % multi == 0, 1, 0)
if metadata['pair'] in('ETH/BTC', 'LTC/BTC'):
dataframe['buy'] = dataframe['buy'].shift(-4)
dataframe['sell'] = dataframe['sell'].shift(-4)
return dataframe
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker)
pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC']
data = history.load_data(datadir=None, ticker_interval='5m', pairs=pairs)
# Only use 500 lines to increase performance
data = trim_dictlist(data, -500)
# Remove data for one pair from the beginning of the data
data[pair] = data[pair][tres:]
# We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True}
default_conf['ticker_interval'] = '5m'

10
freqtrade/tests/rpc/test_rpc.py
View File

@ -51,14 +51,19 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
assert {
'trade_id': 1,
'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': ANY,
'open_rate': 1.099e-05,
'close_rate': None,
'current_rate': 1.098e-05,
'amount': 90.99181074,
'stake_amount': 0.001,
'close_profit': None,
'current_profit': -0.59,
'stop_loss': 0.0,
'initial_stop_loss': 0.0,
'initial_stop_loss_pct': None,
'stop_loss_pct': None,
'open_order': '(limit buy rem=0.00000000)'
} == results[0]
@ -72,14 +77,19 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
assert {
'trade_id': 1,
'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': ANY,
'open_rate': 1.099e-05,
'close_rate': None,
'current_rate': ANY,
'amount': 90.99181074,
'stake_amount': 0.001,
'close_profit': None,
'current_profit': ANY,
'stop_loss': 0.0,
'initial_stop_loss': 0.0,
'initial_stop_loss_pct': None,
'stop_loss_pct': None,
'open_order': '(limit buy rem=0.00000000)'
} == results[0]

5
freqtrade/tests/rpc/test_rpc_telegram.py
View File

@ -189,14 +189,19 @@ def test_status(default_conf, update, mocker, fee, ticker, markets) -> None:
_rpc_trade_status=MagicMock(return_value=[{
'trade_id': 1,
'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': arrow.utcnow(),
'open_rate': 1.099e-05,
'close_rate': None,
'current_rate': 1.098e-05,
'amount': 90.99181074,
'stake_amount': 90.99181074,
'close_profit': None,
'current_profit': -0.59,
'initial_stop_loss': 1.098e-05,
'stop_loss': 1.099e-05,
'initial_stop_loss_pct': -0.05,
'stop_loss_pct': -0.01,
'open_order': '(limit buy rem=0.00000000)'
}]),
_status_table=status_table,

133
freqtrade/tests/test_freqtradebot.py
View File

@ -21,12 +21,13 @@ from freqtrade.state import State
from freqtrade.strategy.interface import SellCheckTuple, SellType
from freqtrade.tests.conftest import (log_has, log_has_re, patch_edge,
patch_exchange, patch_wallet)
from freqtrade.worker import Worker
# Functions for recurrent object patching
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
def patch_freqtradebot(mocker, config) -> None:
"""
This function patch _init_modules() to not call dependencies
This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot
:return: None
@ -35,9 +36,29 @@ def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
patch_exchange(mocker)
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
"""
This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot
:return: FreqtradeBot
"""
patch_freqtradebot(mocker, config)
return FreqtradeBot(config)
def get_patched_worker(mocker, config) -> Worker:
"""
This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot
:return: Worker
"""
patch_freqtradebot(mocker, config)
return Worker(args=None, config=config)
def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None:
"""
:param mocker: mocker to patch IStrategy class
@ -61,7 +82,7 @@ def patch_RPCManager(mocker) -> MagicMock:
# Unit tests
def test_freqtradebot(mocker, default_conf, markets) -> None:
def test_freqtradebot_state(mocker, default_conf, markets) -> None:
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets))
freqtrade = get_patched_freqtradebot(mocker, default_conf)
assert freqtrade.state is State.RUNNING
@ -71,6 +92,16 @@ def test_freqtradebot(mocker, default_conf, markets) -> None:
assert freqtrade.state is State.STOPPED
def test_worker_state(mocker, default_conf, markets) -> None:
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets))
worker = get_patched_worker(mocker, default_conf)
assert worker.state is State.RUNNING
default_conf.pop('initial_state')
worker = Worker(args=None, config=default_conf)
assert worker.state is State.STOPPED
def test_cleanup(mocker, default_conf, caplog) -> None:
mock_cleanup = MagicMock()
mocker.patch('freqtrade.persistence.cleanup', mock_cleanup)
@ -82,28 +113,28 @@ def test_cleanup(mocker, default_conf, caplog) -> None:
def test_worker_running(mocker, default_conf, caplog) -> None:
mock_throttle = MagicMock()
mocker.patch('freqtrade.freqtradebot.FreqtradeBot._throttle', mock_throttle)
mocker.patch('freqtrade.worker.Worker._throttle', mock_throttle)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
worker = get_patched_worker(mocker, default_conf)
state = freqtrade.worker(old_state=None)
state = worker._worker(old_state=None)
assert state is State.RUNNING
assert log_has('Changing state to: RUNNING', caplog.record_tuples)
assert mock_throttle.call_count == 1
# Check strategy is loaded, and received a dataprovider object
assert freqtrade.strategy
assert freqtrade.strategy.dp
assert isinstance(freqtrade.strategy.dp, DataProvider)
assert worker.freqtrade.strategy
assert worker.freqtrade.strategy.dp
assert isinstance(worker.freqtrade.strategy.dp, DataProvider)
def test_worker_stopped(mocker, default_conf, caplog) -> None:
mock_throttle = MagicMock()
mocker.patch('freqtrade.freqtradebot.FreqtradeBot._throttle', mock_throttle)
mocker.patch('freqtrade.worker.Worker._throttle', mock_throttle)
mock_sleep = mocker.patch('time.sleep', return_value=None)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
freqtrade.state = State.STOPPED
state = freqtrade.worker(old_state=State.RUNNING)
worker = get_patched_worker(mocker, default_conf)
worker.state = State.STOPPED
state = worker._worker(old_state=State.RUNNING)
assert state is State.STOPPED
assert log_has('Changing state to: STOPPED', caplog.record_tuples)
assert mock_throttle.call_count == 0
@ -115,17 +146,17 @@ def test_throttle(mocker, default_conf, caplog) -> None:
return 42
caplog.set_level(logging.DEBUG)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
worker = get_patched_worker(mocker, default_conf)
start = time.time()
result = freqtrade._throttle(throttled_func, min_secs=0.1)
result = worker._throttle(throttled_func, min_secs=0.1)
end = time.time()
assert result == 42
assert end - start > 0.1
assert log_has('Throttling throttled_func for 0.10 seconds', caplog.record_tuples)
result = freqtrade._throttle(throttled_func, min_secs=-1)
result = worker._throttle(throttled_func, min_secs=-1)
assert result == 42
@ -133,12 +164,12 @@ def test_throttle_with_assets(mocker, default_conf) -> None:
def throttled_func(nb_assets=-1):
return nb_assets
freqtrade = get_patched_freqtradebot(mocker, default_conf)
worker = get_patched_worker(mocker, default_conf)
result = freqtrade._throttle(throttled_func, min_secs=0.1, nb_assets=666)
result = worker._throttle(throttled_func, min_secs=0.1, nb_assets=666)
assert result == 666
result = freqtrade._throttle(throttled_func, min_secs=0.1)
result = worker._throttle(throttled_func, min_secs=0.1)
assert result == -1
@ -224,7 +255,7 @@ def test_edge_called_in_process(mocker, edge_conf) -> None:
freqtrade = FreqtradeBot(edge_conf)
freqtrade.pairlists._validate_whitelist = _refresh_whitelist
patch_get_signal(freqtrade)
freqtrade._process()
freqtrade.process()
assert freqtrade.active_pair_whitelist == ['NEO/BTC', 'LTC/BTC']
@ -551,8 +582,7 @@ def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_ord
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
result = freqtrade.create_trade()
assert result is False
assert not freqtrade.create_trade()
def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order,
@ -573,11 +603,12 @@ def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order,
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
assert freqtrade.create_trade() is False
assert not freqtrade.create_trade()
assert freqtrade._get_trade_stake_amount('ETH/BTC') is None
def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, markets, mocker) -> None:
def test_create_trade_no_pairs_let(default_conf, ticker, limit_buy_order, fee,
markets, mocker, caplog) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
mocker.patch.multiple(
@ -589,18 +620,17 @@ def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, marke
)
default_conf['exchange']['pair_whitelist'] = ["ETH/BTC"]
default_conf['exchange']['pair_blacklist'] = ["ETH/BTC"]
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
freqtrade.create_trade()
with pytest.raises(DependencyException, match=r'.*No currency pairs in whitelist.*'):
freqtrade.create_trade()
assert freqtrade.create_trade()
assert not freqtrade.create_trade()
assert log_has("No currency pair in whitelist, but checking to sell open trades.",
caplog.record_tuples)
def test_create_trade_no_pairs_after_blacklist(default_conf, ticker,
limit_buy_order, fee, markets, mocker) -> None:
def test_create_trade_no_pairs_in_whitelist(default_conf, ticker, limit_buy_order, fee,
markets, mocker, caplog) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
mocker.patch.multiple(
@ -610,15 +640,12 @@ def test_create_trade_no_pairs_after_blacklist(default_conf, ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
default_conf['exchange']['pair_whitelist'] = ["ETH/BTC"]
default_conf['exchange']['pair_blacklist'] = ["ETH/BTC"]
default_conf['exchange']['pair_whitelist'] = []
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
freqtrade.create_trade()
with pytest.raises(DependencyException, match=r'.*No currency pairs in whitelist.*'):
freqtrade.create_trade()
assert not freqtrade.create_trade()
assert log_has("Whitelist is empty.", caplog.record_tuples)
def test_create_trade_no_signal(default_conf, fee, mocker) -> None:
@ -658,7 +685,7 @@ def test_process_trade_creation(default_conf, ticker, limit_buy_order,
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert not trades
result = freqtrade._process()
result = freqtrade.process()
assert result is True
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
@ -689,10 +716,10 @@ def test_process_exchange_failures(default_conf, ticker, markets, mocker) -> Non
)
sleep_mock = mocker.patch('time.sleep', side_effect=lambda _: None)
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
worker = Worker(args=None, config=default_conf)
patch_get_signal(worker.freqtrade)
result = freqtrade._process()
result = worker._process()
assert result is False
assert sleep_mock.has_calls()
@ -706,14 +733,14 @@ def test_process_operational_exception(default_conf, ticker, markets, mocker) ->
markets=PropertyMock(return_value=markets),
buy=MagicMock(side_effect=OperationalException)
)
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
worker = Worker(args=None, config=default_conf)
patch_get_signal(worker.freqtrade)
assert freqtrade.state == State.RUNNING
assert worker.state == State.RUNNING
result = freqtrade._process()
result = worker._process()
assert result is False
assert freqtrade.state == State.STOPPED
assert worker.state == State.STOPPED
assert 'OperationalException' in msg_mock.call_args_list[-1][0][0]['status']
@ -734,18 +761,18 @@ def test_process_trade_handling(
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert not trades
result = freqtrade._process()
result = freqtrade.process()
assert result is True
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert len(trades) == 1
result = freqtrade._process()
result = freqtrade.process()
assert result is False
def test_process_trade_no_whitelist_pair(
default_conf, ticker, limit_buy_order, markets, fee, mocker) -> None:
""" Test _process with trade not in pair list """
""" Test process with trade not in pair list """
patch_RPCManager(mocker)
patch_exchange(mocker)
mocker.patch.multiple(
@ -782,7 +809,7 @@ def test_process_trade_no_whitelist_pair(
))
assert pair not in freqtrade.active_pair_whitelist
result = freqtrade._process()
result = freqtrade.process()
assert pair in freqtrade.active_pair_whitelist
# Make sure each pair is only in the list once
assert len(freqtrade.active_pair_whitelist) == len(set(freqtrade.active_pair_whitelist))
@ -812,7 +839,7 @@ def test_process_informative_pairs_added(default_conf, ticker, markets, mocker)
freqtrade.strategy.informative_pairs = inf_pairs
# patch_get_signal(freqtrade)
freqtrade._process()
freqtrade.process()
assert inf_pairs.call_count == 1
assert refresh_mock.call_count == 1
assert ("BTC/ETH", "1m") in refresh_mock.call_args[0][0]
@ -3083,5 +3110,5 @@ def test_startup_messages(default_conf, mocker):
'config': {'number_assets': 20}
}
mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
freqtrade = get_patched_freqtradebot(mocker, default_conf)
assert freqtrade.state is State.RUNNING
worker = get_patched_worker(mocker, default_conf)
assert worker.state is State.RUNNING

64
freqtrade/tests/test_main.py
View File

@ -7,8 +7,8 @@ import pytest
from freqtrade import OperationalException
from freqtrade.arguments import Arguments
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.main import main, reconfigure
from freqtrade.worker import Worker
from freqtrade.main import main
from freqtrade.state import State
from freqtrade.tests.conftest import log_has, patch_exchange
@ -43,17 +43,14 @@ def test_main_start_hyperopt(mocker) -> None:
def test_main_fatal_exception(mocker, default_conf, caplog) -> None:
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.freqtradebot.FreqtradeBot',
_init_modules=MagicMock(),
worker=MagicMock(side_effect=Exception),
cleanup=MagicMock(),
)
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
mocker.patch('freqtrade.worker.Worker._worker', MagicMock(side_effect=Exception))
mocker.patch(
'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf
)
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example']
@ -66,17 +63,14 @@ def test_main_fatal_exception(mocker, default_conf, caplog) -> None:
def test_main_keyboard_interrupt(mocker, default_conf, caplog) -> None:
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.freqtradebot.FreqtradeBot',
_init_modules=MagicMock(),
worker=MagicMock(side_effect=KeyboardInterrupt),
cleanup=MagicMock(),
)
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
mocker.patch('freqtrade.worker.Worker._worker', MagicMock(side_effect=KeyboardInterrupt))
mocker.patch(
'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf
)
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example']
@ -89,17 +83,17 @@ def test_main_keyboard_interrupt(mocker, default_conf, caplog) -> None:
def test_main_operational_exception(mocker, default_conf, caplog) -> None:
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.freqtradebot.FreqtradeBot',
_init_modules=MagicMock(),
worker=MagicMock(side_effect=OperationalException('Oh snap!')),
cleanup=MagicMock(),
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
mocker.patch(
'freqtrade.worker.Worker._worker',
MagicMock(side_effect=OperationalException('Oh snap!'))
)
mocker.patch(
'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf
)
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example']
@ -112,21 +106,18 @@ def test_main_operational_exception(mocker, default_conf, caplog) -> None:
def test_main_reload_conf(mocker, default_conf, caplog) -> None:
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.freqtradebot.FreqtradeBot',
_init_modules=MagicMock(),
worker=MagicMock(return_value=State.RELOAD_CONF),
cleanup=MagicMock(),
)
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
mocker.patch('freqtrade.worker.Worker._worker', MagicMock(return_value=State.RELOAD_CONF))
mocker.patch(
'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf
)
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
# Raise exception as side effect to avoid endless loop
reconfigure_mock = mocker.patch(
'freqtrade.main.reconfigure', MagicMock(side_effect=Exception)
'freqtrade.main.Worker._reconfigure', MagicMock(side_effect=Exception)
)
with pytest.raises(SystemExit):
@ -138,19 +129,21 @@ def test_main_reload_conf(mocker, default_conf, caplog) -> None:
def test_reconfigure(mocker, default_conf) -> None:
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.freqtradebot.FreqtradeBot',
_init_modules=MagicMock(),
worker=MagicMock(side_effect=OperationalException('Oh snap!')),
cleanup=MagicMock(),
mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
mocker.patch(
'freqtrade.worker.Worker._worker',
MagicMock(side_effect=OperationalException('Oh snap!'))
)
mocker.patch(
'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf
)
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
freqtrade = FreqtradeBot(default_conf)
args = Arguments(['-c', 'config.json.example'], '').get_parsed_arg()
worker = Worker(args=args, config=default_conf)
freqtrade = worker.freqtrade
# Renew mock to return modified data
conf = deepcopy(default_conf)
@ -160,11 +153,10 @@ def test_reconfigure(mocker, default_conf) -> None:
lambda *args, **kwargs: conf
)
worker._config = conf
# reconfigure should return a new instance
freqtrade2 = reconfigure(
freqtrade,
Arguments(['-c', 'config.json.example'], '').get_parsed_arg()
)
worker._reconfigure()
freqtrade2 = worker.freqtrade
# Verify we have a new instance with the new config
assert freqtrade is not freqtrade2

188
freqtrade/worker.py Executable file
View File

@ -0,0 +1,188 @@
"""
Main Freqtrade worker class.
"""
import logging
import time
import traceback
from argparse import Namespace
from typing import Any, Callable, Optional
import sdnotify
from freqtrade import (constants, OperationalException, TemporaryError,
__version__)
from freqtrade.configuration import Configuration
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType
logger = logging.getLogger(__name__)
class Worker(object):
"""
Freqtradebot worker class
"""
def __init__(self, args: Namespace, config=None) -> None:
"""
Init all variables and objects the bot needs to work
"""
logger.info('Starting worker %s', __version__)
self._args = args
self._config = config
self._init(False)
# Tell systemd that we completed initialization phase
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def _init(self, reconfig: bool):
"""
Also called from the _reconfigure() method (with reconfig=True).
"""
if reconfig or self._config is None:
# Load configuration
self._config = Configuration(self._args, None).get_config()
# Init the instance of the bot
self.freqtrade = FreqtradeBot(self._config)
self._throttle_secs = self._config.get('internals', {}).get(
'process_throttle_secs',
constants.PROCESS_THROTTLE_SECS
)
self._sd_notify = sdnotify.SystemdNotifier() if \
self._config.get('internals', {}).get('sd_notify', False) else None
@property
def state(self) -> State:
return self.freqtrade.state
@state.setter
def state(self, value: State):
self.freqtrade.state = value
def run(self):
state = None
while True:
state = self._worker(old_state=state)
if state == State.RELOAD_CONF:
self.freqtrade = self._reconfigure()
def _worker(self, old_state: State, throttle_secs: Optional[float] = None) -> State:
"""
Trading routine that must be run at each loop
:param old_state: the previous service state from the previous call
:return: current service state
"""
state = self.freqtrade.state
if throttle_secs is None:
throttle_secs = self._throttle_secs
# Log state transition
if state != old_state:
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'{state.name.lower()}'
})
logger.info('Changing state to: %s', state.name)
if state == State.RUNNING:
self.freqtrade.rpc.startup_messages(self._config, self.freqtrade.pairlists)
if state == State.STOPPED:
# Ping systemd watchdog before sleeping in the stopped state
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: STOPPED.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: STOPPED.")
time.sleep(throttle_secs)
elif state == State.RUNNING:
# Ping systemd watchdog before throttling
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: RUNNING.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: RUNNING.")
self._throttle(func=self._process, min_secs=throttle_secs)
return state
def _throttle(self, func: Callable[..., Any], min_secs: float, *args, **kwargs) -> Any:
"""
Throttles the given callable that it
takes at least `min_secs` to finish execution.
:param func: Any callable
:param min_secs: minimum execution time in seconds
:return: Any
"""
start = time.time()
result = func(*args, **kwargs)
end = time.time()
duration = max(min_secs - (end - start), 0.0)
logger.debug('Throttling %s for %.2f seconds', func.__name__, duration)
time.sleep(duration)
return result
def _process(self) -> bool:
state_changed = False
try:
state_changed = self.freqtrade.process()
except TemporaryError as error:
logger.warning(f"Error: {error}, retrying in {constants.RETRY_TIMEOUT} seconds...")
time.sleep(constants.RETRY_TIMEOUT)
except OperationalException:
tb = traceback.format_exc()
hint = 'Issue `/start` if you think it is safe to restart.'
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'OperationalException:\n```\n{tb}```{hint}'
})
logger.exception('OperationalException. Stopping trader ...')
self.freqtrade.state = State.STOPPED
# TODO: The return value of _process() is not used apart tests
# and should (could) be eliminated later. See PR #1689.
# state_changed = True
return state_changed
def _reconfigure(self):
"""
Cleans up current freqtradebot instance, reloads the configuration and
replaces it with the new instance
"""
# Tell systemd that we initiated reconfiguration
if self._sd_notify:
logger.debug("sd_notify: RELOADING=1")
self._sd_notify.notify("RELOADING=1")
# Clean up current freqtrade modules
self.freqtrade.cleanup()
# Load and validate config and create new instance of the bot
self._init(True)
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'config reloaded'
})
# Tell systemd that we completed reconfiguration
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def exit(self):
# Tell systemd that we are exiting now
if self._sd_notify:
logger.debug("sd_notify: STOPPING=1")
self._sd_notify.notify("STOPPING=1")
if self.freqtrade:
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'process died'
})
self.freqtrade.cleanup()

9
mkdocs.yml
View File

@ -3,11 +3,12 @@ nav:
- About: index.md
- Installation: installation.md
- Configuration: configuration.md
- Start the bot: bot-usage.md
- Stoploss: stoploss.md
- Custom Strategy: bot-optimization.md
- Telegram: telegram-usage.md
- Web Hook: webhook-config.md
- Stoploss: stoploss.md
- Start the bot: bot-usage.md
- Control the bot:
- Telegram: telegram-usage.md
- Web Hook: webhook-config.md
- Backtesting: backtesting.md
- Hyperopt: hyperopt.md
- Edge positioning: edge.md

2
requirements-dev.txt
View File

@ -9,4 +9,4 @@ pytest-mock==1.10.3
pytest-asyncio==0.10.0
pytest-cov==2.6.1
coveralls==1.7.0
mypy==0.670
mypy==0.700

4
requirements.txt
View File

@ -1,5 +1,5 @@
ccxt==1.18.425
SQLAlchemy==1.3.1
ccxt==1.18.432
SQLAlchemy==1.3.2
python-telegram-bot==11.1.0
arrow==0.13.1
cachetools==3.1.0