Merge pull request #1863 from xmatthias/feat/flask_rest_retry

Add REST API to control the bot
This commit is contained in:
Matthias 2019-06-02 15:20:12 +02:00 committed by GitHub
commit 4e45aa1564
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 1513 additions and 57 deletions

View File

@ -109,6 +109,13 @@
"token": "your_telegram_token",
"chat_id": "your_telegram_chat_id"
},
"api_server": {
"enabled": false,
"listen_ip_address": "127.0.0.1",
"listen_port": 8080,
"username": "freqtrader",
"password": "SuperSecurePassword"
},
"db_url": "sqlite:///tradesv3.sqlite",
"initial_state": "running",
"forcebuy_enable": false,

View File

@ -21,8 +21,8 @@ Freqtrade is a cryptocurrency trading bot written in Python.
We strongly recommend you to have basic coding skills and Python knowledge. Do not hesitate to read the source code and understand the mechanisms of this bot, algorithms and techniques implemented in it.
## Features
- Based on Python 3.6+: For botting on any operating system — Windows, macOS and Linux.
- Persistence: Persistence is achieved through sqlite database.
- Dry-run mode: Run the bot without playing money.
@ -31,7 +31,7 @@ Freqtrade is a cryptocurrency trading bot written in Python.
- Edge position sizing: Calculate your win rate, risk reward ratio, the best stoploss and adjust your position size before taking a position for each specific market.
- Whitelist crypto-currencies: Select which crypto-currency you want to trade or use dynamic whitelists based on market (pair) trade volume.
- Blacklist crypto-currencies: Select which crypto-currency you want to avoid.
- Manageable via Telegram: Manage the bot with Telegram.
- Manageable via Telegram or REST APi: Manage the bot with Telegram or via the builtin REST API.
- Display profit/loss in fiat: Display your profit/loss in any of 33 fiat currencies supported.
- Daily summary of profit/loss: Receive the daily summary of your profit/loss.
- Performance status report: Receive the performance status of your current trades.
@ -59,7 +59,6 @@ To run this bot we recommend you a cloud instance with a minimum of:
- virtualenv (Recommended)
- Docker (Recommended)
## Support
Help / Slack

193
docs/rest-api.md Normal file
View File

@ -0,0 +1,193 @@
# REST API Usage
## Configuration
Enable the rest API by adding the api_server section to your configuration and setting `api_server.enabled` to `true`.
Sample configuration:
``` json
"api_server": {
"enabled": true,
"listen_ip_address": "127.0.0.1",
"listen_port": 8080,
"username": "Freqtrader",
"password": "SuperSecret1!"
},
```
!!! Danger: Security warning
By default, the configuration listens on localhost only (so it's not reachable from other systems). We strongly recommend to not expose this API to the internet and choose a strong, unique password, since others will potentially be able to control your bot.
!!! Danger: Password selection
Please make sure to select a very strong, unique password to protect your bot from unauthorized access.
You can then access the API by going to `http://127.0.0.1:8080/api/v1/version` to check if the API is running correctly.
To generate a secure password, either use a password manager, or use the below code snipped.
``` python
import secrets
secrets.token_hex()
```
### Configuration with docker
If you run your bot using docker, you'll need to have the bot listen to incomming connections. The security is then handled by docker.
``` json
"api_server": {
"enabled": true,
"listen_ip_address": "0.0.0.0",
"listen_port": 8080
},
```
Add the following to your docker command:
``` bash
-p 127.0.0.1:8080:8080
```
A complete sample-command may then look as follows:
```bash
docker run -d \
--name freqtrade \
-v ~/.freqtrade/config.json:/freqtrade/config.json \
-v ~/.freqtrade/user_data/:/freqtrade/user_data \
-v ~/.freqtrade/tradesv3.sqlite:/freqtrade/tradesv3.sqlite \
-p 127.0.0.1:8080:8080 \
freqtrade --db-url sqlite:///tradesv3.sqlite --strategy MyAwesomeStrategy
```
!!! Danger "Security warning"
By using `-p 8080:8080` the API is available to everyone connecting to the server under the correct port, so others may be able to control your bot.
## Consuming the API
You can consume the API by using the script `scripts/rest_client.py`.
The client script only requires the `requests` module, so FreqTrade does not need to be installed on the system.
``` bash
python3 scripts/rest_client.py <command> [optional parameters]
```
By default, the script assumes `127.0.0.1` (localhost) and port `8080` to be used, however you can specify a configuration file to override this behaviour.
### Minimalistic client config
``` json
{
"api_server": {
"enabled": true,
"listen_ip_address": "0.0.0.0",
"listen_port": 8080
}
}
```
``` bash
python3 scripts/rest_client.py --config rest_config.json <command> [optional parameters]
```
## Available commands
| Command | Default | Description |
|----------|---------|-------------|
| `start` | | Starts the trader
| `stop` | | Stops the trader
| `stopbuy` | | Stops the trader from opening new trades. Gracefully closes open trades according to their rules.
| `reload_conf` | | Reloads the configuration file
| `status` | | Lists all open trades
| `status table` | | List all open trades in a table format
| `count` | | Displays number of trades used and available
| `profit` | | Display a summary of your profit/loss from close trades and some stats about your performance
| `forcesell <trade_id>` | | Instantly sells the given trade (Ignoring `minimum_roi`).
| `forcesell all` | | Instantly sells all open trades (Ignoring `minimum_roi`).
| `forcebuy <pair> [rate]` | | Instantly buys the given pair. Rate is optional. (`forcebuy_enable` must be set to True)
| `performance` | | Show performance of each finished trade grouped by pair
| `balance` | | Show account balance per currency
| `daily <n>` | 7 | Shows profit or loss per day, over the last n days
| `whitelist` | | Show the current whitelist
| `blacklist [pair]` | | Show the current blacklist, or adds a pair to the blacklist.
| `edge` | | Show validated pairs by Edge if it is enabled.
| `version` | | Show version
Possible commands can be listed from the rest-client script using the `help` command.
``` bash
python3 scripts/rest_client.py help
```
``` output
Possible commands:
balance
Get the account balance
:returns: json object
blacklist
Show the current blacklist
:param add: List of coins to add (example: "BNB/BTC")
:returns: json object
count
Returns the amount of open trades
:returns: json object
daily
Returns the amount of open trades
:returns: json object
edge
Returns information about edge
:returns: json object
forcebuy
Buy an asset
:param pair: Pair to buy (ETH/BTC)
:param price: Optional - price to buy
:returns: json object of the trade
forcesell
Force-sell a trade
:param tradeid: Id of the trade (can be received via status command)
:returns: json object
performance
Returns the performance of the different coins
:returns: json object
profit
Returns the profit summary
:returns: json object
reload_conf
Reload configuration
:returns: json object
start
Start the bot if it's in stopped state.
:returns: json object
status
Get the status of open trades
:returns: json object
stop
Stop the bot. Use start to restart
:returns: json object
stopbuy
Stop buying (but handle sells gracefully).
use reload_conf to reset
:returns: json object
version
Returns the version of the bot
:returns: json object containing the version
whitelist
Show the current whitelist
:returns: json object
```

View File

@ -156,6 +156,21 @@ CONF_SCHEMA = {
'webhookstatus': {'type': 'object'},
},
},
'api_server': {
'type': 'object',
'properties': {
'enabled': {'type': 'boolean'},
'listen_ip_address': {'format': 'ipv4'},
'listen_port': {
'type': 'integer',
"minimum": 1024,
"maximum": 65535
},
'username': {'type': 'string'},
'password': {'type': 'string'},
},
'required': ['enabled', 'listen_ip_address', 'listen_port', 'username', 'password']
},
'db_url': {'type': 'string'},
'initial_state': {'type': 'string', 'enum': ['running', 'stopped']},
'forcebuy_enable': {'type': 'boolean'},

375
freqtrade/rpc/api_server.py Normal file
View File

@ -0,0 +1,375 @@
import logging
import threading
from datetime import date, datetime
from ipaddress import IPv4Address
from typing import Dict
from arrow import Arrow
from flask import Flask, jsonify, request
from flask.json import JSONEncoder
from werkzeug.serving import make_server
from freqtrade.__init__ import __version__
from freqtrade.rpc.rpc import RPC, RPCException
logger = logging.getLogger(__name__)
BASE_URI = "/api/v1"
class ArrowJSONEncoder(JSONEncoder):
def default(self, obj):
try:
if isinstance(obj, Arrow):
return obj.for_json()
elif isinstance(obj, date):
return obj.strftime("%Y-%m-%d")
elif isinstance(obj, datetime):
return obj.strftime("%Y-%m-%d %H:%M:%S")
iterable = iter(obj)
except TypeError:
pass
else:
return list(iterable)
return JSONEncoder.default(self, obj)
class ApiServer(RPC):
"""
This class runs api server and provides rpc.rpc functionality to it
This class starts a none blocking thread the api server runs within
"""
def rpc_catch_errors(func):
def func_wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except RPCException as e:
logger.exception("API Error calling %s: %s", func.__name__, e)
return self.rest_error(f"Error querying {func.__name__}: {e}")
return func_wrapper
def check_auth(self, username, password):
return (username == self._config['api_server'].get('username') and
password == self._config['api_server'].get('password'))
def require_login(func):
def func_wrapper(self, *args, **kwargs):
auth = request.authorization
if auth and self.check_auth(auth.username, auth.password):
return func(self, *args, **kwargs)
else:
return jsonify({"error": "Unauthorized"}), 401
return func_wrapper
def __init__(self, freqtrade) -> None:
"""
Init the api server, and init the super class RPC
:param freqtrade: Instance of a freqtrade bot
:return: None
"""
super().__init__(freqtrade)
self._config = freqtrade.config
self.app = Flask(__name__)
self.app.json_encoder = ArrowJSONEncoder
# Register application handling
self.register_rest_rpc_urls()
thread = threading.Thread(target=self.run, daemon=True)
thread.start()
def cleanup(self) -> None:
logger.info("Stopping API Server")
self.srv.shutdown()
def run(self):
"""
Method that runs flask app in its own thread forever.
Section to handle configuration and running of the Rest server
also to check and warn if not bound to a loopback, warn on security risk.
"""
rest_ip = self._config['api_server']['listen_ip_address']
rest_port = self._config['api_server']['listen_port']
logger.info(f'Starting HTTP Server at {rest_ip}:{rest_port}')
if not IPv4Address(rest_ip).is_loopback:
logger.warning("SECURITY WARNING - Local Rest Server listening to external connections")
logger.warning("SECURITY WARNING - This is insecure please set to your loopback,"
"e.g 127.0.0.1 in config.json")
if not self._config['api_server'].get('password'):
logger.warning("SECURITY WARNING - No password for local REST Server defined. "
"Please make sure that this is intentional!")
# Run the Server
logger.info('Starting Local Rest Server.')
try:
self.srv = make_server(rest_ip, rest_port, self.app)
self.srv.serve_forever()
except Exception:
logger.exception("Api server failed to start.")
logger.info('Local Rest Server started.')
def send_msg(self, msg: Dict[str, str]) -> None:
"""
We don't push to endpoints at the moment.
Take a look at webhooks for that functionality.
"""
pass
def rest_dump(self, return_value):
""" Helper function to jsonify object for a webserver """
return jsonify(return_value)
def rest_error(self, error_msg):
return jsonify({"error": error_msg}), 502
def register_rest_rpc_urls(self):
"""
Registers flask app URLs that are calls to functonality in rpc.rpc.
First two arguments passed are /URL and 'Label'
Label can be used as a shortcut when refactoring
:return:
"""
self.app.register_error_handler(404, self.page_not_found)
# Actions to control the bot
self.app.add_url_rule(f'{BASE_URI}/start', 'start',
view_func=self._start, methods=['POST'])
self.app.add_url_rule(f'{BASE_URI}/stop', 'stop', view_func=self._stop, methods=['POST'])
self.app.add_url_rule(f'{BASE_URI}/stopbuy', 'stopbuy',
view_func=self._stopbuy, methods=['POST'])
self.app.add_url_rule(f'{BASE_URI}/reload_conf', 'reload_conf',
view_func=self._reload_conf, methods=['POST'])
# Info commands
self.app.add_url_rule(f'{BASE_URI}/balance', 'balance',
view_func=self._balance, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/count', 'count', view_func=self._count, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/daily', 'daily', view_func=self._daily, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/edge', 'edge', view_func=self._edge, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/profit', 'profit',
view_func=self._profit, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/performance', 'performance',
view_func=self._performance, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/status', 'status',
view_func=self._status, methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/version', 'version',
view_func=self._version, methods=['GET'])
# Combined actions and infos
self.app.add_url_rule(f'{BASE_URI}/blacklist', 'blacklist', view_func=self._blacklist,
methods=['GET', 'POST'])
self.app.add_url_rule(f'{BASE_URI}/whitelist', 'whitelist', view_func=self._whitelist,
methods=['GET'])
self.app.add_url_rule(f'{BASE_URI}/forcebuy', 'forcebuy',
view_func=self._forcebuy, methods=['POST'])
self.app.add_url_rule(f'{BASE_URI}/forcesell', 'forcesell', view_func=self._forcesell,
methods=['POST'])
# TODO: Implement the following
# help (?)
@require_login
def page_not_found(self, error):
"""
Return "404 not found", 404.
"""
return self.rest_dump({
'status': 'error',
'reason': f"There's no API call for {request.base_url}.",
'code': 404
}), 404
@require_login
@rpc_catch_errors
def _start(self):
"""
Handler for /start.
Starts TradeThread in bot if stopped.
"""
msg = self._rpc_start()
return self.rest_dump(msg)
@require_login
@rpc_catch_errors
def _stop(self):
"""
Handler for /stop.
Stops TradeThread in bot if running
"""
msg = self._rpc_stop()
return self.rest_dump(msg)
@require_login
@rpc_catch_errors
def _stopbuy(self):
"""
Handler for /stopbuy.
Sets max_open_trades to 0 and gracefully sells all open trades
"""
msg = self._rpc_stopbuy()
return self.rest_dump(msg)
@require_login
@rpc_catch_errors
def _version(self):
"""
Prints the bot's version
"""
return self.rest_dump({"version": __version__})
@require_login
@rpc_catch_errors
def _reload_conf(self):
"""
Handler for /reload_conf.
Triggers a config file reload
"""
msg = self._rpc_reload_conf()
return self.rest_dump(msg)
@require_login
@rpc_catch_errors
def _count(self):
"""
Handler for /count.
Returns the number of trades running
"""
msg = self._rpc_count()
return self.rest_dump(msg)
@require_login
@rpc_catch_errors
def _daily(self):
"""
Returns the last X days trading stats summary.
:return: stats
"""
timescale = request.args.get('timescale', 7)
timescale = int(timescale)
stats = self._rpc_daily_profit(timescale,
self._config['stake_currency'],
self._config['fiat_display_currency']
)
return self.rest_dump(stats)
@require_login
@rpc_catch_errors
def _edge(self):
"""
Returns information related to Edge.
:return: edge stats
"""
stats = self._rpc_edge()
return self.rest_dump(stats)
@require_login
@rpc_catch_errors
def _profit(self):
"""
Handler for /profit.
Returns a cumulative profit statistics
:return: stats
"""
logger.info("LocalRPC - Profit Command Called")
stats = self._rpc_trade_statistics(self._config['stake_currency'],
self._config['fiat_display_currency']
)
return self.rest_dump(stats)
@require_login
@rpc_catch_errors
def _performance(self):
"""
Handler for /performance.
Returns a cumulative performance statistics
:return: stats
"""
logger.info("LocalRPC - performance Command Called")
stats = self._rpc_performance()
return self.rest_dump(stats)
@require_login
@rpc_catch_errors
def _status(self):
"""
Handler for /status.
Returns the current status of the trades in json format
"""
results = self._rpc_trade_status()
return self.rest_dump(results)
@require_login
@rpc_catch_errors
def _balance(self):
"""
Handler for /balance.
Returns the current status of the trades in json format
"""
results = self._rpc_balance(self._config.get('fiat_display_currency', ''))
return self.rest_dump(results)
@require_login
@rpc_catch_errors
def _whitelist(self):
"""
Handler for /whitelist.
"""
results = self._rpc_whitelist()
return self.rest_dump(results)
@require_login
@rpc_catch_errors
def _blacklist(self):
"""
Handler for /blacklist.
"""
add = request.json.get("blacklist", None) if request.method == 'POST' else None
results = self._rpc_blacklist(add)
return self.rest_dump(results)
@require_login
@rpc_catch_errors
def _forcebuy(self):
"""
Handler for /forcebuy.
"""
asset = request.json.get("pair")
price = request.json.get("price", None)
trade = self._rpc_forcebuy(asset, price)
if trade:
return self.rest_dump(trade.to_json())
else:
return self.rest_dump({"status": f"Error buying pair {asset}."})
@require_login
@rpc_catch_errors
def _forcesell(self):
"""
Handler for /forcesell.
"""
tradeid = request.json.get("tradeid")
results = self._rpc_forcesell(tradeid)
return self.rest_dump(results)

View File

@ -48,6 +48,11 @@ class RPCException(Exception):
def __str__(self):
return self.message
def __json__(self):
return {
'msg': self.message
}
class RPC(object):
"""
@ -465,7 +470,7 @@ class RPC(object):
}
return res
def _rpc_blacklist(self, add: List[str]) -> Dict:
def _rpc_blacklist(self, add: List[str] = None) -> Dict:
""" Returns the currently active blacklist"""
if add:
stake_currency = self._freqtrade.config.get('stake_currency')

View File

@ -29,6 +29,12 @@ class RPCManager(object):
from freqtrade.rpc.webhook import Webhook
self.registered_modules.append(Webhook(freqtrade))
# Enable local rest api server for cmd line control
if freqtrade.config.get('api_server', {}).get('enabled', False):
logger.info('Enabling rpc.api_server')
from freqtrade.rpc.api_server import ApiServer
self.registered_modules.append(ApiServer(freqtrade))
def cleanup(self) -> None:
""" Stops all enabled rpc modules """
logger.info('Cleaning up rpc modules ...')

View File

@ -10,7 +10,7 @@ import arrow
import pytest
from telegram import Chat, Message, Update
from freqtrade import constants
from freqtrade import constants, persistence
from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.edge import Edge, PairInfo
from freqtrade.exchange import Exchange
@ -96,7 +96,7 @@ def patch_freqtradebot(mocker, config) -> None:
:return: None
"""
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
persistence.init(config)
patch_exchange(mocker, None)
mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
@ -112,6 +112,16 @@ def get_patched_worker(mocker, config) -> Worker:
return Worker(args=None, config=config)
def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None:
"""
:param mocker: mocker to patch IStrategy class
:param value: which value IStrategy.get_signal() must return
:return: None
"""
freqtrade.strategy.get_signal = lambda e, s, t: value
freqtrade.exchange.refresh_latest_ohlcv = lambda p: None
@pytest.fixture(autouse=True)
def patch_coinmarketcap(mocker) -> None:
"""
@ -961,3 +971,39 @@ def edge_conf(default_conf):
}
return default_conf
@pytest.fixture
def rpc_balance():
return {
'BTC': {
'total': 12.0,
'free': 12.0,
'used': 0.0
},
'ETH': {
'total': 0.0,
'free': 0.0,
'used': 0.0
},
'USDT': {
'total': 10000.0,
'free': 10000.0,
'used': 0.0
},
'LTC': {
'total': 10.0,
'free': 10.0,
'used': 0.0
},
'XRP': {
'total': 1.0,
'free': 1.0,
'used': 0.0
},
'EUR': {
'total': 10.0,
'free': 10.0,
'used': 0.0
},
}

View File

@ -14,8 +14,7 @@ from freqtrade.persistence import Trade
from freqtrade.rpc import RPC, RPCException
from freqtrade.rpc.fiat_convert import CryptoToFiatConverter
from freqtrade.state import State
from freqtrade.tests.conftest import patch_exchange
from freqtrade.tests.test_freqtradebot import patch_get_signal
from freqtrade.tests.conftest import patch_exchange, patch_get_signal
# Functions for recurrent object patching

View File

@ -0,0 +1,556 @@
"""
Unit test file for rpc/api_server.py
"""
from datetime import datetime
from unittest.mock import ANY, MagicMock, PropertyMock
import pytest
from flask import Flask
from requests.auth import _basic_auth_str
from freqtrade.__init__ import __version__
from freqtrade.persistence import Trade
from freqtrade.rpc.api_server import BASE_URI, ApiServer
from freqtrade.state import State
from freqtrade.tests.conftest import (get_patched_freqtradebot, log_has,
patch_get_signal)
_TEST_USER = "FreqTrader"
_TEST_PASS = "SuperSecurePassword1!"
@pytest.fixture
def botclient(default_conf, mocker):
default_conf.update({"api_server": {"enabled": True,
"listen_ip_address": "127.0.0.1",
"listen_port": "8080",
"username": _TEST_USER,
"password": _TEST_PASS,
}})
ftbot = get_patched_freqtradebot(mocker, default_conf)
mocker.patch('freqtrade.rpc.api_server.ApiServer.run', MagicMock())
apiserver = ApiServer(ftbot)
yield ftbot, apiserver.app.test_client()
# Cleanup ... ?
def client_post(client, url, data={}):
return client.post(url,
content_type="application/json",
data=data,
headers={'Authorization': _basic_auth_str(_TEST_USER, _TEST_PASS)})
def client_get(client, url):
return client.get(url, headers={'Authorization': _basic_auth_str(_TEST_USER, _TEST_PASS)})
def assert_response(response, expected_code=200):
assert response.status_code == expected_code
assert response.content_type == "application/json"
def test_api_not_found(botclient):
ftbot, client = botclient
rc = client_post(client, f"{BASE_URI}/invalid_url")
assert_response(rc, 404)
assert rc.json == {"status": "error",
"reason": f"There's no API call for http://localhost{BASE_URI}/invalid_url.",
"code": 404
}
def test_api_unauthorized(botclient):
ftbot, client = botclient
# Don't send user/pass information
rc = client.get(f"{BASE_URI}/version")
assert_response(rc, 401)
assert rc.json == {'error': 'Unauthorized'}
# Change only username
ftbot.config['api_server']['username'] = "Ftrader"
rc = client_get(client, f"{BASE_URI}/version")
assert_response(rc, 401)
assert rc.json == {'error': 'Unauthorized'}
# Change only password
ftbot.config['api_server']['username'] = _TEST_USER
ftbot.config['api_server']['password'] = "WrongPassword"
rc = client_get(client, f"{BASE_URI}/version")
assert_response(rc, 401)
assert rc.json == {'error': 'Unauthorized'}
ftbot.config['api_server']['username'] = "Ftrader"
ftbot.config['api_server']['password'] = "WrongPassword"
rc = client_get(client, f"{BASE_URI}/version")
assert_response(rc, 401)
assert rc.json == {'error': 'Unauthorized'}
def test_api_stop_workflow(botclient):
ftbot, client = botclient
assert ftbot.state == State.RUNNING
rc = client_post(client, f"{BASE_URI}/stop")
assert_response(rc)
assert rc.json == {'status': 'stopping trader ...'}
assert ftbot.state == State.STOPPED
# Stop bot again
rc = client_post(client, f"{BASE_URI}/stop")
assert_response(rc)
assert rc.json == {'status': 'already stopped'}
# Start bot
rc = client_post(client, f"{BASE_URI}/start")
assert_response(rc)
assert rc.json == {'status': 'starting trader ...'}
assert ftbot.state == State.RUNNING
# Call start again
rc = client_post(client, f"{BASE_URI}/start")
assert_response(rc)
assert rc.json == {'status': 'already running'}
def test_api__init__(default_conf, mocker):
"""
Test __init__() method
"""
mocker.patch('freqtrade.rpc.telegram.Updater', MagicMock())
mocker.patch('freqtrade.rpc.api_server.ApiServer.run', MagicMock())
apiserver = ApiServer(get_patched_freqtradebot(mocker, default_conf))
assert apiserver._config == default_conf
def test_api_run(default_conf, mocker, caplog):
default_conf.update({"api_server": {"enabled": True,
"listen_ip_address": "127.0.0.1",
"listen_port": "8080"}})
mocker.patch('freqtrade.rpc.telegram.Updater', MagicMock())
mocker.patch('freqtrade.rpc.api_server.threading.Thread', MagicMock())
server_mock = MagicMock()
mocker.patch('freqtrade.rpc.api_server.make_server', server_mock)
apiserver = ApiServer(get_patched_freqtradebot(mocker, default_conf))
assert apiserver._config == default_conf
apiserver.run()
assert server_mock.call_count == 1
assert server_mock.call_args_list[0][0][0] == "127.0.0.1"
assert server_mock.call_args_list[0][0][1] == "8080"
assert isinstance(server_mock.call_args_list[0][0][2], Flask)
assert hasattr(apiserver, "srv")
assert log_has("Starting HTTP Server at 127.0.0.1:8080", caplog.record_tuples)
assert log_has("Starting Local Rest Server.", caplog.record_tuples)
# Test binding to public
caplog.clear()
server_mock.reset_mock()
apiserver._config.update({"api_server": {"enabled": True,
"listen_ip_address": "0.0.0.0",
"listen_port": "8089",
"password": "",
}})
apiserver.run()
assert server_mock.call_count == 1
assert server_mock.call_args_list[0][0][0] == "0.0.0.0"
assert server_mock.call_args_list[0][0][1] == "8089"
assert isinstance(server_mock.call_args_list[0][0][2], Flask)
assert log_has("Starting HTTP Server at 0.0.0.0:8089", caplog.record_tuples)
assert log_has("Starting Local Rest Server.", caplog.record_tuples)
assert log_has("SECURITY WARNING - Local Rest Server listening to external connections",
caplog.record_tuples)
assert log_has("SECURITY WARNING - This is insecure please set to your loopback,"
"e.g 127.0.0.1 in config.json",
caplog.record_tuples)
assert log_has("SECURITY WARNING - No password for local REST Server defined. "
"Please make sure that this is intentional!",
caplog.record_tuples)
# Test crashing flask
caplog.clear()
mocker.patch('freqtrade.rpc.api_server.make_server', MagicMock(side_effect=Exception))
apiserver.run()
assert log_has("Api server failed to start.", caplog.record_tuples)
def test_api_cleanup(default_conf, mocker, caplog):
default_conf.update({"api_server": {"enabled": True,
"listen_ip_address": "127.0.0.1",
"listen_port": "8080"}})
mocker.patch('freqtrade.rpc.telegram.Updater', MagicMock())
mocker.patch('freqtrade.rpc.api_server.threading.Thread', MagicMock())
mocker.patch('freqtrade.rpc.api_server.make_server', MagicMock())
apiserver = ApiServer(get_patched_freqtradebot(mocker, default_conf))
apiserver.run()
stop_mock = MagicMock()
stop_mock.shutdown = MagicMock()
apiserver.srv = stop_mock
apiserver.cleanup()
assert stop_mock.shutdown.call_count == 1
assert log_has("Stopping API Server", caplog.record_tuples)
def test_api_reloadconf(botclient):
ftbot, client = botclient
rc = client_post(client, f"{BASE_URI}/reload_conf")
assert_response(rc)
assert rc.json == {'status': 'reloading config ...'}
assert ftbot.state == State.RELOAD_CONF
def test_api_stopbuy(botclient):
ftbot, client = botclient
assert ftbot.config['max_open_trades'] != 0
rc = client_post(client, f"{BASE_URI}/stopbuy")
assert_response(rc)
assert rc.json == {'status': 'No more buy will occur from now. Run /reload_conf to reset.'}
assert ftbot.config['max_open_trades'] == 0
def test_api_balance(botclient, mocker, rpc_balance):
ftbot, client = botclient
def mock_ticker(symbol, refresh):
if symbol == 'BTC/USDT':
return {
'bid': 10000.00,
'ask': 10000.00,
'last': 10000.00,
}
elif symbol == 'XRP/BTC':
return {
'bid': 0.00001,
'ask': 0.00001,
'last': 0.00001,
}
return {
'bid': 0.1,
'ask': 0.1,
'last': 0.1,
}
mocker.patch('freqtrade.exchange.Exchange.get_balances', return_value=rpc_balance)
mocker.patch('freqtrade.exchange.Exchange.get_ticker', side_effect=mock_ticker)
rc = client_get(client, f"{BASE_URI}/balance")
assert_response(rc)
assert "currencies" in rc.json
assert len(rc.json["currencies"]) == 5
assert rc.json['currencies'][0] == {
'currency': 'BTC',
'available': 12.0,
'balance': 12.0,
'pending': 0.0,
'est_btc': 12.0,
}
def test_api_count(botclient, mocker, ticker, fee, markets):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
rc = client_get(client, f"{BASE_URI}/count")
assert_response(rc)
assert rc.json["current"] == 0
assert rc.json["max"] == 1.0
# Create some test data
ftbot.create_trade()
rc = client_get(client, f"{BASE_URI}/count")
assert_response(rc)
assert rc.json["current"] == 1.0
assert rc.json["max"] == 1.0
def test_api_daily(botclient, mocker, ticker, fee, markets):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
rc = client_get(client, f"{BASE_URI}/daily")
assert_response(rc)
assert len(rc.json) == 7
assert rc.json[0][0] == str(datetime.utcnow().date())
def test_api_edge_disabled(botclient, mocker, ticker, fee, markets):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
rc = client_get(client, f"{BASE_URI}/edge")
assert_response(rc, 502)
assert rc.json == {"error": "Error querying _edge: Edge is not enabled."}
def test_api_profit(botclient, mocker, ticker, fee, markets, limit_buy_order, limit_sell_order):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
rc = client_get(client, f"{BASE_URI}/profit")
assert_response(rc, 502)
assert len(rc.json) == 1
assert rc.json == {"error": "Error querying _profit: no closed trade"}
ftbot.create_trade()
trade = Trade.query.first()
# Simulate fulfilled LIMIT_BUY order for trade
trade.update(limit_buy_order)
rc = client_get(client, f"{BASE_URI}/profit")
assert_response(rc, 502)
assert rc.json == {"error": "Error querying _profit: no closed trade"}
trade.update(limit_sell_order)
trade.close_date = datetime.utcnow()
trade.is_open = False
rc = client_get(client, f"{BASE_URI}/profit")
assert_response(rc)
assert rc.json == {'avg_duration': '0:00:00',
'best_pair': 'ETH/BTC',
'best_rate': 6.2,
'first_trade_date': 'just now',
'latest_trade_date': 'just now',
'profit_all_coin': 6.217e-05,
'profit_all_fiat': 0,
'profit_all_percent': 6.2,
'profit_closed_coin': 6.217e-05,
'profit_closed_fiat': 0,
'profit_closed_percent': 6.2,
'trade_count': 1
}
def test_api_performance(botclient, mocker, ticker, fee):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
trade = Trade(
pair='LTC/ETH',
amount=1,
exchange='binance',
stake_amount=1,
open_rate=0.245441,
open_order_id="123456",
is_open=False,
fee_close=fee.return_value,
fee_open=fee.return_value,
close_rate=0.265441,
)
trade.close_profit = trade.calc_profit_percent()
Trade.session.add(trade)
trade = Trade(
pair='XRP/ETH',
amount=5,
stake_amount=1,
exchange='binance',
open_rate=0.412,
open_order_id="123456",
is_open=False,
fee_close=fee.return_value,
fee_open=fee.return_value,
close_rate=0.391
)
trade.close_profit = trade.calc_profit_percent()
Trade.session.add(trade)
Trade.session.flush()
rc = client_get(client, f"{BASE_URI}/performance")
assert_response(rc)
assert len(rc.json) == 2
assert rc.json == [{'count': 1, 'pair': 'LTC/ETH', 'profit': 7.61},
{'count': 1, 'pair': 'XRP/ETH', 'profit': -5.57}]
def test_api_status(botclient, mocker, ticker, fee, markets):
ftbot, client = botclient
patch_get_signal(ftbot, (True, False))
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
rc = client_get(client, f"{BASE_URI}/status")
assert_response(rc, 502)
assert rc.json == {'error': 'Error querying _status: no active trade'}
ftbot.create_trade()
rc = client_get(client, f"{BASE_URI}/status")
assert_response(rc)
assert len(rc.json) == 1
assert rc.json == [{'amount': 90.99181074,
'base_currency': 'BTC',
'close_date': None,
'close_date_hum': None,
'close_profit': None,
'close_rate': None,
'current_profit': -0.59,
'current_rate': 1.098e-05,
'initial_stop_loss': 0.0,
'initial_stop_loss_pct': None,
'open_date': ANY,
'open_date_hum': 'just now',
'open_order': '(limit buy rem=0.00000000)',
'open_rate': 1.099e-05,
'pair': 'ETH/BTC',
'stake_amount': 0.001,
'stop_loss': 0.0,
'stop_loss_pct': None,
'trade_id': 1}]
def test_api_version(botclient):
ftbot, client = botclient
rc = client_get(client, f"{BASE_URI}/version")
assert_response(rc)
assert rc.json == {"version": __version__}
def test_api_blacklist(botclient, mocker):
ftbot, client = botclient
rc = client_get(client, f"{BASE_URI}/blacklist")
assert_response(rc)
assert rc.json == {"blacklist": ["DOGE/BTC", "HOT/BTC"],
"length": 2,
"method": "StaticPairList"}
# Add ETH/BTC to blacklist
rc = client_post(client, f"{BASE_URI}/blacklist",
data='{"blacklist": ["ETH/BTC"]}')
assert_response(rc)
assert rc.json == {"blacklist": ["DOGE/BTC", "HOT/BTC", "ETH/BTC"],
"length": 3,
"method": "StaticPairList"}
def test_api_whitelist(botclient):
ftbot, client = botclient
rc = client_get(client, f"{BASE_URI}/whitelist")
assert_response(rc)
assert rc.json == {"whitelist": ['ETH/BTC', 'LTC/BTC', 'XRP/BTC', 'NEO/BTC'],
"length": 4,
"method": "StaticPairList"}
def test_api_forcebuy(botclient, mocker, fee):
ftbot, client = botclient
rc = client_post(client, f"{BASE_URI}/forcebuy",
data='{"pair": "ETH/BTC"}')
assert_response(rc, 502)
assert rc.json == {"error": "Error querying _forcebuy: Forcebuy not enabled."}
# enable forcebuy
ftbot.config["forcebuy_enable"] = True
fbuy_mock = MagicMock(return_value=None)
mocker.patch("freqtrade.rpc.RPC._rpc_forcebuy", fbuy_mock)
rc = client_post(client, f"{BASE_URI}/forcebuy",
data='{"pair": "ETH/BTC"}')
assert_response(rc)
assert rc.json == {"status": "Error buying pair ETH/BTC."}
# Test creating trae
fbuy_mock = MagicMock(return_value=Trade(
pair='ETH/ETH',
amount=1,
exchange='bittrex',
stake_amount=1,
open_rate=0.245441,
open_order_id="123456",
open_date=datetime.utcnow(),
is_open=False,
fee_close=fee.return_value,
fee_open=fee.return_value,
close_rate=0.265441,
))
mocker.patch("freqtrade.rpc.RPC._rpc_forcebuy", fbuy_mock)
rc = client_post(client, f"{BASE_URI}/forcebuy",
data='{"pair": "ETH/BTC"}')
assert_response(rc)
assert rc.json == {'amount': 1,
'close_date': None,
'close_date_hum': None,
'close_rate': 0.265441,
'initial_stop_loss': None,
'initial_stop_loss_pct': None,
'open_date': ANY,
'open_date_hum': 'just now',
'open_rate': 0.245441,
'pair': 'ETH/ETH',
'stake_amount': 1,
'stop_loss': None,
'stop_loss_pct': None,
'trade_id': None}
def test_api_forcesell(botclient, mocker, ticker, fee, markets):
ftbot, client = botclient
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value=ticker),
get_ticker=ticker,
get_fee=fee,
markets=PropertyMock(return_value=markets)
)
patch_get_signal(ftbot, (True, False))
rc = client_post(client, f"{BASE_URI}/forcesell",
data='{"tradeid": "1"}')
assert_response(rc, 502)
assert rc.json == {"error": "Error querying _forcesell: invalid argument"}
ftbot.create_trade()
rc = client_post(client, f"{BASE_URI}/forcesell",
data='{"tradeid": "1"}')
assert_response(rc)
assert rc.json == {'result': 'Created sell order for trade 1.'}

View File

@ -135,3 +135,32 @@ def test_startupmessages_telegram_enabled(mocker, default_conf, caplog) -> None:
rpc_manager.startup_messages(default_conf, freqtradebot.pairlists)
assert telegram_mock.call_count == 3
assert "Dry run is enabled." in telegram_mock.call_args_list[0][0][0]['status']
def test_init_apiserver_disabled(mocker, default_conf, caplog) -> None:
caplog.set_level(logging.DEBUG)
run_mock = MagicMock()
mocker.patch('freqtrade.rpc.api_server.ApiServer.run', run_mock)
default_conf['telegram']['enabled'] = False
rpc_manager = RPCManager(get_patched_freqtradebot(mocker, default_conf))
assert not log_has('Enabling rpc.api_server', caplog.record_tuples)
assert rpc_manager.registered_modules == []
assert run_mock.call_count == 0
def test_init_apiserver_enabled(mocker, default_conf, caplog) -> None:
caplog.set_level(logging.DEBUG)
run_mock = MagicMock()
mocker.patch('freqtrade.rpc.api_server.ApiServer.run', run_mock)
default_conf["telegram"]["enabled"] = False
default_conf["api_server"] = {"enabled": True,
"listen_ip_address": "127.0.0.1",
"listen_port": "8080"}
rpc_manager = RPCManager(get_patched_freqtradebot(mocker, default_conf))
assert log_has('Enabling rpc.api_server', caplog.record_tuples)
assert len(rpc_manager.registered_modules) == 1
assert 'apiserver' in [mod.name for mod in rpc_manager.registered_modules]
assert run_mock.call_count == 1

View File

@ -22,8 +22,7 @@ from freqtrade.rpc.telegram import Telegram, authorized_only
from freqtrade.state import State
from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_patched_freqtradebot, log_has,
patch_exchange)
from freqtrade.tests.test_freqtradebot import patch_get_signal
patch_exchange, patch_get_signal)
class DummyCls(Telegram):
@ -496,39 +495,7 @@ def test_profit_handle(default_conf, update, ticker, ticker_sell_up, fee,
assert '*Best Performing:* `ETH/BTC: 6.20%`' in msg_mock.call_args_list[-1][0][0]
def test_telegram_balance_handle(default_conf, update, mocker) -> None:
mock_balance = {
'BTC': {
'total': 12.0,
'free': 12.0,
'used': 0.0
},
'ETH': {
'total': 0.0,
'free': 0.0,
'used': 0.0
},
'USDT': {
'total': 10000.0,
'free': 10000.0,
'used': 0.0
},
'LTC': {
'total': 10.0,
'free': 10.0,
'used': 0.0
},
'XRP': {
'total': 1.0,
'free': 1.0,
'used': 0.0
},
'EUR': {
'total': 10.0,
'free': 10.0,
'used': 0.0
}
}
def test_telegram_balance_handle(default_conf, update, mocker, rpc_balance) -> None:
def mock_ticker(symbol, refresh):
if symbol == 'BTC/USDT':
@ -549,7 +516,7 @@ def test_telegram_balance_handle(default_conf, update, mocker) -> None:
'last': 0.1,
}
mocker.patch('freqtrade.exchange.Exchange.get_balances', return_value=mock_balance)
mocker.patch('freqtrade.exchange.Exchange.get_balances', return_value=rpc_balance)
mocker.patch('freqtrade.exchange.Exchange.get_ticker', side_effect=mock_ticker)
msg_mock = MagicMock()

View File

@ -11,8 +11,8 @@ import arrow
import pytest
import requests
from freqtrade import (DependencyException, OperationalException,
TemporaryError, InvalidOrderException, constants)
from freqtrade import (DependencyException, InvalidOrderException,
OperationalException, TemporaryError, constants)
from freqtrade.data.dataprovider import DataProvider
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.persistence import Trade
@ -20,7 +20,8 @@ from freqtrade.rpc import RPCMessageType
from freqtrade.state import State
from freqtrade.strategy.interface import SellCheckTuple, SellType
from freqtrade.tests.conftest import (log_has, log_has_re, patch_edge,
patch_exchange, patch_wallet)
patch_exchange, patch_get_signal,
patch_wallet)
from freqtrade.worker import Worker
@ -59,16 +60,6 @@ def get_patched_worker(mocker, config) -> Worker:
return Worker(args=None, config=config)
def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None:
"""
:param mocker: mocker to patch IStrategy class
:param value: which value IStrategy.get_signal() must return
:return: None
"""
freqtrade.strategy.get_signal = lambda e, s, t: value
freqtrade.exchange.refresh_latest_ohlcv = lambda p: None
def patch_RPCManager(mocker) -> MagicMock:
"""
This function mock RPC manager to avoid repeating this code in almost every tests

View File

@ -10,6 +10,7 @@ nav:
- Control the bot:
- Telegram: telegram-usage.md
- Web Hook: webhook-config.md
- REST API: rest-api.md
- Backtesting: backtesting.md
- Hyperopt: hyperopt.md
- Edge positioning: edge.md

View File

@ -27,3 +27,6 @@ python-rapidjson==0.7.1
# Notify systemd
sdnotify==0.3.2
# Api server
flask==1.0.2

264
scripts/rest_client.py Executable file
View File

@ -0,0 +1,264 @@
#!/usr/bin/env python3
"""
Simple command line client into RPC commands
Can be used as an alternate to Telegram
Should not import anything from freqtrade,
so it can be used as a standalone script.
"""
import argparse
import json
import logging
import inspect
from urllib.parse import urlencode, urlparse, urlunparse
from pathlib import Path
import requests
from requests.exceptions import ConnectionError
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
)
logger = logging.getLogger("ft_rest_client")
class FtRestClient():
def __init__(self, serverurl, username=None, password=None):
self._serverurl = serverurl
self._session = requests.Session()
self._session.auth = (username, password)
def _call(self, method, apipath, params: dict = None, data=None, files=None):
if str(method).upper() not in ('GET', 'POST', 'PUT', 'DELETE'):
raise ValueError('invalid method <{0}>'.format(method))
basepath = f"{self._serverurl}/api/v1/{apipath}"
hd = {"Accept": "application/json",
"Content-Type": "application/json"
}
# Split url
schema, netloc, path, par, query, fragment = urlparse(basepath)
# URLEncode query string
query = urlencode(params) if params else ""
# recombine url
url = urlunparse((schema, netloc, path, par, query, fragment))
try:
resp = self._session.request(method, url, headers=hd, data=json.dumps(data))
# return resp.text
return resp.json()
except ConnectionError:
logger.warning("Connection error")
def _get(self, apipath, params: dict = None):
return self._call("GET", apipath, params=params)
def _post(self, apipath, params: dict = None, data: dict = None):
return self._call("POST", apipath, params=params, data=data)
def start(self):
"""
Start the bot if it's in stopped state.
:returns: json object
"""
return self._post("start")
def stop(self):
"""
Stop the bot. Use start to restart
:returns: json object
"""
return self._post("stop")
def stopbuy(self):
"""
Stop buying (but handle sells gracefully).
use reload_conf to reset
:returns: json object
"""
return self._post("stopbuy")
def reload_conf(self):
"""
Reload configuration
:returns: json object
"""
return self._post("reload_conf")
def balance(self):
"""
Get the account balance
:returns: json object
"""
return self._get("balance")
def count(self):
"""
Returns the amount of open trades
:returns: json object
"""
return self._get("count")
def daily(self, days=None):
"""
Returns the amount of open trades
:returns: json object
"""
return self._get("daily", params={"timescale": days} if days else None)
def edge(self):
"""
Returns information about edge
:returns: json object
"""
return self._get("edge")
def profit(self):
"""
Returns the profit summary
:returns: json object
"""
return self._get("profit")
def performance(self):
"""
Returns the performance of the different coins
:returns: json object
"""
return self._get("performance")
def status(self):
"""
Get the status of open trades
:returns: json object
"""
return self._get("status")
def version(self):
"""
Returns the version of the bot
:returns: json object containing the version
"""
return self._get("version")
def whitelist(self):
"""
Show the current whitelist
:returns: json object
"""
return self._get("whitelist")
def blacklist(self, *args):
"""
Show the current blacklist
:param add: List of coins to add (example: "BNB/BTC")
:returns: json object
"""
if not args:
return self._get("blacklist")
else:
return self._post("blacklist", data={"blacklist": args})
def forcebuy(self, pair, price=None):
"""
Buy an asset
:param pair: Pair to buy (ETH/BTC)
:param price: Optional - price to buy
:returns: json object of the trade
"""
data = {"pair": pair,
"price": price
}
return self._post("forcebuy", data=data)
def forcesell(self, tradeid):
"""
Force-sell a trade
:param tradeid: Id of the trade (can be received via status command)
:returns: json object
"""
return self._post("forcesell", data={"tradeid": tradeid})
def add_arguments():
parser = argparse.ArgumentParser()
parser.add_argument("command",
help="Positional argument defining the command to execute.")
parser.add_argument('--show',
help='Show possible methods with this client',
dest='show',
action='store_true',
default=False
)
parser.add_argument('-c', '--config',
help='Specify configuration file (default: %(default)s). ',
dest='config',
type=str,
metavar='PATH',
default='config.json'
)
parser.add_argument("command_arguments",
help="Positional arguments for the parameters for [command]",
nargs="*",
default=[]
)
args = parser.parse_args()
return vars(args)
def load_config(configfile):
file = Path(configfile)
if file.is_file():
with file.open("r") as f:
config = json.load(f)
return config
return {}
def print_commands():
# Print dynamic help for the different commands using the commands doc-strings
client = FtRestClient(None)
print("Possible commands:")
for x, y in inspect.getmembers(client):
if not x.startswith('_'):
print(f"{x} {getattr(client, x).__doc__}")
def main(args):
if args.get("help"):
print_commands()
config = load_config(args["config"])
url = config.get("api_server", {}).get("server_url", "127.0.0.1")
port = config.get("api_server", {}).get("listen_port", "8080")
username = config.get("api_server", {}).get("username")
password = config.get("api_server", {}).get("password")
server_url = f"http://{url}:{port}"
client = FtRestClient(server_url, username, password)
m = [x for x, y in inspect.getmembers(client) if not x.startswith('_')]
command = args["command"]
if command not in m:
logger.error(f"Command {command} not defined")
print_commands()
return
print(getattr(client, command)(*args["command_arguments"]))
if __name__ == "__main__":
args = add_arguments()
main(args)