robcaulk
255eb71270
start tracking the current candle in FreqAI, add robustness to corr_df caching and inference timer, add test for cache corr_df
2022-11-02 19:32:22 +01:00
robcaulk
a2843165e1
fix leftovers from merge
2022-10-30 10:31:38 +01:00
robcaulk
d1a0874683
merge develop into feat/freqai-rl-dev
2022-10-30 10:13:03 +01:00
robcaulk
3b4402aaab
Merge branch 'move-disk-writing-to-ram' into add-metric-tracker
2022-10-15 13:52:14 +02:00
robcaulk
99dbba6cad
avoid reading from disk to instantiate large objects
2022-10-15 13:50:55 +02:00
robcaulk
d81eef0b70
add timestamps to each metric, use rapidjson
2022-10-15 13:23:01 +02:00
robcaulk
b236e362ba
Merge remote-tracking branch 'origin/develop' into add-metric-tracker
2022-10-14 19:00:49 +02:00
Matthias
eaae9c9e03
Update docstring format
2022-10-10 12:19:29 +00:00
robcaulk
76b33359a9
add an optional metric tracker to collect train timings, inference timings, and cpu load data
2022-10-09 20:22:42 +02:00
Emre
e337d4b78a
Reset dataframe index after slice
2022-10-07 20:00:05 +02:00
robcaulk
9c73411ac2
Merge remote-tracking branch 'origin/develop' into dev-merge-rl
2022-10-05 15:21:45 +02:00
robcaulk
292d72d593
automatically handle model_save_type for user
2022-10-03 18:42:20 +02:00
Robert Caulk
3585742b43
remove trailing whitespace
2022-10-03 17:28:45 +02:00
Robert Caulk
265795824b
make default type for close_price and date_pred np.nan
2022-10-03 11:58:22 +02:00
Robert Caulk
6ecd92de4a
Allow updating without changing identifier
2022-10-03 09:55:57 +02:00
robcaulk
b70f18f4c3
add close price and date to historic_predictions
2022-10-02 18:33:39 +02:00
robcaulk
048cb95bd6
Merge remote-tracking branch 'origin/develop' into dev-merge-rl
2022-10-01 17:48:47 +02:00
robcaulk
cd514cf15d
fix inlier metric in backtesting
2022-10-01 14:18:46 +02:00
robcaulk
48e89e68b9
fix typos
2022-09-25 20:22:19 +02:00
robcaulk
873d2a5069
no model save backtest, plot features backtest, ensure inlier plays nice, doc
2022-09-25 11:18:10 +02:00
paranoidandy
2cc00a1a2c
Allow use of --strategy-list with freqai, with warning ( #7455 )
...
* Allow use of --strategy-list with freqai, with warning
* ensure populate_any_indicators is identical for resused identifiers
* use pair instead of metadata["pair"]
Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-09-24 13:21:01 +02:00
robcaulk
9c361f4422
increase test coverage for RL and FreqAI
2022-09-23 18:04:43 +02:00
robcaulk
ea8e34e192
Merge branch 'develop' into dev-merge-rl
2022-09-22 19:46:50 +02:00
Robert Caulk
f9460c80c2
Merge pull request #7434 from freqtrade/improve-train-queue
...
improve train queue system in FreqAI
2022-09-19 10:55:53 +02:00
robcaulk
eaa43337d2
improve train queue system, ensure crash resilience in train queue.
2022-09-18 17:00:55 +02:00
Matthias
667853c504
Use Alias to type config objects
2022-09-18 13:20:55 +02:00
robcaulk
81417cb795
Merge branch 'develop' into dev-merge-rl
2022-09-14 22:49:11 +02:00
initrv
37dd22c89e
Fixed a bug that prevents clearing old models
...
Corrects the error of clearing old models when the model directory contains directories with names that do not match a regular expression
2022-09-14 03:40:13 +03:00
robcaulk
ec76214d02
backup historical predictions pickle and load the backup in case of corruption
2022-09-04 15:56:07 +02:00
robcaulk
69b3fcfd32
Merge branch 'develop' into dev-merge-rl
2022-09-04 11:23:25 +02:00
robcaulk
fcb5d1cb5a
remove debugging flag
2022-08-28 13:01:39 +02:00
robcaulk
dd628eb525
add tests for outlier detection and removal functions
2022-08-28 12:56:39 +02:00
robcaulk
3199eb453b
reduce code for base use-case, ensure multiproc inherits custom env, add ability to limit ram use.
2022-08-25 19:05:51 +02:00
robcaulk
c0cee5df07
add continual retraining feature, handly mypy typing reqs, improve docstrings
2022-08-24 13:00:55 +02:00
robcaulk
3eb897c2f8
reuse callback, allow user to acces all stable_baselines3 agents via config
2022-08-24 13:00:55 +02:00
robcaulk
16cec7dfbd
fix save/reload functionality for stablebaselines
2022-08-24 13:00:55 +02:00
robcaulk
bf7ceba958
set cpu threads in config
2022-08-24 13:00:55 +02:00
MukavaValkku
01232e9a1f
callback function and TDQN model added
2022-08-24 13:00:55 +02:00
robcaulk
05ed1b544f
Working base for reinforcement learning model
2022-08-24 13:00:40 +02:00
robcaulk
96d8882f1e
Plug mem leak, add training timer
2022-08-22 13:30:30 +02:00
robcaulk
8961b8d560
merge in inference timer and historic predictions handling improvements.
2022-08-14 20:31:15 +02:00
robcaulk
ad846cdb76
fix lock bug, update docstring
2022-08-14 20:24:29 +02:00
robcaulk
58de20af0f
make BaseClassifierModel. Add predict_proba to lightgbm
2022-08-13 20:07:31 +02:00
robcaulk
31be707cc8
clean up code, add docstrings
2022-08-13 19:40:24 +02:00
robcaulk
3e38c1b0bd
take dynamic sized tail off historic_predictions as return dataframe to strategy.
2022-08-13 19:40:24 +02:00
robcaulk
b1b76a2dbe
debug classifier with predict proba
2022-08-13 19:40:24 +02:00
robcaulk
23cc21ce59
add predict_proba to base classifier, improve historic predictions handling
2022-08-13 19:40:24 +02:00
robcaulk
e7de812948
Allow user to user pair_dict for persistent storage of custom data
2022-08-09 16:03:10 +02:00
robcaulk
aef086b02e
Improved dict typing, timeframe parser, collect dates associated with training data points
2022-08-09 15:30:25 +02:00
robcaulk
ea64f43e52
bring back auto DF resizing for okx
2022-08-08 01:13:13 +02:00
Matthias
47a30047eb
Fix typo
2022-08-06 15:03:10 +02:00
Robert Caulk
c172ce1011
improve flexibility of user defined prediction dataframe
2022-08-06 13:51:19 +02:00
Robert Caulk
07763d0d4f
add classifier, improve model naming scheme
2022-08-06 08:33:55 +02:00
Robert Caulk
ce8fbbf743
ensure loading historical df matches frequi indices
2022-08-06 07:25:59 +02:00
Robert Caulk
a3799c4d5d
start frequi with historical data if available
2022-08-05 18:27:47 +02:00
Robert Caulk
51a6b4289f
improve DBSCAN performance for subsequent trainings
2022-08-04 17:41:58 +02:00
robcaulk
95d3009a95
give user ability to analyze live trade dataframe inside custom prediction model. Add documentation to explain new functionality
2022-08-02 20:14:02 +02:00
robcaulk
946d4c7cfc
fix trailing whitespace for flake8
2022-07-31 18:39:46 +02:00
robcaulk
4e68626bcb
ensure convolutional window is prepended for frequi consistency
2022-07-31 17:51:19 +02:00
robcaulk
c84d54b35e
Fix typing issue, avoid using .get() when unnecessary, convert to fstrings
2022-07-29 08:12:50 +02:00
Matthias
a2a0d35a24
Update missing typehints
2022-07-28 07:07:40 +02:00
Matthias
f31106dc61
Minor comment fixes
2022-07-27 07:27:24 +02:00
Matthias
bc760b7eb2
Simplify small segment in datadrawer
2022-07-26 19:41:49 +02:00
robcaulk
ad25a4cb56
reduce number of pair_dict lookups, remove coin_first
2022-07-26 15:58:40 +02:00
robcaulk
fb4e8430cd
isort auto import sorting
2022-07-26 10:51:39 +02:00
robcaulk
e213d0ad55
isolate data_drawer functions from data_kitchen, accommodate tests, add new test
2022-07-26 10:24:14 +02:00
Robert Caulk
c9d46a5237
finish bringing follow_mode up to date
2022-07-25 09:24:40 +02:00
Robert Caulk
ab587747fb
first fix for follower path bug
2022-07-24 23:32:24 +02:00
Matthias
70b7a254af
Update some areas to use default docstring formatting
2022-07-24 16:51:48 +02:00
robcaulk
50d630a155
remove unnecessary comments from data_drawer.py
2022-07-23 13:35:44 +02:00
robcaulk
c2d6a0e891
add record of contribution to doc and source
2022-07-23 13:04:06 +02:00
robcaulk
40f00196eb
use cloudpickle in place of pickle. define Paths once in data_drawer.
2022-07-22 17:37:51 +02:00
robcaulk
e7337728bf
add separator in folder name just incase an asset ends in an integer
2022-07-21 11:25:28 +02:00
Robert Caulk
921f3899f0
revert pickle reading for historic predictions
2022-07-17 16:06:36 +02:00
Robert Caulk
41eeb99177
load pickle file for writing
2022-07-17 10:05:21 +02:00
Robert Caulk
9d184586f1
fix bug in historic prediction saving
2022-07-16 21:16:59 +02:00
Wagner Costa Santos
ca2029a46b
fix issue when bot restarts with same identifier, does not load prediction history
2022-07-14 18:55:24 -03:00
Robert Caulk
fea63fba12
Fix saving/loading historic predictions
2022-07-12 10:12:50 +02:00
Robert Caulk
8ce6b18318
start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available.
2022-07-11 22:01:48 +02:00
robcaulk
ffb39a5029
black formatting on freqai files
2022-07-03 10:59:38 +02:00
robcaulk
106131ff0f
Rehaul organization of return values
2022-07-02 18:09:38 +02:00
robcaulk
93e1410ed9
first step toward cleaning output and enabling multimodel training per pair
2022-07-01 14:00:30 +02:00
robcaulk
051b99791d
reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel
2022-06-26 19:04:23 +02:00
robcaulk
f631ae911b
add model expiration feature, fix bug in DI return values
2022-06-17 14:55:40 +02:00
robcaulk
c5de0c49e4
first functional scanning commit
2022-06-16 00:24:18 +02:00
robcaulk
cab8f517b4
add lock to datadrawer
2022-06-07 01:07:30 +02:00
Robert Caulk
2451ed8c88
Quick bug fix
2022-06-06 15:11:54 -06:00
robcaulk
16b4a5b71f
rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.
2022-06-03 15:19:46 +02:00
robcaulk
0b0dd8dd80
Force high priority pair to be trained before anything else.
2022-06-01 07:55:05 +02:00
robcaulk
bac4ced382
Ensure follower predictions are persistent and uniquely stored
2022-05-31 14:35:04 +02:00
robcaulk
0306f5ca13
Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk
2022-05-31 11:58:21 +02:00
robcaulk
45f4f0f603
ensure follower sends back null arrays in case leader hasnt created a model file yet
2022-05-31 01:48:48 +02:00
robcaulk
606f18e5c1
Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader
2022-05-30 21:35:48 +02:00
robcaulk
5b4c649d43
detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly
2022-05-30 13:55:46 +02:00
robcaulk
e229902381
fix bug in previous commit
2022-05-30 12:48:22 +02:00
robcaulk
a20651efd8
Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).
2022-05-30 11:37:05 +02:00
robcaulk
83dd453723
catch errors occuring on background thread, and make sure to keep the ball rolling. Improve pair retraining queue.
2022-05-28 18:26:19 +02:00
robcaulk
58b5abbaa6
improve multithreaded training queue system
2022-05-24 15:28:38 +02:00
robcaulk
255d35976e
add priority metadata to pairs to avoid a sync of train time + train period
2022-05-24 12:58:53 +02:00
robcaulk
059c285425
paying closer attention to managing live retraining on separate thread without affecting prediction of other coins on master thread
2022-05-24 12:01:01 +02:00
