Commit Graph

715 Commits

Author SHA1 Message Date
robcaulk
387c905a86 merge develop into RL 2022-11-17 21:59:07 +01:00
robcaulk
60fcd8dce2 fix skipped mac test, fix RL bug in add_state_info, fix use of __import__, revise doc 2022-11-17 21:50:02 +01:00
robcaulk
91df79ff44 merge dev into backtest-live-predictions 2022-11-17 21:20:47 +01:00
Robert Caulk
cd6f87be17
Merge pull request #7728 from freqtrade/improve_timerange
Simplify timerange handling
2022-11-17 19:57:48 +01:00
Wagner Costa Santos
3903b04d3f save_live_data_backtest - added docs and tests 2022-11-17 15:20:07 -03:00
Wagner Costa Santos
99bff9cbfa backtesting_from_live_saved_files - code refactoring 2022-11-17 10:30:51 -03:00
Wagner Costa Santos
913749c81b backtesting_from_live_saved_files - code refactoring 2022-11-17 10:30:16 -03:00
Wagner Costa Santos
b01e4e3dbf change default value - save_live_data_backtest as false 2022-11-17 10:14:30 -03:00
Wagner Costa Santos
1a19d90e2e Merge branch 'develop' into backtest_fitlivepredictions 2022-11-17 10:13:11 -03:00
robcaulk
bf4d5b432a ensure model_type is defined 2022-11-13 18:50:25 +01:00
robcaulk
6394ef4558 fix docstrings 2022-11-13 17:43:52 +01:00
robcaulk
96fafb7f56 remove limit_ram_use 2022-11-13 17:14:47 +01:00
robcaulk
b421521be3 help default ReinforcementLearner users by assigning the model_type automatically 2022-11-13 17:12:17 +01:00
robcaulk
90f168d1ff remove more user references. cleanup dataprovider 2022-11-13 17:06:06 +01:00
robcaulk
388ca21200 update docs, fix bug in environment 2022-11-13 16:56:31 +01:00
robcaulk
af9e400562 add test coverage, fix bug in base environment. Ensure proper fee is used. 2022-11-13 15:31:37 +01:00
Matthias
942840da2d Improve setting wording to keep future possibilities open 2022-11-13 15:22:44 +01:00
robcaulk
81f800a79b switch to using FT calc_profi_pct, reverse entry/exit fees 2022-11-13 13:41:17 +01:00
robcaulk
e71a8b8ac1 add ability to integrate state info or not, and prevent state info integration during backtesting 2022-11-12 18:46:48 +01:00
Wagner Costa Santos
27fa9f1f4e backtest saved dataframe from live 2022-11-12 14:37:23 -03:00
Wagner Costa Santos
f9c6c538be Merge branch 'develop' into backtest_fitlivepredictions 2022-11-12 09:37:45 -03:00
robcaulk
9c6b97c678 ensure normalization acceleration methods are employed in RL 2022-11-12 12:01:59 +01:00
robcaulk
6746868ea7 store dataprovider to self instead of strategy 2022-11-12 11:33:03 +01:00
robcaulk
7a4bb040a5 merge develop into feat/freqai-rl-dev 2022-11-12 10:54:34 +01:00
robcaulk
214c622475 move dataframe converter to converter.py 2022-11-12 10:38:25 +01:00
robcaulk
9617d8143d Merge remote-tracking branch 'origin/develop' into add-single-precision-freqai 2022-11-12 10:21:38 +01:00
Matthias
e6172a68d7
Merge pull request #7730 from freqtrade/fix-metric-tracker
fix loading of metric tracker from disk
2022-11-11 20:01:26 +01:00
Robert Caulk
833578716c
Merge pull request #7644 from markdregan/multi-target-classifier
Support for multi target multi-class classification (FreqaiMultiOutputRegressor for Classification)
2022-11-11 18:48:38 +01:00
robcaulk
790ff2a84b merge develop into add-single-prec 2022-11-11 18:08:51 +01:00
robcaulk
e46a57bbd0 move mem logs to debug 2022-11-11 18:05:32 +01:00
robcaulk
66514e84e4 add LightGBMClassifierMultiTarget. add test 2022-11-11 17:45:53 +01:00
robcaulk
054133955b fix loading of metric tracker from disk 2022-11-11 17:24:09 +01:00
Matthias
0f9c5f8d41 Simplify timerange handling 2022-11-10 18:28:18 +01:00
Matthias
57313dd961 Update some usages of timerange to new, simplified method 2022-11-10 18:11:39 +01:00
Wagner Costa Santos
4f0f3e5b64 removed unnecessary code 2022-11-09 10:07:24 -03:00
Wagner Costa Santos
8ee95db927 refactoring backtesting_fit_live_predictions function 2022-11-09 09:51:42 -03:00
Wagner Costa Santos
3e57c18ac6 add fix_live_predictions function to backtesting 2022-11-08 18:20:39 -03:00
Wagner Costa Santos
8d9988a942 enforce date column in backtesting freqai predictions files 2022-11-08 11:06:23 -03:00
Wagner Costa Santos
9c5ba0732a save predictions with date and merge by date 2022-11-08 10:32:18 -03:00
Matthias
884014a4b9 Fix some minor typos 2022-11-07 18:35:28 +00:00
Wagner Costa Santos
6559384286 Merge branch 'develop' into backtest_live_models 2022-11-07 15:14:10 -03:00
Emre
43bdd34964
Optimize reduce_dataframe_footprint function 2022-11-05 19:13:02 +03:00
robcaulk
53df607067 avoid duplicating features with okx/gateio, ensure inference timer gets logged 2022-11-05 15:42:19 +01:00
robcaulk
257c833831 add doc for single precision, dont allow half precision, add test 2022-11-04 18:10:04 +01:00
robcaulk
3ccc120f92 add option to force single precision 2022-11-04 17:42:10 +01:00
robcaulk
8bdc99a3d6 fix self-imposed bug 2022-11-04 16:41:38 +01:00
robcaulk
19d90b813a improve readibility in start_backtesting() 2022-11-04 16:10:46 +01:00
Wagner Costa Santos
8008c63319 Merge branch 'develop' into backtest_live_models 2022-11-04 09:09:39 -03:00
Wagner Costa Santos
a7acfb7ab7 fix dict key error 2022-11-04 09:02:28 -03:00
robcaulk
90c5bfb4b5 add default conv_width 2022-11-03 21:35:12 +01:00
robcaulk
05b309caf2 ensure colon replaced for cached attach 2022-11-03 21:17:48 +01:00
robcaulk
6938ed6584 change default conv_width to 1 2022-11-03 21:11:26 +01:00
robcaulk
444a068481 merge develop into track-current-candle 2022-11-03 21:09:08 +01:00
robcaulk
db942321ad fix bug with lightgbm and colons 2022-11-03 21:03:48 +01:00
robcaulk
d721b50230 flake8 2022-11-03 19:13:24 +01:00
robcaulk
3ba1e221eb fix typo 2022-11-03 19:08:33 +01:00
robcaulk
6c4bdb8f67 remove special characters from feature names 2022-11-03 18:49:39 +01:00
Wagner Costa Santos
17798b3397 Merge branch 'develop' into backtest_live_models 2022-11-03 13:29:25 -03:00
Wagner Costa Santos
356d79b38a verify mean and std exists in model metadata 2022-11-03 13:27:56 -03:00
robcaulk
ce92731132 ensure backwards compatitibility 2022-11-02 20:20:35 +01:00
Wagner Costa Santos
23b6915dde fix issue with different backtesting prediction size 2022-11-02 15:49:51 -03:00
robcaulk
1a38c10fc6 remove old code 2022-11-02 19:37:47 +01:00
robcaulk
255eb71270 start tracking the current candle in FreqAI, add robustness to corr_df caching and inference timer, add test for cache corr_df 2022-11-02 19:32:22 +01:00
robcaulk
97df232ac6 add a warning to __init__ for get_corr_dataframes 2022-10-31 18:18:00 +01:00
robcaulk
66d8ed6c0b Merge remote-tracking branch 'origin/develop' into reduce-indicator-population 2022-10-31 09:42:01 +01:00
robcaulk
d59a7fa2f9 remove analysis_lock and realign example hybrid strat 2022-10-30 17:07:33 +01:00
Matthias
391c3f56f7 Add typehint to corr_pairlist 2022-10-30 13:28:01 +01:00
Mark Regan
7053f81fa8 simplified predict and predict_proba using super(). Added duplicate class label check. 2022-10-30 09:48:30 +00:00
Matthias
cf4af2175c
Merge pull request #7662 from markdregan/backtest_extra_returns_fix
Fix missing f-string from PR #7611
2022-10-30 10:47:06 +01:00
robcaulk
a2843165e1 fix leftovers from merge 2022-10-30 10:31:38 +01:00
robcaulk
52e15b2070 Merge remote-tracking branch 'origin/develop' into feat/freqai-rl-dev 2022-10-30 10:16:39 +01:00
robcaulk
d1a0874683 merge develop into feat/freqai-rl-dev 2022-10-30 10:13:03 +01:00
robcaulk
fc53054d43 leverage list length knowledge, f-string change 2022-10-30 10:12:14 +01:00
Emre
f98c7a2423 Remove loop of normalization from metadata 2022-10-30 10:12:14 +01:00
Mark Regan
c26fda282f fix missing f-string from PR #7611 2022-10-30 08:19:59 +00:00
robcaulk
650bb8b7d7 ensure full pair string is used for caching dataframes. If not, revert to old behavior. Update docs. 2022-10-29 22:26:49 +02:00
Matthias
801e91c39e
Merge pull request #7618 from wizrds/fix/docker-config-record
Update function in FreqAI interface to record FreqAI config params
2022-10-29 08:56:20 +02:00
Mark Regan
6ef82dd8b6 minor change to return 2022-10-27 12:41:12 +01:00
Timothy Pogue
b9bf9edb02 update rapidjson opts 2022-10-25 14:12:13 -06:00
Mark Regan
217add70bd add strat and config for testing on PR 2022-10-25 20:07:39 +01:00
Mark Regan
47056eded3 multi target classifier working but not for parallel 2022-10-25 18:24:27 +01:00
Matthias
1ef38f137d
Fix XGBoost regressor "used before assignment" 2022-10-25 13:37:04 +02:00
Timothy Pogue
51be45547f remove np object, make default str 2022-10-24 12:23:54 -06:00
Matthias
6669714a73 Update mal-formatted docstrings 2022-10-24 18:12:17 +02:00
Robert Caulk
137aa1756b
Merge pull request #7593 from th0rntwig/prediction-shape
Fix constant PCA
2022-10-24 08:33:36 +02:00
robcaulk
4d2b7a74f1 move record params to utils, use rapidjson 2022-10-23 20:51:32 +02:00
Timothy Pogue
07e813dfa0 change param record to only get certain params 2022-10-23 12:09:07 -06:00
th0rntwig
49ff51f11f Change storage loc and fix test fail 2022-10-23 16:24:02 +02:00
Timothy Pogue
4464e91256 use self.identifier in full path 2022-10-21 19:53:33 -06:00
Timothy Pogue
5ee3b8cbbb update config recording to use all configs, fix tests 2022-10-21 19:48:26 -06:00
Robert Caulk
a85826bf24
Merge pull request #7611 from markdregan/backtest_extra_returns
Make extra_returns_per_train data available during backtest
2022-10-21 17:13:22 +02:00
Wagner Costa Santos
6606a0113f refactoring - remove unnecessary config file 2022-10-20 14:53:25 -03:00
Wagner Costa Santos
589944055e fix typos - live predictions candles 2022-10-20 12:15:41 -03:00
Wagner Costa Santos
52b60c5cbb Merge branch 'develop' into backtest_live_models 2022-10-20 11:59:37 -03:00
rcaulk
a9db668082 avoid redundant indicator population for corr_pair list 2022-10-20 16:30:32 +02:00
Mark Regan
073ce1659e remove un-used f-string 2022-10-20 14:26:10 +01:00
Mark Regan
295ba21389 Make extra_returns_per_train values available during backtest 2022-10-20 12:05:37 +01:00
th0rntwig
033c5bd441 Make check constant pred labels agnostic 2022-10-18 12:55:47 +02:00
Matthias
880ddccaa8
Merge pull request #7590 from freqtrade/list-models
List models
2022-10-17 20:40:41 +02:00
Matthias
441032be25 Fix sys.stdout bug for CatboostRegressorMultiTarget 2022-10-17 19:48:27 +02:00
Matthias
6cb14148aa Fix random test failure due to catboost bug
https://github.com/catboost/catboost/issues/2195
2022-10-17 07:00:44 +02:00
Matthias
23a5a516f9
Merge pull request #7557 from freqtrade/add-metric-tracker
Add metric tracker to FreqAI
2022-10-16 18:20:07 +02:00
th0rntwig
20fc521771 Fix constant PCA 2022-10-15 23:30:12 +02:00
Robert Caulk
62ca822597
Merge pull request #7569 from Silur/develop
Add XGBoost random forest predictors to freqai
2022-10-15 16:09:26 +02:00
robcaulk
3b4402aaab Merge branch 'move-disk-writing-to-ram' into add-metric-tracker 2022-10-15 13:52:14 +02:00
robcaulk
99dbba6cad avoid reading from disk to instantiate large objects 2022-10-15 13:50:55 +02:00
robcaulk
d81eef0b70 add timestamps to each metric, use rapidjson 2022-10-15 13:23:01 +02:00
robcaulk
b236e362ba Merge remote-tracking branch 'origin/develop' into add-metric-tracker 2022-10-14 19:00:49 +02:00
Emre
7f05b44376
Add eval set to CatboostClassifier 2022-10-13 23:01:09 +03:00
Wagner Costa Santos
6919f3aa75 Backtest live models - fix utc date convert issue 2022-10-13 15:03:27 -03:00
Wagner Costa Santos
93fe2b6446 Merge branch 'develop' into backtest_live_models 2022-10-13 11:22:58 -03:00
robcaulk
1e31be562e remove whitespace 2022-10-11 21:05:42 +02:00
robcaulk
dba1b573bc remove tensorboard dir from other pred models 2022-10-11 19:49:24 +02:00
robcaulk
5b5bb8aab5 catboost tensorboard bugfix 2022-10-11 19:05:46 +02:00
Robert Caulk
2e34aa9f04
Merge pull request #7544 from th0rntwig/prediction-shape
Remove constant labels from prediction
2022-10-10 21:24:25 +02:00
Robert Caulk
7bcb7d9a1a
Merge pull request #7554 from initrv/add-catboost-tensorboard
Add tensorboard for catboost
2022-10-10 21:03:45 +02:00
Wagner Costa Santos
3081e73f8a Merge branch 'develop' into backtest_live_models 2022-10-10 14:53:45 -03:00
silur
2ad086dd7a add XGBoost random forest predictors to freqai 2022-10-10 14:38:43 +02:00
Matthias
eaae9c9e03 Update docstring format 2022-10-10 12:19:29 +00:00
Matthias
60de192d47 Update Classifier docstrings 2022-10-10 12:13:41 +00:00
robcaulk
a4aa1b972c isolate and standardize location of tensorboard files, add doc, ensure backtesting functionality 2022-10-09 21:11:37 +02:00
robcaulk
76b33359a9 add an optional metric tracker to collect train timings, inference timings, and cpu load data 2022-10-09 20:22:42 +02:00
th0rntwig
4daf0000c7 Move check and add log warning 2022-10-08 16:15:48 +02:00
robcaulk
8d7adfabe9 clean RL tests to avoid dir pollution and increase speed 2022-10-08 12:10:38 +02:00
Emre
e337d4b78a Reset dataframe index after slice 2022-10-07 20:00:05 +02:00
Matthias
d42fb15608 Improve generic exception handler 2022-10-07 16:05:41 +02:00
th0rntwig
a9d5e04a43 Remove constant labels from prediction 2022-10-06 19:26:33 +02:00
initrv
86c781798a Add сatboost train_dir for tensorboard 2022-10-06 19:59:35 +03:00
robcaulk
488739424d fix reward inconsistency in template 2022-10-05 20:55:50 +02:00
robcaulk
936ca24482 separate RL install from general FAI install, update docs 2022-10-05 15:58:54 +02:00
robcaulk
9c73411ac2 Merge remote-tracking branch 'origin/develop' into dev-merge-rl 2022-10-05 15:21:45 +02:00
robcaulk
292d72d593 automatically handle model_save_type for user 2022-10-03 18:42:20 +02:00
Robert Caulk
3585742b43
remove trailing whitespace 2022-10-03 17:28:45 +02:00
Robert Caulk
265795824b
make default type for close_price and date_pred np.nan 2022-10-03 11:58:22 +02:00
Robert Caulk
6ecd92de4a
Allow updating without changing identifier 2022-10-03 09:55:57 +02:00
robcaulk
b70f18f4c3 add close price and date to historic_predictions 2022-10-02 18:33:39 +02:00
robcaulk
cf882fa84e fix tests 2022-10-01 20:26:41 +02:00
robcaulk
048cb95bd6 Merge remote-tracking branch 'origin/develop' into dev-merge-rl 2022-10-01 17:48:47 +02:00
Robert Caulk
3e34f10e3d
Merge pull request #7508 from aemr3/fix-pca-errors
Fix feature list match for PCA
2022-10-01 16:50:29 +02:00
robcaulk
f4c6b99d63 remove commented lines 2022-10-01 14:23:15 +02:00
robcaulk
cd514cf15d fix inlier metric in backtesting 2022-10-01 14:18:46 +02:00
robcaulk
f2b875483f ensure raw features match when PCA is employed 2022-10-01 13:14:59 +02:00
robcaulk
51556e08c3 Merge branch 'develop' into pr/th0rntwig/7495 2022-10-01 12:45:08 +02:00
Emre
cdc01a0781
Fix feature list match for pca 2022-09-30 15:22:05 -07:00
Matthias
cc06c60fd8 Fix pandas deprecation warnings from freqAI 2022-09-30 15:43:23 +02:00
robcaulk
be48131185 make shuffle false in constants 2022-09-30 00:33:08 +02:00
Robert Caulk
6e74d46660
Ensure 1 thread available 2022-09-29 14:02:00 +02:00
Robert Caulk
7ef56e3029
Ensure at least 1 thread is available 2022-09-29 14:01:22 +02:00
Robert Caulk
555cc42630
Ensure 1 thread is available (for testing purposes) 2022-09-29 14:00:09 +02:00
Wagner Costa Santos
6845a5c6ea backtest_live_models - refactoring after PR review 2022-09-29 01:48:38 -03:00
Robert Caulk
dcf6ebe273
Update BaseReinforcementLearningModel.py 2022-09-29 00:37:03 +02:00
robcaulk
83343dc2f1 control number of threads, update doc 2022-09-29 00:10:18 +02:00
th0rntwig
683b084323 Set train-test-split shuffle=False as default and remove stratification 2022-09-28 18:23:56 +02:00
Wagner Costa Santos
55ebbeec18 backtest_live models tests refactoring 2022-09-28 08:48:32 -03:00
Timothy Pogue
099137adac remove hasattr calls 2022-09-27 22:35:15 -06:00
Timothy Pogue
9e36b0d2ea fix formatting 2022-09-27 22:02:33 -06:00
Timothy Pogue
caa47a2f47 close subproc env on shutdown 2022-09-28 03:06:05 +00:00
Wagner Costa Santos
0be115de9c backtest_live_models - added new tests and refactoring 2022-09-27 10:26:57 -03:00
Wagner Costa Santos
72aa47fc51 backtest_live_models - fix issue with timerange BT and 2 trainings within same candle (no data) 2022-09-27 00:14:12 -03:00
Wagner Costa Santos
14b96aaa38 backtesting live models - fix ci issues 2022-09-26 19:52:59 -03:00
Wagner Costa Santos
290afd9699 backtest_live_models - fix typo 2022-09-26 19:21:53 -03:00
Wagner Costa Santos
0318ca9f12 backtest_live_models - fix typo 2022-09-26 19:08:25 -03:00
Wagner Costa Santos
22bef71d5d backtest_live_models - add function comments and tests 2022-09-26 19:01:24 -03:00
Wagner Costa Santos
ec947ad65c remove commented code - backtest_live_models 2022-09-25 23:47:27 -03:00
Wagner Costa Santos
5880f7a638 backtest_live_models - params validation and get timerange from live models in BT 2022-09-25 23:14:00 -03:00
Wagner Costa Santos
f3f3917da3 Merge branch 'develop' into backtest_live_models 2022-09-25 20:05:26 -03:00
robcaulk
48e89e68b9 fix typos 2022-09-25 20:22:19 +02:00
Wagner Costa Santos
0ed7b2bfc3 change start_backtesting to handle backtest_live_models 2022-09-25 10:35:55 -03:00
robcaulk
873d2a5069 no model save backtest, plot features backtest, ensure inlier plays nice, doc 2022-09-25 11:18:10 +02:00
Wagner Costa Santos
7f116db95e added generic function to get timerange from existent models 2022-09-24 13:01:53 -03:00
Wagner Costa Santos
d9c16d4888 Merge branch 'develop' into backtest_live_models 2022-09-24 12:30:55 -03:00
Wagner Costa Santos
3ee7eb63f7 starting backtest live models 2022-09-24 12:28:52 -03:00
Matthias
585342f193
Merge pull request #7454 from freqtrade/fix_backtesting_dfsize_freqai
Ensure the DF has the same size in backtesting FreqAI
2022-09-24 16:17:39 +02:00
paranoidandy
2cc00a1a2c
Allow use of --strategy-list with freqai, with warning (#7455)
* Allow use of --strategy-list with freqai, with warning

* ensure populate_any_indicators is identical for resused identifiers

* use pair instead of metadata["pair"]

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-09-24 13:21:01 +02:00
Matthias
6643d90e64 simplify freqAI start_backtesting 2022-09-24 10:34:14 +02:00
robcaulk
647200e8a7 isort 2022-09-23 19:30:56 +02:00
robcaulk
77c360b264 improve typing, improve docstrings, ensure global tests pass 2022-09-23 19:17:27 +02:00
robcaulk
9c361f4422 increase test coverage for RL and FreqAI 2022-09-23 18:04:43 +02:00
Robert Caulk
f5cd8f62c6
Remove unused code from BaseEnv 2022-09-23 10:24:39 +02:00
Emre
a4eaff4da6
Add training elapsed time 2022-09-23 01:18:34 -07:00
robcaulk
7295ba0fb2 add test for Base4ActionEnv 2022-09-22 23:42:33 +02:00
robcaulk
eeebb78a5c skip darwin in RL tests, remove example scripts, improve doc 2022-09-22 21:16:21 +02:00
robcaulk
ea8e34e192 Merge branch 'develop' into dev-merge-rl 2022-09-22 19:46:50 +02:00
th0rntwig
71e6c54ea4
Normalise distances before Weibull fit (#7432)
* Normalise distances before Weibull

* Track inlier-metric params
2022-09-22 18:11:50 +02:00
Wagner Costa Santos
b1dbc3a65f remove function remove_training_from_backtesting and ensure BT period is correct with startup_candle_count 2022-09-22 12:13:51 -03:00
robcaulk
b5fd11f91b protect against unforeseen issues in scanning thread 2022-09-19 21:18:31 +02:00
robcaulk
42c75b4a7b combine log messages 2022-09-19 19:16:32 +02:00
Robert Caulk
9b66297cc0
Fix append 2022-09-19 12:47:20 +02:00
Robert Caulk
995396c775
Add useful log info 2022-09-19 11:42:56 +02:00
Robert Caulk
ad652817ef
Ensure train ordering after restart
Ensure lowest timestamps get trained first after restart
2022-09-19 11:11:23 +02:00
Robert Caulk
f9460c80c2
Merge pull request #7434 from freqtrade/improve-train-queue
improve train queue system in FreqAI
2022-09-19 10:55:53 +02:00
Matthias
ea58c29ded Add plot_feature_importance to schema definition 2022-09-19 08:13:54 +00:00
Matthias
225f7cd5f8
Merge pull request #7431 from initrv/add-plot-feature-importance
Add plot feature importance
2022-09-19 08:41:10 +02:00
Matthias
d930931000 Bring back sleep - it'll ensure we give up control over the thread. 2022-09-19 07:14:33 +02:00
robcaulk
470d5d8405 ensure full new pairlist is in the queue 2022-09-18 17:08:07 +02:00
robcaulk
eaa43337d2 improve train queue system, ensure crash resilience in train queue. 2022-09-18 17:00:55 +02:00
robcaulk
95457d23ca escape freqai-specific characters from file naming 2022-09-18 13:59:30 +02:00
Matthias
667853c504 Use Alias to type config objects 2022-09-18 13:20:55 +02:00
robcaulk
188f75d8ec set model in models dict 2022-09-18 12:50:44 +02:00
robcaulk
1ef875901a maintian user privacy by keeping plotly offline 2022-09-18 00:01:42 +02:00