Commit Graph

303 Commits

Author SHA1 Message Date
Wagner Costa Santos
93fe2b6446 Merge branch 'develop' into backtest_live_models 2022-10-13 11:22:58 -03:00
robcaulk
1e31be562e remove whitespace 2022-10-11 21:05:42 +02:00
robcaulk
5b5bb8aab5 catboost tensorboard bugfix 2022-10-11 19:05:46 +02:00
Robert Caulk
2e34aa9f04
Merge pull request #7544 from th0rntwig/prediction-shape
Remove constant labels from prediction
2022-10-10 21:24:25 +02:00
Robert Caulk
7bcb7d9a1a
Merge pull request #7554 from initrv/add-catboost-tensorboard
Add tensorboard for catboost
2022-10-10 21:03:45 +02:00
Wagner Costa Santos
3081e73f8a Merge branch 'develop' into backtest_live_models 2022-10-10 14:53:45 -03:00
Matthias
eaae9c9e03 Update docstring format 2022-10-10 12:19:29 +00:00
robcaulk
a4aa1b972c isolate and standardize location of tensorboard files, add doc, ensure backtesting functionality 2022-10-09 21:11:37 +02:00
th0rntwig
4daf0000c7 Move check and add log warning 2022-10-08 16:15:48 +02:00
th0rntwig
a9d5e04a43 Remove constant labels from prediction 2022-10-06 19:26:33 +02:00
robcaulk
048cb95bd6 Merge remote-tracking branch 'origin/develop' into dev-merge-rl 2022-10-01 17:48:47 +02:00
Robert Caulk
3e34f10e3d
Merge pull request #7508 from aemr3/fix-pca-errors
Fix feature list match for PCA
2022-10-01 16:50:29 +02:00
robcaulk
f4c6b99d63 remove commented lines 2022-10-01 14:23:15 +02:00
robcaulk
cd514cf15d fix inlier metric in backtesting 2022-10-01 14:18:46 +02:00
robcaulk
f2b875483f ensure raw features match when PCA is employed 2022-10-01 13:14:59 +02:00
robcaulk
51556e08c3 Merge branch 'develop' into pr/th0rntwig/7495 2022-10-01 12:45:08 +02:00
Emre
cdc01a0781
Fix feature list match for pca 2022-09-30 15:22:05 -07:00
Matthias
cc06c60fd8 Fix pandas deprecation warnings from freqAI 2022-09-30 15:43:23 +02:00
robcaulk
be48131185 make shuffle false in constants 2022-09-30 00:33:08 +02:00
Robert Caulk
6e74d46660
Ensure 1 thread available 2022-09-29 14:02:00 +02:00
Wagner Costa Santos
6845a5c6ea backtest_live_models - refactoring after PR review 2022-09-29 01:48:38 -03:00
robcaulk
83343dc2f1 control number of threads, update doc 2022-09-29 00:10:18 +02:00
th0rntwig
683b084323 Set train-test-split shuffle=False as default and remove stratification 2022-09-28 18:23:56 +02:00
Wagner Costa Santos
55ebbeec18 backtest_live models tests refactoring 2022-09-28 08:48:32 -03:00
Wagner Costa Santos
0be115de9c backtest_live_models - added new tests and refactoring 2022-09-27 10:26:57 -03:00
Wagner Costa Santos
72aa47fc51 backtest_live_models - fix issue with timerange BT and 2 trainings within same candle (no data) 2022-09-27 00:14:12 -03:00
Wagner Costa Santos
ec947ad65c remove commented code - backtest_live_models 2022-09-25 23:47:27 -03:00
Wagner Costa Santos
5880f7a638 backtest_live_models - params validation and get timerange from live models in BT 2022-09-25 23:14:00 -03:00
Wagner Costa Santos
f3f3917da3 Merge branch 'develop' into backtest_live_models 2022-09-25 20:05:26 -03:00
Wagner Costa Santos
0ed7b2bfc3 change start_backtesting to handle backtest_live_models 2022-09-25 10:35:55 -03:00
robcaulk
873d2a5069 no model save backtest, plot features backtest, ensure inlier plays nice, doc 2022-09-25 11:18:10 +02:00
Wagner Costa Santos
7f116db95e added generic function to get timerange from existent models 2022-09-24 13:01:53 -03:00
Wagner Costa Santos
d9c16d4888 Merge branch 'develop' into backtest_live_models 2022-09-24 12:30:55 -03:00
Wagner Costa Santos
3ee7eb63f7 starting backtest live models 2022-09-24 12:28:52 -03:00
Matthias
585342f193
Merge pull request #7454 from freqtrade/fix_backtesting_dfsize_freqai
Ensure the DF has the same size in backtesting FreqAI
2022-09-24 16:17:39 +02:00
th0rntwig
71e6c54ea4
Normalise distances before Weibull fit (#7432)
* Normalise distances before Weibull

* Track inlier-metric params
2022-09-22 18:11:50 +02:00
Wagner Costa Santos
b1dbc3a65f remove function remove_training_from_backtesting and ensure BT period is correct with startup_candle_count 2022-09-22 12:13:51 -03:00
Matthias
667853c504 Use Alias to type config objects 2022-09-18 13:20:55 +02:00
Robert Caulk
73e122ad10
Merge pull request #7391 from wagnercosta/fixBTfreqaiV2
Fix FreaqAI backtesting - startup_candle_count bug
2022-09-10 21:53:00 +02:00
Wagner Costa Santos
311ae8bf1f freqai backtesting - add startup_candle_count at function description 2022-09-10 14:45:42 -03:00
Robert Caulk
4a47c63f71
Merge pull request #7374 from th0rntwig/filter-constant-features 2022-09-10 16:58:58 +02:00
Wagner Costa Santos
79985fda01 fix backtesting freqai startup candles bug 2022-09-10 10:27:17 -03:00
robcaulk
05581db4e3 ensure columns are only dropped in training 2022-09-09 20:37:21 +02:00
Wagner Costa Santos
d38cc06139 Fix freqai backtesting time range issue 2022-09-09 19:27:05 +02:00
Wagner Costa Santos
f4f2884a66 Fix freqai backtesting time range issue 2022-09-07 18:52:58 -03:00
th0rntwig
cdc72bf8ca Correct indexing 2022-09-07 18:14:13 +02:00
th0rntwig
047ded1baa Check for constant columns 2022-09-07 17:47:27 +02:00
Robert Caulk
d7585161b2
Merge branch 'develop' into add-inlier-metric 2022-09-06 20:40:21 +02:00
robcaulk
d44296783e isort datakitchen 2022-09-06 20:10:12 +02:00
robcaulk
e83c9b276d fix whitespace 2022-09-06 19:56:52 +02:00
robcaulk
4b28d0495f fix timestamping, move imports, add words to doc 2022-09-06 19:46:58 +02:00
robcaulk
78d01810ed reduce code redundancy, ensure live always gets the latest data 2022-09-05 00:12:00 +02:00
Wagner Costa Santos
d43ed186fc fix issue with freqai backtesting at slice dataframe 2022-09-05 00:12:00 +02:00
Robert Caulk
8545d74378
Merge pull request #7331 from th0rntwig/pca
Normalise PCA space
2022-09-03 21:49:54 +02:00
robcaulk
5cfb4154eb revert all changes in normalize_data() 2022-09-03 19:48:30 +02:00
robcaulk
c21808ff98 remove metadata redundancy, fix pca bug 2022-09-03 16:54:30 +02:00
robcaulk
c9be66b5b6 increase test coverage for dk, improve function naming, extra cleaning 2022-09-03 15:52:29 +02:00
robcaulk
599c1c79fb reorganized backtest utilities, test new functionality, improve/update doc 2022-09-03 14:00:01 +02:00
th0rntwig
11b2bc269e Added missing s 2022-09-01 22:37:32 +02:00
th0rntwig
3f8400df10 Normalise PCA space 2022-09-01 21:51:33 +02:00
Wagner Costa Santos
d6e115178a refactoring freqai backtesting - remove duplicate code 2022-09-01 07:09:23 -03:00
Wagner Costa Santos
44d3a9140d Merge branch 'develop' of https://github.com/wagnercosta/freqtrade into develop 2022-08-31 15:37:06 -03:00
Wagner Costa Santos
7bed0450d2 pr review - refactoring backtesting freqai 2022-08-31 15:36:29 -03:00
wagnercosta
3d4497467c
Merge branch 'freqtrade:develop' into develop 2022-08-31 14:47:06 -03:00
Wagner Costa Santos
df51da22ee refactoring freqai backtesting 2022-08-31 11:23:48 -03:00
Matthias
a88ffd2c9d Merge branch 'develop' into move_datadownload 2022-08-31 10:23:45 +00:00
robcaulk
7e8e29e42d use continuous value for inlier_metric 2022-08-30 20:41:37 +02:00
robcaulk
a58dd0bbf9 add noise feature, improve docstrings 2022-08-30 18:26:24 +02:00
robcaulk
b11742a4c5 integrate inlier metric function 2022-08-30 18:26:24 +02:00
th0rntwig
d3cb211283 Add inlier metric computation 2022-08-30 18:26:24 +02:00
th0rntwig
50e2808667
Fix bug in DI (#7321) 2022-08-30 12:54:39 +02:00
robcaulk
62c0a174c8 allow users to properly reverse train-test data ordering 2022-08-29 11:04:58 +02:00
robcaulk
dd628eb525 add tests for outlier detection and removal functions 2022-08-28 12:56:39 +02:00
robcaulk
1e41c773a0 fix outlier protection 2022-08-28 12:11:29 +02:00
th0rntwig
71f7d68783 Fixed mypy error 2022-08-27 12:44:55 +02:00
elintornquist
86c5ac44e4 Add outlier percentage check 2022-08-26 23:05:07 +02:00
elintornquist
b2d664c63c Change MinPts calculation 2022-08-26 18:57:27 +02:00
robcaulk
65b552e310 make docs reflect reality, move download_all_data to new utils.py file, automatic startup_candle detection 2022-08-26 15:30:01 +02:00
robcaulk
4b7e640f31 reduce code duplication, optimize auto data download per tf 2022-08-26 13:56:44 +02:00
th0rntwig
5ce1c69803
Improve DBSCAN epsilon identification (#7269)
* Improve DBSCAN epsilon identification
2022-08-22 19:57:20 +02:00
robcaulk
ac42c0153d deprecate indicator_max_period_candles, automatically compute startup candles for FreqAI backtesting. 2022-08-22 18:19:07 +02:00
robcaulk
96d8882f1e Plug mem leak, add training timer 2022-08-22 13:30:30 +02:00
longyu
cfa5b3f12c add new line 2022-08-19 12:39:08 +02:00
longyu
277245c69d remove line 2022-08-19 12:39:00 +02:00
longyu
f70b0bab80 remove line 2022-08-17 23:49:20 +02:00
robcaulk
5155afb4e7 clean up code remnants 2022-08-17 15:22:48 +02:00
robcaulk
0c34104e45 extract download-data from freqai to prepare for future async changes 2022-08-17 15:18:44 +02:00
longyu
9c38c27eed ignore sample itself distance for avg_mean_dist computation 2022-08-17 15:09:57 +02:00
longyu
72c34291e3 newline 2022-08-17 15:09:10 +02:00
robcaulk
006b11e5d5 fix leftover bug in indicator population 2022-08-14 21:42:55 +02:00
robcaulk
ad846cdb76 fix lock bug, update docstring 2022-08-14 20:24:29 +02:00
Matthias
a29402ddde Rename and move analysis_lock to data_kitchen 2022-08-14 17:23:14 +02:00
Matthias
3a9ec76c91 Move "freqai.lock" to backend to simplify user interface 2022-08-14 17:19:50 +02:00
robcaulk
b1b76a2dbe debug classifier with predict proba 2022-08-13 19:40:24 +02:00
robcaulk
23cc21ce59 add predict_proba to base classifier, improve historic predictions handling 2022-08-13 19:40:24 +02:00
robcaulk
fb4b73ce89 ensure dates are saved 2022-08-12 12:03:44 +02:00
robcaulk
2cae3c42e6 remove trade database analyzer, clean up a bit 2022-08-10 17:43:06 +02:00
robcaulk
5a16d5a512 Deactivate database analyzer if user does not use sqlite 2022-08-09 16:36:22 +02:00
robcaulk
aef086b02e Improved dict typing, timeframe parser, collect dates associated with training data points 2022-08-09 15:30:25 +02:00
Matthias
77b3b8a134 Use main exchange instead of creating a separate instance. 2022-08-08 18:34:11 +00:00
Matthias
b16f57cb0d Minor stylistic fixes 2022-08-06 14:55:46 +02:00
Robert Caulk
c172ce1011 improve flexibility of user defined prediction dataframe 2022-08-06 13:51:19 +02:00
Robert Caulk
07763d0d4f add classifier, improve model naming scheme 2022-08-06 08:33:55 +02:00
Robert Caulk
ce8fbbf743 ensure loading historical df matches frequi indices 2022-08-06 07:25:59 +02:00
robcaulk
60d782e5c5 remove unnecessary function 2022-08-05 21:31:32 +02:00
robcaulk
a42a060ab5 fix DB once and for all. Make DBSCAN more efficient and robust. 2022-08-05 21:29:03 +02:00
robcaulk
29b7b014e5 fix bug in DB path initialization 2022-08-05 18:19:26 +02:00
robcaulk
05ec5c5e54 generalize database url path for any db type 2022-08-05 12:19:29 +02:00
Robert Caulk
51a6b4289f improve DBSCAN performance for subsequent trainings 2022-08-04 17:41:58 +02:00
Robert Caulk
fe1b8515a8 fix bug in DBSCAN, update doc 2022-08-04 17:00:59 +02:00
robcaulk
29225e4baf add DBSCAN outlier detection feature, add supporting documentation 2022-08-04 12:15:16 +02:00
robcaulk
eae82d0222 fix bug with database url during backtesting. comment out example trade db analysis. 2022-08-03 16:17:57 +02:00
robcaulk
95d3009a95 give user ability to analyze live trade dataframe inside custom prediction model. Add documentation to explain new functionality 2022-08-02 20:14:02 +02:00
robcaulk
d830105605 *BREAKING CHANGE* remove unnecessary arguments from populate_any_indicators(), accommodate tests 2022-07-31 17:05:29 +02:00
robcaulk
dd8288c090 expose full parameter set for SVM outlier detection. Set default shuffle to false to improve reproducibility 2022-07-30 13:40:05 +02:00
robcaulk
f22b140782 fix backtesting bug, undo move of label stat calc, fix example strat exit logic 2022-07-29 17:27:35 +02:00
robcaulk
c84d54b35e Fix typing issue, avoid using .get() when unnecessary, convert to fstrings 2022-07-29 08:12:50 +02:00
robcaulk
e213d0ad55 isolate data_drawer functions from data_kitchen, accommodate tests, add new test 2022-07-26 10:24:14 +02:00
robcaulk
56b17e6f3c allow user to pass test_size = 0 and avoid using eval sets in prediction models 2022-07-25 19:40:13 +02:00
Robert Caulk
897f18a8c8 ensure proper integer type casting for timestamps. Add check test for backtesting subdaily time periods 2022-07-25 15:07:09 +02:00
Matthias
61c41fd919 Merge branch 'develop' into feat/freqai 2022-07-24 16:18:58 +02:00
Robert Caulk
88e10f7306 add exception for not passing timerange. Remove hard coded arguments for CatboostPredictionModels. Update docs 2022-07-24 09:01:23 +02:00
Robert Caulk
fff39eff9e fix multitarget bug 2022-07-24 08:42:50 +02:00
robcaulk
f3d46613ee move prediction denormalization into datakitchen. remove duplicate associated code. avoid normalization/denormalization for string dtypes. 2022-07-23 17:14:33 +02:00
robcaulk
c91e23dc50 let user avoid normalizing labels 2022-07-23 16:14:13 +02:00
robcaulk
a1cff377ec add record of contribution to data_kitchen.py 2022-07-23 13:32:04 +02:00
robcaulk
40f00196eb use cloudpickle in place of pickle. define Paths once in data_drawer. 2022-07-22 17:37:51 +02:00
robcaulk
afcb0bec00 clean up obsolete comments, move remove_features_from_df to datakitchen 2022-07-22 12:29:20 +02:00
robcaulk
3205788bce extend doc to include descriptions of the return values from FreqAI to the strategy 2022-07-21 22:11:46 +02:00
robcaulk
183dec866a remove ability to backtest open ended timeranges (safer) 2022-07-21 13:02:52 +02:00
robcaulk
ca4dd58642 remove superceded function from datakitchen 2022-07-21 12:40:54 +02:00
robcaulk
8f86b0deaa *breaking change* simplify user strat by consolidating feature loops into backend 2022-07-21 12:24:22 +02:00
robcaulk
e7337728bf add separator in folder name just incase an asset ends in an integer 2022-07-21 11:25:28 +02:00
robcaulk
d43c146676 add more tests for datakitchen functionalities, add regression tests for freqai_interface train/backtest 2022-07-20 12:56:46 +02:00
lolong
9c051958a6
Feat/freqai (#7105)
Vectorize weight setting, log training dates

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-07-19 17:49:18 +02:00
robcaulk
714d9534b6 start adding tests 2022-07-19 16:16:44 +02:00
lolong
ed0f8b1189
Improve FreqAI documentation (#7072)
Improve doc + some other small fixes

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-07-18 11:57:52 +02:00
robcaulk
ef409dd345 Add ground work for TensorFlow models, add protections from common mistakes 2022-07-12 18:09:17 +02:00
Robert Caulk
8ce6b18318 start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available. 2022-07-11 22:01:48 +02:00
Robert Caulk
607455919e Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!) 2022-07-10 12:35:44 +02:00
robcaulk
d9acdc9767 remove excess, increase no model warning clarity 2022-07-06 18:20:21 +02:00
robcaulk
4cac67fd66 Catch infrequent issue associated with grabbing first candle 2022-07-05 12:43:33 +02:00
robcaulk
bd3a6ba2fe update backtesting to handle new output framework 2022-07-03 17:34:44 +02:00
robcaulk
4ff0ef7359 fix bug returning multiple targets for training 2022-07-03 12:15:59 +02:00
robcaulk
ffb39a5029 black formatting on freqai files 2022-07-03 10:59:38 +02:00
robcaulk
106131ff0f Rehaul organization of return values 2022-07-02 18:09:38 +02:00
robcaulk
93e1410ed9 first step toward cleaning output and enabling multimodel training per pair 2022-07-01 14:00:30 +02:00
robcaulk
6c7d02cb18 expose nu in the SVM outlier detection via svm_nu in config 2022-06-28 15:12:25 +02:00
robcaulk
051b99791d reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel 2022-06-26 19:04:23 +02:00
Robert Caulk
852706cd6b
Fix default behavior for expiration_hours 2022-06-21 08:12:51 +02:00