310 Commits

Author	SHA1	Message	Date
robcaulk	15d049cffe	detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check	2022-06-07 19:49:20 +02:00
robcaulk	4b26b6aaec	add lock to any historic data access	2022-06-07 00:54:18 +02:00
robcaulk	d6b8801f41	fix follower bug	2022-06-05 04:40:58 +02:00
robcaulk	16b4a5b71f	rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.	2022-06-03 15:19:46 +02:00
robcaulk	ace951bf7e	another queue bug fix for fresh starts	2022-06-02 13:28:06 +02:00
robcaulk	833d25bda0	Save data after queue reorg	2022-06-01 08:05:38 +02:00
robcaulk	0b0dd8dd80	Force high priority pair to be trained before anything else.	2022-06-01 07:55:05 +02:00
robcaulk	7523ed825e	automatically detect maximum required data based on user fed indicators (to avoid NaNs in dataset for rolling indicators), add new config parameter for backtesting to let users increase their startup_candles to accommodate high timeframe indicators, add docs to explain all. Add new feature for automatic indicator duplication according to user defined intervals (exhibited in example strat and configs now).	2022-05-31 18:42:27 +02:00
robcaulk	9b3b08a2bb	let follower purge old model files	2022-05-31 15:37:38 +02:00
robcaulk	bac4ced382	Ensure follower predictions are persistent and uniquely stored	2022-05-31 14:35:04 +02:00
robcaulk	70adf55643	Automatically detect and change follower data_path to accommodate remote systems	2022-05-31 12:35:09 +02:00
robcaulk	0306f5ca13	Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk	2022-05-31 11:58:21 +02:00
robcaulk	45f4f0f603	ensure follower sends back null arrays in case leader hasnt created a model file yet	2022-05-31 01:48:48 +02:00
robcaulk	606f18e5c1	Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader	2022-05-30 21:35:48 +02:00
robcaulk	5b4c649d43	detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly	2022-05-30 13:55:46 +02:00
robcaulk	e229902381	fix bug in previous commit	2022-05-30 12:48:22 +02:00
robcaulk	a20651efd8	Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).	2022-05-30 11:37:05 +02:00
robcaulk	d59eac3321	revert a79032b	2022-05-29 21:33:38 +02:00
robcaulk	a79032bf75	fixing bug in training queue	2022-05-29 20:19:32 +02:00
robcaulk	3f72263278	allow pairs deeper in the queue to get trained if the higher priority pairs dont need training	2022-05-29 20:02:43 +02:00
robcaulk	cc6cae47ec	allow pairs deeper in the queue to get trained if the higher priority pairs dont need training	2022-05-29 19:49:43 +02:00
robcaulk	4eb4753e20	allow subdaily retraining for backtesting	2022-05-29 17:44:35 +02:00
robcaulk	ce365eb9e3	improve example strat so that it has dynamic buy and sell logic according to original prediction	2022-05-29 14:45:46 +02:00
robcaulk	4eb29c8810	Dont reset pair priority if it doesnt successfully train	2022-05-28 18:34:26 +02:00
robcaulk	83dd453723	catch errors occuring on background thread, and make sure to keep the ball rolling. Improve pair retraining queue.	2022-05-28 18:26:19 +02:00
robcaulk	e54614fa2f	remove remnants of single threaded version, ensure pair queue priority is checked before retraining	2022-05-28 14:55:07 +02:00
robcaulk	2a4d1e2d64	fix bug in setting new timerange for retraining	2022-05-28 12:23:26 +02:00
robcaulk	7870a86e9a	fix live retraining bug	2022-05-28 11:38:57 +02:00
robcaulk	0bf915054d	handle key check correctly	2022-05-28 11:22:32 +02:00
robcaulk	c5a16e91fb	throw user error if user tries to load models but feeds the wrong features (while using PCA)	2022-05-28 11:11:41 +02:00
robcaulk	6193205012	fix bug for target_mean/std array merging in backtesting	2022-05-26 21:07:50 +02:00
robcaulk	b79d4e8876	Allow user to go live and start from pretrained models (after a completed backtest) by simply reusing the identifier config parameter while dry/live.	2022-05-25 14:40:32 +02:00
robcaulk	7486d9d9e2	proper validation of freqai config parameters	2022-05-25 12:37:25 +02:00
robcaulk	7ff3258607	remove assertions, log error if user has not assigned freqai in config, fix stratify bug	2022-05-25 11:43:45 +02:00
robcaulk	35bed842cb	cleanup, add clarity to comments and docstrings	2022-05-25 11:31:03 +02:00
robcaulk	58b5abbaa6	improve multithreaded training queue system	2022-05-24 15:28:38 +02:00
robcaulk	255d35976e	add priority metadata to pairs to avoid a sync of train time + train period	2022-05-24 12:58:53 +02:00
robcaulk	059c285425	paying closer attention to managing live retraining on separate thread without affecting prediction of other coins on master thread	2022-05-24 12:01:01 +02:00
robcaulk	b0d2d13eb1	improve data persistence/mapping for live/dry. This accommodates quick reloads after crash and handles multi-pair cleanly	2022-05-23 21:05:05 +02:00
robcaulk	e1c068ca66	add config asserts, use .get method with default values for optional functionality, move data_cleaning_* to freqai_interface (away from user custom pred model) since it is controlled by config params.	2022-05-23 12:07:09 +02:00
robcaulk	ee3cdd0ffe	more cleanup	2022-05-23 09:55:58 +02:00
robcaulk	af0cc21af9	Enable hourly/minute retraining in live/dry. Suppress catboost folder output. Update config + constants + docs to reflect updates.	2022-05-23 00:06:26 +02:00
robcaulk	42d95af829	Aggregated commit. Adding support vector machine for outlier detection, improve user interface to dry/live, better standardization, fix various other bugs	2022-05-22 17:51:49 +02:00
robcaulk	c5ecf94177	move live retraining to separate thread.	2022-05-19 21:15:58 +02:00
robcaulk	89eacf2f47	Retrain model if FreqAI found a pretrained model but user strategy is not passing the expected features (user has changed the features in the strategy but has passed a the same config[freqai][identifier]). Logger warning output to user.	2022-05-19 17:15:50 +02:00
Matthias	c81b960791	Fix some typos	2022-05-17 19:58:36 +02:00
robcaulk	8664e8f9a3	create a prediction_models folder where basic prediction models can live (similar to optimize/hyperopt-loss. Update resolver/docs/and gitignore to accommodate change	2022-05-17 17:13:38 +02:00
robcaulk	9b3e5faebe	create more flexible whitelist, avoid duplicating whitelist features into corr_pairlist, update docs	2022-05-15 17:42:15 +02:00
robcaulk	22bd5556ed	add self-retraining functionality for live/dry	2022-05-15 17:42:15 +02:00
robcaulk	178c2014b0	appease mypy	2022-05-15 17:42:15 +02:00
robcaulk	f653ace24b	another attempt at fixing datalength bug	2022-05-15 17:42:15 +02:00
robcaulk	b08c0888bb	add USERPATH_FREQAIMODELS, remove return values from @abstract methods	2022-05-15 17:42:15 +02:00
robcaulk	00ff0c9b91	ensure user defined timerange truncates final backtest so that we arent mismatching data lengths upon return to strategy. Rename DataHandler class to FreqaiDataKitchen	2022-05-15 17:41:34 +02:00
robcaulk	def71a0afe	auto build full_timerange and self manage training_timerange	2022-05-15 17:41:34 +02:00
robcaulk	764f9449b4	fix logger, debug some flake8 appeasements	2022-05-15 17:41:34 +02:00
robcaulk	29c2d1d189	use logger in favor of print	2022-05-15 17:38:58 +02:00
robcaulk	99f7e44c30	flake8 passing, use pathlib in lieu of os.path to accommodate windows/mac OS	2022-05-15 17:38:58 +02:00
robcaulk	630d201546	remove trained_stake	2022-05-15 17:38:58 +02:00
robcaulk	b40f8f88ac	cleaning and bug fixing	2022-05-15 17:38:58 +02:00
robcaulk	fc837c4daa	add freqao backend machinery, user interface, documentation	2022-05-15 17:38:58 +02:00