lychiyu/freqtrade_origin
1
0
Fork 0
mirror of https://github.com/freqtrade/freqtrade.git synced 2024-11-15 12:43:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
e0a538f380
freqtrade_origin/en/2019.10/data-download/index.html

1086 lines
26 KiB
HTML
Raw Normal View History

Deployed 159384720 to 2019.10 in en with MkDocs 1.6.0 and mike 2.1.2
2024-08-20 17:56:20 +00:00
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link rel="prev" href="../rest-api/">
<link rel="next" href="../backtesting/">
<link rel="icon" href="../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.0, mkdocs-material-9.5.30">
<title>Data Downloading - Freqtrade</title>
<link rel="stylesheet" href="../assets/stylesheets/main.3cba04c6.min.css">
<link rel="stylesheet" href="../assets/stylesheets/palette.06af60db.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="../stylesheets/ft.extra.css">
<script>__md_scope=new URL("..",location),__md_hash=e=>[...e].reduce((e,_)=>(e<<5)-e+_.charCodeAt(0),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="blue-grey" data-md-color-accent="tear">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#data-downloading" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<div data-md-color-scheme="default" data-md-component="outdated" hidden>
</div>
<header class="md-header md-header--shadow" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href=".." title="Freqtrade" class="md-header__button md-logo" aria-label="Freqtrade" data-md-component="logo">
<img src="../images/logo.png" alt="logo">
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3V6m0 5h18v2H3v-2m0 5h18v2H3v-2Z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
Freqtrade
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Data Downloading
</span>
</div>
</div>
</div>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.516 6.516 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5Z"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11h12Z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41Z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href=".." title="Freqtrade" class="md-nav__button md-logo" aria-label="Freqtrade" data-md-component="logo">
<img src="../images/logo.png" alt="logo">
</a>
Freqtrade
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href=".." class="md-nav__link">
<span class="md-ellipsis">
About
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../installation/" class="md-nav__link">
<span class="md-ellipsis">
Installation
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../docker/" class="md-nav__link">
<span class="md-ellipsis">
Installation Docker
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../configuration/" class="md-nav__link">
<span class="md-ellipsis">
Configuration
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../strategy-customization/" class="md-nav__link">
<span class="md-ellipsis">
Strategy Customization
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../stoploss/" class="md-nav__link">
<span class="md-ellipsis">
Stoploss
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../bot-usage/" class="md-nav__link">
<span class="md-ellipsis">
Start the bot
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_8" >
<label class="md-nav__link" for="__nav_8" id="__nav_8_label" tabindex="0">
<span class="md-ellipsis">
Control the bot
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_8_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_8">
<span class="md-nav__icon md-icon"></span>
Control the bot
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../telegram-usage/" class="md-nav__link">
<span class="md-ellipsis">
Telegram
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../webhook-config/" class="md-nav__link">
<span class="md-ellipsis">
Web Hook
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../rest-api/" class="md-nav__link">
<span class="md-ellipsis">
REST API
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
<span class="md-ellipsis">
Data Downloading
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
<span class="md-ellipsis">
Data Downloading
</span>
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#getting-data-for-backtesting-and-hyperopt" class="md-nav__link">
<span class="md-ellipsis">
Getting data for backtesting and hyperopt
</span>
</a>
<nav class="md-nav" aria-label="Getting data for backtesting and hyperopt">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#pairs-file" class="md-nav__link">
<span class="md-ellipsis">
Pairs file
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#start-download" class="md-nav__link">
<span class="md-ellipsis">
Start download
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#other-notes" class="md-nav__link">
<span class="md-ellipsis">
Other Notes
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#trades-tick-data" class="md-nav__link">
<span class="md-ellipsis">
Trades (tick) data
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#historic-kraken-data" class="md-nav__link">
<span class="md-ellipsis">
Historic Kraken data
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#next-step" class="md-nav__link">
<span class="md-ellipsis">
Next step
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../backtesting/" class="md-nav__link">
<span class="md-ellipsis">
Backtesting
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../hyperopt/" class="md-nav__link">
<span class="md-ellipsis">
Hyperopt
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../edge/" class="md-nav__link">
<span class="md-ellipsis">
Edge Positioning
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../utils/" class="md-nav__link">
<span class="md-ellipsis">
Utility Subcommands
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../faq/" class="md-nav__link">
<span class="md-ellipsis">
FAQ
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_15" >
<label class="md-nav__link" for="__nav_15" id="__nav_15_label" tabindex="0">
<span class="md-ellipsis">
Data Analysis
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_15_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_15">
<span class="md-nav__icon md-icon"></span>
Data Analysis
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../data-analysis/" class="md-nav__link">
<span class="md-ellipsis">
Jupyter Notebooks
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../strategy_analysis_example/" class="md-nav__link">
<span class="md-ellipsis">
Strategy analysis
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../plotting/" class="md-nav__link">
<span class="md-ellipsis">
Plotting
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../sql_cheatsheet/" class="md-nav__link">
<span class="md-ellipsis">
SQL Cheatsheet
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../sandbox-testing/" class="md-nav__link">
<span class="md-ellipsis">
Sandbox Testing
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../deprecated/" class="md-nav__link">
<span class="md-ellipsis">
Deprecated Features
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../developer/" class="md-nav__link">
<span class="md-ellipsis">
Contributors Guide
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#getting-data-for-backtesting-and-hyperopt" class="md-nav__link">
<span class="md-ellipsis">
Getting data for backtesting and hyperopt
</span>
</a>
<nav class="md-nav" aria-label="Getting data for backtesting and hyperopt">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#pairs-file" class="md-nav__link">
<span class="md-ellipsis">
Pairs file
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#start-download" class="md-nav__link">
<span class="md-ellipsis">
Start download
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#other-notes" class="md-nav__link">
<span class="md-ellipsis">
Other Notes
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#trades-tick-data" class="md-nav__link">
<span class="md-ellipsis">
Trades (tick) data
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#historic-kraken-data" class="md-nav__link">
<span class="md-ellipsis">
Historic Kraken data
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#next-step" class="md-nav__link">
<span class="md-ellipsis">
Next step
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1 id="data-downloading">Data Downloading<a class="headerlink" href="#data-downloading" title="Permanent link">&para;</a></h1>
<h2 id="getting-data-for-backtesting-and-hyperopt">Getting data for backtesting and hyperopt<a class="headerlink" href="#getting-data-for-backtesting-and-hyperopt" title="Permanent link">&para;</a></h2>
<p>To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the <code>freqtrade download-data</code> command.</p>
<p>If no additional parameter is specified, freqtrade will download data for <code>"1m"</code> and <code>"5m"</code> timeframes for the last 30 days.
Exchange and pairs will come from <code>config.json</code> (if specified using <code>-c/--config</code>).
Otherwise <code>--exchange</code> becomes mandatory.</p>
<div class="admonition tip updating existing data">
<p class="admonition-title">Tip</p>
<p>If you already have backtesting data available in your data-directory and would like to refresh this data up to today, use <code>--days xx</code> with a number slightly higher than the missing number of days. Freqtrade will keep the available data and only download the missing data.
Be carefull though: If the number is too small (which would result in a few missing days), the whole dataset will be removed and only xx days will be downloaded.</p>
</div>
<h3 id="pairs-file">Pairs file<a class="headerlink" href="#pairs-file" title="Permanent link">&para;</a></h3>
<p>In alternative to the whitelist from <code>config.json</code>, a <code>pairs.json</code> file can be used.</p>
<p>If you are using Binance for example:</p>
<ul>
<li>create a directory <code>user_data/data/binance</code> and copy or create the <code>pairs.json</code> file in that directory.</li>
<li>update the <code>pairs.json</code> file to contain the currency pairs you are interested in.</li>
</ul>
<div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>-p<span class="w"> </span>user_data/data/binance
cp<span class="w"> </span>freqtrade/tests/testdata/pairs.json<span class="w"> </span>user_data/data/binance
</code></pre></div>
<p>The format of the <code>pairs.json</code> file is a simple json list.
Mixing different stake-currencies is allowed for this file, since it's only used for downloading.</p>
<div class="highlight"><pre><span></span><code><span class="p">[</span>
<span class="w"> </span><span class="s2">&quot;ETH/BTC&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="s2">&quot;ETH/USDT&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="s2">&quot;BTC/USDT&quot;</span><span class="p">,</span>
<span class="w"> </span><span class="s2">&quot;XRP/ETH&quot;</span>
<span class="p">]</span>
</code></pre></div>
<h3 id="start-download">Start download<a class="headerlink" href="#start-download" title="Permanent link">&para;</a></h3>
<p>Then run:</p>
<div class="highlight"><pre><span></span><code>freqtrade<span class="w"> </span>download-data<span class="w"> </span>--exchange<span class="w"> </span>binance
</code></pre></div>
<p>This will download ticker data for all the currency pairs you defined in <code>pairs.json</code>.</p>
<h3 id="other-notes">Other Notes<a class="headerlink" href="#other-notes" title="Permanent link">&para;</a></h3>
<ul>
<li>To use a different directory than the exchange specific default, use <code>--datadir user_data/data/some_directory</code>.</li>
<li>To change the exchange used to download the tickers, please use a different configuration file (you'll probably need to adjust ratelimits etc.)</li>
<li>To use <code>pairs.json</code> from some other directory, use <code>--pairs-file some_other_dir/pairs.json</code>.</li>
<li>To download ticker data for only 10 days, use <code>--days 10</code> (defaults to 30 days).</li>
<li>Use <code>--timeframes</code> to specify which tickers to download. Default is <code>--timeframes 1m 5m</code> which will download 1-minute and 5-minute tickers.</li>
<li>To use exchange, timeframe and list of pairs as defined in your configuration file, use the <code>-c/--config</code> option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine <code>-c/--config</code> with most other options.</li>
</ul>
<h3 id="trades-tick-data">Trades (tick) data<a class="headerlink" href="#trades-tick-data" title="Permanent link">&para;</a></h3>
<p>By default, <code>download-data</code> subcommand downloads Candles (OHLCV) data. Some exchanges also provide historic trade-data via their API.
This data can be useful if you need many different timeframes, since it is only downloaded once, and then resampled locally to the desired timeframes.</p>
<p>Since this data is large by default, the files use gzip by default. They are stored in your data-directory with the naming convention of <code>&lt;pair&gt;-trades.json.gz</code> (<code>ETH_BTC-trades.json.gz</code>). Incremental mode is also supported, as for historic OHLCV data, so downloading the data once per week with <code>--days 8</code> will create an incremental data-repository.</p>
<p>To use this mode, simply add <code>--dl-trades</code> to your call. This will swap the download method to download trades, and resamples the data locally.</p>
<p>Example call:</p>
<div class="highlight"><pre><span></span><code>freqtrade<span class="w"> </span>download-data<span class="w"> </span>--exchange<span class="w"> </span>binance<span class="w"> </span>--pairs<span class="w"> </span>XRP/ETH<span class="w"> </span>ETH/BTC<span class="w"> </span>--days<span class="w"> </span><span class="m">20</span><span class="w"> </span>--dl-trades
</code></pre></div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>While this method uses async calls, it will be slow, since it requires the result of the previous call to generate the next request to the exchange.</p>
</div>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>The historic trades are not available during Freqtrade dry-run and live trade modes because all exchanges tested provide this data with a delay of few 100 candles, so it's not suitable for real-time trading.</p>
</div>
<h3 id="historic-kraken-data">Historic Kraken data<a class="headerlink" href="#historic-kraken-data" title="Permanent link">&para;</a></h3>
<p>The Kraken API does only provide 720 historic candles, which is sufficient for FreqTrade dry-run and live trade modes, but is a problem for backtesting.
To download data for the Kraken exchange, using <code>--dl-trades</code> is mandatory, otherwise the bot will download the same 720 candles over and over, and you'll not have enough backtest data.</p>
<h2 id="next-step">Next step<a class="headerlink" href="#next-step" title="Permanent link">&para;</a></h2>
<p>Great, you now have backtest data downloaded, so you can now start <a href="../backtesting/">backtesting</a> your strategy.</p>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"base": "..", "features": [], "search": "../assets/javascripts/workers/search.b8dbb3d2.min.js", "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": {"provider": "mike"}}</script>
<script src="../assets/javascripts/bundle.fe8b6f2b.min.js"></script>
</body>
</html>
Reference in New Issue Copy Permalink