- some work on all pairs, and we don't check protections either so ... just disable them completely
- added info in the docs
Changed pairs-check to if no definition is in the config (but it s maybe in the strategy) it will just force-set it to the proper amount of len(config['pairs']
No default value is specified in the docs for the processing_mode, making it unclear that the default behaviour is to filter out pairs, rather than append.
Fix logical error in the conditional checks for model classes. The `elif` statement that looks for "lightgbm.sklearn" or "xgb" in the model class string is now broken into two separate conditions because the old condition would always evaluate to `True` due to the non-empty string "xgb".
We receive a lot of code that fails the `ruff` checks.
@@ -125,7 +125,7 @@ Exceptions:
Contributors may be given commit privileges. Preference will be given to those with:
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Quantity and quality are considered.
1. Past contributions to Freqtrade and other related open-source projects. Contributions to Freqtrade include both code (both accepted and pending) and friendly participation in the issue tracker and Pull request reviews. Both quantity and quality are considered.
1. A coding style that the other core committers find simple, minimal, and clean.
1. Access to resources for cross-platform development and testing.
The bot has made `429` trades for an average duration of `4:12:00`, with a performance of `76.20%` (profit), that means it has
earned a total of `0.00762792 BTC` starting with a capital of 0.01 BTC.
The column `Avg Profit %` shows the average profit for all trades made while the column `Cum Profit %` sums up all the profits/losses.
The column `Avg Profit %` shows the average profit for all trades made.
The column `Tot Profit %` shows instead the total profit % in relation to the starting balance.
In the above results, we have a starting balance of 0.01 BTC and an absolute profit of 0.00762792 BTC - so the `Tot Profit %` will be `(0.00762792 / 0.01) * 100 ~= 76.2%`.
@@ -464,7 +464,7 @@ It contains some useful key metrics about performance of your strategy on backte
-`Profit factor`: profit / loss.
-`Avg. stake amount`: Average stake amount, either `stake_amount` or the average when using dynamic stake amount.
-`Total trade volume`: Volume generated on the exchange to reach the above profit.
-`Best Pair` / `Worst Pair`: Best and worst performing pair, and it's corresponding `Cum Profit %`.
-`Best Pair` / `Worst Pair`: Best and worst performing pair, and it's corresponding `Tot Profit %`.
-`Best Trade` / `Worst Trade`: Biggest single winning trade and biggest single losing trade.
-`Best day` / `Worst day`: Best and worst day based on daily profit.
-`Days win/draw/lose`: Winning / Losing days (draws are usually days without closed trade).
@@ -618,22 +618,22 @@ To compare multiple strategies, a list of Strategies can be provided to backtest
This is limited to 1 timeframe value per run. However, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this will give a nice runtime boost.
All listed Strategies need to be in the same directory.
All listed Strategies need to be in the same directory, unless also `--recursive-strategy-search` is specified, where sub-directories within the strategy directory are also considered.
This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
This will save the results to `user_data/backtest_results/backtest-result-<datetime>.json`, including results for both `Strategy001` and `Strategy002`.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up to see the details per strategy.
@@ -33,7 +33,6 @@ For spot pairs, naming will be `base/quote` (e.g. `ETH/USDT`).
For futures pairs, naming will be `base/quote:settle` (e.g. `ETH/USDT:USDT`).
## Bot execution logic
Starting freqtrade in dry-run or live mode (using `freqtrade trade`) will start the bot and start the bot iteration loop.
@@ -50,10 +49,12 @@ By default, the bot loop runs every few seconds (`internals.process_throttle_sec
* Call `populate_indicators()`
* Call `populate_entry_trend()`
* Call `populate_exit_trend()`
*Check timeouts for open orders.
* Calls`check_entry_timeout()` strategy callback for open entry orders.
* Calls `check_exit_timeout()` strategy callback for open exit orders.
* Calls `adjust_entry_price()` strategy callback for open entry orders.
*Update trades open order state from exchange.
* Call `order_filled()` strategy callback for filled orders.
* Check timeouts for open orders.
* Calls `check_entry_timeout()` strategy callback for open entry orders.
* Calls `check_exit_timeout()` strategy callback for open exit orders.
* Calls `adjust_entry_price()` strategy callback for open entry orders.
* Verifies existing positions and eventually places exit orders.
* Considers stoploss, ROI and exit-signal, `custom_exit()` and `custom_stoploss()`.
* Determine exit-price based on `exit_pricing` configuration setting or by using the `custom_exit_price()` callback.
@@ -86,8 +87,10 @@ This loop will be repeated again and again until the bot is stopped.
* In Margin and Futures mode, `leverage()` strategy callback is called to determine the desired leverage.
* Determine stake size by calling the `custom_stake_amount()` callback.
* Check position adjustments for open trades if enabled and call `adjust_trade_position()` to determine if an additional order is requested.
* Call `order_filled()` strategy callback for filled entry orders.
* Call `custom_stoploss()` and `custom_exit()` to find custom exit points.
* For exits based on exit-signal, custom-exit and partial exits: Call `custom_exit_price()` to determine exit price (Prices are moved to be within the closing candle).
* Call `order_filled()` strategy callback for filled exit orders.
@@ -14,7 +14,7 @@ You can specify a different configuration file used by the bot with the `-c/--co
If you used the [Quick start](docker_quickstart.md#docker-quick-start) method for installing
the bot, the installation script should have already created the default configuration file (`config.json`) for you.
If the default configuration file is not created we recommend to use `freqtrade new-config --config config.json` to generate a basic configuration file.
If the default configuration file is not created we recommend to use `freqtrade new-config --config user_data/config.json` to generate a basic configuration file.
The Freqtrade configuration file is to be written in JSON format.
Environment variables detected are logged at startup - so if you can't find why a value is not what you think it should be based on the configuration, make sure it's not loaded from an environment variable.
!!! Tip "Validate combined result"
You can use the [show-config subcommand](utils.md#show-config) to see the final, combined configuration.
??? Warning "Loading sequence"
Environment variables are loaded after the initial configuration. As such, you cannot provide the path to the configuration through environment variables. Please use `--config path/to/config.json` for that.
This also applies to user_dir to some degree. while the user directory can be set through environment variables - the configuration will **not** be loaded from that location.
### Multiple configuration files
Multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
@@ -56,6 +63,9 @@ Multiple configuration files can be specified and used by the bot or the bot can
You can specify additional configuration files in `add_config_files`. Files specified in this parameter will be loaded and merged with the initial config file. The files are resolved relative to the initial configuration file.
This is similar to using multiple `--config` parameters, but simpler in usage as you don't have to specify all files for all commands.
!!! Tip "Validate combined result"
You can use the [show-config subcommand](utils.md#show-config) to see the final, combined configuration.
!!! Tip "Use multiple configuration files to keep secrets secret"
You can use a 2nd configuration file containing your secrets. That way you can share your "primary" configuration file, while still keeping your API keys for yourself.
The 2nd file should only specify what you intend to override.
@@ -321,11 +331,13 @@ For example, if you have 10 ETH available in your wallet on the exchange and `tr
To fully utilize compounding profits when using multiple bots on the same exchange account, you'll want to limit each bot to a certain starting balance.
This can be accomplished by setting `available_capital` to the desired starting balance.
Assuming your account has 10.000 USDT and you want to run 2 different strategies on this exchange.
Assuming your account has 10000 USDT and you want to run 2 different strategies on this exchange.
You'd set `available_capital=5000` - granting each bot an initial capital of 5000 USDT.
The bot will then split this starting balance equally into `max_open_trades` buckets.
Profitable trades will result in increased stake-sizes for this bot - without affecting the stake-sizes of the other bot.
Adjusting `available_capital` requires reloading the configuration to take effect. Adjusting the `available_capital` adds the difference between the previous `available_capital` and the new `available_capital`. Decreasing the available capital when trades are open doesn't exit the trades. The difference is returned to the wallet when the trades conclude. The outcome of this differs depending on the price movement between the adjustment and exiting the trades.
!!! Warning "Incompatible with `tradable_balance_ratio`"
Setting this option will replace any configuration of `tradable_balance_ratio`.
@@ -503,13 +515,13 @@ Configuration:
Please carefully read the section [Market order pricing](#market-order-pricing) section when using market orders.
!!! Note "Stoploss on exchange"
`stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
`order_types.stoploss_on_exchange_interval` is not mandatory. Do not change its value if you are
unsure of what you are doing. For more information about how stoploss works please
refer to [the stoploss documentation](stoploss.md).
If `stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
If `order_types.stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new stoploss order.
If stoploss on exchange creation fails for some reason, then an "emergency exit" is initiated. By default, this will exit the trade using a market order. The order-type for the emergency-exit can be changed by setting the `emergency_exit` value in the `order_types` dictionary - however, this is not advised.
### Understand order_time_in_force
@@ -572,9 +584,11 @@ In addition to fiat currencies, a range of crypto currencies is supported.
The valid values are:
```json
"BTC", "ETH", "XRP", "LTC", "BCH", "USDT"
"BTC", "ETH", "XRP", "LTC", "BCH", "BNB"
```
Removing `fiat_display_currency` completely from the configuration will skip initializing coingecko, and will not show any FIAT currency conversion. This has no importance for the correct functioning of the bot.
## Using Dry-run mode
We recommend starting the bot in the Dry-run mode to see how your bot will
@@ -594,7 +608,7 @@ creating trades on the exchange.
```json
"exchange": {
"name": "bittrex",
"name": "binance",
"key": "key",
"secret": "secret",
...
@@ -644,7 +658,7 @@ API Keys are usually only required for live trading (trading for real money, bot
@@ -129,6 +129,8 @@ Below is an outline of exception inheritance hierarchy:
+ FreqtradeException
|
+---+ OperationalException
| |
| +---+ ConfigurationError
|
+---+ DependencyException
| |
@@ -318,6 +320,7 @@ Additional tests / steps to complete:
* Check if balance shows correctly (*)
* Create market order (*)
* Create limit order (*)
* Cancel order (*)
* Complete trade (enter + exit) (*)
* Compare result calculation between exchange and bot
* Ensure fees are applied correctly (check the database against the exchange)
@@ -375,7 +378,7 @@ from pathlib import Path
exchange = ccxt.binance({
'apiKey': '<apikey>',
'secret': '<secret>'
'secret': '<secret>',
'options': {'defaultType': 'swap'}
})
_ = exchange.load_markets()
@@ -418,6 +421,9 @@ This part of the documentation is aimed at maintainers, and shows how to create
### Create release branch
!!! Note
Make sure that the `stable` branch is up-to-date!
First, pick a commit that's about one week old (to not include latest additions to releases).
``` bash
@@ -430,14 +436,11 @@ Determine if crucial bugfixes have been made between this commit and the current
* Merge the release branch (stable) into this branch.
* Edit `freqtrade/__init__.py` and add the version matching the current date (for example `2019.7` for July 2019). Minor versions can be `2019.7.1` should we need to do a second release that month. Version numbers must follow allowed versions from PEP0440 to avoid failures pushing to pypi.
* Commit this part.
* push that branch to the remote and create a PR against the stable branch.
* Push that branch to the remote and create a PR against the **stable branch**.
* Update develop version to next version following the pattern `2019.8-dev`.
### Create changelog from git commits
!!! Note
Make sure that the `stable` branch is up-to-date!
``` bash
# Needs to be done before merging / pulling that branch.
For Binance, it is suggested to add `"BNB/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `BNB` on the account or unless you're willing to disable using `BNB` for fees.
Binance accounts may use `BNB` for fees, and if a trade happens to be on `BNB`, further trades may consume this position and make the initial BNB trade unsellable as the expected amount is not there anymore.
If not enough `BNB` is available to cover transaction fees, then fees will not be covered by `BNB` and no fee reduction will occur. Freqtrade will never buy BNB to cover for fees. BNB needs to be bought and monitored manually to this end.
### Binance sites
Binance has been split into 2, and users must use the correct ccxt exchange ID for their exchange, otherwise API keys are not recognized.
@@ -127,6 +129,8 @@ Freqtrade will not attempt to change these settings.
## Kraken
Kraken supports [time_in_force](configuration.md#understand-order_time_in_force) with settings "GTC" (good till cancelled), "IOC" (immediate-or-cancel) and "PO" (Post only) settings.
!!! Tip "Stoploss on Exchange"
Kraken supports `stoploss_on_exchange` and can use both stop-loss-market and stop-loss-limit orders. It provides great advantages, so we recommend to benefit from it.
You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
@@ -136,6 +140,43 @@ Freqtrade will not attempt to change these settings.
The Kraken API does only provide 720 historic candles, which is sufficient for Freqtrade dry-run and live trade modes, but is a problem for backtesting.
To download data for the Kraken exchange, using `--dl-trades` is mandatory, otherwise the bot will download the same 720 candles over and over, and you'll not have enough backtest data.
To speed up downloading, you can download the [trades zip files](https://support.kraken.com/hc/en-us/articles/360047543791-Downloadable-historical-market-data-time-and-sales-) kraken provides.
These are usually updated once per quarter. Freqtrade expects these files to be placed in `user_data/data/kraken/trades_csv`.
A structure as follows can make sense if using incremental files, with the "full" history in one directory, and incremental files in different directories.
The assumption for this mode is that the data is downloaded and unzipped keeping filenames as they are.
Duplicate content will be ignored (based on timestamp) - though the assumption is that there is no gap in the data.
This means, if your "full" history ends in Q4 2022 - then both incremental updates Q1 2023 and Q2 2023 are available.
Not having this will lead to incomplete data, and therefore invalid results while using the data.
Downloading kraken data will require significantly more memory (RAM) than any other exchange, as the trades-data needs to be converted into candles on your machine.
It will also take a long time, as freqtrade will need to download every single trade that happened on the exchange for the pair / timerange combination, therefore please be patient.
@@ -144,48 +185,6 @@ To download data for the Kraken exchange, using `--dl-trades` is mandatory, othe
Please pay attention that rateLimit configuration entry holds delay in milliseconds between requests, NOT requests\sec rate.
So, in order to mitigate Kraken API "Rate limit exceeded" exception, this configuration should be increased, NOT decreased.
## Bittrex
### Order types
Bittrex does not support market orders. If you have a message at the bot startup about this, you should change order type values set in your configuration and/or in the strategy from `"market"` to `"limit"`. See some more details on this [here in the FAQ](faq.md#im-getting-the-exchange-bittrex-does-not-support-market-orders-message-and-cannot-run-my-strategy).
Bittrex also does not support `VolumePairlist` due to limited / split API constellation at the moment.
Please use `StaticPairlist`. Other pairlists (other than `VolumePairlist`) should not be affected.
### Volume pairlist
Bittrex does not support the direct usage of VolumePairList. This can however be worked around by using the advanced mode with `lookback_days: 1` (or more), which will emulate 24h volume.
Read more in the [pairlist documentation](plugins.md#volumepairlist-advanced-mode).
### Restricted markets
Bittrex split its exchange into US and International versions.
The International version has more pairs available, however the API always returns all pairs, so there is currently no automated way to detect if you're affected by the restriction.
If you have restricted pairs in your whitelist, you'll get a warning message in the log on Freqtrade startup for each restricted pair.
The warning message will look similar to the following:
If you're an "International" customer on the Bittrex exchange, then this warning will probably not impact you.
If you're a US customer, the bot will fail to create orders for these pairs, and you should remove them from your whitelist.
You can get a list of restricted markets by using the following snippet:
``` python
import ccxt
ct = ccxt.bittrex()
lm = ct.load_markets()
res = [p for p, x in lm.items() if 'US' in x['info']['prohibitedIn']]
print(res)
```
## Kucoin
Kucoin requires a passphrase for each api key, you will therefore need to add this key into the configuration so your exchange section looks as follows:
For Kucoin, it is suggested to add `"KCS/<STAKE>"` to your blacklist to avoid issues, unless you are willing to maintain enough extra `KCS` on the account or unless you're willing to disable using `KCS` for fees.
Kucoin accounts may use `KCS` for fees, and if a trade happens to be on `KCS`, further trades may consume this position and make the initial `KCS` trade unsellable as the expected amount is not there anymore.
## Huobi
## HTX (formerly Huobi)
!!! Tip "Stoploss on Exchange"
Huobi supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
HTX supports `stoploss_on_exchange` and uses `stop-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
## OKX (former OKEX)
@@ -265,6 +264,24 @@ We do strongly recommend to limit all API keys to the IP you're going to use it
Bybit (futures only) supports `stoploss_on_exchange` and uses `stop-loss-limit` orders. It provides great advantages, so we recommend to benefit from it by enabling stoploss on exchange.
On futures, Bybit supports both `stop-limit` as well as `stop-market` orders. You can use either `"limit"` or `"market"` in the `order_types.stoploss` configuration setting to decide which type to use.
## Bitmart
Bitmart requires the API key Memo (the name you give the API key) to go along with the exchange key and secret.
It's therefore required to pass the UID as well.
```json
"exchange": {
"name": "bitmart",
"uid": "your_bitmart_api_key_memo",
"secret": "your_exchange_secret",
"password": "your_exchange_api_key_password",
// ...
}
```
!!! Warning "Necessary Verification"
Bitmart requires Verification Lvl2 to successfully trade on the spot market through the API - even though trading via UI works just fine with just Lvl1 verification.
## All exchanges
Should you experience constant errors with Nonce (like `InvalidNonce`), it is best to regenerate the API keys. Resetting Nonce is difficult and it's usually easier to regenerate the API keys.
@@ -128,15 +128,9 @@ This warning can point to one of the below problems:
* Barely traded pair -> Check the pair on the exchange webpage, look at the timeframe your strategy uses. If the pair does not have any volume in some candles (usually visualized with a "volume 0" bar, and a "_" as candle), this pair did not have any trades in this timeframe. These pairs should ideally be avoided, as they can cause problems with order-filling.
* API problem -> API returns wrong data (this only here for completeness, and should not happen with supported exchanges).
### I'm getting the "RESTRICTED_MARKET" message in the log
Currently known to happen for US Bittrex users.
Read [the Bittrex section about restricted markets](exchanges.md#restricted-markets) for more information.
### I'm getting the "Exchange XXX does not support market orders." message and cannot run my strategy
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Bittrex and Gate.io).
As the message says, your exchange does not support market orders and you have one of the [order types](configuration.md/#understand-order_types) set to "market". Your strategy was probably written with other exchanges in mind and sets "market" orders for "stoploss" orders, which is correct and preferable for most of the exchanges supporting market orders (but not for Gate.io).
To fix this, redefine order types in the strategy to use "limit" instead of "market":
@@ -32,6 +32,9 @@ FreqAI is configured through the typical [Freqtrade config file](configuration.m
A full example config is available in `config_examples/config_freqai.example.json`.
!!! Note
The `identifier` is commonly overlooked by newcomers, however, this value plays an important role in your configuration. This value is a unique ID that you choose to describe one of your runs. Keeping it the same allows you to maintain crash resilience as well as faster backtesting. As soon as you want to try a new run (new features, new model, etc.), you should change this value (or delete the `user_data/models/unique-id` folder. More details available in the [parameter table](freqai-parameter-table.md#feature-parameters).
## Building a FreqAI strategy
The FreqAI strategy requires including the following lines of code in the standard [Freqtrade strategy](strategy-customization.md):
@@ -162,7 +165,8 @@ Below are the values you can expect to include/use inside a typical strategy dat
| `df['&*_std/mean']` | Standard deviation and mean values of the defined labels during training (or live tracking with `fit_live_predictions_candles`). Commonly used to understand the rarity of a prediction (use the z-score as shown in `templates/FreqaiExampleStrategy.py` and explained [here](#creating-a-dynamic-target-threshold) to evaluate how often a particular prediction was observed during training or historically with `fit_live_predictions_candles`). <br> **Datatype:** Float.
| `df['do_predict']` | Indication of an outlier data point. The return value is integer between -2 and 2, which lets you know if the prediction is trustworthy or not. `do_predict==1` means that the prediction is trustworthy. If the Dissimilarity Index (DI, see details [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di)) of the input data point is above the threshold defined in the config, FreqAI will subtract 1 from `do_predict`, resulting in `do_predict==0`. If `use_SVM_to_remove_outliers` is active, the Support Vector Machine (SVM, see details [here](freqai-feature-engineering.md#identifying-outliers-using-a-support-vector-machine-svm)) may also detect outliers in training and prediction data. In this case, the SVM will also subtract 1 from `do_predict`. If the input data point was considered an outlier by the SVM but not by the DI, or vice versa, the result will be `do_predict==0`. If both the DI and the SVM considers the input data point to be an outlier, the result will be `do_predict==-1`. As with the SVM, if `use_DBSCAN_to_remove_outliers` is active, DBSCAN (see details [here](freqai-feature-engineering.md#identifying-outliers-with-dbscan)) may also detect outliers and subtract 1 from `do_predict`. Hence, if both the SVM and DBSCAN are active and identify a datapoint that was above the DI threshold as an outlier, the result will be `do_predict==-2`. A particular case is when `do_predict == 2`, which means that the model has expired due to exceeding `expired_hours`. <br> **Datatype:** Integer between -2 and 2.
| `df['DI_values']` | Dissimilarity Index (DI) values are proxies for the level of confidence FreqAI has in the prediction. A lower DI means the prediction is close to the training data, i.e., higher prediction confidence. See details about the DI [here](freqai-feature-engineering.md#identifying-outliers-with-the-dissimilarity-index-di). <br> **Datatype:** Float.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br> **Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%`. <br> **Datatype:** Depends on the output of the model.
| `df['%*']` | Any dataframe column prepended with `%` in `feature_engineering_*()` is treated as a training feature. For example, you can include the RSI in the training feature set (similar to in `templates/FreqaiExampleStrategy.py`) by setting `df['%-rsi']`. See more details on how this is done [here](freqai-feature-engineering.md). <br> **Note:** Since the number of features prepended with `%` can multiply very quickly (10s of thousands of features are easily engineered using the multiplictative functionality of, e.g., `include_shifted_candles` and `include_timeframes` as described in the [parameter table](freqai-parameter-table.md)), these features are removed from the dataframe that is returned from FreqAI to the strategy. To keep a particular type of feature for plotting purposes, you would prepend it with `%%` (see details below). <br> **Datatype:** Depends on the feature created by the user.
| `df['%%*']` | Any dataframe column prepended with `%%` in `feature_engineering_*()` is treated as a training feature, just the same as the above `%` prepend. However, in this case, the features are returned back to the strategy for FreqUI/plot-dataframe plotting and monitoring in Dry/Live/Backtesting <br> **Datatype:** Depends on the feature created by the user. Please note that features created in `feature_engineering_expand()` will have automatic FreqAI naming schemas depending on the expansions that you configured (i.e. `include_timeframes`, `include_corr_pairlist`, `indicators_periods_candles`, `include_shifted_candles`). So if you want to plot `%%-rsi` from `feature_engineering_expand_all()`, the final naming scheme for your plotting config would be: `%%-rsi-period_10_ETH/USDT:USDT_1h` for the `rsi` feature with `period=10`, `timeframe=1h`, and `pair=ETH/USDT:USDT` (the `:USDT` is added if you are using futures pairs). It is useful to simply add `print(dataframe.columns)` in your `populate_indicators()` after `self.freqai.start()` to see the full list of available features that are returned to the strategy for plotting purposes.
@@ -7,7 +7,7 @@ Low level feature engineering is performed in the user strategy within a set of
| Function | Description |
|---------------|-------------|
| `feature_engineering_expand_all()` | This optional function will automatically expand the defined features on the config defined `indicator_periods_candles`, `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `include_periods_candles`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `indicator_periods_candles`.
| `feature_engineering_standard()` | This optional function will be called once with the dataframe of the base timeframe. This is the final function to be called, which means that the dataframe entering this function will contain all the features and columns from the base asset created by the other `feature_engineering_expand` functions. This function is a good place to do custom exotic feature extractions (e.g. tsfresh). This function is also a good place for any feature that should not be auto-expanded upon (e.g., day of the week).
| `set_freqai_targets()` | Required function to set the targets for the model. All targets must be prepended with `&` to be recognized by the FreqAI internals.
@@ -74,9 +74,8 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| | **Reinforcement Learning Parameters within the `freqai.rl_config` sub dictionary**
| `rl_config` | A dictionary containing the control parameters for a Reinforcement Learning model. <br> **Datatype:** Dictionary.
| `train_cycles` | Training time steps will be set based on the `train_cycles * number of training data points. <br> **Datatype:** Integer.
| `cpu_count` | Number of processors to dedicate to the Reinforcement Learning training process. <br> **Datatype:** int.
| `max_trade_duration_candles`| Guides the agent training to keep trades below desired length. Example usage shown in `prediction_models/ReinforcementLearner.py` within the customizable `calculate_reward()` function. <br> **Datatype:** int.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentaiton. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `model_type` | Model string from stable_baselines3 or SBcontrib. Available strings include: `'TRPO', 'ARS', 'RecurrentPPO', 'MaskablePPO', 'PPO', 'A2C', 'DQN'`. User should ensure that `model_training_parameters` match those available to the corresponding stable_baselines3 model by visiting their documentation. [PPO doc](https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html) (external website) <br> **Datatype:** string.
| `policy_type` | One of the available policy types from stable_baselines3 <br> **Datatype:** string.
| `max_training_drawdown_pct` | The maximum drawdown that the agent is allowed to experience during training. <br> **Datatype:** float. <br> Default: 0.8
| `cpu_count` | Number of threads/cpus to dedicate to the Reinforcement Learning training process (depending on if `ReinforcementLearning_multiproc` is selected or not). Recommended to leave this untouched, by default, this value is set to the total number of physical cores minus 1. <br> **Datatype:** int.
@@ -142,7 +142,7 @@ Parameter details can be found [here](freqai-parameter-table.md), but in general
As you begin to modify the strategy and the prediction model, you will quickly realize some important differences between the Reinforcement Learner and the Regressors/Classifiers. Firstly, the strategy does not set a target value (no labels!). Instead, you set the `calculate_reward()` function inside the `MyRLEnv` class (see below). A default `calculate_reward()` is provided inside `prediction_models/ReinforcementLearner.py` to demonstrate the necessary building blocks for creating rewards, but this is *not* designed for production. Users *must* create their own custom reinforcement learning model class or use a pre-built one from outside the Freqtrade source code and save it to `user_data/freqaimodels`. It is inside the `calculate_reward()` where creative theories about the market can be expressed. For example, you can reward your agent when it makes a winning trade, and penalize the agent when it makes a losing trade. Or perhaps, you wish to reward the agent for entering trades, and penalize the agent for sitting in trades too long. Below we show examples of how these rewards are all calculated:
!!! note "Hint"
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occuring at a single point in time.
The best reward functions are ones that are continuously differentiable, and well scaled. In other words, adding a single large negative penalty to a rare event is not a good idea, and the neural net will not be able to learn that function. Instead, it is better to add a small negative penalty to a common event. This will help the agent learn faster. Not only this, but you can help improve the continuity of your rewards/penalties by having them scale with severity according to some linear/exponential functions. In other words, you'd slowly scale the penalty as the duration of the trade increases. This is better than a single large penalty occurring at a single point in time.
This way, you can return to using any model you wish by simply specifying the `identifier`.
!!! Note
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire backtest timerange. This means that you should be sure that features do look-ahead into the future.
Backtesting calls `set_freqai_targets()` one time for each backtest window (where the number of windows is the full backtest timerange divided by the `backtest_period_days` parameter). Doing this means that the targets simulate dry/live behavior without look ahead bias. However, the definition of the features in `feature_engineering_*()` is performed once on the entire training timerange. This means that you should be sure that features do not look-ahead into the future.
More details about look-ahead bias can be found in [Common Mistakes](strategy-customization.md#common-mistakes-when-developing-strategies).
@@ -114,6 +114,11 @@ Here we compile some external materials that provide deeper looks into various c
- [Real-time head-to-head: Adaptive modeling of financial market data using XGBoost and CatBoost](https://emergentmethods.medium.com/real-time-head-to-head-adaptive-modeling-of-financial-market-data-using-xgboost-and-catboost-995a115a7495)
- [FreqAI - from price to prediction](https://emergentmethods.medium.com/freqai-from-price-to-prediction-6fadac18b665)
## Support
You can find support for FreqAI in a variety of places, including the [Freqtrade discord](https://discord.gg/Jd8JYeWHc4), the dedicated [FreqAI discord](https://discord.gg/7AMWACmbjT), and in [github issues](https://github.com/freqtrade/freqtrade/issues).
## Credits
FreqAI is developed by a group of individuals who all contribute specific skillsets to the project.
@@ -337,11 +337,15 @@ There are four parameter types each suited for different purposes.
* `CategoricalParameter` - defines a parameter with a predetermined number of choices.
* `BooleanParameter` - Shorthand for `CategoricalParameter([True, False])` - great for "enable" parameters.
!!! Tip "Disabling parameter optimization"
Each parameter takes two boolean parameters:
* `load` - when set to `False` it will not load values configured in `buy_params` and `sell_params`.
* `optimize` - when set to `False` parameter will not be included in optimization process.
Use these parameters to quickly prototype various ideas.
### Parameter options
There are two parameter options that can help you to quickly test various ideas:
* `optimize` - when set to `False`, the parameter will not be included in optimization process. (Default: True)
* `load` - when set to `False`, results of a previous hyperopt run (in `buy_params` and `sell_params` either in your strategy or the JSON output file) will not be used as the starting value for subsequent hyperopts. The default value specified in the parameter will be used instead. (Default: True)
!!! Tip "Effects of `load=False` on backtesting"
Be aware that setting the `load` option to `False` will mean backtesting will also use the default value specified in the parameter and *not* the value found through hyperoptimisation.
!!! Warning
Hyperoptable parameters cannot be used in `populate_indicators` - as hyperopt does not recalculate indicators for each epoch, so the starting value would be used in this case.
@@ -435,7 +439,7 @@ While this strategy is most likely too simple to provide consistent profit, it s
??? Hint "Performance tip"
During normal hyperopting, indicators are calculated once and supplied to each epoch, linearly increasing RAM usage as a factor of increasing cores. As this also has performance implications, there are two alternatives to reduce RAM usage
* Move `ema_short` and `ema_long` calculations from `populate_indicators()` to `populate_entry_trend()`. Since `populate_entry_trend()` gonna be calculated every epochs, you don't need to use `.range` functionality.
* Move `ema_short` and `ema_long` calculations from `populate_indicators()` to `populate_entry_trend()`. Since `populate_entry_trend()` will be calculated every epoch, you don't need to use `.range` functionality.
* hyperopt provides `--analyze-per-epoch` which will move the execution of `populate_indicators()` to the epoch process, calculating a single value per parameter per epoch instead of using the `.range` functionality. In this case, `.range` functionality will only return the actually used value.
These alternatives will reduce RAM usage, but increase CPU usage. However, your hyperopting run will be less likely to fail due to Out Of Memory (OOM) issues.
@@ -922,6 +926,12 @@ Once the optimized strategy has been implemented into your strategy, you should
To achieve same the results (number of trades, their durations, profit, etc.) as during Hyperopt, please use the same configuration and parameters (timerange, timeframe, ...) used for hyperopt `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
Should results not match, please double-check to make sure you transferred all conditions correctly.
Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy.
You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
### Why do my backtest results not match my hyperopt results?
Should results not match, check the following factors:
* You may have added parameters to hyperopt in `populate_indicators()` where they will be calculated only once **for all epochs**. If you are, for example, trying to optimise multiple SMA timeperiod values, the hyperoptable timeperiod parameter should be placed in `populate_entry_trend()` which is calculated every epoch. See [Optimizing an indicator parameter](https://www.freqtrade.io/en/stable/hyperopt/#optimizing-an-indicator-parameter).
* If you have disabled the auto-export of hyperopt parameters into the JSON parameters file, double-check to make sure you transferred all hyperopted values into your strategy correctly.
* Check the logs to verify what parameters are being set and what values are being used.
* Pay special care to the stoploss, max_open_trades and trailing stoploss parameters, as these are often set in configuration files, which override changes to the strategy. Check the logs of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss`, `max_open_trades` or `trailing_stop`).
* Verify that you do not have an unexpected parameters JSON file overriding the parameters or the default hyperopt settings in your strategy.
* Verify that any protections that are enabled in backtesting are also enabled when hyperopting, and vice versa. When using `--space protection`, protections are auto-enabled for hyperopting.
@@ -6,7 +6,7 @@ In your configuration, you can use Static Pairlist (defined by the [`StaticPairL
Additionally, [`AgeFilter`](#agefilter), [`PrecisionFilter`](#precisionfilter), [`PriceFilter`](#pricefilter), [`ShuffleFilter`](#shufflefilter), [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) act as Pairlist Filters, removing certain pairs and/or moving their positions in the pairlist.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You should always configure either `StaticPairList` or `VolumePairList` as the starting Pairlist Handler.
If multiple Pairlist Handlers are used, they are chained and a combination of all Pairlist Handlers forms the resulting pairlist the bot uses for trading and backtesting. Pairlist Handlers are executed in the sequence they are configured. You can define either `StaticPairList`, `VolumePairList`, `ProducerPairList`, `RemotePairList` or `MarketCapPairList` as the starting Pairlist Handler.
Inactive markets are always removed from the resulting pairlist. Explicitly blacklisted pairs (those in the `pair_blacklist` configuration setting) are also always removed from the resulting pairlist.
@@ -24,6 +24,7 @@ You may also use something like `.*DOWN/BTC` or `.*UP/BTC` to exclude leveraged
* [`VolumePairList`](#volume-pair-list)
* [`ProducerPairList`](#producerpairlist)
* [`RemotePairList`](#remotepairlist)
* [`MarketCapPairList`](#marketcappairlist)
* [`AgeFilter`](#agefilter)
* [`FullTradesFilter`](#fulltradesfilter)
* [`OffsetFilter`](#offsetfilter)
@@ -67,7 +68,7 @@ When used in the leading position of the chain of Pairlist Handlers, the `pair_w
The `refresh_period` setting allows to define the period (in seconds), at which the pairlist will be refreshed. Defaults to 1800s (30 minutes).
The pairlist cache (`refresh_period`) on `VolumePairList` is only applicable to generating pairlists.
Filtering instances (not the first position in the list) will not apply any cache and will always use up-to-date data.
Filtering instances (not the first position in the list) will not apply any cache (beyond caching candles for the duration of the candle in advanced mode) and will always use up-to-date data.
`VolumePairList` is per default based on the ticker data from exchange, as reported by the ccxt library:
@@ -80,12 +81,14 @@ Filtering instances (not the first position in the list) will not apply any cach
"number_assets":20,
"sort_key":"quoteVolume",
"min_value":0,
"max_value":8000000,
"refresh_period":1800
}
],
```
You can define a minimum volume with `min_value` - which will filter out pairs with a volume lower than the specified value in the specified timerange.
In addition to that, you can also define a maximum volume with `max_value` - which will filter out pairs with a volume higher than the specified value in the specified timerange.
##### VolumePairList Advanced mode
@@ -112,8 +115,8 @@ For convenience `lookback_days` can be specified, which will imply that 1d candl
!!! Warning "Performance implications when using lookback range"
If used in first position in combination with lookback, the computation of the range based volume can be time and resource consuming, as it downloads candles for all tradable pairs. Hence it's highly advised to use the standard approach with `VolumeFilter` to narrow the pairlist down for further range volume calculation.
??? Tip "Unsupported exchanges (Bittrex, Gemini)"
On some exchanges (like Bittrex and Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
??? Tip "Unsupported exchanges"
On some exchanges (like Gemini), regular VolumePairList does not work as the api does not natively provide 24h volume. This can be worked around by using candle data to build the volume.
To roughly simulate 24h volume, you can use the following configuration.
Please note that These pairlists will only refresh once per day.
@@ -192,14 +195,15 @@ The RemotePairList is defined in the pairlists section of the configuration sett
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"bearer_token": "my-bearer-token"
"bearer_token": "my-bearer-token",
"save_to_file": "user_data/filename.json"
}
]
```
The optional `mode` option specifies if the pairlist should be used as a `blacklist` or as a `whitelist`. The default value is "whitelist".
The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append".
The optional `processing_mode` option in the RemotePairList configuration determines how the retrieved pairlist is processed. It can have two values: "filter" or "append". The default value is "filter".
In "filter" mode, the retrieved pairlist is used as a filter. Only the pairs present in both the original pairlist and the retrieved pairlist are included in the final pairlist. Other pairs are filtered out.
@@ -207,6 +211,42 @@ In "append" mode, the retrieved pairlist is added to the original pairlist. All
The `pairlist_url` option specifies the URL of the remote server where the pairlist is located, or the path to a local file (if file:/// is prepended). This allows the user to use either a remote server or a local file as the source for the pairlist.
The `save_to_file` option, when provided with a valid filename, saves the processed pairlist to that file in JSON format. This option is optional, and by default, the pairlist is not saved to a file.
??? Example "Multi bot with shared pairlist example"
`save_to_file` can be used to save the pairlist to a file with Bot1:
```json
"pairlists": [
{
"method": "RemotePairList",
"mode": "whitelist",
"pairlist_url": "https://example.com/pairlist",
"number_assets": 10,
"refresh_period": 1800,
"keep_pairlist_on_failure": true,
"read_timeout": 60,
"save_to_file": "user_data/filename.json"
}
]
```
This saved pairlist file can be loaded by Bot2, or any additional bot with this configuration:
The user is responsible for providing a server or local file that returns a JSON object with the following structure:
```json
@@ -227,6 +267,25 @@ The optional `bearer_token` will be included in the requests Authorization Heade
!!! Note
In case of a server error the last received pairlist will be kept if `keep_pairlist_on_failure` is set to true, when set to false a empty pairlist is returned.
#### MarketCapPairList
`MarketCapPairList` employs sorting/filtering of pairs by their marketcap rank based of CoinGecko. It will only recognize coins up to the coin placed at rank 250. The returned pairlist will be sorted based of their marketcap ranks.
```json
"pairlists": [
{
"method": "MarketCapPairList",
"number_assets": 20,
"max_rank": 50,
"refresh_period": 86400
}
]
```
`number_assets` defines the maximum number of pairs returned by the pairlist. `max_rank` will determine the maximum rank used in creating/filtering the pairlist. It's expected that some coins within the top `max_rank` marketcap will not be included in the resulting pairlist since not all pairs will have active trading pairs in your preferred market/stake/exchange combination.
`refresh_period` setting defines the period (in seconds) at which the marketcap rank data will be refreshed. Defaults to 86,400s (1 day). The pairlist cache (`refresh_period`) is applicable on both generating pairlists (first position in the list) and filtering instances (not the first position in the list).
#### AgeFilter
Removes pairs that have been listed on the exchange for less than `min_days_listed` days (defaults to `10`) or more than `max_days_listed` days (defaults `None` mean infinity).
@@ -312,6 +371,11 @@ As this Filter uses past performance of the bot, it'll have some startup-period
Filters low-value coins which would not allow setting stoplosses.
Namely, pairs are blacklisted if a variance of one percent or more in the stop price would be caused by precision rounding on the exchange, i.e. `rounded(stop_price) <= rounded(stop_price * 0.99)`. The idea is to avoid coins with a value VERY close to their lower trading boundary, not allowing setting of proper stoploss.
!!! Tip "PerformanceFilter is pointless for futures trading"
The above does not apply to shorts. And for longs, in theory the trade will be liquidated first.
!!! Warning "Backtesting"
`PrecisionFilter` does not support backtesting mode using multiple strategies.
@@ -393,6 +457,8 @@ If the trading range over the last 10 days is <1% or >99%, remove the pair from
]
```
Adding `"sort_direction": "asc"` or `"sort_direction": "desc"` enables sorting for this pairlist.
!!! Tip
This Filter can be used to automatically remove stable coin pairs, which have a very low trading range, and are therefore extremely difficult to trade with profit.
Additionally, it can also be used to automatically remove pairs with extreme high/low variance over a given amount of time.
@@ -403,7 +469,7 @@ Volatility is the degree of historical variation of a pairs over time, it is mea
This filter removes pairs if the average volatility over a `lookback_days` days is below `min_volatility` or above `max_volatility`. Since this is a filter that requires additional data, the results are cached for `refresh_period`.
This filter can be used to narrow down your pairs to a certain volatility or avoid very volatile pairs.
This filter can be used to narrow down your pairs to a certain volatility or avoid very volatile pairs.
In the below example:
If the volatility over the last 10 days is not in the range of 0.05-0.50, remove the pair from the whitelist. The filter is applied every 24h.
@@ -420,6 +486,8 @@ If the volatility over the last 10 days is not in the range of 0.05-0.50, remove
]
```
Adding `"sort_direction": "asc"` or `"sort_direction": "desc"` enables sorting mode for this pairlist.
### Full example of Pairlist Handlers
The below example blacklists `BNB/BTC`, uses `VolumePairList` with `20` assets, sorting pairs by `quoteVolume` and applies [`PrecisionFilter`](#precisionfilter) and [`PriceFilter`](#pricefilter), filtering all assets where 1 price unit is > 1%. Then the [`SpreadFilter`](#spreadfilter) and [`VolatilityFilter`](#volatilityfilter) is applied and pairs are finally shuffled with the random seed set to some predefined value.
@@ -40,9 +40,9 @@ Freqtrade is a free and open source crypto trading bot written in Python. It is
Please read the [exchange specific notes](exchanges.md) to learn about eventual, special configurations needed for each exchange.
- [X] [Binance](https://www.binance.com/)
- [X] [Bittrex](https://bittrex.com/)
- [X] [Bitmart](https://bitmart.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [Huobi](http://huobi.com/)
- [X] [HTX](https://www.htx.com/) (Former Huobi)
- [X] [Kraken](https://kraken.com/)
- [X] [OKX](https://okx.com/) (Former OKEX)
- [ ] [potentially many others through <img alt="ccxt" width="30px" src="assets/ccxt-logo.svg" />](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_
@@ -95,11 +95,13 @@ Make sure that the following 2 lines are available in your docker-compose file:
### Consuming the API
You can consume the API by using the script `scripts/rest_client.py`.
The client script only requires the `requests` module, so Freqtrade does not need to be installed on the system.
You can consume the API by using `freqtrade-client` (also available as `scripts/rest_client.py`).
This command can be installed independent of the bot by using `pip install freqtrade-client`.
This module is designed to be lightweight, and only depends on the `requests` and `python-rapidjson` modules, skipping all heavy dependencies freqtrade otherwise needs.
By default, the script assumes `127.0.0.1` (localhost) and port `8080` to be used, however you can specify a configuration file to override this behaviour.
@@ -120,9 +122,27 @@ By default, the script assumes `127.0.0.1` (localhost) and port `8080` to be use
| `reload_config` | Reloads the configuration file.
| `trades` | List last trades. Limited to 500 trades per call.
| `trade/<tradeid>` | Get specific trade.
| `trade/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trade/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trade/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `trades/<tradeid>` | DELETE - Remove trade from the database. Tries to close open orders. Requires manual handling of this trade on the exchange.
| `trades/<tradeid>/open-order` | DELETE - Cancel open order for this trade.
| `trades/<tradeid>/reload` | GET - Reload a trade from the Exchange. Only works in live, and can potentially help recover a trade that was manually sold on the exchange.
| `show_config` | Shows part of the current configuration with relevant settings to operation.
| `logs` | Shows last log messages.
| `status` | Lists all open trades.
| `count` | Displays number of trades used and available.
| `entries [pair]` | Shows profit statistics for each enter tags for given pair (or all pairs if pair isn't given). Pair is optional.
| `exits [pair]` | Shows profit statistics for each exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `mix_tags [pair]` | Shows profit statistics for each combinations of enter tag + exit reasons for given pair (or all pairs if pair isn't given). Pair is optional.
| `locks` | Displays currently locked pairs.
| `delete_lock <lock_id>` | Deletes (disables) the lock by id.
| `profit` | Display a summary of your profit/loss from close trades and some stats about your performance.
@@ -11,34 +11,129 @@ The call sequence of the methods described here is covered under [bot execution
!!! Tip
Start off with a strategy template containing all available callback methods by running `freqtrade new-strategy --strategy MyAwesomeStrategy --template advanced`
## Storing information
## Storing information (Persistent)
Storing information can be accomplished by creating a new dictionary within the strategy class.
Freqtrade allows storing/retrieving user custom information associated with a specific trade in the database.
The name of the variable can be chosen at will, but should be prefixed with `custom_` to avoid naming collisions with predefined strategy variables.
Using a trade object, information can be stored using `trade.set_custom_data(key='my_key', value=my_value)` and retrieved using `trade.get_custom_data(key='my_key')`. Each data entry is associated with a trade and a user supplied key (of type `string`). This means that this can only be used in callbacks that also provide a trade object.
For the data to be able to be stored within the database, freqtrade must serialized the data. This is done by converting the data to a JSON formatted string.
Freqtrade will attempt to reverse this action on retrieval, so from a strategy perspective, this should not be relevant.
The data is not persisted after a bot-restart (or config-reload). Also, the amount of data should be kept smallish (no DataFrames and such), otherwise the bot will start to consume a lot of memory and eventually run out of memory and crash.
The above is a simple example - there are simpler ways to retrieve trade data like entry-adjustments.
!!! Note
If the data is pair-specific, make sure to use pair as one of the keys in the dictionary.
It is recommended that simple data types are used `[bool, int, float, str]` to ensure no issues when serializing the data that needs to be stored.
Storing big junks of data may lead to unintended side-effects, like a database becoming big (and as a consequence, also slow).
!!! Warning "Non-serializable data"
If supplied data cannot be serialized a warning is logged and the entry for the specified `key` will contain `None` as data.
??? Note "All attributes"
custom-data has the following accessors through the Trade object (assumed as `trade` below):
*`trade.get_custom_data(key='something', default=0)` - Returns the actual value given in the type provided.
*`trade.get_custom_data_entry(key='something')` - Returns the entry - including metadata. The value is accessible via `.value` property.
*`trade.set_custom_data(key='something', value={'some': 'value'})` - set or update the corresponding key for this trade. Value must be serializable - and we recommend to keep the stored data relatively small.
"value" can be any type (both in setting and receiving) - but must be json serializable.
## Storing information (Non-Persistent)
!!! Warning "Deprecated"
This method of storing information is deprecated and we do advise against using non-persistent storage.
Please use [Persistent Storage](#storing-information-persistent) instead.
It's content has therefore been collapsed.
??? Abstract "Storing information"
Storing information can be accomplished by creating a new dictionary within the strategy class.
The name of the variable can be chosen at will, but should be prefixed with `custom_` to avoid naming collisions with predefined strategy variables.
The data is not persisted after a bot-restart (or config-reload). Also, the amount of data should be kept smallish (no DataFrames and such), otherwise the bot will start to consume a lot of memory and eventually run out of memory and crash.
!!! Note
If the data is pair-specific, make sure to use pair as one of the keys in the dictionary.
@@ -760,26 +761,38 @@ The `position_adjustment_enable` strategy property enables the usage of `adjust_
For performance reasons, it's disabled by default and freqtrade will show a warning message on startup if enabled.
`adjust_trade_position()` can be used to perform additional orders, for example to manage risk with DCA (Dollar Cost Averaging) or to increase or decrease positions.
`max_entry_position_adjustment` property is used to limit the number of additional buys per trade (on top of the first buy) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment buys.
The strategy is expected to return a stake_amount (in stake currency) between `min_stake` and `max_stake` if and when an additional buy order should be made (position is increased).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
Additional orders also result in additional fees and those orders don't count towards `max_open_trades`.
This callback is **not** called when there is an open order (either buy or sell) waiting for execution.
`adjust_trade_position()` is called very frequently for the duration of a trade, so you must keep your implementation as performant as possible.
Additional Buys are ignored once you have reached the maximum amount of extra buys that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade.
Adjustment orders can be assigned with a tag by returning a 2 element Tuple, with the first element being the adjustment amount, and the 2nd element the tag (e.g. `return 250, 'increase_favorable_conditions'`).
Position adjustments will always be applied in the direction of the trade, so a positive value will always increase your position (negative values will decrease your position), no matter if it's a long or short trade. Modifications to leverage are not possible, and the stake-amount is assumed to be before applying leverage.
Modifications to leverage are not possible, and the stake-amount returned is assumed to be before applying leverage.
### Increase position
The strategy is expected to return a positive **stake_amount** (in stake currency) between `min_stake` and `max_stake` if and when an additional entry order should be made (position is increased -> buy order for long trades, sell order for short trades).
If there are not enough funds in the wallet (the return value is above `max_stake`) then the signal will be ignored.
`max_entry_position_adjustment` property is used to limit the number of additional entries per trade (on top of the first entry order) that the bot can execute. By default, the value is -1 which means the bot have no limit on number of adjustment entries.
Additional entries are ignored once you have reached the maximum amount of extra entries that you have set on `max_entry_position_adjustment`, but the callback is called anyway looking for partial exits.
### Decrease position
The strategy is expected to return a negative stake_amount (in stake currency) for a partial exit.
Returning the full owned stake at that point (`-trade.stake_amount`) results in a full exit.
Returning a value more than the above (so remaining stake_amount would become negative) will result in the bot ignoring the signal.
!!! Note "About stake size"
Using fixed stake size means it will be the amount used for the first order, just like without position adjustment.
If you wish to buy additional orders with DCA, then make sure to leave enough funds in the wallet for that.
Using 'unlimited' stake amount with DCA orders requires you to also implement the `custom_stake_amount()` callback to avoid allocating all funds to the initial order.
!!! Warning
!!! Warning "Stoploss calculation"
Stoploss is still calculated from the initial opening price, not averaged price.
Regular stoploss rules still apply (cannot move down).
@@ -789,6 +802,11 @@ Position adjustments will always be applied in the direction of the trade, so a
During backtesting this callback is called for each candle in `timeframe` or `timeframe_detail`, so run-time performance will be affected.
This can also cause deviating results between live and backtesting, since backtesting can adjust the trade only once per candle, whereas live could adjust the trade multiple times per candle.
!!! Warning "Performance with many position adjustments"
Position adjustments can be a good approach to increase a strategy's output - but it can also have drawbacks if using this feature extensively.
Each of the orders will be attached to the trade object for the duration of the trade - hence increasing memory usage.
Trades with long duration and 10s or even 100ds of position adjustments are therefore not recommended, and should be closed at regular intervals to not affect performance.
``` python
from freqtrade.persistence import Trade
@@ -822,7 +840,8 @@ class DigDeeperStrategy(IStrategy):
@@ -1003,3 +1023,33 @@ class AwesomeStrategy(IStrategy):
All profit calculations include leverage. Stoploss / ROI also include leverage in their calculation.
Defining a stoploss of 10% at 10x leverage would trigger the stoploss with a 1% move to the downside.
## Order filled Callback
The `order_filled()` callback may be used to perform specific actions based on the current trade state after an order is filled.
It will be called independent of the order type (entry, exit, stoploss or position adjustment).
Assuming that your strategy needs to store the high value of the candle at trade entry, this is possible with this callback as the following example show.
Additional technical libraries can be installed as necessary, or custom indicators may be written / invented by the strategy author.
@@ -173,7 +173,7 @@ You can use [recursive-analysis](recursive-analysis.md) to check and find the co
In this example strategy, this should be set to 400 (`startup_candle_count = 400`), since the minimum needed history for ema100 calculation to make sure the value is correct is 400 candles.
By letting the bot know how much history is needed, backtest trades can start at the specified timerange during backtesting and hyperopt.
@@ -367,6 +367,11 @@ class AwesomeStrategy(IStrategy):
}
```
??? info "Orders that don't fill immediately"
`minimal_roi` will take the `trade.open_date` as reference, which is the time the trade was initialized / the first order for this trade was placed.
This will also hold true for limit orders that don't fill immediately (usually in combination with "off-spot" prices through `custom_entry_price()`), as well as for cases where the initial order is replaced through `adjust_entry_price()`.
The time used will still be from the initial `trade.open_date` (when the initial order was first placed), not from the newly placed order date.
### Stoploss
Setting a stoploss is highly recommended to protect your capital from strong moves against you.
@@ -486,17 +491,18 @@ for more information.
:param timeframe: Informative timeframe. Must always be equal or higher than strategy timeframe.
:param asset: Informative asset, for example BTC, BTC/USDT, ETH/BTC. Do not specify to use
current pair.
current pair. Also supports limited pair format strings (see below)
:param fmt: Column format (str) or column formatter (callable(name, asset, timeframe)). When not
specified, defaults to:
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {column}_{timeframe} if asset is not specified.
Format string supports these format variables:
* {asset} - full name of the asset, for example 'BTC/USDT'.
Pair format supports these format variables:
* {base} - base currency in lower case, for example 'eth'.
* {BASE} - same as {base}, except in upper case.
* {quote} - quote currency in lower case, for example 'usdt'.
* {QUOTE} - same as {quote}, except in upper case.
Format string additionally supports this variables.
* {asset} - full name of the asset, for example 'BTC/USDT'.
* {column} - name of dataframe column.
* {timeframe} - timeframe of informative dataframe.
:param ffill: ffill dataframe after merging informative pair.
@@ -1003,11 +1009,15 @@ This is a common pain-point, which can cause huge differences between backtestin
The following lists some common patterns which should be avoided to prevent frustration:
- don't use `shift(-1)`. This uses data from the future, which is not available.
- don't use `.iloc[-1]` or any other absolute position in the dataframe, this will be different between dry-run and backtesting.
- don't use `shift(-1)` or other negative values. This uses data from the future in backtesting, which is not available in dry or live modes.
- don't use `.iloc[-1]` or any other absolute position in the dataframe within `populate_` functions, as this will be different between dry-run and backtesting. Absolute `iloc` indexing is safe to use in callbacks however - see [Strategy Callbacks](strategy-callbacks.md).
- don't use `dataframe['volume'].mean()`. This uses the full DataFrame for backtesting, including data from the future. Use `dataframe['volume'].rolling(<window>).mean()` instead
- don't use `.resample('1h')`. This uses the left border of the interval, so moves data from an hour to the start of the hour. Use `.resample('1h', label='right')` instead.
!!! Tip "Identifying problems"
You may also want to check the 2 helper commands [lookahead-analysis](lookahead-analysis.md) and [recursive-analysis](recursive-analysis.md), which can each help you figure out problems with your strategy in different ways.
Please treat them as what they are - helpers to identify most common problems. A negative result of each does not guarantee that there's none of the above errors included.
### Colliding signals
When conflicting signals collide (e.g. both `'enter_long'` and `'exit_long'` are 1), freqtrade will do nothing and ignore the entry signal. This will avoid trades that enter, and exit immediately. Obviously, this can potentially lead to missed entries.
@@ -59,7 +59,7 @@ For the Freqtrade configuration, you can then use the the full value (including
"chat_id": "-1001332619709"
```
!!! Warning "Using telegram groups"
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasent surprises.
When using telegram groups, you're giving every member of the telegram group access to your freqtrade bot and to all commands possible via telegram. Please make sure that you can trust everyone in the telegram group to avoid unpleasant surprises.
## Control telegram noise
@@ -175,11 +175,13 @@ official commands. You can ask at any moment for help with `/help`.
| `/status` | Lists all open trades
| `/status <trade_id>` | Lists one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/status table` | List all open trades in a table format. Pending buy orders are marked with an asterisk (*) Pending sell orders are marked with a double asterisk (**)
| `/order <trade_id>` | Lists orders of one or more specific trade. Separate multiple <trade_id> with a blank space.
| `/trades [limit]` | List all recently closed trades in a table format.
| `/count` | Displays number of trades used and available
| `/locks` | Show currently locked pairs.
| `/unlock <pair or lock_id>` | Remove the lock for this pair (or for this lock id).
| `/marketdir [long | short | even | none]` | Updates the user managed variable that represents the current market direction. If no direction is provided, the currently set direction will be displayed.
| `/list_custom_data <trade_id> [key]` | List custom_data for Trade ID & Key combination. If no Key is supplied it will list all key-value pairs found for that Trade ID.
| **Modify Trade states** |
| `/forceexit <trade_id> | /fx <tradeid>` | Instantly exits the given trade (Ignoring `minimum_roi`).
| `/forceexit all | /fx all` | Instantly exits all open trades (Ignoring `minimum_roi`).
@@ -6,7 +6,7 @@ To update your freqtrade installation, please use one of the below methods, corr
Breaking changes / changed behavior will be documented in the changelog that is posted alongside every release.
For the develop branch, please follow PR's to avoid being surprised by changes.
## docker
## Docker
!!! Note "Legacy installations using the `master` image"
We're switching from master to stable for the release Images - please adjust your docker-file and replace `freqtradeorg/freqtrade:master` with `freqtradeorg/freqtrade:stable`
Show configuration file (with sensitive values redacted by default).
Especially useful with [split configuration files](configuration.md#multiple-configuration-files) or [environment variables](configuration.md#environment-variables), where this command will show the merged configuration.
Output reduced for clarity - supported and available exchanges may change over time.
!!! Note "missing opt exchanges"
Values with "missing opt:" might need special configuration (e.g. using orderbook if `fetchTickers` is missing) - but should in theory work (although we cannot guarantee they will).
* Example: see all exchanges supported by the ccxt library (including 'bad' ones, i.e. those that are known to not work with Freqtrade):
Example: see all exchanges supported by the ccxt library (including 'bad' ones, i.e. those that are known to not work with Freqtrade)
@@ -65,7 +65,7 @@ You can set the POST body format to Form-Encoded (default), JSON-Encoded, or raw
The result would be a POST request with e.g. `{"text":"Status: running"}` body and `Content-Type: application/json` header which results `Status: running` message in the Mattermost channel.
When using the Form-Encoded or JSON-Encoded configuration you can configure any number of payload values, and both the key and value will be ouput in the POST request. However, when using the raw data format you can only configure one value and it **must** be named `"data"`. In this instance the data key will not be output in the POST request, only the value. For example:
When using the Form-Encoded or JSON-Encoded configuration you can configure any number of payload values, and both the key and value will be output in the POST request. However, when using the raw data format you can only configure one value and it **must** be named `"data"`. In this instance the data key will not be output in the POST request, only the value. For example:
```json
"webhook":{
@@ -134,6 +134,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -155,6 +156,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -176,6 +178,7 @@ Possible parameters are:
*`stake_amount`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`order_type`
*`current_rate`
@@ -199,6 +202,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -224,6 +228,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -249,6 +254,7 @@ Possible parameters are:
*`profit_ratio`
*`stake_currency`
*`base_currency`
*`quote_currency`
*`fiat_currency`
*`exit_reason`
*`order_type`
@@ -302,6 +308,7 @@ You can configure this as follows:
```
The above represents the default (`exit_fill` and `entry_fill` are optional and will default to the above configuration) - modifications are obviously possible.
To disable either of the two default values (`entry_fill` / `exit_fill`), you can assign them an empty array (`exit_fill: []`).
Available fields correspond to the fields for webhooks and are documented in the corresponding webhook sections.
Install ta-lib according to the [ta-lib documentation](https://github.com/mrjbq7/ta-lib#windows).
Install ta-lib according to the [ta-lib documentation](https://github.com/TA-Lib/ta-lib-python#windows).
As compiling from source on windows has heavy dependencies (requires a partial visual studio installation), Freqtrade provides these dependencies (in the binary wheel format) for the latest 3 Python versions (3.9, 3.10 and 3.11) and for 64bit Windows.
These Wheels are also used by CI running on windows, and are therefore tested together with freqtrade.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Matthias
