bbgo_origin/doc/topics/developing-strategy.md

Developing Strategy

There are two types of strategies in BBGO:

1. built-in strategy: like grid, bollmaker, pricealert strategies, which are included in the pre-compiled binary.
2. external strategy: custom or private strategies that you don't want to expose to public.

For built-in strategies, they are placed in pkg/strategy of the BBGO source repository.

For external strategies, you can create a private repository as an isolated go package and place your strategy inside it.

In general, strategies are Go struct, defined in the Go package.

Quick Start

To add your first strategy, the fastest way is to add it as a built-in strategy.

Simply edit pkg/cmd/builtin.go and import your strategy package there.

When BBGO starts, the strategy will be imported as a package, and register its struct to the engine.

You can also create a new file called pkg/cmd/builtin_short.go and import your strategy package.

import (
    _ "github.com/c9s/bbgo/pkg/strategy/short"
)

Create a directory for your new strategy in the BBGO source code repository:

mkdir -p pkg/strategy/short

Open a new file at pkg/strategy/short/strategy.go and paste the simplest strategy code:

package short

import (
	"context"
	"fmt"

	"github.com/c9s/bbgo/pkg/bbgo"
	"github.com/c9s/bbgo/pkg/types"
)

const ID = "short"

func init() {
	// Register our struct type to BBGO
	// Note that you don't need to field the fields.
	// BBGO uses reflect to parse your type information.
	bbgo.RegisterStrategy(ID, &Strategy{})
}

type Strategy struct {
    Symbol string `json:"symbol"`
    Interval types.Interval `json:"interval"`
}

func (s *Strategy) Subscribe(session *bbgo.ExchangeSession) {
	session.Subscribe(types.KLineChannel, s.Symbol, types.SubscribeOptions{Interval: s.Interval})
}

func (s *Strategy) Run(ctx context.Context, orderExecutor bbgo.OrderExecutor, session *bbgo.ExchangeSession) error {
	session.MarketDataStream.OnKLineClosed(func(k types.KLine) {
	    fmt.Println(k)
	})
    return nil
}

This is the most simple strategy with only ~30 lines code, it subscribes to the kline channel with the given symbol from the config, And when the kline is closed, it prints the kline to the console.

Note that, when Run() is executed, the user data stream is not connected to the exchange yet, but the history market data is already loaded, so if you need to submit an order on start, be sure to write your order submit code inside the event closures like OnKLineClosed or OnStart.

Now you can prepare your config file, create a file called bbgo.yaml with the following content:

exchangeStrategies:
- on: binance
  short:
    symbol: ETHUSDT
    interval: 1m

And then, you should be able to run this strategy by running the following command:

go run ./cmd/bbgo run

The Strategy Struct

BBGO loads the YAML config file and re-unmarshal the settings into your struct as JSON string, so you can define the json tag to get the settings from the YAML config.

For example, if you're writing a strategy in a package called short, to load the following config:

externalStrategies:
- on: binance
  short:
    symbol: BTCUSDT

You can write the following struct to load the symbol setting:

package short

type Strategy struct {
    Symbol string `json:"symbol"`
}

To use the Symbol setting, you can get the value from the Run method of the strategy:

func (s *Strategy) Run(ctx context.Context, session *bbgo.ExchangeSession) error {
    // you need to import the "log" package
    log.Println("%s", s.Symbol)
    return nil
}

Now you have the Go struct and the Go package, but BBGO does not know your strategy, so you need to register your strategy.

Define an ID const in your package:

const ID = "short"

Then call bbgo.RegisterStrategy with the ID you just defined and a struct reference:

func init() {
	bbgo.RegisterStrategy(ID, &Strategy{})
}

Note that you don't need to fill the fields in the struct, BBGO just need to know the type of struct.

(BBGO use reflect to parse the fields from the given struct and allocate a new struct object from the given struct type internally)

Exchange Session

The *bbgo.ExchangeSession represents a connectivity to a crypto exchange, it's also a hub that connects to everything you need, for example, standard indicators, account information, balance information, market data stream, user data stream, exchange APIs, and so on.

By default, BBGO checks the environment variables that you defined to detect which exchange session to be created.

For example, environment variables like BINANCE_API_KEY, BINANCE_API_SECRET will be transformed into an exchange session that connects to Binance.

You can not only connect to multiple different crypt exchanges, but also create multiple sessions to the same crypto exchange with few different options.

To do that, add the following section to your bbgo.yaml config file:

---
sessions:
  binance:
    exchange: binance
    envVarPrefix: binance
  binance_cross_margin:
    exchange: binance
    envVarPrefix: binance
    margin: true
  binance_margin_ethusdt:
    exchange: binance
    envVarPrefix: binance
    margin: true
    isolatedMargin: true
    isolatedMarginSymbol: ETHUSDT
  okex1:
    exchange: okex
    envVarPrefix: okex
  okex2:
    exchange: okex
    envVarPrefix: okex

You can specify which exchange session you want to mount for each strategy in the config file, it's quiet simple:

exchangeStrategies:

- on: binance_margin_ethusdt
  short:
    symbol: ETHUSDT

- on: binance_margin
  foo:
    symbol: ETHUSDT

- on: binance
  bar:
    symbol: ETHUSDT

Market Data Stream and User Data Stream

When BBGO connects to the exchange, it allocates two stream objects for different purposes.

They are:

• MarketDataStream receives market data from the exchange, for example, KLine data (candlestick, or bars), market public trades.
• UserDataStream receives your personal trading data, for example, orders, executed trades, balance updates and other private information.

To add your market data subscription to the MarketDataStream, you can register your subscription in the Subscribe of the strategy code, for example:

func (s *Strategy) Subscribe(session *bbgo.ExchangeSession) {
	session.Subscribe(types.KLineChannel, s.Symbol, types.SubscribeOptions{Interval: "1m"})
}

Since the back-test engine is a kline-based engine, to subscribe market trades, you need to check if you're in the back-test environment:

func (s *Strategy) Subscribe(session *bbgo.ExchangeSession) {
    if !bbgo.IsBackTesting {
        session.Subscribe(types.MarketTradeChannel, s.Symbol, types.SubscribeOptions{})
    }
}

To receive the market data from the market data stream, you need to register the event callback:

func (s *Strategy) Run(ctx context.Context, orderExecutor bbgo.OrderExecutor, session *bbgo.ExchangeSession) error {
	session.MarketDataStream.OnKLineClosed(func(kline types.KLine) {
	    // handle closed kline event here
	})
	session.MarketDataStream.OnMarketTrade(func(trade types.Trade) {
	    // handle market trade event here
	})
}

In the above example, we register our event callback to the market data stream of the current exchange session, The market data stream object here is a session-wide market data stream, so it's shared with other strategies that are also using the same exchange session, so you might receive kline with different symbol or interval.

It's better to add a condition to filter the kline events:

func (s *Strategy) Run(ctx context.Context, orderExecutor bbgo.OrderExecutor, session *bbgo.ExchangeSession) error {
    session.MarketDataStream.OnKLineClosed(func(kline types.KLine) {
        if kline.Symbol != s.Symbol || kline.Interval != s.Interval {
			return
        }
		// handle your kline here
    })
}

You can also use the KLineWith method to wrap your kline closure with the filter condition:

func (s *Strategy) Run(ctx context.Context, orderExecutor bbgo.OrderExecutor, session *bbgo.ExchangeSession) error {
	session.MarketDataStream.OnKLineClosed(types.KLineWith("BTCUSDT", types.Interval1m, func(kline types.KLine) {
	    // handle your kline here
	})
}

Note that, when the Run() method is executed, the user data stream and market data stream are not connected yet.

Submitting Orders

To place an order, you can call SubmitOrders exchange API:

createdOrders, err := session.Exchange.SubmitOrders(ctx, types.SubmitOrder{
    Symbol:   "BTCUSDT",
    Type:     types.OrderTypeLimit,
    Price:    fixedpoint.NewFromFloat(18000.0),
    Quantity: fixedpoint.NewFromFloat(1.0),
})
if err != nil {
    log.WithError(err).Errorf("can not submit orders")
}

log.Infof("createdOrders: %+v", createdOrders)

There are some pre-defined order types you can use:

• types.OrderTypeLimit
• types.OrderTypeMarket
• types.OrderTypeStopMarket
• types.OrderTypeStopLimit
• types.OrderTypeLimitMaker - forces the order to be a maker.

Although it's crypto market, the above order types are actually derived from the stock market:

A limit order is an order to buy or sell a stock with a restriction on the maximum price to be paid or the minimum price to be received (the "limit price"). If the order is filled, it will only be at the specified limit price or better. However, there is no assurance of execution. A limit order may be appropriate when you think you can buy at a price lower than--or sell at a price higher than -- the current quote.

A market order is an order to buy or sell a stock at the market's current best available price. A market order typically ensures an execution, but it does not guarantee a specified price. Market orders are optimal when the primary goal is to execute the trade immediately. A market order is generally appropriate when you think a stock is priced right, when you are sure you want a fill on your order, or when you want an immediate execution.

A stop order is an order to buy or sell a stock at the market price once the stock has traded at or through a specified price (the "stop price"). If the stock reaches the stop price, the order becomes a market order and is filled at the next available market price.

UserDataStream

UserDataStream is an authenticated connection to the crypto exchange. You can receive the following data type from the user data stream:

• OrderUpdate
• TradeUpdate
• BalanceUpdate

When you submit an order to the exchange, you might want to know when the order is filled or not, user data stream is the real time notification let you receive the order update event.

To get the order update from the user data stream:

session.UserDataStream.OnOrderUpdate(func(order types.Order) {
	if order.Status == types.OrderStatusFilled {
		log.Infof("your order is filled: %+v", order)
    }
})

However, order update only contains status, price, quantity of the order, if you're submitting market order, you won't know the actual price of the order execution.

One order can be filled by different size trades from the market, by collecting the trades, you can calculate the average price of the order execution and the total trading fee that you used for the order.

If you need to get the details of the trade execution. you need the trade update event:

session.UserDataStream.OnTrade(func(trade types.Trade) {
    log.Infof("trade price %f, fee %f %s", trade.Price.Float64(), trade.Fee.Float64(), trade.FeeCurrency)
})

To monitor your balance change, you can use the balance update event callback:

session.UserDataStream.OnBalanceUpdate(func(balances types.BalanceMap) {
    log.Infof("balance update: %+v", balances)
})

Note that, as we mentioned above, the user data stream is a session-wide stream, that means you might receive the order update event for other strategies.

To prevent that, you need to manage your active order for your strategy:

activeBook := bbgo.NewActiveOrderBook("BTCUSDT")
activeBook.Bind(session.UserDataStream)

Then, when you create some orders, you can register your order to the active order book, so that it can manage the order update:

createdOrders, err := session.Exchange.SubmitOrders(ctx, types.SubmitOrder{
    Symbol:   "BTCUSDT",
    Type:     types.OrderTypeLimit,
    Price:    fixedpoint.NewFromFloat(18000.0),
    Quantity: fixedpoint.NewFromFloat(1.0),
})
if err != nil {
    log.WithError(err).Errorf("can not submit orders")
}

activeBook.Add(createdOrders...)

Notification

You can use the notification API to send notification to Telegram or Slack:

bbgo.Notify(message)
bbgo.Notify(message, objs...)
bbgo.Notify(format, arg1, arg2, arg3, objs...)
bbgo.Notify(object, object2, object3)

Note that, if you're using the third format, simple arguments (float, bool, string... etc) will be used for calling the fmt.Sprintf, and the extra arguments will be rendered as attachments.

For example:

bbgo.Notify("%s found support price: %f", "BTCUSDT", 19000.0, kline)

The above call will render the first format string with the given float number 19000, and then attach the kline object as the attachment.

Handling Trades and Profit

In order to manage the trades and orders for each strategy, BBGO designed an order executor API that helps you collect the related trades and orders from the strategy, so trades from other strategies won't bother your logics.

To do that, you can use the *bbgo.GeneralOrderExecutor:

var profitStats = types.NewProfitStats(s.Market)
var position = types.NewPositionFromMarket(s.Market)
var tradeStats = &types.TradeStats{}
orderExecutor := bbgo.NewGeneralOrderExecutor(session, s.Symbol, ID, instanceID, position)

// bind the trade events to update the profit stats
orderExecutor.BindProfitStats(profitStats)

// bind the trade events to update the trade stats
orderExecutor.BindTradeStats(tradeStats)
orderExecutor.Bind()

Graceful Shutdown

When BBGO shuts down, you might want to clean up your open orders for your strategy, to do that, you can use the OnShutdown API to register your handler.

bbgo.OnShutdown(func(ctx context.Context, wg *sync.WaitGroup) {
    defer wg.Done()

    _, _ = fmt.Fprintln(os.Stderr, s.TradeStats.String())
    if err := s.orderExecutor.GracefulCancel(ctx) ; err != nil {
		log.WithError(err).Error("graceful cancel order error")
    }
})

Persistence

When you need to adjust the parameters and restart BBGO process, everything in the memory will be reset after the restart, how can we keep these data?

Although BBGO is written in Golang, BBGO provides a useful dynamic system to help you persist your data.

If you have some state needs to preserve before shutting down, you can simply add the persistence struct tag to the field, and BBGO will automatically save and restore your state. For example,

type Strategy struct {
	Position    *types.Position    `persistence:"position"`
	ProfitStats *types.ProfitStats `persistence:"profit_stats"`
	TradeStats  *types.TradeStats  `persistence:"trade_stats"`
}

And remember to add the persistence section in your bbgo.yaml config:

persistence:
  redis:
    host: 127.0.0.1
    port: 6379
    db: 0

In the Run method of your strategy, you need to check if these fields are nil, and you need to initialize them:

	if s.Position == nil {
		s.Position = types.NewPositionFromMarket(s.Market)
	}

	if s.ProfitStats == nil {
		s.ProfitStats = types.NewProfitStats(s.Market)
	}

	if s.TradeStats == nil {
		s.TradeStats = types.NewTradeStats(s.Symbol)
	}

That's it. Hit Ctrl-C and you should see BBGO saving your strategy states.