Following the article Guideline to backtest with custom datasets, we learnt how to upload custom data to the system for backtest analysis. This post will further discuss how to upload data to the cloud data directory via API.
This feature is particularly useful to these usage scenarios:
- Massive data upload programmatically
- Real-time external data feed for live trading
Suppose we have our own data source and want to live feed the data to a file "ABC". Let's say a particular trading record contains the following information.
- Timestamp: 2022-08-02 15:05:00
- Open price: 102.3
- High price: 104.7
- Low price: 101.4
- Close price: 104.1
- Volume: 123
- user: your ALGOGENE user id
- api_key: your API key
- file: specify the file to store your data
- mode: 'a' for data append; 'w' for data overwrite to your specified file
- msg: the data message to write to the file
We can post data from a local machine to the cloud data directory as follows:
Upon successful data upload, you will receive this message.
Now, we can check the details under [My History] > [Custom File Viewer]. The system will create the "ABC" file for us if it doesn't exist.
Update meta data
Similar to the previous post, we need to update the file "_meta_.json" to let the system know how to process our data.
In above example, our file contains 6 columns separated by comma.
|Column Index||Column Name||Data Type|
|0||Timestamp||in format of YYYY-MM-DD HH:MM:SS|
We can define the meta data as follows:
How to use custom data for Live Trading?
As long as the configuration is setup properly, we will then be able to get the custom real-time data stream in live-testing or real-trading environment.
- Go to [Live Test] or [Real Trade] > [Account Details]
- Select a trading account and click "Show algo details"
- Our "ABC" is now available under the instrument dropdown.
If there is new update in our custom data file "ABC", we will receive the market data feed under 'on_marketdatafeed' or 'on_bulkdatafeed.
On the other hand, we can use the API function 'getHistoricalBar' to query from our custom data files.