admin

Guideline to upload custom market data for portfolio analysis

How it work


This article provides demonstrate how to analyze an investment portfolio beyond the trading universe of ALGOGENE.

  • import custom data files
  • specify user-defined instrument and file formats
  • add new trading instruments for portfolio analysis/optimization

Data Preparation

Suppose we want to build a portfolio for several stocks that are currently not available on ALGOGENE. We downloaded the market data from Yahoo Finance.

For example, we got the daily prices in 2020 for

The downloaded files from Yahoo Finance are in csv format. When we open the files using Notepad or other plain text editors, we can see the data structure as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
Date,Open,High,Low,Close,Adj Close,Volume
2020-01-02,60.849998,60.950001,60.599998,60.900002,60.479397,14629077
2020-01-03,60.900002,61.200001,60.250000,60.400002,59.982849,14419537
2020-01-06,60.099998,60.400002,59.799999,60.000000,59.585613,13809308
2020-01-07,60.200001,60.299999,59.799999,59.900002,59.486305,8818594
2020-01-08,59.299999,59.400002,58.849998,59.299999,58.890450,16826669
2020-01-09,59.549999,59.900002,59.549999,59.849998,59.436649,17802374
2020-01-10,59.950001,60.049999,59.750000,59.849998,59.436649,19011475
2020-01-13,59.500000,60.299999,59.500000,59.950001,59.535961,40492594
...

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
Date,Open,High,Low,Close,Adj Close,Volume
2020-01-02,6.730000,6.830000,6.730000,6.800000,6.420695,239292513
2020-01-03,6.830000,6.850000,6.720000,6.720000,6.345157,277420033
2020-01-06,6.700000,6.710000,6.580000,6.650000,6.279061,260518248
2020-01-07,6.640000,6.700000,6.600000,6.610000,6.241293,204806676
2020-01-08,6.540000,6.590000,6.480000,6.560000,6.194082,267421237
2020-01-09,6.650000,6.710000,6.600000,6.670000,6.297946,215859440
2020-01-10,6.730000,6.750000,6.680000,6.720000,6.345157,223748680
2020-01-13,6.740000,6.830000,6.740000,6.800000,6.420695,481124394
...

As we can see, the files contains 7 columns in total and structured below.

Column Index Column Name Data Type
0 Date in format of YYYY-MM-DD
1 Open float
2 High float
3 Low float
4 Close float
5 Adj Close float
6 Volume integer

Data Import

Now, let's import our data files as follows:

  • After login the portal, go to [My History] > [Custom File Viewer]
  • Select '/data' directory, then upload data files
  • import

  • We can then 'Edit' to view the uploaded content
  • view

  • Now, we need to create a meta file '_meta_.json' to instruct the system how to process the data files.
    • '_meta_.json' is in JSON format where we can specify the instrument name in the first key
      • in this example, we label them as 'HSBC' and 'CCB' respectively
      • it should be noticed that our specified name has to be distinct from ALGOGENE's existing instruments. Otherwise, the system will skip processing our data files.
    • the second key of '_meta_.json' should contain the following
      • 'file': the file name on the cloud directory
      • 'file_delimiter': the file delimiter used in a data file
      • 'period_start': the starting date of a data file, in the format of YYYY-MM-DD
      • 'period_end': the end date of a data file, in the format of YYYY-MM-DD
      • 'settleCurrency': the settlement currency of the instrument (it is HKD in our example)
      • 'contractSize': the number of share per each lot of the instrument
      • 'fmt_time': specified the date format in a data file, in Python date encoding
        • %Y: the year in four-digit format, eg. "2018"
        • %y: the year in two-digit format, that is, without the century. For example, "18" instead of "2018"
        • %m: the month in 2-digit format, from 01 to 12
        • %b: the first three characters of the month name. eg. "Sep"
        • %d: day of the month, from 1 to 31
        • %H: the hour, from 0 to 23
        • %M: the minute, from 00 to 59
        • %S: the second, from 00 to 59
        • %f: the microsecond from 000000 to 999999
        • %Z: the timezone
        • %z: UTC offset
        • %j: the number of the day in the year, from 001 to 366
        • %W: the week number of the year, from 00 to 53, with Monday being counted as the first day of the week
        • %U: the week number of the year, from 00 to 53, with Sunday counted as the first day of each week
        • %a: the first three characters of the weekday, e.g. Wed
        • %A: the full name of the weekday, e.g. Wednesday
        • %B: the full name of the month, e.g. September
        • %w: the weekday as a number, from 0 to 6, with Sunday being 0
        • %p: AM/PM for time
      • 'col_time': the column position of data date (first column index = 0)
      • 'col_open': the column position of open price (first column index = 0)
      • 'col_high': the column position of high price (first column index = 0)
      • 'col_low': the column position of low price (first column index = 0)
      • 'col_close': the column position of closing price (first column index = 0)
      • 'col_volume': the column position of volume (first column index = 0)
    meta

The sample meta file used in the example can be copied here:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
{
   "HSBC": {
       "file": "0005.HK.csv",
       "file_delimiter": ",",
       "period_start": "2020-01-01",
       "period_end": "2020-12-31",
       "settleCurrency": "HKD",
       "contractSize": 400,
       "fmt_time": "%Y-%m-%d",
       "col_time": 0,
       "col_open": 1,
       "col_high": 2,
       "col_low": 3,
       "col_close": 5,
       "col_volume": 6
   },
   "CCB": {
       "file": "0939.HK.csv",
       "file_delimiter": ",",
       "period_start": "2020-01-01",
       "period_end": "2020-12-31",
       "settleCurrency": "HKD",
       "contractSize": 1000,
       "fmt_time": "%Y-%m-%d",
       "col_time": 0,
       "col_open": 1,
       "col_high": 2,
       "col_low": 3,
       "col_close": 5,
       "col_volume": 6
   }
}

Portfolio Optimization

After we properly setup the '_meta_.json', we can then include our custom instruments for portfolio analysis.

  • Go to [Market Analysis] > [Portfolio Optimizer]
  • asset

  • Choose *My Instrument* under asset class
  • Then you will see our defined symbol "HSBC" and "CCB"
  • Now you can conduct portfolio analysis as in previous article