Skip to content

Include any Intraday data reported on or after the close #539

Description

@maread99

The following 1min data for symbol QAN.AX covers the minutes running into the close and post close prints:

date open high low close volume
2026-01-12 04:57:00+00:00 10.22 10.23 10.22 10.225 5918
2026-01-12 04:58:00+00:00 10.225 10.23 10.22 10.22 9515
2026-01-12 04:59:00+00:00 10.225 10.23 10.22 10.23 9721
2026-01-12 05:00:00+00:00 10.23 10.23 10.23 10.23 3
2026-01-12 05:10:00+00:00 10.2 10.2 10.2 10.2 1.15023e+06

(Recreate using prices._request_yahoo(TDInterval.T1, start, end) with same values as below and then droplevel(0) from the index.)

However:

prices = PricesYahoo("QAN.AX")
start = pd.Timestamp('2026-01-12 10:00:00+1100', tz='Australia/Sydney')
end = pd.Timestamp('2026-01-13 10:00:00+1100', tz='Australia/Sydney')
prices.get("1min", start, end, lose_single_symbol=True, tzout="UTC").tail(3)

returns:

open high low close volume
[2026-01-12 04:57:00+00:00, 2026-01-12 04:58:00+00:00) 10.22 10.23 10.22 10.225 5918
[2026-01-12 04:58:00+00:00, 2026-01-12 04:59:00+00:00) 10.225 10.23 10.22 10.22 9515
[2026-01-12 04:59:00+00:00, 2026-01-12 05:00:00+00:00) 10.225 10.23 10.22 10.23 9721

i.e. neither the 05:00 print on the close nor the subsequent 05:10 post-auction print are included or otherwise represented in the returned price data.

The same is true for data at other intraday intervals, for example for 5mins the available price data is:

date open high low close volume
2026-01-12 04:45:00+00:00 10.2 10.21 10.2 10.2 15544
2026-01-12 04:50:00+00:00 10.2 10.23 10.2 10.225 79604
2026-01-12 04:55:00+00:00 10.225 10.23 10.22 10.23 40911
2026-01-12 05:00:00+00:00 10.23 10.23 10.23 10.23 3
2026-01-12 05:10:00+00:00 10.2 10.2 10.2 10.2 1.15023e+06

Although

prices.get("5min", start, end, lose_single_symbol=True, tzout="UTC").tail(3)

returns:

open high low close volume
[2026-01-12 04:45:00+00:00, 2026-01-12 04:50:00+00:00) 10.2 10.21 10.2 10.2 15544
[2026-01-12 04:50:00+00:00, 2026-01-12 04:55:00+00:00) 10.2 10.23 10.2 10.225 79604
[2026-01-12 04:55:00+00:00, 2026-01-12 05:00:00+00:00) 10.225 10.23 10.22 10.23 40911

And for 1h the available data is:

date open high low close volume
2026-01-12 01:00:00+00:00 10.24 10.24 10.14 10.21 352551
2026-01-12 02:00:00+00:00 10.215 10.22 10.195 10.205 158861
2026-01-12 03:00:00+00:00 10.205 10.225 10.18 10.195 199303
2026-01-12 04:00:00+00:00 10.195 10.23 10.19 10.23 279855
2026-01-12 05:00:00+00:00 10.23 10.23 10.2 10.2 1.15023e+06

while

df = prices.get("1h", start, end, lose_single_symbol=True, tzout="UTC").tail(4)

returns:

open high low close volume
[2026-01-12 01:00:00+00:00, 2026-01-12 02:00:00+00:00) 10.24 10.24 10.14 10.21 352550
[2026-01-12 02:00:00+00:00, 2026-01-12 03:00:00+00:00) 10.215 10.22 10.195 10.205 158861
[2026-01-12 03:00:00+00:00, 2026-01-12 04:00:00+00:00) 10.205 10.225 10.18 10.195 199303
[2026-01-12 04:00:00+00:00, 2026-01-12 05:00:00+00:00) 10.195 10.23 10.19 10.23 279855

In short, when there are prints of intraday data at or after the market close (i.e. here 05:00 and later) the close prices and any auction prices and volume are not being represented in the data.

What to implement

Clearly this data should be included. However, it doesn't make sense to simply include these post-close indices - unlike other indices, they do not represent trades over a duration (or at least not over the same duration as the data's interval). I think I'm currently of the view that all post-close data:

  • must be represented
  • the most appropriate way to represent post-close data is by consolidating it into the data for the last indice of market hours (i.e. the last indice with a left side earlier than the market close); for the above example this would be [2026-01-12 04:00:00+00:00, 2026-01-12 05:00:00+00:00) for the 1h data and [2026-01-12 04:59:00+00:00, 2026-01-12 05:00:00+00:00) for the 1min data.

How to implement

There's likely a lot of things that will need to be considered - will need to look through the whole implementation to get a handle on them all. Those that initially occur to me:

  • could probably assume there's a final print on the close and just look for one and if it's there then consolidate into the prior indice. ALTHOUGH would need to handle the edge case of 24 hour markets,
  • would need knowledge of the expected time of final print. This is exchange-specific and therefore would be better accommodated on exchange-calendars, perhaps by providing for a 'final_print' property. Similar to opens and closes would need to be able to set and query by session.
  • daterange.GetterIntraday would need to return datetimes through to the 'final_print' whenever the requested period would otherwise go to a session close. But what about a request with an end explicitly defined as the session close - including data from the post-auction would then be including data from after the requested end - a big no. Really should only incorporate post close data if end defined as a session or as a datetime that's >= the 'final_print'.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions