Refresh fund NAVs from AMFI, and report how old prices are - #4
Merged
Conversation
Prices were never updated except by a broker sync, and nothing said so. `app/navs.py` fetches AMFI's daily NAV file — public, no key, nothing personal sent — and marks every mutual fund position. This matters most for Paytm, which reports no NAV at all: those funds were marked at the NAV of their last transaction, and one was 24 days stale. Refreshing corrected India from 97,31,673 to 97,20,028. Matching refuses to guess. A NAV on the wrong fund is worse than a stale one, because a stale price is visibly stale and a wrong one is not. Funds with an ISIN match on it. Paytm funds have none, so the scheme name must reduce to exactly one Direct/Growth scheme after filler words are dropped — which correctly matched "HDFC Index Fund - BSE Sensex Plan" to AMFI's "HDFC BSE Sensex Index Fund" despite the word order, and refuses anything ambiguous. Regular plans and IDCW options are excluded, since an IDCW NAV would permanently understate a growth holding. Resolved ISINs are stored so a name is matched once and can be audited. "Data through" measured the wrong thing. It reported the last transaction date, so a trade this morning made the header look healthy while fund NAVs were a month old. Price age is now reported per asset class as the OLDEST mark in each group, not the newest — one fund priced today does not make the fund holdings current — shown beside the transaction date and bannered past a week. The quotes table was dead: written by upsert_quotes, read by nothing, 0 rows. It now backs pricing. A quote is used only when dated later than the broker snapshot, so a stale quote can never override a fresh sync. Writing the tests found a real bug. results() had a second pricing path for holdings with no transactions — which is every mutual fund — that read positions.price directly and ignored quotes entirely. So the funds this feature exists to reprice were the one group it would not have reached. Both paths now go through _positions(). 22 new tests: feed parsing, the matcher's refusals, and that a stale quote loses to a fresh broker price. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three problems, all from the same root: prices were only ever updated by a broker sync,
and nothing told you how old they were.
1. Fund NAVs now refresh from AMFI
AMFI publishes every Indian scheme's NAV daily as one public text file — no key, no
account, nothing personal sent.
This matters most for Paytm, which reports no NAV at all. Those funds were marked at
the NAV of their last transaction; the oldest was 24 days stale. Refreshing moved the
India total by about 0.1%.
Matching refuses to guess. A NAV on the wrong fund is worse than a stale one — a stale
price is visibly stale, a wrong one is not.
after filler words are dropped. Correctly matched
HDFC Index Fund - BSE Sensex Plan→AMFI's
HDFC BSE Sensex Index Funddespite the word order.growth holding.
2. "Data through" measured the wrong thing
It reported the last transaction date. A trade this morning made the header look healthy
while fund NAVs were a month old. Price age is now reported per asset class as the
oldest mark in each group — one fund priced today does not make the fund holdings
current — shown beside the transaction date and bannered past a week.
3. The
quotestable was deadWritten by
upsert_quotes, read by nothing, 0 rows. It now backs pricing. A quote is usedonly when dated later than the broker snapshot, so a stale quote can never override a
fresh sync.
A bug the tests found
results()had a second pricing path for holdings with no transactions — which isevery mutual fund — that read
positions.pricedirectly and ignored quotes entirely. Thefunds this feature exists to reprice were the one group it would not have reached. Both
paths now go through
_positions().Evidence
quotesrowsrealized + unrealized + dividends == net_profitholds in both markets.Not covered: equities. AMFI is funds only; stock prices still come from the broker at
sync time.