Failure with Linux and USAA

[Dameon87]

I'm unsure what the exact cause of this may be, but I'm running into trouble using this with Linux (Ubuntu 20.10) on both python3.8 and python3.9. I can get the commands to work correctly on windows. However, every command via linux is outputting:

~/.local/bin$ ofxget stmt usaa -u 011111111 --all
Traceback (most recent call last):
File "/home/wallet/.local/bin/ofxget", line 8, in
sys.exit(main())
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/scripts/ofxget.py", line 1568, in main
REQUEST_HANDLERSargs["request"]
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/scripts/ofxget.py", line 646, in request_stmt
_merge_acctinfo(args, acctinfo)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/scripts/ofxget.py", line 617, in _merge_acctinfo
acctinfos: List[AcctInfo] = sorted(extract_acctinfos(markup), key=sortKey)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/scripts/ofxget.py", line 1380, in extract_acctinfos
parser.parse(markup)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/Parser.py", line 85, in parse
self.header, message = self._read(source)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/Parser.py", line 118, in _read
header, message = parse_header(source)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/header.py", line 294, in parse_header
header, header_end_index = OFXHeaderV1.parse(rawheader)
File "/home/wallet/.local/lib/python3.9/site-packages/ofxtools/header.py", line 81, in parse
raise OFXHeaderError(f"OFX header is malformed:\n{rawheader}")
ofxtools.header.OFXHeaderError: OFX header is malformed:

<title>

        System Error
      | USAA</title>
<style media="screen" type="text/css">

This, again, only seems to happen on linux. The same command works fine on my windows install.

[csingley]

Hmm, I can't reproduce this on Python 3.9 under Linux. Is this occurring on master, or an older version of ofxtools?

[Dameon87]

I've tried this on master and the most current release version. I'm quite unsure why this is happening. Only thing I can think of is it's an issue in Ubuntu. Are you able to attempt to give this a try on Ubuntu 20.10?

[csingley]

If you're running identical Python and ofxtools on 2 different OS'es, but on one of them you're getting different results, I'd eliminate differences in user config before turning to the OS.

The problem is that USAA isn't sending you OFX. Your Ubuntu machine is sending different requests. Try comparing the output of ofxget list usaa and ofxget stmt --dryrun usaa on both hosts to find the differences.

[gthole]

I'm also encountering this issue.

Docker python:3.7-alpine
reqs.pip ofxtools
Then ofxget acctinfo usaa --user <uid> throws an invalid OFX header. It looks like USAA is returning an HTML doc in response to this endpoint, so maybe it's an issue on their end?

[csingley]

Well shoot, now it's failing for me as well. I'll try to look into this a bit more.

[jackpot51]

@csingley bad news, USAA completely changed their OFX authentication so now only Quicken can connect

https://communities.usaa.com/t5/Finances/USAA-Creates-Quicken-Monopoly/td-p/243850

[csingley]

Well that's a drag. USAA seems to be following Schwab to the "OFX Secure Plus" dark side... "You'll use Quicken and like it".

The original reports on this ticket were probably formatting differences that were fixable by tweaking connection parameters. This latest change is something qualitatively different, and it ain't gonna be fixable from my end.

[gthole]

Haven't tested this yet, but someone claims to have got this working from the master branch:
https://lists.gnucash.org/pipermail/gnucash-devel/2021-February/045704.html

[cfazzini]

@csingley I've been able to get USAA to work again with some information from the GnuCash-devel mailing list. The following config seems to work for me with latest version from here with some addl notes.

The clientuid I extracted from the URL when authenticating via https://df3cx-services.1fsapi.com/casm/usaa/enroll. After entering USAA credentials the page is redirected to https://www.usaa.com/inet/ent_oauth_consent/authorize?0&client_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&.... It would seem there may be a link between the account and this clientuid. Also ClientUID must be in uppercase.

DTACCTUP of 19900101 is required to get good data. The default of 19901231 returned success, but only "Client up to date", no acct info.

pretty = true is also required (assuming the newline requirement).

I'm not sure if or where these changes should be incorporated, but hoping the information is useful none the less.

[USAA-NEW]
url = https://df3cx-services.1fsapi.com/casm/usaa/access.ofx
clientuid = XXXXXXXX-XXXX-XXXX-XXXXX-XXXXXXX
org = USAA Federal Savings Bank
fid = 67811
bankid = 314074269
user = xxxxxxxxx
version = 103
unclosedelements = true
appid = QMOFX
appver = 2300
pretty = true

@gthole That was me. Was updating this post here at same time you posted. 👍

[jackpot51]

@cfazzini do you know what user agent is being used?

I ended up getting the IP block described on https://lists.gnucash.org/pipermail/gnucash-devel/2021-February/045690.html, thank fuck my assets are at Chase now...

[cfazzini]

@jackpot51 Best I can tell from the code in ofxget.py, default is a blank User-Agent, so it would appear I'm not sending one. But, could use confirmation on that. Sucks on the IP ban. Hopefully it's short term.

[csingley]

@cfazzini - thanks for the report! Setting CLIENTUID is not too crazy.

I'm thinking I may need to patch OFXClient to uppercase the UUID and send the truncated DTACCTUP.

Should we perhaps set up a wiki, to serve as a repository for this kind of info that's helpful to users?

[cfazzini]

Sorry just realized I wasn't too clear. It seemed to not have an issue with the full date format, the value of the date seemed to matter more. I just used the 19900101 format with the --dtacctup CLI flag and it was fine.

[csingley]

@cfazzini - thanks for the clarification.

From ofxtools.Client.OFXClient, by default useragent: str = "InetClntApp/3.0"

[cfazzini]

Thanks for clarification, I was looking at this in ofxget.py

parser.add_argument(
            "--useragent",
            dest="useragent",
            help="Value to use in HTTP 'User-Agent' header (defaults to empty string)",
        )