Error during HIBP search: list index out of range

[inurbits]

Describe the bug
When running lil-pwny I'm getting a list index out of range error. It seems to process the HIBP hashes and the AD hashes but crashes when it tries to compare them.

To Reproduce
Steps to reproduce the behavior:

1. Generate HIBP password file, I used hibp-downloader to download a generate the single file. Generated hash file starts with these and looks ok:
   000000005AD76BD555C1D6D771DE417A4B87E4B4:10
   00000000A8DAE4228F821FB418F59826079BF368:4
   00000000DD7F2A1C68A35673713783CA390C9E93:876
2. Get NTLM hashes from IFM backup. I used secretsdump to create the hash input file.
3. Run lil-pwny with the following options:
   lil-pwny -hibp hibp_hashes.txt -ad PWD_Audit_Aug/Active_Directory/corp.ntds -output json --verbose
4. Results below:
   {"localtime": "2024-08-21 14:21:51,711", "level": "SUCCESS", "source": "lil pwny", "detection_data": "Lil Pwny started execution"}
   {"localtime": "2024-08-21 14:21:51,711", "level": "INFO", "source": "lil pwny", "message": "Version: 3.2.0"}
   {"localtime": "2024-08-21 14:21:51,711", "level": "INFO", "source": "lil pwny", "message": "Created by: PaperMtn"}
   {"localtime": "2024-08-21 14:21:51,711", "level": "INFO", "source": "lil pwny", "message": "Loading AD user hashes..."}
   {"localtime": "2024-08-21 14:21:51,853", "level": "SUCCESS", "source": "lil pwny", "detection_data": "Finding users using passwords that are a variation of their username..."}
   {"localtime": "2024-08-21 14:21:52,005", "level": "DEBUG", "source": "lil pwny", "message": "502978 username variants generated "}
   {"localtime": "2024-08-21 14:21:52,990", "level": "DEBUG", "source": "lil pwny", "message": "Converting username variants to NTLM hashes "}
   {"localtime": "2024-08-21 14:21:53,047", "level": "DEBUG", "source": "lil pwny", "message": "Split into 1 parallel jobs "}
   {"localtime": "2024-08-21 14:21:53,047", "level": "DEBUG", "source": "lil pwny", "message": "4 cores being utilised"}
   {"localtime": "2024-08-21 14:21:53,793", "level": "SUCCESS", "source": "lil pwny", "detection_data": "Size of HIBP file provided 38.48 GB"}
   {"localtime": "2024-08-21 14:21:53,793", "level": "SUCCESS", "source": "lil pwny", "detection_data": "Comparing 84423 AD users against HIBP compromised passwords..."}
   {"localtime": "2024-08-21 14:21:53,803", "level": "DEBUG", "source": "lil pwny", "message": "Split into 395 parallel jobs "}
   {"localtime": "2024-08-21 14:21:53,803", "level": "DEBUG", "source": "lil pwny", "message": "4 cores being utilised"}
   {"localtime": "2024-08-21 14:21:54,223", "level": "CRITICAL", "source": "lil pwny", "message": Error during HIBP search: list index out of range}

Expected behavior
Expected to go through the list of hashes and print results.

Desktop (please complete the following information):

• OS: Red Hat Enterprise Linux
• Version Lil-pwny version 3.2.0
• Python version: 3.11.9

[PaperMtn]

Hi @inurbits

Looking at the command: -ad PWD_Audit_Aug/Active_Directory/corp.ntds it seems like you're missing a step in getting the user hashes. You will need to recover the NTLM hashes from the .ntds file into a formatted .txt file.

Instructions on how to do this are here

Let me know how it goes once you've got the hashes and are passing them in a .txt file.

[inurbits]

Hi @PaperMtn, thanks for the response.

That file does have NTLM hashes but maybe not in the format the tool is expecting. The entries have the format:
::::::

This is what I would usually pass to John for cracking. I can check the source also, but is it looking for a different format to parse, like :?

[inurbits]

@PaperMtn Looks like Github didn't like my formatting, I was able to get a little further looking into the expected format. The Hashcat NT format is just username:NT hash. I updated my input file and lil-pwny goes through ~60 entries before exiting with the same "List index out of range" error. Since it works on some of them this might just be an issue with something in my data. I tried to figure out which line it was having issues with but I'm not great at debugging multiprocessing code.

[cwebcc]

It looks like the actual problem is that sysinternals got updated and now the command to extract the hashcat formatted hashes has changed. See Step 2.

This is why (from DSInterals):

parameter called -ExportFormat, which replaces the Format-Custom cmdlet when exporting data to hashcat

@PaperMtn Maybe you can update your sample command in the usage part.