cacher.py Traceback errors on 10.11.6 server 5.2

[elflames]

G'day eriking,
I have tried to use you cacher.py script on my caching server as I too need a nice way to to show this is saving us bandwidth.
Ever time i call cacher.py in terminal, i get the following output;

Traceback (most recent call last):
File "/usr/local/bin/cacher.py", line 913, in
main()
File "/usr/local/bin/cacher.py", line 896, in main
cacherdata = cacher(rawLog.readlines(), targetDate, friendlyNames)
File "/usr/local/bin/cacher.py", line 167, in cacher
frompeers_size = linesplit[17]
IndexError: list index out of range

python is not my strong suit and I hope this error is a simple misconfig on my behalf.
Any input from yourself would be greatly appreciated.

cheers Damien C

[erikng]

Ping @Hunty1 - this is due to your code changes.

@elflames can you post your logs?

[erikng]

Also @elflames can you try https://github.com/erikng/Cacher/pull/37/commits?

[calum-github]

will need the logs to check.

10.11 doesnt have the metrics sql db so that commit above you'll need to change the flag to use the logs for getting metric information

o.add_option('--measurement',
 +                 help=('Optional: Measurement source, use either'
 +                       ' logs or metrics to specify either the Debug'
 +                       ' logs or metrics sqlite db as the source for'
 +                       ' calculating bandwith statistics'
 +                       ' Defaults to use metrics sqlite db'))```

[elflames]

will the last 500 lines of the current debug log be enough?

[erikng]

More than likely not. Some days cacher would look through 20,000+ lines. It just takes one bad line to cause this error. On a side note I may need to stuff the main function into a try and print out the bad log line to help debugging. Thanks, Erik Gomez

…

________________________________ From: D.Cooper <notifications@github.com> Sent: Tuesday, April 25, 2017 10:28:28 PM To: erikng/Cacher Cc: Erik Gomez; Comment Subject: Re: [erikng/Cacher] cacher.py Traceback errors on 10.11.6 server 5.2 (#40) will the last 500 lines of the current debug log be enough? — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#40 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFa_GCADBlY07l6rdlsbLjW-naOphTTPks5rztX8gaJpZM4NIXYo>.

[calum-github]

Just zip up the entire logs directory and send

[elflames]

logs attached in first message. please advise once you have them.
cheers

[calum-github]

Why are all your values in bytes?

2017-04-26 14:18:22.464 #jaHYUDXIeXSh Served all 98273652 of 98273652 bytes; 13657 from cache, 98259995 stored from Internet, 0 from peers
2017-04-26 14:19:13.107 #ymOfzFFE5UKR Served all 2882611 of 2882611 bytes; 2882611 from cache, 0 stored from Internet, 0 from peers
2017-04-26 14:27:19.772 #91qUgiZ/EzS5 Served all 5027 of 5027 bytes; 0 from cache, 0 stored from Internet, 5027 from peers
2017-04-26 14:34:23.079 #sVBZswC2kppm Served all 77119519 of 77119519 bytes; 13591 from cache, 77105928 stored from Internet, 0 from peers
2017-04-26 14:46:40.244 #+sQfhSd9CTUM Served all 150 of 150 bytes; 150 from cache, 0 stored from Internet, 0 from peers
2017-04-26 14:46:41.185 #LD38A0U+OCjq Served all 150 of 150 bytes; 150 from cache, 0 stored from Internet, 0 from peers
2017-04-26 14:47:27.290 #g3Go2/rnlyxN Served all 5979 of 5979 bytes; 5979 from cache, 0 stored from Internet, 0 from peers
2017-04-26 14:57:23.897 #GvOF11Z+eIkm Served all 553037340 of 553037340 bytes; 13594 from cache, 553023746 stored from Internet, 0 from peers
2017-04-26 14:59:48.712 #Om6oNCLGNDVF Served all 21102703 of 21102703 bytes; 13685 from cache, 21089018 stored from Internet, 0 from peers```

[calum-github]

It should look like this

2017-04-26 15:13:08.749 #Q4WwsSNFJl49 Served all 135.1 MB of 135.1 MB; 135.1 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:13:20.750 #ysv4GA9ze4WG Served all 7.1 MB of 7.1 MB; 7.1 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:14:00.652 #wEBKcqU+VZ3r Served all 7.9 MB of 7.9 MB; 7.9 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:14:02.451 #rkweQBxYTyr4 Served all 9.3 MB of 9.3 MB; 9.3 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:14:08.235 #pzZVBmo9yeo6 Served all 9.1 MB of 9.1 MB; 9.1 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:14:38.253 #oGaLe2oy1rQg Served all 8.6 MB of 8.6 MB; 8.6 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:15:04.222 #/hcddy1RH1F5 Served all 135.1 MB of 135.1 MB; 135.1 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:16:31.658 #zbJ65SmHMHCp Served all 9.4 MB of 9.4 MB; 9.4 MB from cache, 0 bytes stored from Internet, 0 bytes from peers
2017-04-26 15:24:00.890 #2LSSrK+vNLRt Served all 2.3 MB of 2.3 MB; 0 bytes from cache, 2.3 MB stored from Internet, 0 bytes from peers

This is why its tripping, its expecting a bandwidth type after the number

ie MB after 2.3

but yours are all in bytes already

ie

Served all 21102703 of 21102703 bytes

instead of what it should be like

Served all 21.1 MB of 21.1 MB

And this is why the line split is tripping up.

[elflames]

I am not suer why mine has defaulted to bytes. checking the server admin settings now..
using commit #37 code, I was able to get this to stdout;

Cacher has retrieved the following stats for 2017-04-24:

7.5 GB of bandwith served to client devices.
4.3 GB of bandwith requested from Apple
146.0 MB of bandwith requested from other Caching Servers

IP Addresses hit the Caching Server on 2017-04-24 consisting of:
Unique IP Addresses.

A total of 77 iOS downloads were requested from the Caching Server on 2017-04-24 consisting of:
A total of 17 iPad downloads
A total of 60 iPhone downloads
2 iPad Air 1st Generation [Wifi]
10 iPad Air 2nd Generation [Wifi + Cellular]
1 iPad Air 2nd Generation [Wifi]
4 iPad Mini 4th Generation [Wifi + Cellular]
9 iPhone 5 [CDMA]
4 iPhone 5C [Global]
5 iPhone 5S [China Model]
6 iPhone 6 Plus
10 iPhone 6
3 iPhone 6S Plus
5 iPhone 6S
8 iPhone 7 Plus [GSM]
6 iPhone 7 [GSM]
1 iPhone 7 [Global]
3 iPhone SE

A total of 861 OS downloads were requested from the Caching Server on 2017-04-24 consisting of:
77 iOS downloads:
9 iOS 8.2
1 iOS 8.4.1
2 iOS 9.0.1
1 iOS 9.3.2
1 iOS 9.3.4
1 iOS 9.3.5
1 iOS 10.0.3
21 iOS 10.1.1
13 iOS 10.2.1
1 iOS 10.2
26 iOS 10.3.1
784 macOS downloads:
2 macOS 10.11.5
13 macOS 10.11.6
28 macOS 10.12.0
85 macOS 10.12.1
27 macOS 10.12.2
12 macOS 10.12.3
3 macOS 10.12.4
2 macOS 10.12.5
612 macOS 16.5.0

A total of 861 files were downloaded from the Caching Server on 2017-04-24 consisting of:
766 personal icloud files
18 .pkg files
54 .ipa files
23 .zip files

A total of 558 unique files were downloaded from the Caching Server on 2017-04-24 consisting of:
16 .pkg files
50 .ipa files
13 .zip files

Cacher version: 3.0.5
Uptime: 1:36, 2

[calum-github]

Please confirm what OS version and what version of Server app you using. and what region are you from or using? US English?

[elflames]

macOS: 10.11.6
ServerApp: 5.2
Region: AUS
happy to jump onto slack channel or free node channel.

[calum-github]

Are you sure its 5.2?

5.1.7 is the latest for 10.11.6

5.2 won't install/run on 10.11.6

I have a 10.11.6 caching server here with the latest version of server 5.1.7 and i am not seeing the output you are.

Perhaps you have manually or through some kind of automation have installed 5.2 on 10.11.6 somehow and thats causing you issues.

[calum-github]

My apologies, just built a 10.11.6 caching server here and 5.2 came down from the app store.

¯_(ツ)_/¯

Running a quick test not seeing the values in bytes here..... Must be something on your machine

[elflames]

I will rebuilding the server and see what I get.. Thank you for the quick replies and your continued development of this tool.