NB 6.70B6 seems slow to process headers:
Posted: Mon Mar 07, 2016 6:12 pmPerhaps it's due to an increased amount of spam posting but I've noticed that NB 6.70B6 is quite slow to sort through headers (as in more than 22 minutes to process a.b.mm, a.b.hdtv, and a.b.teevee together)
At the moment of posting it's been 38 minutes since initiating a 16 hour header download, NB reports (4) under cache, is using 1.13GB (and rising) of memory, 7-11MB/s of disk and ranging between 5 and 14% of my cpu (I7-4790) FWIW disk access on the drive that the headers are stored can peak at 120MB+ (when I'm compressing headers for example)
After this processing finally finishes I will go back to 6.6X to see if it's any faster for the same selection of groups, but on the bright side, I can post again
Note: activated verbose logging and bounced the program (before going back to 6.6x) it started with a (3) in cache and is using the cpu and disk to presumably process the already downloaded headers as things propagate into the log I'll post here:
[14:22:55] DEBUG High Speed File Creation Mode: Disabled - "Run as Administrator" to Activate this mode
[14:22:55] HIGH PAR Pausing: Enabled
[14:22:55] HIGH Par2Repair - Multi-core 1.5
[14:22:55] HIGH MessageId Filter: Disabled
[14:22:55] HIGH FileType Version: 2.01
[14:41:28] DEBUG Checking Autodownload: alt.binaries.hdtv
[14:43:58] DEBUG Checking Autodownload: alt.binaries.hdtv
[14:48:57] DEBUG Checking Autodownload: alt.binaries.multimedia
(Note, I don't use autodownload....) sometime prior to 1504 (but after 1448) it finished as cache now shows (0) and the group list shows a combined total of 92798 New files under the main screen (before choosing to view the groups) so it *might* be down to so much spam being posted, but that sort of delay is still ghastly. Will try 6.6X when I grab more headers in 7 or 8 hours.
At the moment of posting it's been 38 minutes since initiating a 16 hour header download, NB reports (4) under cache, is using 1.13GB (and rising) of memory, 7-11MB/s of disk and ranging between 5 and 14% of my cpu (I7-4790) FWIW disk access on the drive that the headers are stored can peak at 120MB+ (when I'm compressing headers for example)
After this processing finally finishes I will go back to 6.6X to see if it's any faster for the same selection of groups, but on the bright side, I can post again
Note: activated verbose logging and bounced the program (before going back to 6.6x) it started with a (3) in cache and is using the cpu and disk to presumably process the already downloaded headers as things propagate into the log I'll post here:
[14:22:55] DEBUG High Speed File Creation Mode: Disabled - "Run as Administrator" to Activate this mode
[14:22:55] HIGH PAR Pausing: Enabled
[14:22:55] HIGH Par2Repair - Multi-core 1.5
[14:22:55] HIGH MessageId Filter: Disabled
[14:22:55] HIGH FileType Version: 2.01
[14:41:28] DEBUG Checking Autodownload: alt.binaries.hdtv
[14:43:58] DEBUG Checking Autodownload: alt.binaries.hdtv
[14:48:57] DEBUG Checking Autodownload: alt.binaries.multimedia
(Note, I don't use autodownload....) sometime prior to 1504 (but after 1448) it finished as cache now shows (0) and the group list shows a combined total of 92798 New files under the main screen (before choosing to view the groups) so it *might* be down to so much spam being posted, but that sort of delay is still ghastly. Will try 6.6X when I grab more headers in 7 or 8 hours.
Another option is instead of having one giant database in each group file for all headers, you go (back?) to having databases based on dates (for example, monthly) that way unless someone is searching through or loading old headers there's no reason to touch headers from more than say 30 days old if you went monthly. (Edit: or allow manual separation of the headers, but that's a lot more effort for you if you also would then allow manual searching of the separated headers)