6.50B12 high CPU and download stall issue

[Quade]

Dthor on IRC found me a test case where certain files can stall the download and cause high CPU usage.

The workaround till I fix this, is to go into "Really Advanced" in the server options for each server and click "Body Mode" then restart Newsbin.

[olog_hai]

RAM usage gets up there as well. Maybe something isn't getting cleaned up?

[itimpi]

RAM usage gets up there as well. Maybe something isn't getting cleaned up?

RAM usage can easily get that high if you have a lot of items loaded. What is the size of your download list? Also do you have headers for any large multi-media groups loaded?

[olog_hai]

The download list was empty. NB was minimized and had open only the groups list (90 groups and 3 GOGs) tab and not actively doing anything. That's why I think that it's not doing a great job of releasing memory. It doesn't bother me because I have 12 gigs, but I've noticed that some applications that consume lots of RAM become slow and unstable eventually, and then you have to relaunch them.

[DThor]

If you had group tabs open - that gobbles up the memory. Try closing them. The download queue contents doesn't in and of itself really take up much memory unless you have a massive amount of stuff there. Another thing to check is any anti-virus - some of them snoop on everything you download and hang on to it. When you quit Newsbin, does the memory free up?

DT

[olog_hai]

I had no groups tabs open, just the Groups List. No sense in closing that. For anti-virus I have only MSE, which I think is lightweight and unobtrusive. The memory is indeed released when I close Newsbin. Honestly, this a minor issue for me, but one I thought worth a brief mention.

I do have more serious problems, especially with how Newsbin handles dropped groups from one of my providers as well as the fact that it keeps showing me some old stuff with "hide old" enabled. In Agent if you delete or mark something as read, you never ever see it again while the filter is enabled. Also it would be great if I could assign newsgroups to specific servers or vice versa; Agent has this feature. But anyway, this paragraph is off the topic of this particular thread.

[Quade]

How many cache entries are you showing down in the status bar? Cache blocks are probably 500-800K each when they get used.

[possamai]

Not sure if this is the 'stall issue' you mentioned but when a file is downloaded it stops downloading for a while, then continues.
It's during the par2 scan. See for yourself:

[20:30:18] DEBUG FileIOThread - AssembleFile: release info + screenshots + password can be found at: www.bd25.org - free registration - "Something_very_legal.part112.rar" yEnc
[20:30:39] DEBUG AutoPAR: ScanFile Blocks:18 File is Perfect: Something_very_legal.part112.rar

If you look at the timestamps.. Between those times the downloadspeed is zero.

[Quade]

There's an option in the performance options to prevent download during repair and unrar. Do you have it set?

[possamai]

Nope, and that just pauses during repair/unrar..
this pauses between every file while autopar checks if the file checksum is okay.

The files are 259,2MB and 683 records.. so maybe that's where it goes wrong. too much too handle

[Quade]

[20:30:18] DEBUG FileIOThread - AssembleFile: release info + screenshots + password can be found at: www.bd25.org - free registration - "Something_very_legal.part112.rar" yEnc
[20:30:39] DEBUG AutoPAR: ScanFile Blocks:18 File is Perfect: Something_very_legal.part112.rar

Doesn't prevent or indicate it's not downloading. It just means on your setup, it takes 20 seconds to scan this file (which is pretty slow). In the mean time the download data should be feeding into the cache blocks.

[possamai]

Odd... I could see the network load dropping and raising at the same time as those messages appeared in the log..
Maybe something weird in the files or something.. I'll download something else, see if it happens.

And yes it's slow.. highly fragmented disk and the size of the files won't help either..

[Kent_Diego]

I have 6.53 and been seeing this a lot lately. I never saw this in 6.51. Thought is was Astraweb until I saw this thread and checked the CPU usage. The groups will not update and one CPU core will run newsbin 100% forever until I close Newsbin. I will try the work-around. Please post when a new version fixes this.

[Quade]

Are you downloading headers? There are a couple groups, one is a.b.frogs, that simply breaks Newsbin. You should probably delete that group if you sub to it. Basically the posters are posting using completely random subjects with no file information so, there's no way to actually download files from this group using headers anymore. Only with NZB files.

[Kent_Diego]

Are you downloading headers? There are a couple groups, one is a.b.frogs, that simply breaks Newsbin. You should probably delete that group if you sub to it. Basically the posters are posting using completely random subjects with no file information so, there's no way to actually download files from this group using headers anymore. Only with NZB files.

Thanks for replying. New updates: With 6.5.3 and with latest beta I saw 100% CPU usage on one core for newsbinpro64.exe. I even rebooted computer and started NB without doing anything else. Just starting NB caused high CPU usage. I let it run for a long time and after an hour or so the CPU went to zero. Now when I double click a group the new headers show. It looks like it is all fixed. I guess NB had some important house keeping activities it needed to perform before I could do anything and it took a long, long time to get straightened out. The download bodies setting did not help.

Thanks, again.

[Quade]

Newsbin downloads headers as quickly as it can to disk, then imports them with a single thread into the database. What you download headers, you'll get CPU load till the import finishes. If you download all headers from a new group for instance, this can take awhile.

If you look down where it says "Cache" there's a number in parens (XX). This number is the number of blocks of headers it's importing.