Quade wrote:Look at the size of the import folder, multiply it by 10 and that's how much data was downloaded from the news server. Depending on the groups, it's a bunch of data. I have about 200 groups and even though my machine is pretty powerful it takes 10-15 minutes to import them all.
I could import them faster by say doing 2 GZ's at a time but I think many PC's would groan under the load. One thing you can do it set Newsbin to download headers every 30 minutes to an hour. Then there's never a huge lump of headers to process in one go.
As a follow-up to my previous query, I've noticed in groups with huge post counts that while downloading headers, the Import folder creates what I'm guessing is the queue of headers it is downloading, usually storing them in 25-35 MB gz files. However, every so often it creates a 1 KB file and once the program gets to this file, it really grinds to a halt with a ton of disc activity, spending at least 15 minutes and sometimes more than 30 minutes doing something in the background once it gets to that 1 KB file. Is this by design? This morning I finished downloading headers from a group that had almost 900 million of them. It then took the program more than 6 hours to place them all into the group's spool .DB file, each time stalling as I've described each time it encounters a 1 KB file. Does this sound correct, and if not, any idea what's causing this? Does anyone else have this issue?
I know there's a new beta, but just wanted to post here since I haven't installed Beta 6 yet. I do know it does the same thing on the current release version, which is why I decided to install Beta 5, hoping this would clear it up.