version 6.62 'Download all headers' problem

[Tattoo]

Hello,

I just started using NewsBin again after a few years so I went and downloaded the latest version which is 6.62 and am having a problem with it when downloading all headers that I don't remember having before. The last version I used was 6.42.

This 6.62 version also uses every bit of my resources which is maxed out at 100% which I don't remember being like that.

I use 2 hard drives, C: & D:, and use my D: drive for the NewsBin data folder. If I remember correctly, I'm not suppose to post group names, right? This group has a bit over 2 billion headers and I have the Download age set to 1500 and using Giganews. I also have cable internet at 2 mps, if that matters.

What I do, since there are so many headers for this group, as well as others but am working with just this one group atm, I'll start it to download all headers and go to sleep. It's usually not finished since there are so many headers but a lot is downloaded. In ver. 6.42, the headers would be added to the list in the group window but now I have all these .gz files in the Import folder and they're not moving. Right now there are 1399 .gz files and they're not going into the group list.

I've read in the forums here to add the line ChunkCacheSize= to the Settings section of the .ini and use less than 200 because NewBin was showing 10/200 for the Cache in the Status Bar, but it didn't explain enough of what the Cache is use for meaning headers or downloads so not sure. So I tried that and used 100 but it didn't work. It'll work decent for a bit only keeping 1 or 2 .gz files in Import but when I wake up, the Import folder is filled.

So is there a work around to fix this problem? I can't seem to fix this after a week now and need assistance, please. I'm thinking I may have to go back down to 6.42 unless you have a solution. Could this be happening because I use my second drive for the Data folder instead of the C: drive? They're both Western Digital, Caviar, Black, drives, but the pc is not new. Not exactly sure what it is but don't think it is even a quad core. I picked it up cheap to do odd things. Could that be the problem?

What I did yesterday was, I'd let it build up to 20-30, then cut the internet and let them read into NewsBin, then restart it and do another 20-30. but after while it was only keeping 1 .gz with a temp file so I thought it was working fine and went to sleep.

If one of the .gz files are getting corrupted, why would that happen and is there a fix for it?

Thank you,

John

EDIT: I should add that both drives were defragged prior to installing NewsBin and my D: drive had like 500 GB free of 698 GB so it's basically empty.

EDIT2: I had to restart NewsBin and got this message when showing all posts for this group
[06:41:55] ERROR WorkerThread Died: DbWorker
Now that I got that message. no more .gz files will read into the group so it's looking like I'll have to start all over again. Very disappointing as this has been going on for a week now with no solution. Is there a way NOT to lose the headers I have already and still get all the remaining headers without going thru this all over again?

[Quade]

If one of the .gz files are getting corrupted, why would that happen and is there a fix for it?

What security software do you use? Kaspersky damages compressed header downloads. You can disable "XFeatures" in the server advanced options and then restart Newsbin. At that point the headers downloads should be untouched. The better solution would be to make the security software ignore Newsbin's downloads altogether. This won't fix your existing GZ files though.

That's the only thing that should damage a GZ file. It's like anything else computing through. The quality of the header download depends on your machine.

There's no setting you can set to improve import speed. Disabling XFeatures will probably slow header downloads enough for your PC to keep up. It depends on your net speed though.

20-30 25 meg GZ files represent 7.5 TB of actual header data. It takes awhile to crunch through that. You might want to seriously consider whether you really want to download all of them. A group like Boneless alone can use 100 gigs of disk space. Most News servers aren't really working that well out beyond 1500 days too.

[Tattoo]

If one of the .gz files are getting corrupted, why would that happen and is there a fix for it?

What security software do you use? Kaspersky damages compressed header downloads. You can disable "XFeatures" in the server advanced options and then restart Newsbin. At that point the headers downloads should be untouched. The better solution would be to make the security software ignore Newsbin's downloads altogether. This won't fix your existing GZ files though.

That's the only thing that should damage a GZ file. It's like anything else computing through. The quality of the header download depends on your machine.

There's no setting you can set to improve import speed. Disabling XFeatures will probably slow header downloads enough for your PC to keep up. It depends on your net speed though.

20-30 25 meg GZ files represent 7.5 TB of actual header data. It takes awhile to crunch through that. You might want to seriously consider whether you really want to download all of them. A group like Boneless alone can use 100 gigs of disk space. Most News servers aren't really working that well out beyond 1500 days too.

Thank you for the speedy reply, Quade.

I use ZoneAlarm for firewall and didn't have a problem until I went to sleep. I will try to disable XFeatures and see if that helps.

I ended up just deleting the .gz files since they were useless and started over again. I reinstalled 6.42 and tried that but the headers weren't showing while downloading them like 6.62 does so I went back to it. I'm also liking the new look of it as well with the commas in the header count, in the status bar, that makes things easier.

I also went with a clean install this time to see if it helps. I'll see what happens this time and get back if still having problems.

Thank you again,

John

[Quade]

I'd still recommend making them ignore Newsbin's connections. ZA has been problematic in the past too.

[Tattoo]

I'd still recommend making them ignore Newsbin's connections. ZA has been problematic in the past too.

OK, Quade. I'm not sure how to do that but will look into it. It downloads fine so not sure what to do with it. Thank you.