Cache header backlog using filters - all files fail

[Bexley]

Hi, I am using positive filters for a group (accept certain users only). I continually end up with blocked cache. For example now I have 21 files on Import for a group. Following your procedure to clear this I remove the locked file by copying the Import files elsewhere to see what is locked. The problem is Newsbin fails to process on every single file so I end up doing stop and restart 21 times, or I can just delete all the Import files and not get an update. Since these are filtered updates - no spam, much lower volume - why is Newsbin failing consistently here? Any thoughts on what to do? Regards

[Quade]

I'm skeptical the filters have anything to do with the fact your GZ files are stalling. It's more likely they're corrupt on disk and that's what's causing them to stall.

You could try to un-GZ them with WinRAR. That might tell you if they're damaged or not. If you're running out of disk space while doing header downloads, that could corrupt them too.

[Bexley]

Thanks, I will check. Only mentioned filters to indicate lower volumes, nothing to do with them I think. Regards

[Bexley]

Hi, there appeared to be nothing wrong with the gz files, able to open with 7z no problem. Disk is fast SSD with download and decoding sent to other disks. Over 100GB free. Regards

[Quade]

Maybe we should revisit your filtering idea then.

The question is whether the GZ files are really getting stuck or whether they filters are slowing things down enough to make them appear stuck.

How complicated are the filters? Filtering the header imports is much more expensive than filtering later.

[Bexley]

Hi, thanks as always for the prompt reply. First, I opened and tested the gz, it opened but the crc failed. This is 100MB download but the SSD is fast, etc (gaming hardware). The filter is accept if string match (no regex). I can try poster match? So today there were 6 chunks 40MB, on test all failed crc check. I will try poster match filter.

[Bexley]

Negatory on poster filter, 4 small chunks, 1.4m, 10.5kb, 1.56m, 141kb all fail crc. Really it seems to fail with even the smallest update, mystified here.

[Quade]

Hi, thanks as always for the prompt reply. First, I opened and tested the gz, it opened but the crc failed. This is 100MB download but the SSD is fast, etc (gaming hardware). The filter is accept if string match (no regex). I can try poster match? So today there were 6 chunks 40MB, on test all failed crc check. I will try poster match filter.

If you have bad GZ files that probably explains the stalling. The main question then is why your GZ files get corrupted but mine don't. GZ files aren't that complicated. Newsbin reads data from the internet and writes it out compressed as GZ files. There's not a bunch of room for it to get corrupted. Typically I suspect either something like bad RAM or your security software damaging the files.