Cache Flush?

[TBlack]

Beta 12

NB is not doing anything after a file and header download....it is totally idle and the Cache entry on the Status Bar says:

Cache: 143/200 (0)

Is something possibly not be flushed? Could this be contributing to the hang that some are reporting?

If I close NB and then reopen, after the Header Download is parsed, I see Cache: 200/200 (0)

Thanks

[Quade]

Just some chunks hanging out. Might be partial files or signature matches. You can flush it by hitting "Help/About".

Partial or filtered files can leave stragglers, they get flushed out when the space is needed by something else.

Thanks.

[TBlack]

You can flush it by hitting "Help/About"

A hidden feature of Help/About? Don't see anything that says Flush the Cache.

Thanks

[Quade]

I just use it for testing. All it does is force stragglers to disk.

[RayMark]

I continue having problems with downloads stalling @ 0/200 even with beta 13. Only beta 3 is working without stalling.
I have to do Help/About many many times to download anything - and that is with books only. I don't even try anything above beta 3 with large files.

Can you please hook up help/about so that it is done continuously without manual interventions? As it is now, 6.60 certainly cannot be released as final until that issue is fixed.


P.S.

While I was typing this message, NewsBin stalled again―obviously―I was not doing Help/About every 3 seconds while I was typing this...

[Quade]

It is automatic for me. You're just having another weird problem.

I accept that there's a problem in Newsbin because this shouldn't happen. The question is why is it happening to you? Something is different about how you download. Something is happening that prevents the cache flush during normal operation.

I'd suggest removing your V scanner and firewall (except the MS supplies ones, they're OK). Then reboot and try again running naked. Then try some test downloads and see if this still happens.

I'd like to get to the bottom of this but it's hard because it doesn't happen to me.

[Quade]

I have an idea. Something to try anyway.

In the performance options if you have "Pause Download during unrar" checked, uncheck it and then restart Newsbin.

[RayMark]

Yes, pause during unrar was switched on. I will try without it.
However, when it happened today, autopar was disabled and rar files were not unrarred at all, so not sure if it can help.
I was mostly downloading lots of very small rar files and not extracting. Many less than 1 MB, but some 30 MB size files began causing the problems.


Here is a hint: the problems began after beta 3. With beta 3 there is no stalling. I missed beta 4, but the problems began in beta 5 and are continuing up to and including beta 13.
Think what you changed in beta 4 or beta 5

[Quade]

Turn autopar back in. Disable unrar in the group properties. See what happens.

I never run with autopar disabled.

[RayMark]

I did. Autopar is on, pause during unrarring is off.

It is worse with beta 13 now if anything. Downloads stopped both with 700 small files (ebooks) (with unrarring switched off locally) and also with regular large files added from the internet search (with unrarring on). Not only Help/About was needed to move forward (but only very little forward, soon again stalls) but also now I noticed that while doing nothing with 0/200, NewsBin was still using 75-92% CPU (quad-core). I never noticed such CPU usage before. Not sure if it was related to the 0/200 stall or maybe just coincidence.
Generally, it seems that with autopar on and pausing during unrarring off it takes longer for that 0/200 stall to occur. But sometimes it occurs pretty quickly. Maybe it is caused by some specific files being downloaded.

[RayMark]

Unfortunately, it occurred again while I was typing the previous post. In minutes. With regular large files. Forgot to check the CPU usage....

And again - in a couple of minutes. So it probably has to be because of some specific combination of files. I basically loaded a bunch of web-dl posts from the internet search.
I am watching the CPU usage now: Initially after stalling it was below 4% but gradually grew up to 15% but then stabilized at 11%. Still a lot, considering that NewsBin has stalled and does nothing (nothing useful to me, at least).

And again - this time the CPU usage stays @ 1-2%. Of course, each time before NewsBin stalls, it downloads some files so the situation is different each time.

[RayMark]

Quade, it might be important?

Now I had the stall with 6.60 beta 3 - I always thought that the problem began AFTER beta3.
I really don't remember downloading stalling with beta 3 ever before. I am using beta 3 specifically for stall-free downloads.
And Help/About helped very little. The download speed got up from 0, but almost immediately went back to 0 again.

I have another thought about the reason and why it might be difficult to reproduce for you:
I am using 4 servers simultaneously, and after re-starting NewsBin now I got lots and lots of "430 No such article" messages from two of them.
Although the posts were complete on the other servers and downloaded successfully.
Perhaps multiple servers + multiple "430 No such article" messages can cause this stalling?

[RayMark]

UPDATE:

I take it back about beta 3 stalling - turns out, it encountered par2 file and began a long process of creating almost 5 GB worth of rar files that were next in the download list and could have been downloaded much faster. CPU usage: 17%
It also does not show any progress so I don't even know how long to wait. But I counted the downloaded rar files and they all are present even though 17 of them are not yet downloaded.
I moved those par2 files to the end of the list but that does not make the rar files to be downloaded. I guess, I will restart again, thus interrupting that par2 process.
The newer betas perhaps handle this better. I just forgot this side of beta 3...

However, the reason for stalling of newer betas perhaps might be related to multiple servers + multiple 430 "No such article" messages?

[RayMark]

NewsBin 6.60 RC4 stalled @ 0/200. After that, a single Help->About downloads 3-4 MB only. I have to press repeatedly (continuously) Help->About in order to finish downloads.
Why can't it be done automatically? If I can click and click and click Help->About, the same thing can be done automatically when downloading stalls @ 0/200. Probably not a solution but a work-around at least.

I had big hopes that the fix in RC2 against stalling @ 0/200 was general enough. But no, no such luck.
This time the problem is certainly not with duplicate spam. Or is it?
It seems to be rather with my headers that sometimes have duplicate entries: when downloading headers from 4 different servers somehow it happened. Although possibly sometimes even with different message IDs.
So perhaps duplicate headers can be considered as spam....
Anyway. There is something very wrong with 6.60 if it cannot be fixed once and for all not to stall @ 0/200 whatever the garbage input.

I am very afraid that also future versions such as 6.70, 6.80, 7.00, 8.00, 10.00, etc. will continue with 6.60 stalling problems and the last good NewsBin will forever remain 6.55 Final...

[Quade]

It seems to be rather with my headers that sometimes have duplicate entries: when downloading headers from 4 different servers somehow it happened. Although possibly sometimes even with different message IDs.

Ok, download headers from a single server then and tell me if the problem goes away. I have no problem working on this, just no way to reproduce it. I download headers from 2 servers. I only really need to download from one but I do 2 simply to test it.

Since you feel it's something about how you download that causes this. I suggest changing how you download till it goes away. For example if you download headers from a single server and it goes away, that's a good clue.

[TBlack]

Quade,

Any chance you can put the 6.56 Cache Code in 6.60 with a Option Selection in Options for Testing and/or Use? 6.56 seems to be stable. Was there a reason to change it?

Thanks

[Quade]

No, no chance. The problem probably has nothing to do with the actual cache code. More than likely something else is crashing internally and losing a cache block. Just patching code in and hoping it'll fix this is kinda pointless. The only way to fix this is to characterize the problem and then go from there. It's not happening to me so, clearly my PC or something I'm doing is different than Raymark.

Are you having download stalling issue too Tblack? With Raymark, it never recovers. Going to zero is normal. Not recovering from going to zero isn't.

[Quade]

Just for shits and grins. Disable "Use Recycle Bin" in Newsbin and disable it for the download drive too and see if the symptoms change. I just had a long stall that I thought was Raymarks problem but in fact I had an unrar what was doing a shitload of post-unrar deletes and this process stalled the rest of the download.

The filter from a single unrar tossed 6000 files in the recycle bin. Took forever and the download stalled while it was happening.

6.56 didn't recycle this stuff.

[TBlack]

Are you having download stalling issue too Tblack? With Raymark, it never recovers. Going to zero is normal. Not recovering from going to zero isn't.

No, I am not. Think I only ever had one stall and reported it, but have not for a long time. No recent issues.

Thanks

[TBlack]

Just had this happen three times just today, same group:

[10:58:57] ERROR UseNet alt.binaries.boneless Header Download Failed: ReadFailure

Cache is: 200/200 (1)

There currently is a file stuck is a Boneless file stuck in the Import Folder.

Untouched file is here:

https://onedrive.live.com/redir?resid=5 ... =folder%2c

[Quade]

[10:58:57] ERROR UseNet alt.binaries.boneless Header Download Failed: ReadFailure

1 - Is a network read failure. Not a problem reading from the Import file.

2 - Does the import file end in .txt? If not, you can just delete it. Newsbin only imports GZ and TXT files. Header download failures like this can leave stragglers that won't ever import. The periodic data folder purge should eventually kill them.

Yeah looked at the file. No txt extension. It's basically junk at this point. Thanks.

[TBlack]

Thanks