[01:54:02] DEBUG - Cache Miss: Larger chunk cache might pe

[haileris]

Hi

Now and again newsbin just pauses. When I switch on debugging I get constant error messages as per the title - and I mean constant - so over 10-15 seconds 4000 of them will rack up! Any reason that one can thing of why this might be happening?

Cheers

[Quade]

I removed the message.... It didn't seem to help.

Basically the message means Newsbin is waiting for the disk drive to let it write more data to the disk. If it's unraring or repairing for example, there simply might not be any time to write the download data. There's not much you can do about it. You can disable writing while it's unraring/repairing. Download speed will go down but, unrar/repair speed will go up. It's in the performance options.

[haileris]

Hi

The trouble is there is no downloading going on. Cache is 0/100 but downloading is 0/0. I'm running 6.40RC1 at the moment and after a spell of downloading everything just stops. There is nothing in the logging tab, even at debug level. If I close and restart downloads commence again for a while. If I revert to a lower newsbin version (6.32 for example) there is no such issue. I've set some of the performance configs but as above there isn't anything going on that I can tell (CPU newsbin process 0% at time of "pause".)

Urrgh!!!

Cheers
D

[Quade]

That sounds like it's not able to write to disk at all. It downloads 100 chunks of data and just stalls out. If that what it looks like, 100 chunks and then stall? Is the download path a normal path? Does it have enough free space?

I'd clear out the download folder (assuming you don't have much there) make a download folder on the C drive, maybe "C:\DL\". Pick one file right click, "Download to Existing Folder" and then download to the new download folder.

If on the other hand it runs out after awhile, it might be a leak but, I'm not getting one here so, the hard part will be finding it.

In the performance options you could also try checking "Write Downloaded Files with no Memory Buffering" and see if your symptoms change.

[haileris]

I have 3 drives, 1 system, 1 for the downloads and 1 for the decoded files. There is enough space (over 100 GB on both the 2nd and 3rd drives). It stopped downloading today around 6am. No errors in the log file. I'm not sure if the PAR process is the issue - there was a lot of files with the status "PAR PROCESS" in the download list? I left the debug logging on and the last message was for a successful par check.

Anyway, restart and everything starts again - downloading and unparing. I have about 11k download files in the list and the process can consume up to 1.5GB (I have 4GB).

[Quade]

In the performance options you could also try checking "Write Downloaded Files with no Memory Buffering" and see if your symptoms change.

Try this.

[haileris]

Hi

Thanks for the reply. I did have that set but not sure when I set it. Do I need to reboot or does it take effect straight away?

Cheers

[Quade]

Should restart Newsbin probably.

[haileris]

Thanks - unfortunately didn't work. Got home around 12:30 am this morning, newsbin had stopped for about an hour and a half. Plenty of valid downloads in the pipeline. Last messages were a bunch file tracker clear messages on a particular file. Stopped and started - downloading fine again.

[haileris]

Same thing, worked from 12:30 to 3pm then stopped. Last messages were a bunch file tracker clear messages on a particular file. I'm going to clean up my downloads a bit and try one more time - if not will have to revert to old version for now and hope it fixes itself

Btw, maybe this is something that I am doing wrong? On some of the newsgroups the PAR files seem to be separate downloads, so that over time one can have a bunch of "orphaned" PAR files just sitting around in the downloads area. Am I missing something here about the way in which I am downloading stuff?

[Quade]

After you switched it to "no buffer" mode, do the cache numbers still go to zero? I assume you're still running "no memory buffering" mode.

[haileris]

Hi

Yes it does and yes I am in no memory buffering mode. I have taken a few screenshots if it will help - suitably disfigured. is it ok to post them here?

Cheers

[Quade]

I don't know that they'll help. I know what the problem is. You're running out of packet buffers. The only question is why. I don't think pictures will tell me that.

[haileris]

I have one observation. I have a disk allocated as a download drive (D) and another as my decode drive (E). Sometimes the E drive becomes full up before I have had a chance to look at it, so stuff will continue downloading onto the D drive but of course autopar will fail. Is this something that is handled differently in the newer versions of newsbin? I ask that because I had a low space warning on the 30th "roughly" at the same time as newsbin "stopped" and today whilst I didn't have a low space condition warning I only had 21 GB left on the E drive so there is a possibility that unrars failed. I'll check this out more tonight.

Having said this, not very useful but if I try to do anything with newsbin apart from looking or closing it after its "stopped" then it hangs (so moving downloads around or moving stuff from failed back into the downloads.)

[itimpi]

At the moment as far as I know Newsbin only monitors the DATA and DOWNLOAD drives for free space. Perhaps Quade could be persuaded that the UNRAR drive (if neither of the two previous ones) should also be monitored?

[Quade]

Well, the question is, does it stall if it's not running out of disk space?

[haileris]

Hi

Sorry for the delay - it stalls whether it has sufficient disk space or not.

I've just done some testing / watching with an older client. I'll probably buy another (separate) drive for this VM's download drive as the existing one is performing poorly. The behaviour I have observed is that in the older client the cache numbers fluctuate and sometimes drop down close to or actually at zero for a short while, with a corresponding loss of speed. But the (cache) numbers do pick up eventually (and then drop / pick up again at times etc etc) and when they do there is the corresponding speed increase to about 15MBps at full pelt. On the newer client I'm afraid I've not observed when that stalls but as we know it does not appear to attempt anything with the cache settings at zero. I guess the would stay at zero if nothing is being downloaded?

[Quade]

It stays at zero when it runs out of buffers. At idle is should be (100 - connection count). If you pause for example while downloading, it'll settle down to (100 - connection count).

If for example, Newsbin can't write to the disk, it'll go to zero and stay there. That's what this sounds like, working, working, working, can't write to disk, runs out of packet buffers and is done. The fact disabling the packet buffers also stalls out also suggests something like this.

[haileris]

Sounds good except I don't get the stall if I use an older copy of Newsbin. I'm using version 6.30 and its fine. So even if there were a condition that for a period of time Newsbin could not write to the disk does it give up completely? There seems to be a change in behaviour somewhere down the line?

[Quade]

The fact that it's just happening to you at this point points to something specific about you . If other people were reporting 0 packet stalls then I would be more suspicious of an internal Newsbin problem. There's not enough evidence one way or the other. It's not stalling for me.

Apparently you're using a VM and apparently you've had weird disk issues with the VM (or so I thought you said). I'd suggests you try it on another PC without using a VM and see what happens. The fact it used to work with 6.30 doesn't mean much these days because firewalls treat new versions like New programs.

I'd love for you to find some way for me to reproduce this. Then I could fix it. As it is, I have nothing to work with.

[haileris]

Not weird disk issues with the VM, just poor performance as its shared with another VM. I say poor but its still sufficient to download simultaneously from 2 different VM's at a combined total of 132mbps when the applications are working. I'm going install a separate drive in the near future so that will remove any contention so I'll see what happens then. Really don't want to go down the separate PC route - it just isn't cost effective, not to mention a step backwards (for my setup at least!) The firewall isn't the problem, that's for sure. Windows firewall isn't that intelligent and besides it wouldn't work for a bit and then stop.

[Quade]

Why not run it in the host OS, then see of the stalling goes away? That would give you some clues.

[haileris]

The host OS is ESXi.