newsbin.com

On a single core machine, I see Newsbin taking a very long time decoding RARs while on a dual core machine I see unrar time being up to ten times faster. When comparing the CPUs, I would expect the dual core machine being only slightly faster considering that unrar is single-threaded and that the dual core machines single-threaded performance is not that much greater.

Is there a special reason for it to be like this? Is there maybe a setting I can change?
It seems that at least in some situations, it helps if I change the RAR decode priority. Could that be the solution?

The machines are identical other than CPU? I'd be much more likely to think this is a disk drive issue than CPU.

Actually, the single core machine is a desktop while the dual core is a laptop. The desktop machine has considerably higher performance in the disk drive department than the laptop.

Anyway, it turns out I was way off. Both the single core and dual core machines will exhibit similar behavior if the DL throughput is high enough. I'm not sure if it this a linux/wine issue or if it is a newsbin/config issue. It seems that if the Internet bandwidth is low enough, then decode has no problem on the same machine. Looks like Newsbin gets behind on something in the DL/decode process, which then overloads newsbin in such a way that unrar suffers.

I live quite far from my main server (Easynews), so I run newsbin with 30-50 connections as that will give me a DL rate of about 4-6MB/s. (Don't you just love fiber? )

Any ideas/suggestions?

Some people turn off automatic decode and either do it with an external tool or defer it till after all the files finish downloading. I just think you're running into some inherent performance limits of your setup.

Probably. I just wonder where the limitation comes from, if there might be something in wine doing it. There is definitely a disk I/O aspect of it too, since files that fit in cache are usually ok. Luckily Linux handles disk cache much better than Windows (possibly just pre-7), so I gain something there.

Typically it's ALL disk IO. SATA doesn't like simultaneous access.

Yeah, the problem's not wine...I run linux/wine(32 bit of course)/newsbin on a crazy-ass fast system(cpu and IO) at one location and it positively screams. Everything is *blip*, done. At home, on a slightly more humble system running 64 bit Windows, quad core intel with a more typical consumer hard drive, it has all the same IO symptoms you're describing. I'm pretty sure it's all about disk IO. Bring on moar SSD's!

DT

I see. Recommendation for running Newsbin with 100Mbps: Crazy fast SSD

I've run it at 100+ on my machine to a normal disk drive. You just have to be careful what else you're doing with this disk.

How many connections were you running then? I have a long distance to my service provider, so I need many connections to get a decent speed. I assume Newsbin does file I/O for each connection independently, which would cause excessive I/O when running 20-30 connections at high speed.

10 connections probably. Disk IO is locked so, only one thread writes at a time.

I just got a new PC for running Newsbin (not the reason to buy though), and it looks like it has nothing at all to do with disk I/O or number of cores.
My new PC is a Core i7-860, and when I get it in the slow state, there is almost no disk I/O going on, probably because it is very slow to decode.

When I managed to provoke the state for the decoder, I increased the number of connections to 50, which increased my throughput to about 7-8MB/s. The throughput at this point seems to be limited by CPU, since the CPU load I can measure from Newsbin is approximately 1 core 100% loaded. I'm guessing this means that all download threads run in a single CPU thread?
Furthermore, I can see that the wine service takes a fair amount of CPU during download, and increases a bit during decode. First part of decode is relatively fast, and hits a load of 150% for the Newsbin process. But after going to slow state, this changes radically: The Newsbin process goes down to about 66% CPU load, but the wine process doubles its CPU load. It looks like Newsbin gets into some state where it does excessive system calls, and since Wine might be a bit slower than Windows to service the calls, this can increase the effect.

I guess it is time to update this one:
I have been running with 5.55 for a couple of months now, and the first thing to notice is that the slow decode problem is gone. Automatic unrar/repair is very fast in 5.55, even with Newsbin running heavy downloads.
Now a new problem is introduced in 5.55:
When the download runs with a large list, it sometimes stops moving completed files from the chunks folder to the download location, and they just stay in the list as paused/completed. They way I have managed to work around this is to shutdown Newsbin saving the download list, and then start it up again. The files will then stay in the download list as paused, but if I resume them, they will save properly. Only problem is that I have to resume them a small group at a time, otherwise my problem comes back. Another thing is that I see indications that some files have been missed in the process.

jacob733 wrote:When the download runs with a large list, it sometimes stops moving completed files from the chunks folder to the download location, and they just stay in the list as paused/completed.

Have you tried to work without a chunk folder, so the files have to rename only.

It should make no difference - even when using a Chunks folder the .nb! files are simply renamed (as long as the post was completel downloaded).

Ubuntu 8.04

I have the OS, but I would like to make dual booting System. I have windows on a 15 GB partition. I have another 80GB on the other partition that i just use for PlayStation related things. Is there a way I can install Ubuntu on my 80GB partition and still be able to boot windows. Or perhaps have some sort of choice to boot an OS when cpu boots from a hard shutdown?