newsbin.com

Hi

Iam planing to make a RAM drive of 20GB ram for my Processing folder (Dl/unrar) and then it will unrar to a normal harddrive.

Now newbinpro is downloading and unraring same time, but with only 20GB (ramdrive) I need it to Download,unrar,delete and then download the next item in que
Is this possible ?

Not as far as I know. I am assuming that you have Newsbin set to Decode to a different drive and automatically delete the rar/par files afterwards? Might also need the option to put them into the Recycle bin disabled to make sure space is freed.

Great effort has gone into making sure that the AutoPar features can run in parallel with Downloading to maximise total throughput for the average user. You are now asking for the opposite Be interesting to see if Quaded thinks introducing such a mode might be possible.

itimpi wrote:Not as far as I know. I am assuming that you have Newsbin set to Decode to a different drive and automatically delete the rar/par files afterwards? Might also need the option to put them into the Recycle bin disabled to make sure space is freed.

Great effort has gone into making sure that the AutoPar features can run in parallel with Downloading to maximise total throughput for the average user. You are now asking for the opposite Be interesting to see if Quaded thinks introducing such a mode might be possible.

The problem is when you have download speeds 200-500Mbits the hard drive willl go nuts when its unraring and trying to save the dl data same time. Then it's better todo one thing at a time.
(users on slow connection will prefer the dual task system as now, but not for people using fast connections)

The problem is when you have download speeds 200-500Mbits the hard drive willl go nuts when its unraring and trying to save the dl data same time.

What do you have the RAM cache set to ?

[SETTINGS]
MemCacheLimit=?

If you set this to say, 200 Newsbin will download files up to 200 megs into memory and only write them out when each is completed in memory.

I don't have"MemCacheLimit" under [SETTINGS] in my NewsBin.nbi

it will help some I guess to have it at 200 (biggest single rar files I seen are around 150mb) but the problem is the unrar and dl same time I think. Then the disk need to read/assemble the files on disk and same time getting data saved at the max speed the disk can handle.

Files are unrared to a separate disk,

If Newsbin is caching them in memory, there's a bunch more disk IO time for the unrar to happen. I have to say, I consider download speed far more important than unrar speed. I'd be more inclined to throttle the unrar than to throttle the download.

Quade wrote:If Newsbin is caching them in memory, there's a bunch more disk IO time for the unrar to happen. I have to say, I consider download speed far more important than unrar speed. I'd be more inclined to throttle the unrar than to throttle the download.

If you dl a HD-documentry where every rar is 150MByte x 30 at a speed of 300Mbit , storing in memory is easy, but when the disk drive need to handle over 35MB/s to write and then same time access other files for Unraring it's problem.
Soo the download speed gets killed of the disk IO. And its faster and more healthy for the disk to process one item at a time when you got extream download speeds.

Why not just add a function that we can force newsbinpro to dl,unrar,delete and then take next item in que. (Singleprocessing=1)
The only workaround for me now is to pick one file at a time or lower the download speed, and both solutions feels the wrong way to go.

The only solution I can think of, is to create a ramdrive ,but then I must beable to force newsbinpro to single process the files for me. Else I would need like 100GB ram memory, Iam planing to stay around 20GB for the ram drive.

You have to understand, there's a lot of users that get their panties in a serious twist when Newsbin even shows up in the top ten memory usage, so there's a sizable group out there that want the reliability of data stored to disk over memory. Typically, those concerned about this are on one lung machines which makes the odds of system problems and losing the lot even higher. Aiming for 12G systems isn't typically in the majority's interest.
Now, I *do* have a 12G system with lots of horsepower, so I've certainly toyed with the notion of virtualizing the lot into memory, just pointing out that it's hard to please everyone and it's easy to say 'make it an option '. Doesn't mean it won't happen.

DT

The only solution I can think of, is to create a ramdrive ,but then I must beable to force newsbinpro to single process the files for me.

Why not just let it download the way it wants to download and not mess with it? I'm not getting why this is an issue. I'm not seeing a problem that needs solving. Let it download and unrar and come back later to look at your files. If you want to improve performance, unrar to a different drive than you download too.

It's only storing ONE file in memory so, if you set the cache to 200, it'll typically only consume 200 megs.

I agree with the OP.

Why can't there be an option atleast ?
An option to either do as OP says, or one that does unrar'ing when all downloads are done.
That way the ones that are having isses can just enable it. And the ones that like the current way can just ignore it.

Why? Because it stalls the system.
Downloading and Unraring a 5 GB set single: About 20 minutes.
Downloading and Unraring two 5 GB sets added in queue at the same time: About 60 minutes
By trying to do it parallel instead of sequential, it's actually INCREASING the total time.

You are making the BIG assumption that it will be relatively trivial to support both modes of operation.

Having said that I guess it is up to Quade to make any definitive statements about this.

Quade wrote:

The only solution I can think of, is to create a ramdrive ,but then I must beable to force newsbinpro to single process the files for me.

Why not just let it download the way it wants to download and not mess with it? I'm not getting why this is an issue. I'm not seeing a problem that needs solving. Let it download and unrar and come back later to look at your files. If you want to improve performance, unrar to a different drive than you download too.

It's only storing ONE file in memory so, if you set the cache to 200, it'll typically only consume 200 megs.

With very fast Internet connection you have to understand downloading is many times faster then unraring, and then you get disk IO that will kill your harddrive and all that IO on the disk will slowdown the downloads.
Then single file processing is the way to go.

itimpi wrote:You are making the BIG assumption that it will be relatively trivial to support both modes of operation.
Having said that I guess it is up to Quade to make any definitive statements about this.

There already is an auto-throttle system in place afaik. I'm sure that can be expanded on to make it a sequential-process system and be made toggleable, if he chooses to go for it.

I guess there are some feelings involved here.

I know I'd personally much rather either download _or_ unpack, not both at the same time (I even requested it a few years back, I believe). Either on an item by item basis or by downloading the entire list before post processing anything. I can't say whether there's any speed benefit to it as opposed to the drives grinding like nuts trying to both download and unpack at the same time, but it _feels_ so much better to not hear the poor drives grinding and grinding.

Me too I would like not to hear my HDD going nuts
So an option like in SABnzbd will be great:
Pause the download list until the repair/unrar is done ....

I've not said no. I never do. It's not something that's high on my list though. Getting 6 out the door is the current priority.