Pause after each file

[JSchuricht]

I recently upgraded to a 300mb connection, I can pull about 200mb from the news servers I use, and noticed that NB is pausing after each file which is basically halving my speed. I am running 6.52B5 but saw the same thing with B3. The system is 6x 3.0GHz Xeon, 12GB RAM, 8x 450GB 15K SAS for the data folder and the main download folder set an 24x 2TB 7.2k drive array over a 10gb network. The pause seems to be during the transfer from the 15k drives to the network. Is there any way to set NB to start downloading the next file during the file transfer to improve speed?

[Quade]

1 - I'd look in the performance options and see if "Pause Download during unrar/repair is checked"

2 - Down at the bottom, I'd look at the Cache line. If the first number goes to zero during unrar/repair, it means Newsbin can't write the data it has in memory to disk. My cache is typically set to 400. You want it to be larger than the largest number of chunks in the files you download.

[PERFORMANCE]
ChunkCacheSize = 1000;

If you have the RAM, you might want to bump the size up. Each cache block is probably 500-600K of RAM while actively downloading.

and the main download folder set an 24x 2TB 7.2k drive array over a 10gb network.

Set the download path to the local drive and see what happens. You might be better off downloading locally and unraring remotely.

Re-reading your comment it sounds like your slowdown is the process of assembling the files from chunks. Running a larger cache means they can all assemble out of RAM but, still going over the network for assembly will be slower than local. There's also a hit because autopar has to re-read the assembled file to collect the PAR status. I think downloading locally and unraring remotely would be more efficient.

You're on the edge of the performance envelope though so, there's probably not too many people experienced with this out there. You ought to be able to sustain 200 without too much trouble. Going to a local server, downloading to a local drive, I've often got well over 2 times that speed so, it's doable. I get 110 Mbps day in and day out downloading locally.

[JSchuricht]

Thanks, the pause option was unchecked but the cache was at 200 and dropping to 0 during the pause I was seeing. I tried 1000 and it was still dropping to 0 occasionally so I went for 6000, I am guessing that equates to MB. That fixed the issue.

Just a suggestion but if you setup the counter at the bottom so you could right click or double click on it to adjust cache size it would be much easier to get to.

[Quade]

As I said in my post, it's about 500-600K per cache block so, assuming 600K

600K * 6000 = 3.6 gigs of RAM. Which is fine as long as you realize how much RAM you're using.

The issue is likely that network fileIO is too slow so, you're using a bunch of memory buffering to compensate for the network.

[itimpi]

As I said in my post, it's about 500-600K per cache block so, assuming 600K

600K * 6000 = 3.6 gigs of RAM. Which is fine as long as you realize how much RAM you're using.

The issue is likely that network fileIO is too slow so, you're using a bunch of memory buffering to compensate for the network.

Just thought that it was worth pointing out that if by any chance one is not running a 64-bit version of both Windows and Newsbin, trying to use this much RAM would cause issues of its own.

[ddaniel51]

I took your suggestions, increased the cache size and set my download path to a local ssd but noticed the chunks still load to the network drive slowing the process down. Can we specify in the ini where the chunks folder is located?

Tnx
Dave

[Quade]

With a huge cache, chunks should never even land on disk. I wonder if you're just seeing Newsbin try to clean up the chunks?

If you move the data folder to the local drive, the chunks will also go to the local drive.

[ddaniel51]

Thanks Quade

After increasing the cache to 2k there are no more written to disk.

Dave