6.62 Performance Issues

[haileris]

Hi

Is anyone else having issues with 6.62 releases and performance. I use 3 different ISP's and none of them can push anywhere near the maximum on a machine using 6.62. Older versions I see the kind of speed I expect i.e. pushing maximum bandwidth for the connection. The strange thing is for the first minute or so the speed is as expected... then it rapidly moves downwards e.g. 100Mbps to 9 Mbps within a few minutes. No missing posts issues and cache is nowhere near zero. Pulling my hair out here!

Cheers
D

Btw, not sure if this helps but when I look in task manager and observe the network traffic there are spikes when I get to 100Mbps and then it drops down to 0 for 30 seconds, then another spike. It almost looks as though newsbin is doing something else during that time?

[Quade]

What news servers are you using?

[haileris]

Hi

Sorry for the massive delay in replying.

eweka
newshosting
astraweb
extremeusenet

The selection of a usenet provider doesn't seem to make any difference. I'm doing a few changes at the moment but for now regressed some of the versions which has restored the performance .

Kind regards
D

[haileris]

Uurghh got around to looking at this again due to newsbin crashing for some reason on a download. Still struggling. Saved all my old downloads to NZB files and imported them into latest newsbin version. Old version 6.52 downloading at 71Mbps. New version 6.62 downloading at 15-22 Mbps. All on newshosting.

[Quade]

Make sure you tell your security software to completely ignore Newsbin. Was an issue the other day where Kaspersky was proxying and screwing up all the connections for one user. He had to tell it to leave Newsbin alone. Same for the data folder.

I'm getting 120 Mbps days in and day out so, it's very possible to get high speeds. Some users are getting twice that speed.

The fact old Newsbin goes fast and new Newsbin goes slow suggests it's security software.

[haileris]

Other than Windows firewall I have no other security software on this host. I'll disable it to check. Is there any extra logging I can put on to show what newbin is doing?

[Quade]

Windows firewall isn't known to cause any issues.

What kind of drive is the data folder on? 6.62 uses the data folder more heavily than older versions. You can go into the options and select "Open Data Folder" to tell you where the data folder is.

I'll see if I can figure something out. I noticed my Wife's PC was running a bit slow too.

[haileris]

All my newsbin machines are virtual. This particular machine it isn't too bad. Its a slice of a RAID=10 array of 6 x 3TB 7200 rpm drives. There is nothing else using that RAID set at the moment. The machine is not CPU bound at any point. There are no datastore alarms. There's no un-raring or repairing going on at the time of the captures nor is there any header downloads (disabled header downloads, deleted all the groups info and flushed the spool folder entirely. Running out of ideas!)

The below captures show the difference in behaviour between 6.62 and 6.56 for me for the same news provider setup and downloads (using the same config file):

6.62


6.56


Took this on a 6.62 logging session. Why is there a gap of 13 seconds before downloads commence again?


Thanks for the help - much appreciated!

[Quade]

I'll bet the lull between each Ethernet peak is Newsbin assembling the files. This is particularly true if the files are large with many chunks.

In the next beta, the assembly of files and the download of chunks is now independent. In 6.22, the assembly of files can gate the download of chunks if you run out of cache blocks. I think this issue is going to come down to slow disk IO gating the download. 6.52 simply assembled files differently. VM's are known for slowish disk IO.

Wonder if increasing the cache would help? How much RAM does this VM have to work with?

Look at the cache line on the status bar, how low does it go?

[haileris]

I can pretty much scale the memory up how I please - that particular VM has 12GB RAM at the moment but could easily go up to 24GB. I already have cache chunks set to 2000 though. Lowest I have seen is 1604/2000 - that is on 6.56 though.

Respectfully have to disagree on the VM / disk IO comment - this is ESXi 6 on a server with a decent RAID card, 6Gbps backplane etc. It ain't new (LGA 1366 dual sockets) but it should cope. I've got the OS drive, download drive and unrar drive separated to reduce disk contention as well. I am going to put some SSD's into the mix at some point - at the very least use them as a caching drive.

Anyway, paused the download that was crashing newsbin so its back to full working order on 6.56.I hope you change the behaviour back in the next version, or at least provide an option? Thanks again for responding so quickly.

[Quade]

Respectfully have to disagree on the VM / disk IO comment - this is ESXi 6 on a server with a decent RAID card, 6Gbps backplane etc. It ain't new (LGA 1366 dual sockets) but it should cope. I've got the OS drive, download drive and unrar drive separated to reduce disk contention as well. I am going to put some SSD's into the mix at some point - at the very least use them as a caching drive.

You could use procmon and see what's happening with the disk when it stalls out like this. If it's assembling files from chunks, it should be relatively easy to tell. In my experience, VM disk IO is simply slower than naked disk IO. Raid 10 disk IO is likely slower than writing to a single disk. Something is preventing you, with a machine about as powerful as mine, from getting full speed. It's clear from your pictures that download totally stalls out and is only using 1/4 or less of the download time available to it.

Just for grins, how about exiting Newsbin, right click "Run as Adminstrator" and see if anything changes. Windows by default will zero out any disk files Newsbin opens and re-sizes. If you run it in "Admin" mode, Newsbin can disable this zeroing. Basically, every file gets written twice, once to zero the disk sector and the second time with the real data.

Bottom line is I can get full speed and you can't get close to that. The exact mechanism is unknown just yet.

[haileris]

Migrate this workload to some directly presented disks. Didn't help. Will look more later on - back to previous version again for now

[TMM]

I am also experiencing this issue using modern sky Lake i7 with 16gb of Ram plenty of system
resources available
Newshosting My current provider
Thanks again for continuing to develop and evolve nb