newsbin.com

Trying to figure out how to maximize GigE connection, but the most I'm able to tap is about 300-400mbps despite using about 20 connections (astraweb). Even using 30 more connections from a backup provider doesn't get me that much more of a speed bump. The target hard drive is raid 0 (2 hdd) with sequential write speeds of 95MB/s and 512k writes of 28MB/s.

Anyone getting faster sustained download speeds? What provider are you using? How many hops for trace rt and avg ms? I've gotten slight blips of 500mbps but those are far and few between and not sustained.

I tried using the native IPv6 routing but couldn't get it it work with newsbin but did work with other newsreader but oddly enough the speed was slower and astraweb claims they don't officially support ipv6 which probably has something to do with it, I suppose.

The native speedtest:

On my 6 core desktop to my local server to a single spinning hard disk, I've gotten as fast as 800 Mbps.

In my experience, SATA raid performs worse most of the time over a single fast spinning disk. I'd guess though that the bottleneck is still the internet.

Yeah I suspect it is the internet connection as well since I changed the target drive to SSD and it still tops out to < 500mbps for downloading from most news servers I tried.

I would have thought piling up 30-50 connections over two providers with different backbones would be enough to saturate gigE connection, but apparently not.

Omg all I can say is I wish I had net speeds that fast!

sourceavenger wrote:Omg all I can say is I wish I had net speeds that fast!

Can't complain for $70/month. Just wish I had something to do with all this bandwidth though

I use 110 connections myself so my 70Mbp/s is always maxed out, the main issue I have is my hard drives read/write speed is slower cause its an encrypted external drive. I would recommend a few things to boost your download speed.
1) Disable SSL encryption for your download sites, you would be suprised how much faster it appears to make it consistantly.
2) Use a VPN service like VyprVPN or SlickVPN where 256bit OpenVPN encryption is used, that bypasses your ISP's throttling methods.
3) Enable your network interfaces extra features such as checksum offloading, moderation control, etc. All of that can reduce CPU usage and improve speed.
4) If your using windows Vista/7/8 disable any of the Aero effects and set your windows theme to Windows Classic, this reduces allot of the OS memory and cpu work.
5) Run computer tweaking tools to change some settings to improve performance
6) Defragment the machine regularly to allow for improved reading and write speeds.
7) Conisder using superfetch/prefetch to make windows booting and program use speeds increase.
8 ) Disable any firewall or antivirus real time protections to allow unfiltered downloads.
9) Add your computer to the DMZ on your routers settings to make it bypass the routers firewall all together, this allows for less overhead needed on the router itself.
10 ) In newsbin I would recommend increasing your cache from 200 to something higher. I personally use 1024 which seems to allow me to never hit 0 cache use state anymore.
11) Run newsbin with Realtime priority and it will improve the programs speed, I have noticed more consistant download speeds with this setting.

I use 110 connections myself so my 70Mbp/s is always maxed out,

I wouldn't. I get 112 Mbps all day long with 20 connections. There's no good reason to run that many connections. The more connections you have, the more problems you're likely to have. I would run exactly as many as give you max speed and no more.

Okay I can reduce my connection count. I am using Newshosting(60 connections), Astraweb(20 connections) and giganews(50 connections). Would it be best to set astraweb and newshosting as backups? And that way im only using giganews for the majority of posts and the rest the other two fill?

I use 10 each, AW and Giga. You can set it so, you only download headers from one. I typically only download headers from AW because they download so quickly.

110 connections!!! OMG, that is way too much for 70mbps. Previously when I had TWC, I could easily max out the 100mbps line with 6-8 connections easily using astra. I have a quad core core i5 powering the box with ssd and raid. Don't think computing power is holding it back as cpu util is 10-20% or less depending on what else I'm doing.

Wow really? I have TWC lol, wish I had Google Fiber though :p. The stupid Insight/TWC merger sucked because they started throttling traffic soon after lol. I use VyprVPN to bypass the throttling lol. Anyways I will try changing my connection settings to a lower value to see if it improves my speed.

Also, I am lost on trying to filter this spam so far the only way I have prevented most of it from displaying is I changed my crossposts from 3 to 1 and added a few words and reg expressions Quade posted a long time ago(ive been looking for hours lol). The main spam I am hitting now is Newsbin will download a image from for example a newsguy.com posted image instead of giganews, that newsguy.com one is complete but the giganews one just sits and never downloads. It seems to be consistant between all my incomplete and multi-part downloads. Giganews never downloads the duplicate(same file name and size) file but also never deletes it from queue, after about 300 or so of these get queued up then it locks up.

I can get my full rated/paid for bandwidth with my ISP (about 28Mbps) using even a single connection to Astraweb so, I'd say Newsbin Pro is pretty damned good at what it does. I keep it at 10 connections but there's no real reason I suppose (they say I can use 20 but I've yet to find a situation where I wasn't maxed out consistently at 10). I've done tests with just 2 connections and it still goes right to full speed on my connection as verified by looking at the throughput with Task Manager's network panel.

Never been an issue for me, but in terms of what Newsbin Pro reports for speed the "fastest" I've noted was about 230Mbps in the recent past and I'm fully aware that's just because of the header compression when I was pulling down a few bazillion headers on one occasion. Even so, it was pretty funny to see such a high number being reported.

As if my connection would ever be that fast...

FWIW
I have done a test on 30 and 10 connections, think it does not make much of a difference.

-

FttH with 300Mbit/s up/down, but I did not get anything above 260Mbit/s download.


Peculiarly enough though, upload often exceeds 300Mbit/s


Just my 2 cents
=

Great illustration that running tons of connections is not necessary to obtain your full datarate. We often recommend that you adjust your connection down to the lowest number that consistently gives the best datarate as that turns out to be less resource intensive and leads to fewer server and connectivity errors.

Thanks for the post.

Question:
Anyone out there who cares to download the portable freeware tool (NetWorx, one of the best tools in this freeware segment!)
and download a -say- 2.3GB file from Newshosting.

My speed jumps up and down heavily... Newhosting advised that they have no bandwidth limits.

With me it is like this:

=


Note: this is without Flash Player - Flash Player uninstalled, pc rebooted
=


I have been testing/studying this matter in depth for a few weeks now and spent an awfull lot of time on this.
It is 100% sure no hardware matter. It is a software issue.

I have another PC, which I now temporarily use for testing.
XP installed, clean, absolutely no additional software, no SP, no video driver, no audio, nothing at all.
(it is a restore of an image that I made immediately after installing XP from CD-ROM)

1. launch IE9 or IE10, go to a speedtest site, it asks for Flash Player, install Flash Player - speedtest show 315Mbit/s down and 320Mbps/s up- as a straight line, really perfect!
2. install+launch Firefox v26, additional flashplayer needed(!), speedtest does not run, flashplayer for Firefox installed, download speed crap.

Like this

=

=

again, both on a clean XP.

Tried to download a .bin-file from some site, but it did not get higher than 140Mbps. it is fairly stable around that speed, but I have a 300Mbps.
Have not found a site offering a download .bin file at high speed.

=

At that speed, you might need multiple news servers to fill in the pipe. When downloading binary files, you're also at the mercy of your disk drives too. XP is pretty old these days. Not sure I'd expect it to be the fastest.


In 6.50 if you dedicate enough ram to it, the downloads can all go to RAM and then get written to disk in one blast.

If you are saying a speedtest through a web browser gives a consistent download speed and running with Newsbin is up and down, that's understandable since they use different protocols. The website test is probably one connection downloading one big file 100% of the time. With Newsbin you are running multiple connections using NNTP which downloads chunks of different files. There is also processing time between each chunk that is downloaded. Your machine performance can impact Usenet download speeds as well as the type of content you are downloading. You also don't say what version of Newsbin you are using or how many connections you are using. Version 6.50 will try to keep more of the chunk to chunk processing in memory so should smooth out download speed more than 6.42 or older. Those dips to 0 may have been Newsbin assembling chunks into a file. I don't think it is feasible to expect a straight line of bandwidth usage when monitoring at these sample intervals. If you average over time I wouldn't expect as big a difference.

Quade and Dexter, thank you both for the feedback.

The screenshot is taken from a speedtest website which, I think my ISP seems to have 'licensed' from Ookla.
At one point in time it said that the license would expire after xx days. The url has a domain from my ISP in it.

Just for your info.

The screenshots were taken whilst using a XP machine, it is fairly clean pc, used for some basic stuff and testing.

Normally I use my Windows 7 x64 pc 8GB RAM.
The speedtests whilst using this pc (win7) are also crap, no matter what browser I use.
It is just a straight line on a clean XP machine, using IE and flash player that works together it IE.

As for Newsbin Pro: I am using 6.40 b.2059
I am also using Newshosting as primary server and, 25 connections, I am not sure about SSL,
sofar I had it tagged, but I think I read somewhere it would down the downloads???
I am using Easynews as fill server, 20 connections.

Any recommendations?

Thank you!

=

Is there really an issue? What is the average datarate Newsbin is reporting? With that type of connection, anything over 100mbps is exciting. I think Quade can get like 800mbps to his local server so we know Newsbin is up for it as long as the machine and network is. At those kinds of speeds, there are lots of potential bottlenecks. If you are saying you have a 300mbps connection and only get 30mbps download then I'd say it's worth looking for an issue.

Well, okay, the thing is that I was 'worried' about the speed jumping up and down so strongly.
If the download would have been stable within -let's say- 20Mbps up and down, I would not worry.
But these strong fluctuations made me wonder if it is normal and tfore post here.
I did some testing (using the "Speedmeter" component of Networx) and depending on what you download
the average is roughly somewhere around 200-220 or so.
(I agree, if you are used to 10, 20 or 30Mbps, the above looks nice, definitely, but is 2/3 of the subscribed speed).

Bear with me, I am no expert on this, if this is okay, then it is okay with me. DOnot have anyone else to check with
Would an upgrade improve and what is your idea about tagging SSL again?

Thanks!
=

I would say the up and down you are seeing is normal as long as the average datarate is high. Version 6.50 may be smoother because it does more in memory by default. A higher powered machine with faster hard drives and alot of memory would give the best performance.

I don't believe SSL would make a noticeable difference in your observations unless the server is overloaded and they can't handle the load.

Okay, many thanks. I will install v 6.50 one of these days.

Again thanks!
=