Slow Compressed Header Download Speed Issues

[awilson]

I need some guidance on dealing with slow header downloads.

I am on a 111mbit line (Stofanet.dk) and get that speed when downloading constantly maxed out, but header compression does not seem to be working with Giganews anymore. I am sure that it has been working before so I am not sure why it has now suddenly stopped.

I see guys saying they get 450 mbps on a 100 mbit line when downloading headers but I am lucky to see 30mbit when downloading headers. I have tried the latest beta RC4 and the normal trial download also on separate computers and reinstalled several times, turned firewalls off etc ... but still the poor header speeds.

I have also tried different news servers at Giganews both news.giganews.com and news-europe.giganews.com, SSL on / off

I haven't changed any routers or hardware setup so I am now stumped as to why it no longer works!

Any guidance or advice appreciated!

Thanks

[itimpi]

You need to decide whether you want to optimise header downloads for speed, or for getting headers ready for use quickly. Newsbin v6 does a lot of processing of headers after downloading them to insert them into its local spool databases, and if this is done in realtime then header downloads can become constrained by disk I/O speed rather than your link speed.

What do you have set for Preferences->Switches->Background processing? If this is set then the headers are downloaded as rapidly as possible and stored in holding files on disk (rather like v5 works) with the additional disk processing then being run later as a background task. This mode of operation means that headers are not necessarily ready for use immediately the download finishes. This is indicated by a figure in square brackets appearing against the cache usage figures in the Status bar indicating how many blocks of headers are awaiting processing into the database.

[awilson]

I would prefer to optimise for speed of header download rather than geeting them ready quickly for use.

I have tried both options for Background processing but the data still arrives at relatively the same speed of 50mbit.

The cache is 150/200 (742) shown in the lower section. Xfeatures are enabled (not checked) and I have tried 10-50 connections with Giganews server with same results.

Could my ISP be throttling header retreival - whilst not throttling download?

Thanks again ...

[itimpi]

If you want to optimise for download speed then you need the background processing mode to bet to on.

If anybody would be doing throttling on header downloads it would be the news server, not your ISP.

I notice that you have a large figure in brackets (742) in your figure for the cached headers awaiting processing. This is much larger than I ever see and it suggests that your disk is likely to be the limiting factor. Also you did not give the spec of your machine - maybe it is just not capable of handling the very high download speeds using Newsbin v6 that some are quoting.

I have headers set to download every 6 hours without any intervention from me so it is possible that I am simply not watching at the relevant moment. Do you use the option to get headers automatically at regular intervals? It does tend to make getting them much less of an impact on ones workflow if they are already sitting on disk when you want to think of viewing them.

[awilson]

The machine is a Intel Q6600 with 4GB Ram running Windows 7 x86 - The primary disk is a Samsung 500GB whilst the storage disk I am using is a 2000GB Samsung also ...

[Quade]

If I download headers from enough groups at once, I get good header download speed. 300-600 Mbps from Astraweb. Lately my giganews header download speeds have seemed slower. On a single connection basis, I'm probably not even getting 100 Mbps which is my line speed. It's not clear why. Using both SSL and Header compression, the server has to work twice as hard. I might try disabling SSL and see if I get more consistent speeds.

Edit:

[08:42:00] HIGH NNTPSocket - XOVER 245974302-246274302
[08:42:00] HIGH NNTPSocket - 224 data follows [COMPRESS=GZIP]
[08:42:04] HIGH NNTPSocket - XOVER 92348770-92648770
[08:42:04] HIGH NNTPSocket - 224 data follows [COMPRESS=GZIP]

Watching the logs, with show server commands enabled, I can see that the data is delivered compressed.

[john58871]

I have the same problems,I lost all the headers from a total group,and by downloading newsgroup a.b.multimedia cost my 5 hours ,2 times.
when downloading suddenly stops you lost your downloading,you see nothing in the group when you it opens.
It's maybe Newsbin Pro 6.11 - Build: 1342 problem with unpacking [COMPRESS=GZIP]

I go return to a earlier version,this is shit,I try 5.59 build 9826 no difference.
SSL or regular no difference.

I have complaint by EWEKA ,I hope that they do something.

HIGH NNTPSocket - XOVER 1220793568-1221093568
HIGH NNTPSocket - 224 Overview Information Follows [COMPRESS=GZIP]
HIGH NNTPSocket - XOVER 1221093568-1221393568

I will try your solution.
Another problem the Newsbin download speed they is not correct.


Solved header download is the debt of the provider , speed was to low,i was confused by errors newsbin speed report

[Quade]

when downloading suddenly stops you lost your downloading,you see nothing in the group when you it opens.

1 - Just re-start the download with a "Download Latest". Not a "Download All". It'll eventually finish the header download.

2 - What's your "storage age" set to? If it's not long enough, newsbin will discard records older than the storage age.

3 - Might try right click "show all posts"