Errors coming up...

[mdelig00]

I'm on an unlimited cable modem connection using supernews.com as my usenet client. Once in a while I get these errors and I don't know why. When the errors come up I'll check the Newsbin program and it is still running (it seems normally). I get these errors when downloading a lot of new headers in groups. I'm thinking they are passive non-critial errors that aren't in anyone's control maybe? Maybe these errors do happen normally and aren't logged in other programs?? Maybe someone can shed a little light?

What the real bad thing is here, some times it could take hours to get all hearders from multiple groups...when these errors come up does that mean the timeout reset the download all the way back to 0 and started all over again? It is hard to tell since there isn't much information displayed as in header downloading.

Using the latest version on a Windows 64bit machine. Thanks.



[02:10:34] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Compressed Headers - Timeout Waiting for Server to send Data: Socket - Read or Write Timeout
[02:10:34] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Timeout - Waiting for Server to send Data: Socket - Read or Write Timeout
[04:16:05] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Compressed Headers - Connection Failed while reading Data: Socket - Read From Server Failed
[04:17:19] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Compressed Headers - Timeout Waiting for Server to send Data: Socket - Read or Write Timeout
[04:17:19] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Timeout - Waiting for Server to send Data: Socket - Read or Write Timeout

[DThor]

Which release are you using?

Odds are it's just a dodgy connection to the server, which might even be on their end since header downloads have been tripling or more lately since the demise of nzb matrix. A lot of users out there have started to use headers. It's also possible an update to your antivirus is getting between you and the server, but unless you're getting fail across the board, this isn't likely.

Newsbin picks up wherever it left off downloading headers by design, so you don't need to worry about wasted downloads. If this is really prominent, and especially if you're having to constantly kick newsbin and restart header downloads, this might be something you'll want to look into a little deeper, but otherwise it's probably safe to ignore.

DT

[mdelig00]

Which release are you using?

Odds are it's just a dodgy connection to the server, which might even be on their end since header downloads have been tripling or more lately since the demise of nzb matrix. A lot of users out there have started to use headers. It's also possible an update to your antivirus is getting between you and the server, but unless you're getting fail across the board, this isn't likely.

Newsbin picks up wherever it left off downloading headers by design, so you don't need to worry about wasted downloads. If this is really prominent, and especially if you're having to constantly kick newsbin and restart header downloads, this might be something you'll want to look into a little deeper, but otherwise it's probably safe to ignore.

DT

Using 6.41 RC1 2122.

When ever I have problems with connection issues with Newsbin Pro I'll close down Newsbin and switch over to another usenet client. I do this basically for testing and trying to figure out what the issue is. The timeout issues and disconnects happen only with Newsbin Pro. The other news client I use also downloads headers in the same fashion. Compressed also. I've been able to download headers non-stop without timeout issues, but if it doesn't reset the header download or really cause a serious issue then I guess it isn't that big of an issue right now.

[Quade]

Does the other client do header compression? Is it using SSL? I think it's a firewall that's stepping on the connection. With compressed headers, the header download is binary. That's not usenet spec. It's a new feature between me and the news servers for high speed header downloads. If the V scanner or firewall is expecting text, it might kill the connection if it see's something it doesn't like.

I use giganews and astraweb. Regularly get compressed header downloads up around 400 Mbps and get few timeouts.

One thing you could try would be to disable XFeatures in the server options, restart Newsbin and see if you still get timeouts.

[mdelig00]

Does the other client do header compression? Is it using SSL? I think it's a firewall that's stepping on the connection. With compressed headers, the header download is binary. That's not usenet spec. It's a new feature between me and the news servers for high speed header downloads. If the V scanner or firewall is expecting text, it might kill the connection if it see's something it doesn't like.

I use giganews and astraweb. Regularly get compressed header downloads up around 400 Mbps and get few timeouts.

One thing you could try would be to disable XFeatures in the server options, restart Newsbin and see if you still get timeouts.

Yes, I stated above the other client uses compression also. It is also using SSL, port 563. I do not have any firewalls turned on, nor do I run any anti-virus software. By disabling the X features, what is being lost?

[Quade]

I have a theory which might explain why it would only be an issue for some people. Newsbin downloads N headers and then writes them out to disk, then downloads some more. I wonder if for some people it takes an inordinate amount of time to actually write the data so, the connections time out. It would explain why I never see it. I only download headers with 10 connections and have an exceptionally fast computer. I've done alot testing with this issue, even switched to a different SSL implementation and was never able to repro it. That's another thing that suggests it's timing related.

I'm going to some up with a test version that writes less data per write but, writes more often and see if that makes a difference for the people who have this issue. I find it hard to believe that the writes are that slow but, every PC is different. It might even be a V scanner or something that slows the writes.

[mdelig00]

I have a theory which might explain why it would only be an issue for some people. Newsbin downloads N headers and then writes them out to disk, then downloads some more. I wonder if for some people it takes an inordinate amount of time to actually write the data so, the connections time out. It would explain why I never see it. I only download headers with 10 connections and have an exceptionally fast computer. I've done alot testing with this issue, even switched to a different SSL implementation and was never able to repro it. That's another thing that suggests it's timing related.

I'm going to some up with a test version that writes less data per write but, writes more often and see if that makes a difference for the people who have this issue. I find it hard to believe that the writes are that slow but, every PC is different. It might even be a V scanner or something that slows the writes.

Well the computer is a medium powered machine. 64bit with 8GB of ram and the main OS drive is an SSD drive. I did change the data folder to a mechanical drive because the headers take so much space up. If you want me to try anything, let me know. My trial period is over in a couple of days and the only thing I can't decide on is what search package to get. I find the unlimited to be pretty expensive compared to others, but maybe 500/month may be fine. Does that 500 mean what it says. 500 searches per month regardless on what results come back? I've been using the software heavily more and more and slowly getting more familiar with all the options/settings. Thanks.

[Quade]

Most people don't hit the 500 limit. I don't know your usage though. Some people have a couple TB in the download list at all times and download 24x7. People like that probably need unlimited. Normal people probably don't.

I'll get a test version to Dex tomorrow for people to try.

Unlimited search works out to 16 cents a day or $5 a month. Regular is 1/2 that.

[mdelig00]

Most people don't hit the 500 limit. I don't know your usage though. Some people have a couple TB in the download list at all times and download 24x7. People like that probably need unlimited. Normal people probably don't.

I'll get a test version to Dex tomorrow for people to try.

Unlimited search works out to 16 cents a day or $5 a month. Regular is 1/2 that.

I'll probably just end up getting the unlimited for 3 months and then decide if I even need unlimited. I doubt I'll ever do that many searches at all to be honest. I'm tapped into several good usenet indexers with a large retention and connected with Sickbeard/Sabnzbd. The only time I would really use searching would be for music and/or personal stuff.

I did install the Newsbin Pro on my main/fast computer. Added some groups and went to download headers and the critical error timeouts started to pop up again. This is a very fast computer, with SSD drives installed. So maybe the timeout issues are server related? I haven't switched off SSL yet. I use Supernews news service. I had Giganews, but after using Supernews and finding out I was getting the same speed, retention and threads all for a huge less amount of money I dropped Giga and went with Super. They use the same backbone Giga uses anyway. Maybe I'll try to turn off SSL?

[00:34:55] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Compressed Headers - Timeout Waiting for Server to send Data: Socket - Read or Write Timeout
[00:34:55] ERROR NNTPSocket - Server:news.supernews.com NNTPServer: Timeout - Waiting for Server to send Data: Socket - Read or Write Timeout

**update**

After turning off SSL, there hasn't been any timeouts at all. They would usually come up pretty quickly and consistently. Now, nothing so far and it's been about 10 minutes. What would SSL have to with getting timeouts and no timeouts with SSL turned off? I would of course rather use SSL.

**update**

No errors still, and this is like 30-40 minutes later with straight downloading headers. So using SSL does change something within the program and how it connects to the server? Are there built in timeouts that are different between the two? NON SSL -vs- SSL?

[Quade]

What virus scanner/firewall do you use?

Some of them do a "man in the middle" attack on SSL to monitor encrypted data. They tend to F things up. SSL changes time internally too.

I normally suggest to people, complete removal of any non-MS virus scanner and firewall, reboot. Then test downloads. See if the problem is still there. Just disabling them doesn't seem to work. They're woven into the OS at the driver level.

Then if that makes the problem go away, re-install the protection and then see if it continues working. If not, find a different security product.

[mdelig00]

What virus scanner/firewall do you use?

Some of them do a "man in the middle" attack on SSL to monitor encrypted data. They tend to F things up. SSL changes time internally too.

I normally suggest to people, complete removal of any non-MS virus scanner and firewall, reboot. Then test downloads. See if the problem is still there. Just disabling them doesn't seem to work. They're woven into the OS at the driver level.

Then if that makes the problem go away, re-install the protection and then see if it continues working. If not, find a different security product.

As I stated before I don't use anti-virus software. Literally, nothing. Not even MS security essentials.

[Quade]

Well, lets get more detailed then. The occasional connection timeout just doesn't matter. It matters if it prevents a header or file download. Long term TCP connections, like those used to download from news servers fail sometimes. That's the nature of the beast. It's only a problem if it prevents download. Both header downloads and file downloads have internal retries so, failed connections just try again.

Do your timeouts prevent download or do they just appear from time to time and then go away while download continues?

I get the occasional timeout too. Maybe one or two a session. It has no impact on downloading. I could just not report it in the logs but, I think more information is better than less information.

[mdelig00]

Well, lets get more detailed then. The occasional connection timeout just doesn't matter. It matters if it prevents a header or file download. Long term TCP connections, like those used to download from news servers fail sometimes. That's the nature of the beast. It's only a problem if it prevents download. Both header downloads and file downloads have internal retries so, failed connections just try again.

Do your timeouts prevent download or do they just appear from time to time and then go away while download continues?

I get the occasional timeout too. Maybe one or two a session. It has no impact on downloading. I could just not report it in the logs but, I think more information is better than less information.

I've seen the timeout issues mainly when downloading headers and not when downloading stuff. I use many different usenet client programs, and none of them have timeout issues. Maybe because Newsbin reports them and the others do not? It is something I can live with if it doesn't "kill" the download or reset the header counts. Thanks.