failed to clear socket should never happen

[pjcamp]

That's the error message. What does it mean? It happens in the middle of a header download and all the headers downloaded up to that point disappear.

[Quade]

It means something is probably killing the connection out from under Newsbin. If you use SSL, you might try turning it off and seeing if that changes anything. If it does, it would suggest that maybe your security software is stepping on the connection.

[pjcamp]

It means something is probably killing the connection out from under Newsbin. If you use SSL, you might try turning it off and seeing if that changes anything. If it does, it would suggest that maybe your security software is stepping on the connection.

I tried two things. First I turned off the security software. Same problem. Then I turned off SSL as well. Same problem.

I'm wondering if it has to do with the number of headers. This is a biggish group, probably hit by the recent spammers (which started this whole cycle in the first place). I nuked the whole installation and started over. So it is trying to grab a fairly large number of headers all at once.

[Quade]

I'm wondering if it has to do with the number of headers. This is a biggish group, probably hit by the recent spammers (which started this whole cycle in the first place). I nuked the whole installation and started over. So it is trying to grab a fairly large number of headers all at once.

I suppose anything is possible. Newsbin doesn't really do anything with the downloaded headers except write them to disk. The actual header processing happens at a different time.

What server are you using? You might want to try disabling "Xfeatures" in the server advanced options.

[pjcamp]

What server are you using? You might want to try disabling "Xfeatures" in the server advanced options.

Newshosting.

I cut the time back from 180 days to 30 days. That's more than enough to get me caught back up. If that doesn't work, I'll try xfeatures.

[pjcamp]

Well, turning off xfeatures caused a different error. Now I get this:

[02/11 11:18:57] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded
[02/11 20:41:34] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded
[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded

[dexter]

I've seen a couple of reports of this. I wonder if NewsHosting implemented some new connection timeout or something. You should be able to just do a "Download Latest Headers" on the group and it will pick up again from where the server errored out.

[pjcamp]

I've seen a couple of reports of this. I wonder if NewsHosting implemented some new connection timeout or something. You should be able to just do a "Download Latest Headers" on the group and it will pick up again from where the server errored out.

Thanks! That's what I'm doing. It comes in stages. If they have done such a thing, I'll change providers.

[dexter]

I'm curious which plan you have with them. I wonder if only their metered plan does this or if it is across their tiers of service.

[pjcamp]

I'm curious which plan you have with them. I wonder if only their metered plan does this or if it is across their tiers of service.

I have the unlimited one.

The error message appears to be on Newsbin's end. The log shows that it sends a bunch of requests for data that go unanswered and then times out. So other than the spam problem slowing it to a crawl, this seems to be on Newshosting's side.

[Quade]

[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded

This error is from the server. It's considered a fatal error in usenet. It suggests they're thrown some max connection time onto each connection.

How long does the connection go before this kicks in?

If you're already caught up with your header downloads, I'm surprised you'd see this.

Might ask Newshosting what the whole point of this error is.

[pjcamp]

[02/12 06:00:31] ERROR NEWS SERVER ERROR: Newshosting SendCommand - Failed: 500 max connect time exceeded

This error is from the server. It's considered a fatal error in usenet. It suggests they're thrown some max connection time onto each connection.

How long does the connection go before this kicks in?

If you're already caught up with your header downloads, I'm surprised you'd see this.

Might ask Newshosting what the whole point of this error is.

I did ask Newshosting. Their first claim was that it was a problematic interaction between Newsbin and my security software. I agree with you that it is more likely to be on their end. It went for several hours before bailing.

[dexter]

Their first claim was that it was a problematic interaction between Newsbin and my security software.

I suppose this is possible if your security software is holding a persistent connection or something. Try configuring your security software to ignore Newsbin and see if that makes a difference. Or if you can live without it for the duration of a test download, disable your security software altogether, reboot your machine, then start Newsbin and see if you can get this failure again.

[pjcamp]

The log file is a little more explicit now in that there is a new line. It goes like this.

Code: Select all

[02/17 15:54:45]  DEBUG XOVER Numbers: <newsgroup> Min: 272082411 Max: 273082411 Newshosting
[02/17 15:54:45]  DEBUG Newshosting <newsgroup> XOVER: <newsgroup> 272082411:273082411
[02/17 15:55:00]  DEBUG DbWorker: Db Gz to Db  Failed: <newsgroup>

After trying this a buncha buncha times, it also says:

Code: Select all

[02/18 04:51:35]  ERROR Server:Newshosting NNTPServer: Failed to read text line:

once.

So I'm assuming DbWorker is local and is failing to load information into a database, perhaps because that information is corrupt?

[Quade]

I count at least 3 issues which look semi-unrelated.

1 - GZ to DB error. Has nothing to do with downloading headers. It happens after a successful download when headers are imported.

2 - 500 error from the server for old connection

3 - Read timeouts while reading headers indicating the connection dropped out.

Might be worth disabling xfeatures in the server advanced options. Then restart Newsbin. I'm not seeing any obvious security software symptoms but since security software does tend to F up downloading, it wouldn't shock me that it's related to that.

[pjcamp]

I count at least 3 issues which look semi-unrelated.

1 - GZ to DB error. Has nothing to do with downloading headers. It happens after a successful download when headers are imported.

2 - 500 error from the server for old connection

3 - Read timeouts while reading headers indicating the connection dropped out.

Might be worth disabling xfeatures in the server advanced options. Then restart Newsbin. I'm not seeing any obvious security software symptoms but since security software does tend to F up downloading, it wouldn't shock me that it's related to that.

xfeatures is disabled already. Security software also disabled, though that doesn't make it go completely away.

[pjcamp]

OK now it timed out while paused. That doesn't seem like it would be server related.

Just to be sure, I'm going to get a free 3 day trial from Supernews. That's a different Tier 1 provider entirely.

[Quade]

OK now it timed out while paused. That doesn't seem like it would be server related.

Pause doesn't pause header downloads. Headers are different then file downloads. There's no good way to stop the download on the fly. The news server protocol doesn't support it. If you want to stop a header download you're better off just deleting it, then updating the groups again when you're ready for more. Newsbin will continue downloading from where it left off as long as you don't "Download all Headers" next time you download headers.

[pjcamp]

OK now it timed out while paused. That doesn't seem like it would be server related.

Pause doesn't pause header downloads. Headers are different then file downloads. There's no good way to stop the download on the fly. The news server protocol doesn't support it. If you want to stop a header download you're better off just deleting it, then updating the groups again when you're ready for more. Newsbin will continue downloading from where it left off as long as you don't "Download all Headers" next time you download headers.

Ah. Then Supernews will be a good test. So far so good.

But if you can resume a download by deleting and restarting, why doesn't that happen on a timeout as well?

[pjcamp]

Well, it is interesting so far. Still haven't got a full set of headers, but there are no timeouts from Supernews. It has stopped downloading three times but each time was the same error message: Failed to read text line. That's progress of a sort.

[Quade]

But if you can resume a download by deleting and restarting, why doesn't that happen on a timeout as well?

A 500 error is considered to be a permanent error. Any lesser error, Newsbin just retries the download. Other news servers don't do this. It's worth thinking about trying to mitigate this but since it's a new issue, it would have to examined first.

. It has stopped downloading three times but each time was the same error message: Failed to read text line. That's progress of a sort.

This is likely a simple read timeout without the 500 error.

[pjcamp]

So this was interesting.

I got an account on Supernews. That improved the situation a little bit. I could at least see headers long enough to see who the offending poster was and put him in the filter accordingly.

So does that mean he's now invisible, banished from Newsbin? Well, no. There are a couple and they still cause the headers to load painfully slowly, and they still appear in the list alongside a red icon and a status message "Poster Filtered." That seems somewhat less than helpful. If he's filtered, should he not be gone altogether? Totally absent? I strongly suspect I'm missing a configuration somewhere but damned if I can find it.

[dexter]

Depends on where you are applying the filter. Sounds like you are just using a display filter which means the header info is still in the database. If you turn filters on for the post list by clicking the icon that looks like a power button, it will actively hide any headers matching your filter. Otherwise, the filtered posts are still displayed and are just tagged as filtered as you are seeing now.

If you want to keep these posts from getting in to the database at all, you need to apply the filter as a header download filter.

[pjcamp]

Depends on where you are applying the filter. Sounds like you are just using a display filter which means the header info is still in the database. If you turn filters on for the post list by clicking the icon that looks like a power button, it will actively hide any headers matching your filter. Otherwise, the filtered posts are still displayed and are just tagged as filtered as you are seeing now.

If you want to keep these posts from getting in to the database at all, you need to apply the filter as a header download filter.

Thank you! That did the trick. I knew I was not finding something.