newsbin.com

by **jackholexxxx** » Wed Oct 14, 2015 11:12 pm

Version: 6.6.1 Build 4309
News server: Newshosting

Over the last week or so I have been having a lot of files go to the failed list. There is nothing in the error column or logging tab. Usually I just double click them and they download. Now I have one that even with <CTRL-Y> it immediately fails again. Also, it seems lately that many files that are shown complete from headers are needing repair after download. I did try deleting the entire spool folder thinking something may be corrupt, but the problem persists. Any suggestions?

edit: I found another file that immediately failed and was able to download with <CTRL-Y>. I ran MacPar against the files and most of them showed invalid checksum. This was downloaded from the search tab (internet search) and was shown as complete in both the search and download list.

by **Quade** » Wed Oct 14, 2015 11:25 pm

I have another user from Newshosting with exactly the same symptoms. On the other hand Dex tried some of his downloads using Newshosting and they were perfect. So, I'm aware that something is happening but I'm not sure what yet. Were you using the Beta's up to now?

How are you adding the downloads?

by **thwartme** » Wed Oct 14, 2015 11:30 pm

This sounds identical to the problem I've been having, except I'm downloading headers from Giganews instead of Newshosting. I use both Giganews and Newshosting to download actual files.

by **jackholexxxx** » Wed Oct 14, 2015 11:32 pm

I have been using the betas and recently installed the final build. I am using headers mostly and just double clicking the files.

I just restarted newsbin and was able to download the first file I mentioned with <CTRL-Y>. When I ran a par check most of the files gave the invalid checksum and could not be repaired with the par files provided.

I have an astraweb block account and was able to download the file without issue or repair. I hope newshosting isn't crapping out. They have been a really good provider for me since I left giganews.

by **thwartme** » Wed Oct 14, 2015 11:49 pm

Disabled Newshosting, refreshed headers, all files downloaded fine. I know correlation is not causality, but it's looking like Newshosting may be the problem.

by **kalzekdor** » Thu Oct 15, 2015 12:29 am

I'm having this same issue, off and on for about two weeks. I'm using v6.61 (though behavior was present in v6.60 as well), and my usenet provider is Easynews.

Downloads will randomly fail, either right away or partway though, without any error. Only thing that shows in the logging tab is successful article retrievals like:

Code: Select all: [23:15:32] HIGH Easynews ARTICLE <1444874300.60301.12@reader.easyusenet.nl> | 220 0 <1444874300.60301.12@reader.easyusenet.nl>

No clue what's causing it, as I can usually download the same files over their web interface.

by **Quade** » Thu Oct 15, 2015 12:31 am

Disabled Newshosting, refreshed headers, all files downloaded fine. I know correlation is not causality, but it's looking like Newshosting may be the problem.

It might be the passage of time too. Like the files weren't completely on newshosting yet or something but then became complete. I'm watching this issue closely. I suspect a 6.62 will be required but I don't have a handle on it yet. I'm not to the point where I'd blame the server yet. I think there's something weird going on inside Newsbin.

Do you see any weird logging messages? The guy over email, every file was tagged "bad" by the PAR scanner. He was getting "bad yenc format" messages too.

by **Questar_99** » Thu Oct 15, 2015 1:17 am

Same here on Easynews, ever since I upgraded to 6.61. Downgraded back to 6.60 and problem solved.

by **thirtybird** » Thu Oct 15, 2015 1:19 am

I came here for the same issue. It was present in 6.60, and is in 6.61 as well. I'm downloading headers every 20 minutes with 4k overlap from Usenetserver. Files are being added to the download list by an auto-download filter, and a number of them immediately or shortly fail with no error. Manually adding it with ctrl-E or ctrl-Y still fails, letting time pass and reloading the past 50-100k headers I am usually able to download it eventually. I took one of the files that was causing this trouble and exported it to nzb and attempted to download it with an older version, it did not fail, but it was incomplete and missing some blocks even though newsbin reported it as complete.

Is there any setting we can enable to help debug more thoroughly?

by **kalzekdor** » Thu Oct 15, 2015 5:13 pm

I suspect it has something to do with the way NewsBin detects incomplete files. Usually waiting an hour or two and then retrying the download will work. It's incorrectly stating that all parts are available, when they're not. At some point it tries to download a part that doesn't exist, but doesn't know how to handle that because it assumes it does exist, and so it fails the download silently.

If that's the case, then even a couple of missing posts, which would normally be repaired thanks to parity files, could cause an entire download to fail.

Just my intuition based on observed behavior. Might help point you in the right direction, at least.

by **Quade** » Thu Oct 15, 2015 6:01 pm

I think you're close to the heart of it. Here's what I think is happening. All the chunks download but some of the chunks are corrupted so every file is damaged. Damaged enough a repair is impossible which is why adding them back to the download list doesn't make them continue the download. Because all the downloaded chunks are still there, both the good and the bad.

I've been testing this last night and today and the one thing I notice is more damaged chunks than I've ever seen before in a particular download. I can tell they're damaged because most are missing about 500K of the 700K chunk but the download completed (it had the right end tag to complete it). Newsbin doesn't fail chunks with yEnc format errors because some posters simply didn't post correctly. I'm wondering at this point whether I should start rejecting posts that have format errors and perhaps let any other servers try it.

So there are really 3 failure modes for chunks.

1 - Chunk not on server, classic 430 error.
2 - Chunk damaged. It's there but it's not exactly what it should be.
3 - Server delivers the wrong chunk when requesting the correct one.

#1 is common.

#2 seems to have become a problem just recently.

#3 is exceptionally rare.

I think what we've been seeing over the past couple days is #2. Chunk downloads but you still result in a corrupted download.

I'm probably going to slap a beta up soon for people to try. It'll have more logging and you'll be able to turn on strict yEnc format checking,

by **Questar_99** » Thu Oct 15, 2015 8:18 pm

I think you may have hit on it Quade. Maybe my downgrade fixing the problem was coincidence, and in the interim the correct posts appeared on the server?

BTW, I'm using NZB files and not headers.

If you post a release with more logging I'll give it a shot.

by **Quade** » Thu Oct 15, 2015 9:47 pm

I'm hoping to have something up tomorrow.

by **S3v3n11** » Fri Oct 16, 2015 12:46 am

I want to add a me too to this problem. Giganews and Usenet.

by **sfetchit** » Fri Oct 16, 2015 1:46 pm

I've been having this problem for a number of days now - it appears to be 2 separate problems occurring - one is when the posts are not complete on one server and there are multiple servers in use - for example I have astraweb, usenetserver, and newshosting. When downloading from astraweb from a newsbin search that showed as complete, the download failed immediately with no error message as to why - when I switched off astraweb and just used usenetserver, the post downloaded complete no problem. The second problem is that when downloading a post set from the newsbin search now, sometimes instead of downloading the correct contained posts, it duplicates the first post in the list (usually the rar) multiple times, resulting in a failure where it's trying to download the same file multiple times. In these cases I've had to expand the post set and select/download the individual files where it doesn't then duplicate the first post. Hope this helps.

by **S3v3n11** » Fri Oct 16, 2015 5:50 pm

S3v3n11 wrote:I want to add a me too to this problem. Giganews and Usenet.

To add some information when I disable Usenet and just use Giganews the problem seems to go away. Also I did not have this problem in 6.60.

by **MichaelW** » Fri Oct 16, 2015 10:45 pm

I might be able to add some more information. It seems to be host specific. It started for me on usenetserver.com 3 days ago. The files would go immediately to failed with nothing for a reason in the logging tab, even if on debug (though it would show up in logging with no info attached). If I disabled usenetserver and let it use my ISP newshost it would work flawlessly (they are also a highwinds host like usenetserver so the problem is definitely on usenetserver's end)/ The problem also seems to be time dependent, when it occurs seems to be 0-8 hours after the initial post and then seems to resolve which is weird. I just downloaded build 4314 and it seems to have alleviated at least some of the problem, I will continue to test it tonight.

by **Moley** » Sat Oct 17, 2015 1:18 pm

Another Data Point

I'm having this problem on 6.61 and 6.56, I use the Blackhole, 75% of files download w/o issue, the remaining 25% stay at what appears to be 100% in Newsbin but unable to unrar. If I delete the Folder and remove the listing from download list in Newsbin, re-add the original NZB from the blackhole folder , the files download w/o error. Aint that some shit eh Image link not allowed for unregistered users. I can repeat this with all the failed files, but it is random, it happens mostly on non-truncated file names.

by **Quade** » Sat Oct 17, 2015 4:06 pm

I'd give the new beta a try and if need be crank up the retries. They gate when Newsbin decides to try to repair.

by **sabalon** » Sat Oct 17, 2015 6:13 pm

Just wanted to add a me too. Using usenetserver. I had started the download before the headers were completely downloaded, I guess. I pulled the nzb out of the post, and opened it and started the download. Somewhere it failed, and just stopped, moving it to failed. Usually I could tell it to try again (Ctrl-Y) and it works, but it is like it refused to get past one missing bit somewhere. The same nzb downloaded fine from the same server via sabnzbd. Not sure how much it had to rely on PARs, if any, but the file is fine.

Same thing with another nzb the other night as well.

by **tacomaguy20** » Sat Oct 17, 2015 6:29 pm

MichaelW wrote:I might be able to add some more information. It seems to be host specific. It started for me on usenetserver.com 3 days ago. The files would go immediately to failed with nothing for a reason in the logging tab, even if on debug (though it would show up in logging with no info attached). If I disabled usenetserver and let it use my ISP newshost it would work flawlessly (they are also a highwinds host like usenetserver so the problem is definitely on usenetserver's end)/ The problem also seems to be time dependent, when it occurs seems to be 0-8 hours after the initial post and then seems to resolve which is weird. I just downloaded build 4314 and it seems to have alleviated at least some of the problem, I will continue to test it tonight.

I primarily use usenetserver and this has been going on for about a week or so. I did try blocknews and had a limited amount of data left that I could download, but I seemed to be having the same issue. I just sent usenetserver an email asking if they are aware of the issue and if they are working to resolve it.

by **thirtybird** » Sat Oct 17, 2015 7:41 pm

Grabbed and installed 6.62 B1. File I was just trying to DL that was going immediately to the failed list before the upgrade is downloading in the new beta, but wow, is it showing massive missing pieces even though it's supposedly complete.

EDIT: So I'm watching it do retries, and each time it gts to the end, it clears the missing pieces and retries, and with each pass it finds more and more pieces. It went from about 30% complete on pass 1 to 65% complete by pass #4. Incredibly bizarre to watch

by **Quade** » Sat Oct 17, 2015 8:58 pm

The servers are all screwed up at this point. The only way to get all of the files is to keep trying. It's looking like the server the files are getting posted to is struggling to feed the new files to every other news server.

by **xenutv** » Sun Oct 18, 2015 5:07 am

I'm having the same problem here with Usenetserver. I'm using the latest release on a mac mini through the Cross Over app and it has always worked great. Lately a large number of downloads have been failing with invalid checksums stopping the files from working. I installed the new release today with hopes it would help but it hasn't. When I turn to a different program (Unison) it downloads the files fine from Usenetserver. I just hate using Unison instead of Newsbin.

by **Quade** » Sun Oct 18, 2015 10:54 am

Every other users is telling me B1 is working for them and working around the server problem.

First I'd confirm you're really running 6.62B1 and then I'd make sure the retries are at least 2. You might actually want to try 5-10 retries until the servers are fixed.

by **enchant** » Sun Oct 18, 2015 12:08 pm

I originally posted a problem here:
viewtopic.php?f=43&t=39015
using 6.55. I upgraded and now have the same problem as everyone else.
Usenetserver.

I'll download the new beta and see if that helps.

Edit: sorry, I must be dense. I can't find the beta.
Edit2: Found it.

by **Quade** » Sun Oct 18, 2015 12:34 pm

With 6.55, turning up the retries might have been enough to ride through this too.

by **delboy007** » Sun Oct 18, 2015 4:21 pm

same problem here with astraweb for the last 3 days. Goes straight to failed with logs saying "server reports parts of the file are missing". same for anything I try and download. I have set it to 5 retries.

by **Quade** » Sun Oct 18, 2015 5:51 pm

You should try the new beta. The servers are messed up. The new beta has code changes to make it work even with the messed up servers.

by **delboy007** » Sun Oct 18, 2015 6:35 pm

just tried that and have the same problem. Cleared the existing files etc as instructed.

by **enchant** » Mon Oct 19, 2015 6:56 am

I've got the beta installed and retries set to 20. Last night, all of my shows downloaded ok, but they didn't unrar. I was nervous when I first opened Newsbin and saw this:

But I was able to go to the download directory and manually unrar them. All complete.

I believe I've got my options set correctly, but maybe not?

by **Quade** » Mon Oct 19, 2015 9:38 am

Retries set to 20 is a long assed time. You have the retry counter covered up so I can't see how many retries have happened. If it's less than 20 then Newsbin is doing what you asked of it. Which is to keep trying.

I'd experiment with reducing the retries.

by **enchant** » Mon Oct 19, 2015 9:42 am

I've dropped it to 8.

Edit:
What is Newsbin waiting for on the reties? If it has enough to extract the file, what more is there?

by **enchant** » Mon Oct 19, 2015 10:39 am

FWIW...

Two shows just downloaded for me. The first came through without any problems. The second was like my previous post. It appears all the rars came, but then it stalled for a while. So I tried screwing with the options a little.

First, I tried Aggressive Assembly mode. Didn't make a difference, so I turned it back off.
Unchecked Pause Pars ...
Initially, this didn't make a difference.
I right-clicked on the compacted show line item and "Resume download" - nothing.
Expanded the show, scrolled down to the first par file which said "Paused" and tried Resume download on that. Immediately things sprang to life. It downloaded about 3 par files, repaired the file and automatically unrared it.

I was wondering what it needed to repair, since all the rar files I've seen are ok, but it also downloaded a .srr and .nfo file. So somehow it's not automatically downloading these little extra files and hanging on that? This brings up a couple possible questions.

For one thing, since upgrading last week, my filters don't seem to be working anymore. All these little extra files are getting downloaded. Here's what I have:

by **Quade** » Mon Oct 19, 2015 11:06 am

I run 2 retries. I'd try that. If you still have files going to the failed list, set it to 3. Retries tell Newsbin to keep trying. You have files in the compacted set that didn't complete. I can see the red parts of the progress bar. Newsbin's going to keep trying to get them till you run out of retries. That's what "retries" is for.

The filtered files get deleted after the unrar. Because Newsbin might need them as part of the repair process.

by **enchant** » Mon Oct 19, 2015 11:09 am

Originally when I had the problem, I had 2 retries. I'll try 3.

by **Quade** » Mon Oct 19, 2015 11:42 am

2 retries in 6.61 isn't the same as 2 retries in 6.62. That's the main reason we have a 6.62. 6.61 was more aggressive about assembly.

by **enchant** » Tue Oct 20, 2015 7:03 am

Nine shows downloaded overnight trouble free. Thanks for the help getting this resolved. Still getting all the little files that used to get filtered out, but I'll address that in a separate post.

by **thwartme** » Wed Oct 21, 2015 11:48 pm

Been running 6.62B1 build 4314 for a couple of days now, and it seems to have fixed my problems. No issues with downloads going to the failed list, or posts failing to unrar.

Thanks!

Thanks,
thwartme

by **Quade** » Thu Oct 22, 2015 1:26 am

Thanks for the feedback. I think the servers are recovering too.

by **Questar_99** » Thu Oct 22, 2015 2:31 am

Quade wrote:Thanks for the feedback. I think the servers are recovering too.

Using a provider (Easynews) that has servers located both in the US and Europe, files that silently fail in the US are downloadable from the European servers. I assume there's replication issues coming out of Europe.

Upgrading to 6.62 now.

by **Questar_99** » Sat Oct 24, 2015 7:50 pm

Here's what Easynews has to say on the subject

"Recently we found that people are posting some type of
encoded/encrypted data to Usenet masked as yEnc, but what looks like
UUencode. There are false subject lines and even the start of the
article body says yEnc when it's clearly not."

by **Quade** » Sat Oct 24, 2015 9:24 pm

The question might be, what client can download that properly?

The problem that I'm seeing is that valid posts are being delayed. People are getting the NZB files well before the posts reach their news server so, the clients timeout for lack of data.

If you can get me an example of one of these I wouldn't mind checking it out. Newsbin can do yEnc or UU but it depends on the header to ID the data. I'd have to change things some of this becomes popular.

by **Questar_99** » Mon Oct 26, 2015 10:09 pm

Well the posts kicked Easynews' yenc decoder in the head, their admins had work around them. Do you know any of the admins there?

by **Quade** » Mon Oct 26, 2015 10:39 pm

Well the posts kicked Easynews' yenc decoder in the head, their admins had work around them. Do you know any of the admins there?

It won't work in Newsbin either but that doesn't mean I can't make it work. I just need to see some examples. I'm not seeing any sign of these posts in anything I download. Not seeing any reports of them from my users either.

There is a type of post that encodes yEnc in "quotable-printable" which Newsbin does handle. I wonder if that's what they're talking about?

by **Questar_99** » Thu Oct 29, 2015 9:57 pm

I posted in the Easynews support forum to have an admin read this thread. We'll see what they say / do.

newsbin.com

Who is online