newsbin.com

by **netstrader** » Fri Mar 11, 2011 4:59 pm

Hi,

I installed newsbin v6.0b5 for a quick review. The interface is very nice. I noticed that the tool skips files.
I had a nzb file. I dropped it in the downloading box. After the tool downloaded +3GB (the nzb was 4+ GB), it skipped and continued to the next nzb file. I expanded the nzb and saw at least 10+ files hadn't been downloaded. I restarted
the tool and it went back and continued where it left off.

Anyone else see this problem with v6.0B5?

by **Quade** » Fri Mar 11, 2011 6:14 pm

See any errors in the logging tab?

by **netstrader** » Fri Mar 11, 2011 8:21 pm

Quade wrote:See any errors in the logging tab?

There are no constant error messages. I just have "220 0" and "ARTICLE" messages.

[15:01:37] HIGH nhxl.newshosting.com - 220 0 <part109of132.iDEfdKnIFBCb3gRxVoza@powerpost2000AA.local>
[15:01:37] HIGH nhxl.newshosting.com - 220 0 <part110of132.iDEfdKnIFBCb3gRxVoza@powerpost2000AA.local>
[15:01:38] HIGH nhxl.newshosting.com - ARTICLE <part111of132.iDEfdKnIFBCb3gRxVoza@powerpost2000AA.local>

I just tested again with three nzb files. Newsbin reported that it downloaded 4.5GB for the first one and 4.3GB for the second one. It started to download the three one. I expanded the first nzb and saw that everything was downloaded except for the par2 files. I checked the files by manually running quickpar and it reported one file was missing. On the second nzb, it reported that everything was downloaded. I checked the files and there were only 10 files out of 90+ files. I restarted newsbin. Newsbin then downloaded the needed par2 and did the autopar on the first nzb. On the second nzb, newsbin reported that it only downloaded 500MB and continued where it left off. It looked like the index is messing up.

I didn't upgrade the storage header format. I upgraded from 5.5 -> 6.0B4 -> 6.0B5.

by **Quade** » Fri Mar 11, 2011 11:08 pm

I think the issue is probably that 6 doesn't do retries yet. You might be able to make it work better by adding another server, a copy of your main server with a different name and then divide the connections between them. So, if your main server has 10 connections make it 5 each. That'll make it look like 6 is retrying because it'll try both servers. This is just a work around though, apparently you get errors on these files and that's why you need retries. I don't have the same issue using either giga or astraweb.

by **jayd09** » Sat Mar 12, 2011 8:04 pm

I too am experiencing the same issue. What i did notice was that the file is actually downloaded to the NB2 file and then when it goes to compile the file as a rar it just disappears.
This issue never happened before B5.

by **Quade** » Sat Mar 12, 2011 10:50 pm

How big are the files? Each of the rars.

With B5, by default, with files under 20 megs I believe, you'll only see the NB2's for an instant before they get renamed to the real filename. If these are QPMS files (.001,.002 split by quickpar) then it's probably a bug. A work around for the bug, is to specify an unrar folder that's not the same as the download folder.

You issue sounds nothing like the original topic...just saying, I don't care if you post it here.

by **jayd09** » Sun Mar 13, 2011 1:23 am

Each Rar file is 50 MB and they are not split with QuickPar, they are filename.r##
I tried increase the memory cache to do files larger than 50 and still experienced the same issue.
If i close newsbin and re-open it, the file looks like it was skipped and re-downloads. Most of the time on the second try it works fine.

The issue sounds exactly like what the other user was experiencing which is why I did not start another post. Newsbin shows that all files were downloaded but they never get written to disc.
If i manually run quickpar, it will show missing files.

I am currently running windows 7 64 bit with SP1 installed.

by **Quade** » Sun Mar 13, 2011 10:16 am

Yeah, I guess it goes sound the same. What news server are you using?

I'd do what I suggested to the OP then.

by **jayd09** » Sun Mar 13, 2011 2:58 pm

I am using astraweb.

The issue does not seem to be related to the download but the writing of the file to disk. I have watched it create the full NB2 file and then when it goes to write it to the disk the NB2 file disappears but the Rar file is never created. If you close and re-open the program it thinks the file is missing and re-downloads.

This seems to be a new issue with B5, as i did not have this problem with any other Version 6 betas.

by **Quade** » Sun Mar 13, 2011 5:11 pm

I have an idea. I'd edit the NBI file and add this to the settings section

[SETTINGS] - Already in the NBI
MemCacheLimit=200 - You need to add.

What this will do is store more of the file in memory, up to 200 megs
before it writes it out to disk.

- Exit Newsbin
- Edit the NBI file and add this to the "Settings" section
- Save
- Restart Newsbin.

There's basically 2 download modes now, one for large files (like rars, not the set) and one for smaller files. This setting changes the threshold of what a "Large" file is. It's telling Newsbin "anything over 200 megs is a large file" by default it's set to 20.

It's probably a bug but, I think this will bypass the bad code.

by **mbake5** » Sun Mar 13, 2011 5:53 pm

I tried MemCacheLimit=200 and the first 3 files did NOT get written to disk. The 4th and 5th file did get written to disk. My observation is that this problem seems to happen primarily (or only?) on restarts. The files that did not get written to disk show downloaded in the "Downloads" tab but they don't appear in the "Files" tab.

by **Quade** » Sun Mar 13, 2011 6:33 pm

I don't know then.

I'd dig out procmon and see if you can figure out when and how the files are getting deleted. With that setting, for any files under 200 megs, no NB2's are written. Well, the NB2 is there for an instant. Written and then renamed.

by **mbake5** » Sun Mar 13, 2011 9:32 pm

I was running process monitor when the problem occurred. I can't see that any attempt was made to write the file to disk. The files that don't get written always display the icon that it has been downloaded but the two numbers in the size field don't match (i.e. 153,600/158,903 KB). It seems like it gets flushed from memory before it even attempts to write it to disk.

by **Quade** » Sun Mar 13, 2011 10:41 pm

You mean they're still sitting there in the download list? What happens if you right click and "Assemble Incompletes" then? Maybe they're not done.

by **mbake5** » Sun Mar 13, 2011 11:32 pm

Yes, they are still in the downloads list but that is normal behavior since these files are part of a nzb set. The files don't get removed until the set is complete. Assemble Incompletes changed the status bar from solid orange to grey but the "incomplete" file isn't saved.

by **Quade** » Mon Mar 14, 2011 12:09 am

I've seen 5 or so reports of this but, it's not happening to me (though I'm using B6 and not B5).

So, at this point I don't know what to tell you. Not enough information to really see what's going on. It's like it's silently failing to open the output file. Still the failure should show up in procmon. The fact it happens whether it's downloading the new way or the old is odd too.

You're not downloading files that have to be renamed by par files to their real names are you? Some idiot posters obfuscate the filenames and expect the PAR file to rename the files. Newsbin does the rename but, that's about the only reason I can think of that you'd see this but, none of the IRC testers reported it. Do you have the encrypted file filter turned on?

by **blanketparty** » Mon Mar 14, 2011 3:39 pm

I'm having a different but similar problem.

My multipart files look like they finish. I have had several files do this. It shows all the parts downloaded but doesn't auto extract. I tried clicking on assemble incompletes, but still doesn't auto extract. Then, if I exit NewsBin and reopen, it shows some of the parts not downloaded and continues as normal.

Edit: Should have fuzzed out the filenames - don't post information about what you download.

This didn't happen in B4, only started when I upgraded to B5

by **Quade** » Mon Mar 14, 2011 3:49 pm

I looked at your image and the files in the middle aren't complete. They're still downloading. My guess is they weren't complete and you restarted and what you see is them re-downloading. If they show up again with red or black blocks, you'll need to right click "Assemble Incompletes" on the whole set to turn them into files on disk.

by **blanketparty** » Mon Mar 14, 2011 4:57 pm

They show up as 100% complete.

If I right click and auto assemble, they turn grey, But it still doesn't extract. Then when I close and re-open, they re-download and all is fine.

I can watch it happen and reliably reproduce it.

by **Quade** » Mon Mar 14, 2011 5:22 pm

They're re-downloading because they aren't being save to disk. Why they aren't saving is the 64 dollar question.

by **mbake5** » Mon Mar 14, 2011 10:31 pm

No, I am not downloading files that have to be renamed. I do not have encrypted file filter turned on. I am also running Windows 7 64-bit. I have that in common with an earlier poster. Is anyone else experiencing this problem running a different OS?

by **Quade** » Mon Mar 14, 2011 11:20 pm

That's the only OS I use too.

by **rbswft** » Tue Mar 15, 2011 8:19 am

Yes I am too, lots of files downloaded and saved nowhere. Large parts toa multipart download

Running XP 64 bit.

What I have noticed is if the speed gets throttled or paused, this could be casuing the issue. Maybe just coincidence.

by **rbswft** » Tue Mar 15, 2011 10:48 am

Just tested and seems to happen if the downloads are paused for a period of time.

When you go back, the files just download and not saved.

Ammendment.

Afraid happening all the time with/without throttling.
Looks like the nb2 file is not being created, and thus no file is created after download.

Hope this helps

by **Gripen** » Tue Mar 15, 2011 10:56 am

I'm having the same problem but I have finished all downloads by doing assemble incompletes, pausing the rest, saving and restarting newsbin doing assemble incompletes again, this a few times and it eventually if you let it go it works. I've just purchased Newsbin 2 days ago and I must say I'm really impressed by it. Very stable but there are some obvious bugs in this program/beta! Anyway, hope this gets sorted soon. Don't wanna go back to using Newsleecher :')

by **jayd09** » Tue Mar 15, 2011 11:05 am

Hi Quade,
How far away is B6 from being publicly available? You said you are currently running B6 and do not experience the issue. I am curious if the issue is resolved entirely in B6.
This issue was not there in B4.

Did everyone having the issue run the option to reset the gui? I know i did and maybe the issue stems from that. Just speculating at this point.

by **Quade** » Tue Mar 15, 2011 1:22 pm

I never experienced it with B5 either so, I'm not convinced B6's going to fix things for you. I seriously suggest reverting to B4 for those of you who have this problem and then sample some downloads when B6 comes out. I'll try to add some more logging too. Maybe something will popup.

Do you guys use "Use Chunk Folder"? Someone mentioned that symptoms changed when they set that and I've no tried chunk folder in awhile.

by **rbswft** » Tue Mar 15, 2011 3:04 pm

chunks folders does not work in B5. Tried and still same issue.

Could you explain what the nb2 files are? When they appear it works, when they dont , the files do not get created.

Thanks

by **Quade** » Tue Mar 15, 2011 3:16 pm

There's two ways files can download now.

1 - Directly to the NB2 file.
2 - To RAM

The setting MemCacheLimit determines what mode it's using per file. So, if the file is smaller than the limit, it's downloaded to ram, then written out to the NB2 in one shot. If it's larger than the limit, it's written directly to the NB2, one chunk at a time.

With either mode, when the file completes, the resulting data is fed into autopar and signature matching and other things. Then the NB2 is renamed to the real filename and the proper length is set in the file.

by **rbswft** » Tue Mar 15, 2011 4:26 pm

Can you force it all to nb2 file?

What setting should I give the MemCacheLimit to do this?

Add that line in when did the upgrade to B5.

That would make sense if there is a memory issue involved in why the files disappears.

Gone back to b2 and do not have the problem at all.

That line not in the ini file.

by **Quade** » Tue Mar 15, 2011 5:44 pm

This 2 mode download is new as of B5. So, it doesn't matter if that setting is in the NBI or not for older versions.

Turns out you can't turn it completely off in B5. You can't set it any smaller than 20 megs.

by **jayd09** » Wed Mar 16, 2011 1:53 am

I seemed to have resolved the issue by one of two things.
1. I completely uninstalled newsbin and deleted all folders/registry settings
2. Created a new nbi file

I believe step 2 is actually the one that resolved the issue not the uninstall/re-install.

Also just an FYI, A clean install of B5 does not create program shortcuts. I know others have posted about it but I just thought i would mention it.

by **daguenet** » Wed Mar 16, 2011 9:22 am

I have had newsbin not save files that it downloaded and the only way I got it to work around this was to delete the multipart download from the download list and go back into the group and re-add it to the download list. I has taken up to 3 tries to get it to work for one download. It has not happened in a couple of days and I have not changed any setting in either the .nbi file or the options menu.

By the way I am running windows 7 and newsbin b5 both 64 bit.

by **daguenet** » Wed Mar 16, 2011 10:50 am

Here is another strange happening. I have had some files that are downloaded, but newsbin reports them as downloading with 0/xxx and only one hash showing. There files are in the download directory and seem to be complete. At least quick par reports them as complete. Most of the time this happens it is the first few files of a multipart download. Everything seems to work fine other then newsbin will not do the autopar and autorar on these files, due to the files that waiting to download.

P.S. This seems to only happen after newsbin has been stopped and restarted.

by **mbake5** » Wed Mar 16, 2011 9:40 pm

I found and resolved the problem in beta6 (should apply to beta5 too). It was McAfee On-Access virus scanner that was causing the problems. I was able see that in procmon. McShield.exe was interrupting the I/O stream and causing partial .rar files to be written. I disabled McAfee On-Access and the problem has been resolved. There must be something that was changed in beta5 that is causing McAfee to prevent writing some or all of the data to disk. I have never had a problem with the Newsbin/McAfee combination prior to beta5. I am not sure if other virus scanners have a problem with beta5 but this would explain why some users are having problems and others are not.

by **Quade** » Wed Mar 16, 2011 11:04 pm

Wow thanks, knew it had to be something weird because IRC guys didn't report it.

by **DThor** » Thu Mar 17, 2011 8:14 am

To most scanners, running the beta is the equivalent of running a completely different program. The truly low level activity of requesting data from a usenet server is really the same. Perhaps there's a way to bypass it snooping on a given port, or the program itself.

DT

by **chaznet** » Fri Mar 18, 2011 5:53 am

"mbake5"...I disabled McAfee On-Access and the problem has been resolved...I am not sure if other virus scanners have a problem with beta5 but this would explain why some users are having problems and others are not.

I use Microsoft Security Essentials and I'm having the problem. Not sure if it's related as I have the exluded the \appdata\local\newsbin directory; excluded the Newsbinpro64.exe process; and have it set to ignore *.nb! files.

by **DThor** » Fri Mar 18, 2011 7:38 am

Completely disabling and trying some downloads is an easy test. A handful of av are so odious you have to completely uninstall it to really stop the pain, and they don't even make that trivial.
I would think the MS product has a simple 'disable ' switch, though.

DT

by **Quade** » Fri Mar 18, 2011 9:53 am

\appdata\local\newsbin

Is that where the files download to? That's the folder you need to exclude in addition to the data folder. I've never heard of MS Security getting in the way of downloads.

I have MS Security installed but, disabled right now. If you disable the service, you pretty much shut it down.

by **chaznet** » Fri Mar 18, 2011 1:16 pm

Quade wrote:
\appdata\local\newsbin

Is that where the files download to? That's the folder you need to exclude in addition to the data folder. I've never heard of MS Security getting in the way of downloads.

I have MS Security installed but, disabled right now. If you disable the service, you pretty much shut it down.

No, I excluded that directory as it was Newsbin related, not because the files downloaded there; I didn't exclude download directories as I do want files scanned after they're downloaded and extracted -- I excluded the *.nb! files as I assumed those were temporary files used while the download was occurring.

I didn't mention this because I thought it as causing an issue, just that the OP thought he'd found the solution.

by **Quade** » Fri Mar 18, 2011 1:36 pm

I didn't exclude download directories as I do want files scanned after they're downloaded and extracted

6.XB6 downloads to NB2 files so, you want to exclude these files in addition to the NB!. Download problems are related to the download folder so, by not excluding the download folder. You're permitting the scanner to get in the way of download. Maybe you should let the files download, then scan after the fact?

by **chaznet** » Fri Mar 18, 2011 9:56 pm

Quade wrote:
I didn't exclude download directories as I do want files scanned after they're downloaded and extracted

6.XB6 downloads to NB2 files so, you want to exclude these files in addition to the NB!. Download problems are related to the download folder so, by not excluding the download folder. You're permitting the scanner to get in the way of download. Maybe you should let the files download, then scan after the fact?

I don't think it's even related, as it was never an issue until v6b5.

by **Quade** » Fri Mar 18, 2011 10:08 pm

I disabled McAfee On-Access and the problem has been resolved. There must be something that was changed in beta5 that is causing McAfee to prevent writing some or all of the data to disk. I have never had a problem with the Newsbin/McAfee combination prior to beta5. I am not sure if other virus scanners have a problem with beta5 but this would explain why some users are having problems and others are not.

You could do the same thing, use procmon and see what happens when Newsbin writes the NB2 out. By default, any file over 20 megs downloads just like it did in B4. B5's new method only applies to smaller files by default.

The question is, why can Dex and I and the IRC testers download with no issue with the exact same version and you can't. Lets say it is something else. What exactly could it be?

by **chaznet** » Fri Mar 18, 2011 10:11 pm

Quade wrote:
I disabled McAfee On-Access and the problem has been resolved. There must be something that was changed in beta5 that is causing McAfee to prevent writing some or all of the data to disk. I have never had a problem with the Newsbin/McAfee combination prior to beta5. I am not sure if other virus scanners have a problem with beta5 but this would explain why some users are having problems and others are not.

You could do the same thing, use procmon and see what happens when Newsbin writes the NB2 out. By default, any file over 20 megs downloads just like it did in B4. B5's new method only applies to smaller files by default.

The question is, why can Dex and I and the IRC testers download with no issue with the exact same version and you can't. Lets say it is something else. What exactly could it be?

You mentioned you had MSE installed but turned off -- you should turn it on and see if it has any effect -- I don't think it does, but maybe you'll discover differently?

by **Quade** » Sat Mar 19, 2011 12:38 am

I tried a single set with essentials turned on and everything downloaded file. Win7 64. Upgraded to the newest version of security essentials, added no rules and everything download fine.

So, I don't know why you're having problems. Are you SURE your problems are like what the original poster posted about?

by **chaznet** » Sat Mar 19, 2011 1:19 am

Quade wrote:Are you SURE your problems are like what the original poster posted about?

...no, I thought this topic seemed similar but not exactly the same so I started another topic, you directed me to this one...

by **Quade** » Sat Mar 19, 2011 7:19 am

How about re-stating your issue and how it differs from the original issue? Kind of "back to square one" and then I'll think about it some more.

by **chaznet** » Sat Mar 19, 2011 1:09 pm

Quade wrote:How about re-stating your issue and how it differs from the original issue? Kind of "back to square one" and then I'll think about it some more.

viewtopic.php?f=43&t=27928

by **Quade** » Sat Mar 19, 2011 2:57 pm

Sounds like the same issue as the OP to be honest. I'm sure that's why I suggested you post here. Newsbin thinks it wrote the data out but, his virus scanner blocked the write.

Next version will let you totally disable, the high speed in ram download mode. Maybe you just can't use it.

newsbin.com

Who is online