Newsbin queue messes up Win7 x64

[skoolrumble]

Hi, I'm using the 64-bit of newsbin 6.2 pro, and this seems to always mess up my windows.

I'll queue a whole bunch of stuff, some of it will not unpar/unrar (they'll just show a bunch of files in green and won't auto assemble unless I do it manually). Newsbin will take up a ludicrous amount of RAM (over 1GB, I have 8GB total). And will also screw up my windows. I'll have problems clicking on other programs, especially windows explorer. It will just hang when I try to do so, with the perpetual ring of death. Task manager can't even seem to close these programs anymore. Not even newsbin. Attempting to do so will just cause my computer to lock and I'll be forced to do a power button reboot.

Help. Is there some setting I can change in autopar that will prevent this? It seems to come down to having a bunch of files in the queue that were complete (rar files downloaded, maybe with some in green still, but no pars are downloaded) but weren't unparred. I think it's because those files in green aren't doing the auto-assemble incomplete bit. They're just staying there. I also wind up with tons (sometimes hundreds) of of those temporary unrar folders (I assume they unrarred properly), but just weren't moved.

[DThor]

You'll need to provide more information, what's 'a whole bunch of stuff'? I almost never see Newsbin hit that much memory, but maybe you're adding a lot more than me.

As far as 'messing up your windows', well, it's hard to tell what that is. You might be doing some very serious IO, you might have a problem with your windows installation. Without specifics it's hard to say. 8G should be fine, but you need to troubleshoot. For example, you seem to assume it's a memory thing. 1G out of 8 isn't a big deal. If you're not running craploads of other things, then it's likely IO is hammering your system. 'Some green' means chunks are downloading. If you're doing par repairs, that will hit your cpu and IO simultaneously. Again, this is all guessing on my part. Really, you need to debug the situation, try to isolate what's happening when you're having specific issues. What cpu?

DT

[skoolrumble]

whole bunch of stuff is just tens or maybe even over a hundred gigs of stuff. I queue the stuff through nzb files, if that matters. some of the stuff to be unrarred are over 10 GB in size. I think the problem comes down to a file failing to unrar/repair in autopar. When that happens, it seems to cause the files following it to also fail to autopar (and I'm guessing, the memory to balloon)?

I'm reading here that it waits 48 hours to assemble incompletes. I can download 50GB in 2 days. So maybe that's causing the problem and the backlog?

The problem I'm having is that it seems to lock up my windows. I can't close any program or even select them. I wind up getting that Windows ring indicating it's waiting for something. And then eventually, I can do nothing as the whole thing seems to have locked up. I then have to reboot.

I'm not saying it's a memory thing that's causing the lockups (unless newsbin is also grabbing a lot of the cached memory and also forcing system resources onto the HDD page file? And this usually happens when I leave it alone for a while and nothing else is running, except for some windows explorers.

I have an intel core i5-2500k. Thanks.

Someone has a related problem here,
viewtopic.php?f=43&t=28826&p=176343&hilit=max+retries#p176343

[DThor]

Nothing you've mentioned is particularly shocking in terms of scale, certainly 10G downloads aren't a big deal, and say 20 of those in the queue, again, no biggie. The issue of adding a whole mess of incompletes can indeed cause issues if they get backed up, since AFAIK there's a limit to how far Newsbin will 'look ahead', so if you had many hundreds of them sitting there, it could cause Newsbin to pause and might need intervention.

None of this, to me anyway, explains the effect it's having on the rest of the system. Basically, check and see just how much memory is free, how much Newsbin is using, that should indicate if memory is related. Sounds to me, though, that something else is up. Certainly extremely intense IO can trigger similar situations on some configurations. Unless I missed it, you never mentioned disk activity. Is it running like mad? Are you constantly updating headers at the same time? Updating headers and par repairs will push your IO bandwidth, hard. Your system *shouldn't* be lagging like that, but some systems, I've seen that behaviour when disk IO is insane.

One other quick thing to check - ensure that your auto-download nzb folder is *not* the same as your download folder. With users downloading a lot, that will kill your IO.

DT

[Quade]

You know, I'd suggest not doing whatever if is you're doing that causes this. I'd stop adding shitloads of downloads until you can get one or 2 working properly. If you have a bunch of incompletes waiting there, you probably need to clean them out. Either by "Assemble Incompletes" or deleting them if they're too bad to repair. Newsbin doesn't wait 48 hours any more. If there a sufficient blocks and what looks like sufficient PARS. It'll assemble and attempt the repair. If sounds like you're not getting enough of these files for Newsbin to consider them "repairable".

You might try right clicking them and selecting "Properties" and see what the par counts look like. Lots of partial downloads will eat up ram so, you need to do something about preventing them from queuing up. I think if you solve the "incomplete files" problem, all the other problems will solve themselves.

[skoolrumble]

You know, I'd suggest not doing whatever if is you're doing that causes this. I'd stop adding shitloads of downloads until you can get one or 2 working properly. If you have a bunch of incompletes waiting there, you probably need to clean them out. Either by "Assemble Incompletes" or deleting them if they're too bad to repair. Newsbin doesn't wait 48 hours any more. If there a sufficient blocks and what looks like sufficient PARS. It'll assemble and attempt the repair. If sounds like you're not getting enough of these files for Newsbin to consider them "repairable".

You might try right clicking them and selecting "Properties" and see what the par counts look like. Lots of partial downloads will eat up ram so, you need to do something about preventing them from queuing up. I think if you solve the "incomplete files" problem, all the other problems will solve themselves.

Hi. It works fine most of the time. It's only occasionally that the incompletes will build up (in like a half-day or a day). And I do clear off the incompletes as soon as I can.

However, I don't know how to make sure the incompletes do not cause the autopar to leave them alone for some inexplicable reason? Most of the time there are more than enough pars & files. Sometimes I'll manually assemble-incompletes, download the pars and manually repair them (with more than enough pars to spare); so I'm not sure why it stopped in the first place?

I'm not sure if it's related to bleeping up my windows, but I do notice that sometimes the auto-parring will stop. I'll have a bunch of stuff still there unrarred (plenty of space on both source and destination drives), even though most of the stuff doesn't even need repairs. I close newsbin, restart it and then it will autopar the stuff that had been stopped before.

BTW, when the above happens and I restart newsbin, is it really necessary for it to download off the servers even though the files are already there 100% complete? I'm not sure what that's about? Thanks.

[skoolrumble]

Nothing you've mentioned is particularly shocking in terms of scale, certainly 10G downloads aren't a big deal, and say 20 of those in the queue, again, no biggie. The issue of adding a whole mess of incompletes can indeed cause issues if they get backed up, since AFAIK there's a limit to how far Newsbin will 'look ahead', so if you had many hundreds of them sitting there, it could cause Newsbin to pause and might need intervention.

None of this, to me anyway, explains the effect it's having on the rest of the system. Basically, check and see just how much memory is free, how much Newsbin is using, that should indicate if memory is related. Sounds to me, though, that something else is up. Certainly extremely intense IO can trigger similar situations on some configurations. Unless I missed it, you never mentioned disk activity. Is it running like mad? Are you constantly updating headers at the same time? Updating headers and par repairs will push your IO bandwidth, hard. Your system *shouldn't* be lagging like that, but some systems, I've seen that behaviour when disk IO is insane.

One other quick thing to check - ensure that your auto-download nzb folder is *not* the same as your download folder. With users downloading a lot, that will kill your IO.

DT

TX for replying. I think I've had at most a 15 or so unfinished autopars, so it's never that backed up. as I said, in those times, newsbin will use up ~1GB of memory, and there's I believe still quite a bit of memory left since I'm not really running much else. Although one thing that does pop up (not related to memory, but maybe relevant?) is that I tend to have 4-6 Windows Explorers open, not doing anything.

HDDs aren't running afaik, and no header downloads at all.

Not sure what the auto-download nzb folder is? "Autoload NZB Path"? That one I've left blank as I don't use it. However, the drive I download to and the one I autopar are usually different ones. Occasionally, I'll do it to the same drive, but in different folders. Thanks.

[DThor]

Everything you describe seems fine, except the windows issue. At the risk of sounding like passing the buck, it doesn't sound to me like there's anything Newsbin is doing wrong, at the very worse maybe it's triggering a system issue. It does push some aspects of your computer fairly hard. Sorry, I'm out of ideas.

DT

[skoolrumble]

Everything you describe seems fine, except the windows issue. At the risk of sounding like passing the buck, it doesn't sound to me like there's anything Newsbin is doing wrong, at the very worse maybe it's triggering a system issue. It does push some aspects of your computer fairly hard. Sorry, I'm out of ideas.
DT

Hmm.. think I've narrowed down the reason why it screws up. It probably has to do with Truecrypt. I was downloading and unparring to the same Truecrypt drive and it seems to occasionally lock up or just reboots my computer when I'm asleep. Is there a specific reason why it might be Truecrypt? And yes, I kinda need to download to a Truecrypt drive at times (cough*cough). Thanks.

[Quade]

I download 100% of the time to a truecrypt drive and have no issues. I do think if the system is crashing there's something wrong with the PC.

In Control Panel/System/Advanced System Settings/Start Up and Recovery. Uncheck "Automatically Restart".

Now if it crashes it should blue screen and hopefully show you where it's crashing.