Win 8 Strange memory problem

[pcdave21]

Hi long time user of newsbin woke up this morning to the download list doing nothing and windows 8 telling me to close newsbin as memory was low. Checked task manager and it was 90% of 8GB as soon as I closed it dropped down to 0.8% any ideas on what could of caused it. Running windows 8 pro x64 full release using 6.40 build 2049 x64.

Now when running it again I get unable to create output files with this and it starts immediately eating the ram again.
Got a screenshot thats not very helpful sorry. Lots of these messages as well.

[11:41:43] ERROR - Assemble -Spooler: MemIO Unable to Open file: [ ]-[02/27] -

[dcg]

[DThor]

Someone recently had memory explode on them and it was the antivirus. A key thing here is whether or not it's newsbin that's actually using the memory. I'm betting not - it might well be triggering the behaviour, but it's not the cause. You should try using a more detailed process monitor to track down exactly what is gobbling memory. Are you using McAfee?



DT

[Quade]

Now when running it again I get unable to create output files with this and it starts immediately eating the ram again.
Got a screenshot thats not very helpful sorry. Lots of these messages as well.

I think if you solve this, the unable to open the file thing, you'll probably solve the memory usage issue as well. Newsbin is probably buffering up download data and can't write it out to disk so, it sits there. I agree that it's not Win8 specific. I've been using win8 for production and devel since MS released it on MSDN.

First thing I'd check is the download paths. Make sure the drive is there and that Newsbin can write to it.

[pcdave21]

You got it in one bud had a look at my nas logs and it seems it had a hairy in the middle of the night and would not let anything write to it till I rebooted it. (thanks synology) so I guess it was just gobbling as much memory as possible till it ran out.

Cheers bud for quick response

[lorac]

I've started having this problem under W7 recently as well. I'm downloading locally to a raid6 array. I tried turning off MS security essentials, but that didn't help.

What it looks like is some bad crc on certain files, but i don't get why that would cause the memory usage to spike. Rebooting the PC does not help.

[13:42:10] ERROR - CNNTPWorkerServer - internal crash
[13:42:10] ERROR - CNNTPWorkerServer - internal crash
[13:42:10] ERROR - CNNTPWorkerServer - internal crash
[13:42:10] ERROR - CNNTPWorkerServer - internal crash
[13:42:10] ERROR - CNNTPWorkerServer - internal crash

[Quade]

I've seen that before on 32 bit platforms when you run into the 2 Gb total process ram limit. If you're running a 32 bit version of Windows and Newsbin, you have to keep Newsbin's memory footprint below 2 gigs. The thing is, you have to try pretty hard to hit 2 gigs. I can do it but, I have to do silly things like loading up boneless in one go. Large download lists are a large memory hit too.

If you're running 32 bit newsbin under 64 bit windows, Newsbin has over 3 gigs to play with.

What version are you using? Older versions used "memory mapped files" which could blow you over 2 gigs pretty easily to. It's been removed from 6.34 and 6.40.

[lorac]

64 bit. 6.40, problems happening with 6.34 as well. Newsbin uses over 6GB of ram. The system becomes very unstable, but eventually I can close NB. That recovers the ram.

My problem seems very similar to this thread as well. http://forums.newsbin.com/viewtopic.php?f=43&t=31027

I'll try it on another PC.

[Quade]

The system becomes very unstable, but eventually I can close NB. That recovers the ram.

His problem is that there was an issue writing to disk so, all of memory was used holding chunks of the downloaded files. Are the files landing on disk?

[lorac]

Well, files are being downloaded. Once memory usage starts going insane i don't believe it is downloading.

The hard drive is functional. The problem seems related to files it doesn't seem to like.

It only started yesterday. I'll download some other media to see if continues.

[Quade]

Once memory usage starts going insane i don't believe it is downloading.

I suspect the sequence is:

1 - Writing to disk stops working, for whatever reason

2 - Download continues till you run out of RAM.

So, the question is why does the writing stop. I'd try downloading to a different drive and see what happens.

Just for grins, in the performance options, you might try "Write downloading files with no memory buffering". See if the symptoms change. You'll have to restart Newsbin for it to take affect.

[lorac]

The problem is related to what i'm downloading, not the location. I tried both your suggestions, but the same thing happens. Almost nothing gets written to the disk. I was able to download 25+ gb of other media w/o a problem last night. I'm not sure why it just doesn't like this particular media...

[Quade]

PM me the subject and group and I can check it out. Don't post it here.

If you hit the + icon and see a bunch of reds or black blocks, that means the files might not be on the server any more.

What it tells me is that I should consider flushing partial downloads to disk when they've been in memory long enough.

[TMM]

I am also having this issue
it randomly starts taking up all my ram 16 gig
without no warning, but the majority of time it is okay
I was in downloading that many files at the time, only a few 20 MB files files. This problem seems to occur sporadically. I don't think it has anything to do with Windows 8 because I'd been running it in various incarnations throughout development life cycle
running Windows 8 professional 64-bit. Please note my speech recognition software have become less accurate lately, so I apologise if my post is hard to read

[Quade]

In the performance options, try checking "Write Downloaded Files with No Memory Buffering" then restart Newsbin. See what happens.

[TMM]

In the performance options, try checking "Write Downloaded Files with No Memory Buffering" then restart Newsbin. See what happens.

I will give it a go and report back

[Quade]

This basically disabled buffering in the program itself. It uses the disk instead.

[Quade]

I got a confirmation this morning that disabling buffering seems to solve this for one use who had this issue. Theoretically it's a bit slower but, running this mode here, I'm not really noticing it. If you download to an SSD, it's actually the preferred way to download.

[TMM]

I got a confirmation this morning that disabling buffering seems to solve this for one use who had this issue. Theoretically it's a bit slower but, running this mode here, I'm not really noticing it. If you download to an SSD, it's actually the preferred way to download.

that seems to have worked thank you so much. Also an added bonus as I use a ssd as my primary download drive
Ocz Vertex3 120 gb or something close to that


I cannot remember the exact amount of available space
but thank you once again for creating and maintaining such a wonderful piece of software
and the support is second to none. It's like you guys don't even have a life
just curious beside writing and maintaining nb what are your interests/hobbies
as always, keep up the good work

[Quade]

I'm in the midst of re-installing the differential in my old BMW. I've had the rear suspension apart every year I've owned the car. Old BMW's (mid-late 90's) tend to fall apart. They need constant maintenance. Had the motor out last season. Thinking about doing a V8 swap but, I haven't found the right wrecked Camaro yet.

[TMM]

I'm in the midst of re-installing the differential in my old BMW. I've had the rear suspension apart every year I've owned the car. Old BMW's (mid-late 90's) tend to fall apart. They need constant maintenance. Had the motor out last season. Thinking about doing a V8 swap but, I haven't found the right wrecked Camaro yet.

That's interesting. I am also in to cars unfortunately, due to my disability cerebral palsy spastic quadriplegia. I will never be able to drive. However, I know almost everything about if I told someone what to do. I think I could easily help with most mechanical issues
but I can't do that, so I focus on computers. Everything from security to get more general IT helpdesk. But my main focus has always been security, whether it be Windows , Linux, IThink you for letting me have a brief glimpse into your life outside nb
Please note I have edited my post
to simplify and to maintain complete accuracy

[alexander038]

Please note my speech recognition software have become less accurate lately, so I apologise if my post is hard to read

I never have have problems to understand your posts TMM and my english is far from great

Best regards, alex

[TMM]

Please note my speech recognition software have become less accurate lately, so I apologise if my post is hard to read

I never have have problems to understand your posts TMM and my english is far from great

Best regards, alex

That is great news
that gives me
a lot more confidence in what I write
All the best.
Paul