Filemap failure, NB2 files and the /3gb switch.

[Quade]

You know, some people reported Autopar didn't work in 6 and I could never reproduce it. I added some logging and some interesting log messages started to show up in some reports.

[16:06:01] ERROR - INT_AssembleFile: Unable to MAP file <filename>[274/306] - "<filename>.part272.rar" yEnc

For example. This happens to people running 32 bit windows who have Newsbin using a bunch of RAM (loaded groups, wish and download list). The problem is, 32 bit windows is limited to 2 GB of RAM use per process. When a file downloads, Newsbin maps it into memory and does processing on it like autopar and signature generation. If you already have a bunch of stuff in memory, say 1.5 gigs and Newsbin needs to map a 200 meg file into memory, there might not be a contiguous block of address space to make this happen. You get a filemap error and autopar doesn't get the data it needs. In 6.1, it also means the file doesn't get renamed to the final filename.

http://technet.microsoft.com/en-us/libr ... 0(EXCHG.65).aspx
http://msdn.microsoft.com/en-us/library ... 32(v=vs.85).aspx

This is some information on setting the /3gb switch in 32 bit windows. It lets you have an extra 1 GB of address space for your process and makes this problem less likely to happen. So, keeping RAM use down and/or setting the /3gb switch should hopefully prevent filemap errors.

64 bit windows doesn't seem to have this problem. It has a vast address space.

[RayMark]

So what else goes wrong when this "Unable to MAP" occurs?

Never mind autopar. Is the file always decoded and assembled correctly?

200 MB is a pretty common standard size for rar parts these days. But sometimes people post a large single file, not split into parts.
I came across several really large files, 1.5 GB and larger - they will never be mapped, with /3GB or not.
I would not be surprised if there are 30-60 GB Bluray ISOs posted as single files.
If not, I could post one for testing purposes
Or split into rar parts 20 GB each. Try to auto-unrar it.

[Quade]

Dex was telling me the other day that there are more 64 bit installs than 32 these days. That tells me this is a problem that will eventually fix itself. 64 bit windows doesn't have this problem at all. Even mapping in 1.5 GB and larger shouldn't be a problem. As I've tried to explain but, people don't seem to get, this isn't a RAM problem. It's an address space problem. 64 bit windows has nearly unlimited address space. Mapped files that aren't currently accessed get paged in and paged out just like virtual RAM.

As for what happens. Prior to 6.11, you'd end up with an NB2 file. With 6.11, it gets renamed properly but, autopar will ignore it. All post processing ignores it in fact.

As for these 60 Gb single files, I think you'll need a better PC at that point.

[Quade]

Got a new version that removes filemapping. We'll be putting it into the forum as soon as 6.21 is released. If any of you 32 bit people having problems with this want to PM. I'll get a list and maybe we can send you a 6.3B1 to try.