Dealing with incompletes

[bruce73]

I have the v.6.0RC1 on a Win 7/64 machine and am having problems with large files and how it manages incompletes. I understand that it doesn't auto-assemble relatively new files and understand the rationale for that. But the issue I'm having is when the program is closed and what happens when it is re-opened and the download list still has this file.

Here's an example: this morning after an all-night download of a 40GB file, there were ~20 incompletes. Before I could manually assemble them, I needed to reboot for some other issue. When I restarted NB, the program re-downloaded all of those files (about 1GB worth) -- only then was I able to manually assemble. IIRC, v.5.xx would simply have the incompletes listed, ready for assembly. Am I missing something somewhere? Is this how it's supposed to work?

FWIW, I don't have auto-RAR enabled, because I'd rather pick the time to do that, and not have it be automatic. That shouldn't make a difference, should it?

[Quade]

What do you have the "MemCacheLimit" set to? If you don't know then it's probably the default which I believe is 15 megs. Files that are cached in memory probably don't get saved on restart. I'll check the large files out and report back.

[bruce73]

I assume that setting is in the .ini file, which I haven't edited, so it would be at the default. So, since my first post -- after assembling all the incompletes, the program just sat there. So I ran the PAR2 file and saw how many blocks I needed (44). Added that many to the download list, they downloaded but just sat in the list. So, long story short, I uninstalled v.6 and re-installed v.5.59 and I finally got the file to repair.

I really like v.6 -- the GUI, the improvements with headers, etc. -- but I don't like the way incompletes work. Maybe it's just me and that I'm used to doing things a certain way, but for large files I still prefer v.5.59.

[Moondawgie]

This issue, and one involving the deletion of message parts by an nntp provider ( ) before Newsbin has finished downloading and assembling all of the parts into a complete file, are the remaining big problems which I encounter with my use of NewsBin 6.

The problem with files being stored in memory, rather than being written to disk, when my computer being automatically shutdown during the night by my backup power supply when the power has gone out, is what has led to my receipt of two warnings from Charter Communications that I'd exceeded my monthly 100GB allotment of "bandwidth". With the last warning they informed me that the next time will result in my loss of their cable internet service for 6 months. Unfortunately for me, Charter is the only broadband internet service available at my location.

In those situations with Newsbin 5 where the nntp provider would delete files, I would contact friends and acquaintances who use other nntp providers from whose nntp servers the parts might not yet have been deleted, asking them to retrieve and send me the missing parts, which I'd add to those downloaded and saved to my computer by Newsbin 5, and use to assemble a complete file. Some might find it interesting that some such files sometimes are deleted from the nntp servers at Easynews, and Giganews, but remain, at least for awhile longer, on the http server at Easynews. I've not yet figured out how to save the individual parts of multi-part postings for use in such situations, but I've a feeling that it can be done.

Such problems have led to my resorting to the use of my old (s l o w) newsreader, Forte Agent, to download such files, so that far less is lost in the event of a loss of power, or computer crash.

[itimpi]

The problem with files being stored in memory, rather than being written to disk, when my computer being automatically shutdown during the night by my backup power supply when the power has gone out, is what has led to my receipt of two warnings from Charter Communications that I'd exceeded my monthly 100GB allotment of "bandwidth". With the last warning they informed me that the next time will result in my loss of their cable internet service for 6 months. Unfortunately for me, Charter is the only broadband internet service available at my location.

You can turn off the caching of files in RAM if you do not ming the performance hit by setting the size for the memory cache to 0 in the .nbi file. You can also set the upper size for files that will be cached by a nbi file entry. Sounds as if these settings might be worthwhile to you.

[Moondawgie]

I'll certainly give this suggestion a shot.

Thank you.

[Quade]

Ok, did find something with saving partials. A single missing block was enough to make it miss the rest of the downloaded blocks. Fixed. Be in the next RC.

All the stuff about memory mode is still true though.

[bruce73]

So the thing about the Memory Cache is that partials are not written to disc at all? Only when they are combined? And that increases performance? I gather this is new to v.6, right?

[Quade]

Yeah, memory mode caches the entire file in memory and writes it out in one write. It'll cache files up to the size you set. Then when the file is finished, you manually assemble or it assembles itself, it's written out to disk in a single write.

New. The problem is most disks are the bottlenecks these days. Even SSD's are too slow to handle simultaneous, writing, unraring and repairing.