Download list / state not saving (Cache issue?)

[astrax]

Hi,
one thing which isn't working right the last couple of days - for whatever reason, since I haven't had a crash or anything, is that NB doesn't remember the order in which files are sorted in DL after the program is closed.
And as if that wasn't enough, it also shows the files that were already downloaded as idle. Then you have to unpause them. Then it tries to d/l them and then it realizes that they're already present, marks it downloaded and skips to the next file. That takes long, if you have already downloaded most of the files of a set. Also NB usually does this with more connections, than entered in the preferences. I have no idea what could cause this...

Yesterday I watched my Cache go down to 0, even though I added MemCacheLimit=120 to settings... after the whole shebang, described above, it goes back to around 192/200.

I'm running V6.11: 00 3E E2 87 F0 DA on Windows 7, 64bit, 8GB RAM.

Thanks for reading!

[Quade]

MemcacheLimit and "Cache" are two different things. "Memcachelimit" is how much of the file it'll keep in memory". "Cache" represents how many outstanding downloaded blocks it'll cache waiting to write to disk. I've toyed with the idea of eliminating "Cache" because the "MemCacheLimit" should be more than enough. Cache made sense before "MemCacheLimit" existed. Now it doesn't really.

When "Cache" goes to zero, it means your disks can't keep up with the download speed. You can only get 100% out of the disks and if you have a repair or unrar going on, much of the disk bandwidth is already being used up.

You might want to delete the "Downloads.db3". It's where all this information about download list items and order is stored.

[astrax]

hm, thanks.
I saved all the files in the downloadlist to an NZB, deleted the Downloads.db3 and started again. The order is not saved, but that's not a problem. Now, some files are shown as paused, some start downloading right away. But when it comes to the files that are already downloaded, in the folder, I unpause them and NB starts downloading masses of files at once (that's where the cache goes down) - even though i have only 8 connections to the server!

And I just watched multiple downloads disappear from the list, which were paused and then started to download. I have no idea , what's going on here.
When I try to download them again, either by selecting from a group or as an NZB, same thing happens: after checking the first few files, the whole bunch disappears.

*edit - about two hours later:
added only the files from a NZB, that *weren't* already downloaded. Downloads start flawless. Closed NB to see what happens - after restart file order isn't maintained and files that were added by hand disappeared...

[itimpi]

, I unpause them and NB starts downloading masses of files at once (that's where the cache goes down) - even though i have only 8 connections to the server!

Newsbin is still only downloading 8 files at a time (as this is your connection limit). However as only one chunk is downloaded initially for each, nNwsbin will normally be moving onto the next file before the previous one downloaded is written to disk (using a different buffer from the cache) and Newsbin can complete the checks it carries out on the first chunk of a file. This is why the cache is going to zero - the buffers are being used faster than they are being written to disk and thus freed up for re-use.

[Quade]

How much do you have in the download list? I'd wipe it again, deleted the downloads.db3, again. Add a couple files and then restart Newsbin and see if they're they're. If they're not, there's something wrong with your setup. If it worked, I'd pause and add some more.

I have a feeling Newsbin either can't write to the downloads.db3 or you have a shitload of files in the downloads.db3 and there's some timing issue.

To trouble-shoot, you have to start small.

[thwartme]

Just did a re-start of my whole system, and when I relaunched Newsbin, it had forgotten my DL order as well. The trouble is, due to not being able to do a 'partial sort' and Newsbin's habit of jumping par2 files up the queue all on it's own, I had spent a lot of time manually positioning each and every file in that list... and that effort is now gone.

I'm running Newsbin 6.10, build 1339. I had previously posted about other problems with the way Newsbin is handling download order, and now this one crops up.

http://forums.newsbin.com/viewtopic.php?f=43&t=29228

Specifically, I had 51 items in my DL list, including a group of 19 rar sets with 19 associated par sets, which had been posted separately. I had them listed in a specific order, and I had each par set adjacent to it's rar set in the list. When I restarted, Newsbin had jumped all the par sets up the list, not to the top, but above all the rar sets.

This isn't a buffer problem, or a disc reading problem. Newsbin is making it's own decisions about file order. Can we please have it stop doing that? Files should go into the list in the order they are added, and if the user moves them, they should stay where the user puts them.

Thanks,
thwartme

[itimpi]

I have never seen Newsbin "jump" par sets in the way you describe. The files always appear in the Download list in added order for me.

THe list order not being saved on exit is almost certainly an issue with updating the database during the exit processing.

[Quade]

I think I get what you're saying now. Autopar works best if the pars come before the files so, when adding to the download list they get promoted to the front of the line. With sets that have the rars and pars combined this doesn't happen but, when the files are split out it does. When it loads the download list, it's just like adding a bunch manually so, the pars go up front. So, I'll bet the RAR groups are in the correct order and just the pars are at the top.

It's not clear if this issue is the same thing as what the OP is reporting.

[thwartme]

Okay, so loading a saved download list is essentially the same as dropping an NZB list in. That's why when it loaded my list, the posts were not in the same order as when I shut down. I understand your response.

Now, back to my actual question... can we make it stop doing that? I don't want par2 files jumping ahead, I want them where I put them. In V5, this didn't happen. I would prefer it like that.

Thanks,
thwartme

[Quade]

I'll have to think about it. Look at how the download list loads and see whether I can let if just leave the absolute order.

[astrax]

thanks for the replies -

, I unpause them and NB starts downloading masses of files at once (that's where the cache goes down) - even though i have only 8 connections to the server!

Newsbin is still only downloading 8 files at a time (as this is your connection limit).

OK, thanks. So, I confused: NB isn't actually downloading these files, but checking? Maybe in a future version, that state could be titled that, or similar and not "downloading".

How much do you have in the download list? I'd wipe it again, deleted the downloads.db3, again. Add a couple files and then restart Newsbin and see if they're they're. If they're not, there's something wrong with your setup. If it worked, I'd pause and add some more.

I have a feeling Newsbin either can't write to the downloads.db3 or you have a shitload of files in the downloads.db3 and there's some timing issue.

To trouble-shoot, you have to start small.

thanks, Quade. I don't know what's considered "a shitload" in terms of file numbers but I definitely had more files in the list at other times before and NB didn't behave that way. Up to now I had around 10, 12 downloads (700 - 1000mb each, containing multiple files and one big 50GIG with files around 400mb) in the list, but I took them out.
Before I deleted it the first time, the downloads.db3 was around 170mb - is that a lot? But deleting it didn't help. I deleted it again and did what you suggested. Right now it seems to work (remembering the order).

I'll keep an eye on it...

[astrax]

dang, it's happening again. Not remembering the order after closing, I mean.

I had removed/downloaded all files in the list and now, with new downloads added, Newsbin still doesn't remember the order the files were in.

[Quade]

The files are there but, the order is just wrong? The order is set when Newsbin exits.