NewsBin is doing something - but what?

[RayMark]

I think I wrote about this issue some time ago. The problem is not new
Sometimes after starting NewsBin, downloads (both posts and new headers) do not show up in the download list for a very long time. NewsBin is using about 1% of CPU and only 50-70 MB of memory. CPU usage sometimes drops to 0, but mostly in the range 0.8-1.6%

NewsBin is clearly doing something and it would be nice if it informed the user about it. Now, even the log shows 0 messages.
So yesterday, I started NewsBin 6.61 beta 1 - I needed to download a file quickly. No such luck. It did not show up in the download list. I made a nzb file of it - no difference - from the group of from the nzb file.
So, after 30 minutes or so I closed 6.61 beta 1 and simply downloaded the file with 6.55 Final.

Today I installed 6.61 beta 3 (build 4226) - after starting it I tried to download new headers for a group - still trying. The same situation - NewsBin is doing something but nothing is showing up in the download list. I assume that after some hours it will finish whatever it is doing and begin working normally, but I cannot be sure.
So please, inform the users what newsbin is doing after starting. Do not assume that it takes a few seconds only. Perhaps sometimes it takes hours.

Additional information: this situation occurred after closing NewsBin 6.61 beta 1 normally. However I closed it in order to reboot windows because the system ran out of resources and the internet was no longer working properly.
Not only NewsBin stalled and was not downloading anything, but uTorrent as well - it was informing something about not enough resources. After trying to close uTorrent and Opera (old 12.17 64 bit version) - both did not terminate and their processes could not be killed.

However, NewsBin closed normally.
I do not suspect that NewsBin caused the problem. It was likely uTorrent or Opera (I like using that old version of Opera but it often causes CPU and memory leaks).

But NewsBin closed normally, and 6.55 Final had no problems after rebooting. Only 6.61 betas are doing something for hours.
So I want to know what they are doing and how long approximately it would take - if it is possible, some progress indicator?
Again, I seem to recall it occurred before, with some older NewsBin version (but not with 6.55 F, it seems?) - NewsBin takes time after starting before it begins to work normally again.

I guess, I could experiment with removing some db3 files? I see 11 of them, not counting headers.
I tried removing Downloads.db3 - it did not solve the problem...
Then I tried removing DownloadMarker.db3 - again, it did not solve the problem...
SOLVED: removing all Downloaded (Downloaded.db3, Downloaded.db3-journal, Downloaded.db3.bad (is it a legitimate file, or my own renamed backup, hm?), DownloadedFiles.txt) revived NewsBin - seeing that -journal file caused me to suspect it.
QUESTIONs: why these files are needed? Is it safe to remove them? I don't want to lose the visual indication in opened groups which files are already downloaded and which are not! - The first impression is that the downloaded posts are still marked as downloaded. So why then these files are needed: to show in the Files List tab, to detect duplicates probably?


But the problem I am reporting here is that NewsBin is keeping me in the dark and does not tell what it is doing. I will keep in mind that it is likely related to Downloaded.db3 when it happens again. But surely NewsBin could inform what it is doing when keeping me waiting to begin using it for hours?

UNRELATED:
BTW, I had at least 2 occurrences of 0/200 download stalling with 6.61 beta 1 (Help->About revived downloading - and seems that for longer than previously) - and not with infected books, but after a longer usage, it is difficult to pinpoint what caused it. Not yet with beta 3 - I just installed it.

[Quade]

If I had to guess, I'd guess your DB3's are simply getting corrupted over and over again. So the question might not be "what is Newsbin doing" but "why do my DB3's get corrupted and stall newsbin on startup".

The fact that you see a ".bad" file in the folder tells me Newsbin is detecting some of the corrupt DB3's.

[Desfarges]

Same for me, newsbin (6.6x) using one of my core, is nearly unresponsive, and no info into the log. Is the number of my filter that affect it?

[RayMark]

The same with 6.61 RC1 - nothing appears in the download list, nothing in the log, but CPU is being used - even more, over 4% this time.
Again, 6.55 Final is working fine - even with the same Downloaded.db3.
With 6.6.1 I need to delete Downloaded.db3 each time before I start it to be able to use it. Interesting that this problem surfaced recently in 6.6.1 beta 1. I mean, beta 1 was working fine until at some point this began to happen.
And from that point forward, it always happens with 6.61 even though I delete Downloaded.db3 each time, so if it was corrupt - it is deleted. Still, the next time I need to delete newly created Downloaded.db3 again.

So, 6.61 now creates bad Downloaded.db3 files each time, it seems. Not bad for 6.55 but bad enough so that 6.61 itself cannot handle them.

[RayMark]

If I had to guess, I'd guess your DB3's are simply getting corrupted over and over again. So the question might not be "what is Newsbin doing" but "why do my DB3's get corrupted and stall newsbin on startup".

The fact that you see a ".bad" file in the folder tells me Newsbin is detecting some of the corrupt DB3's.

Exactly.
First question: how come 6.61 creates bad (if it is really bad) Downloaded.db3 file each time even when it is started without a Downloaded.db3 file at all. (the file with .bad extension is long gone - deleted days ago now).
Second question: how come 6.55 has no problems with handling that Downloaded.db3 file. I guess - 6.61 performs more checking than 6.55.

As to the first question: perhaps a special flag is set that some checking of Downloaded.db3 has been initiated. And, because I never allow it to complete (it would take hours?), NewsBin 6.61 always tries to continue running that check when it finds Downloaded.db3 present.


BTW, 0/200 stall with RC1 occcurred very quickly (without any books, just regular non-infected multi-part rars as far as I can tell. Well, not exactly regular - I think I have some corrupted headers, so that after re-downloading headers from other servers somehow posts are duplicated - each rar part is present twice and in the download list each rar parts is also shown as a collapsed but expandable, consisting of two identical rar parts. The whole thing is shown as half-downloaded, but after expanding, one is downloaded 100%, the other one is not downloaded. Anyway, NewsBin extracts such rars and deletes them correctly, but does not remove them from the download list.
And this seems to be causing the 0/200 stall.
Unfortunately, I don't see how to replicate this situation without such duplicated headers.
I don't want to re-download the headers because I would probably lose the indication what is already downloaded and what is not.