Download Latest - when is it complete?

[thuvia]

I saw this problem in 6.0 and in 6.0b1: when I load a GoG and Download Latest, eventually the header downloads in the download list clear and I think the post list is up to date. In V5 there would still be a progress bar at the bottom of the post list showing that it was actually still doing "something", so I'd wait for that. (In fact, I found that even after that, often only a fraction of posts would actually be shown, so I'd do another Download Latest on the Gog and they would all magically appear.)

In V6, I got burned by waiting until I couldn't see a progress bar, re-sorting the list, browsing and adding downloads until after a point in the listing I knew there was nothing more I wanted, then Ctrl-A Del to delete the posts. What I didn't realise was that NewsBin had sneakily been adding more new posts to the bottom of the list in the background, and I ended up deleting them without ever seeing them!

On closer inspection, even after I think the update is complete (no obvious progress bar), I'm actually still seeing the progress bar flicker momentarily on from time to time (blink and you miss it), and I can hear disk activity.

How can I tell when it's REALLY done updating? I've tried using V6 with the Background Updates both enabled and disabled.

[Quade]

I see what you're saying. It's a timing issue. It always tries to pop the progress bar but, if the update only loads up 20 new sets for example, the progress bar's up for such a short time it's invisible. In the old 5.59 days, the loading of new posts into the post list was far more intensive so, it always took long enough to see the progress. You could almost say it's simply too fast now.

It's asynchronous. Meaning the download engine downloads some new headers, they get inserted into the DB and the new/updated record numbers get sent to the post lists. The post list worker thread then loads the new data out of the DB and appends it. The time between header update and finish update of the post lists is indeterminate. The post lists don't even know that an update is coming until they get it so, I'm not sure how to signal this. It gets even more complicated if you use "BackGround Update" because then the connection between the download of headers and the insert into DB also becomes indeterminate.

If you did a Ctrl-A del before you updated the headers, you ought to be able to keep ahead of this. How often are you updating? couple times a minute?

[thuvia]

If you did a Ctrl-A del before you updated the headers, you ought to be able to keep ahead of this. How often are you updating? couple times a minute?

More like once every couple of days.

Not sure I understand the question! Every couple of days I double-click the GoG. I just want it to do it's stuff and let me know when it's finished, then I download what I want from the post list and Ctrl-A Del to get rid of all the other stuff I didn't want, close the GoG until next time. The extra update (once) and sort I just got used to doing in V5 to make sure I was seeing everything new, and all that should be unnecessary.

If I just Ctrl-L the GoG as a first step, it should be empty, then I Download Updates.

[Quade]

I was trying to figure out if you were a "bouncer". Some people bounce up and down on the update button every 10 seconds or so. Don't ask me why, I don't question my users any more.

In your case,

- if you right click show posts, Ctrl-A del, then hit the update button. You could control the sequencing.

- Alternatively you could Ctrl-A Del when you're done with the group then, next time you double-click only the new stuff will show up.

- If you don't use background update mode, the download list progress and the loading of posts into the post list is close to synchronous. When the progress in the download list goes away, the last notification will have already been sent to the post list. If you do use background update, you'll have to wait.

[thuvia]

I was using the "Alternatively..." method.

Could be that I had background update turned on when I tested, and didn't test it right after I turned it off. I have it turned off now and I'll see what happens in a day or so when there a few new headers to download (can't test it with another GoG right now since I'm in the middle of a bi-weekly header update for all groups...).

I also normally have the Hide Progress option turned on, but I tried it with that off too (I didn't know what the "tagging" part meant - I had it turned off because the modal delete progress bar is just frustrating!) - no difference.

Thanks for the explanation.

[Quade]

Hide Progress is for when you mark and delete vast numbers of posts. It doesn't impact updating the groups or the post list updates.

[thuvia]

Hide Progress is for when you mark and delete vast numbers of posts. It doesn't impact updating the groups or the post list updates.

Good, I can keep it enabled then!

[itimpi]

You might want to enable Newsbin automatic header downloads at pre-determined intervals. That way headers would tend to already be downloaded before you load a group,

[thuvia]

OK, I ran through this (load empty groups, grab latest couple of days of updates) with Background Updates disabled, and didn't see the problematic behaviour.

I had Background Updates enabled (but tried it briefly disabled before) because I read in another topic that otherwise my disk might be getting thrashed during a long update, and I'm often trying to watch HD video from the same disk while I'm updating. I haven't yet determined if this is actually going to be a problem.

[Quade]

Foreground updates at 300 Mbps makes my machine grind pretty hard so, it might be a problem. I do the periodic update thing though as D suggested so, my headers downloads are never that intensive.

[thuvia]

Foreground updates at 300 Mbps makes my machine grind pretty hard so, it might be a problem. I do the periodic update thing though as D suggested so, my headers downloads are never that intensive.

Updates "at random" would take me back to the "don't know when it's done updating" scenario I started this topic with...

[thuvia]

It's asynchronous. Meaning the download engine downloads some new headers, they get inserted into the DB and the new/updated record numbers get sent to the post lists. The post list worker thread then loads the new data out of the DB and appends it. The time between header update and finish update of the post lists is indeterminate. The post lists don't even know that an update is coming until they get it so, I'm not sure how to signal this. It gets even more complicated if you use "BackGround Update" because then the connection between the download of headers and the insert into DB also becomes indeterminate.

How about some sort of indication somewhere (status bar?) that there are any worker threads running at all? That way, unless I've got some scheduled NewsBin actions (which I don't, at least not deliberately!), I can be assured that the application behaviour is going to be deterministic.

I presume this would also let me know when deleting the post list is really done too (since I have that popup progress bar hidden - why does it even exist in the first place?), and would also suffice if I turn background updates back on.