Cant get NB to download headers for specified amout of days

[dash]

Hi,

even though i have set "Download Age" to 30 days, NB keeps downloading much older headers for new groups...

Tried this suggestion from NB-wiki:

If you have previously downloaded headers for a group and want to apply a different Download Age, you have to Delete Stored Posts for the group, then do a Download Latest to download the headers limited by the new Download Age setting.

but it didn't help. NB keeps downloading headers from 600+ Days ago.
Using the latest v6.56.

http://a.pomf.se/uvcakb.png

[Quade]

1 - Exit newsbin.

2 - Restart Newsbin.

3 - Right click the group "Post Storage\Use Download Age" say yes.

Normally, you don't have to restart but, I want to start from Scratch.

Double-click the group. See what happens. What news server are you using? Just keep in mind "Download all Headers" always means "All". You probably know that but, I've had some people set 30 days then "Download all", then ask me why it was "Downloading all".

[dash]

Hi Quade,
I'm using ssl-news.info. Unfortunately your suggestion above didn't work.
(I didn't use "Download all" feature, i used either double click on a newly added group, or "download latest".)
I also tried deleting everything from "Newsbin Data\SPOOL_V6" and "Newsbin Data\Import" and clearing groups entries from Newsbin.nbi, which didnt work out as well.


As far as I know NB always worked like that for me...(most of the time I've been using .nbz files to get my downloads, so it wasn't a big issue for me.)

[Quade]

I have a feeling the issue is how Newsbin probes the groups. If the server lies about how many headers it has and Newsbin probes into empty regions. It'll sometimes just says "fuck it" and downloads what it can. You could give 6.60 a try. I did work on the probing some between 6.56 and 6.60.

How are you around SQL?

It's possible to manually override the range but it's not automatic. It involves editing a file called "range.db3" using "Sqlite3.exe" or you can use a graphical front end to Sqlite.

Keep in mind that 30 days of boneless is probably still many gigs of header downloads. That's the highest traffic group on usenet.

[dash]

Will try that, thanks

[Quade]

Since you want to try. I'll come up with some details. Should post them up some time today.

[dash]

Tried v6.60 but its the same.
The other thing in v6.6 was: the "Size" values were stuck at 0/0 during download of headers for 20+mins (the gauge still showed the progress though):
http://a.pomf.se/eylwjr.jpg
Only after ca 30mins NB started showing the "Size"-values. If I remember correctly, in v6.56 the values were showing almost immediately after starting the header download.
Generaly v6.6 was much more sluggish, so I had to revert to 6.56:
http://a.pomf.se/atcgwe.webm (<-only took 3 sec to display 18k posts in 6.56. In v6.6 it takes minutes to load 1k posts)


Anyway, I did some more testing by changing download age, and there was one curious thing that i noticed:
after changing the download age to 10 days, NB first started downloading headers that were 10 days old and older.
After downloading all those old headers (10-1xxx days), NB then downloaded newer headers that were 0-9 days old.

Then I deleted stored posts and set download age to 5 days. Now NB first started downloading headers that were 5 days old and older.
After downloading all those older posts, NB then downloaded newer headers that were 0-4 days old...

And about editing range.db, what should i change in there?
There is currently only one Table "GroupRange" with following fields/values:
Fields:
NN_Server;NN_Group;NN_MinGroup;NN_MaxGroup
Values:
ssl-news.info;alt.binaries.boneless;0;0

(Is this the right file?: \Newsbin Data\SPOOL_V6\alt.binaries.boneless\Range.db3)

[Quade]

Then I deleted stored posts and set download age to 5 days. Now NB first started downloading headers that were 5 days old and older.
After downloading all those older posts, NB then downloaded newer headers that were 0-4 days old...

It always downloads oldest to newest. Whatever oldest happens to be.

1 - In Newsbin's options under network select "Show server commands".

2 - Do a "Download Latest" on boneless. Let is run for 20 seconds them delete it.

3 - Look in the logging tab.

[14:31:00] HIGH AW alt.binaries.boneless GROUP alt.binaries.boneless
|211 12686498095 2256535552 14943033646 alt.binaries.boneless

211 - OK
12686498095 - Count
2256535552 - Min record
14943033646 - Max Record
alt.binaries.boneless - group

sqlite3.exe" Range.db3
SQLite version 3.8.6 2014-08-15 11:46:33
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE GroupRange (NN_Server TEXT,NN_Group TEXT,NN_MinGroup INTEGER, NN_MaxGroup INTEGER, PRIMARY KEY(NN_Server,NN_Group));
sqlite> select * from GroupRange;
AW|alt.binaries.boneless|13655439747|14941351030

After header download.

sqlite> select * from GroupRange;
AW|alt.binaries.boneless|13655439747|14943033646

NN_MaxGroup is the starting point when it goes to download the next range of headers. It checks the group then determines the difference and downloads the new headers. Then sets a new Max.

sqlite> update GroupRange set NN_MaxGroup = 14943033646;
sqlite> select * from GroupRange;
AW|alt.binaries.boneless|13655439747|14943033646

sqlite> update GroupRange set NN_MaxGroup = (select NN_MaxGroup - 100000 from GroupRange);
sqlite> select * from GroupRange;
AW|alt.binaries.boneless|13655439747|14942933646
sqlite>

I'd set the NN_MaxGroup to the max number you see in the group response (the 211 thing) from the logs. Then by manipulating NN_MaxGroup you can control exactly how many headers you want it to pull.