Not seeing new headers in all groups

[confusednewb]

Just what I've toppost. I'm running NB 6.21 Pro (real, paid version, thank-you) and I'm only seeing new (0 day) posts in a few of the groups, with most of them showing 2-7 days old as the newest post.

This is using Easynews. I've gone to Easynews, and obviously, there are new posts every day in the groups I'm trying to access. Filters are off; I've done refresh on all the groups.

Any ideas what is up, and how to fix it?

[Quade]

Looks down at the "Cache" line on the status bar (at the bottom). Do you see (NNN) with the N's being numbers next to the cache entry? If so, it's still importing headers. Until that number goes away, you won't see the latest records in the group.

[confusednewb]

OK - in the download list, I can see the downloading of headers. When that stops, I thought things were done.

On the bottom, I have

Cache:92/100 (1485)

where the 1485 number has been changing; up to 1487 as I typed this and went back to check.

At what point is it done; also - I REALLY thought that once the updates were downloaded, I used to see all the posts. Has something changed that I'm unaware of?

thanks

[confusednewb]

Ok, now its even worse; since I was seeing a 7 day delay in alt.bin.lossless, I just deleted the group, then readded it, and downloaded all headers.

All headers are downloaded, but I get no posts whatsoever.

Current Cache: 92/100 (1498)

What do I have to do to make it finish - or whatever, and when will I see my posts?

fwiw - NB is also no longer displaying any posts from some of the smaller groups I subscribe to - they only get a few posts a day, so one would think I'd at least see the old ones - but I get nothing.

[DThor]

Under Options/Setup, what is Storage Age set to?

DT

[confusednewb]

storage age set to 600 days, since Easynews has finally increased retention.

my Chunk Cache Use State is "ready"

[Quade]

If you're still downloading headers, they might still continue going up. Currently they download and store as quickly as the network can deliver the data. Then newsbin packs them away into the DB as quickly as it can. What that number is telling is that 1400 blocks of records are waiting to be imported into the DB. You can disable "Background Update" in the options if you want. That'll make them insert in real time but, it'll really slow your header downloads.

What kind of disk is the data folder on? On my machine, inserts into the DB can keep pace to about 100 Mbps at which point they start backlogging like this.

[confusednewb]

If you're still downloading headers, they might still continue going up. Currently they download and store as quickly as the network can deliver the data. Then newsbin packs them away into the DB as quickly as it can. What that number is telling is that 1400 blocks of records are waiting to be imported into the DB. You can disable "Background Update" in the options if you want. That'll make them insert in real time but, it'll really slow your header downloads.

What kind of disk is the data folder on? On my machine, inserts into the DB can keep pace to about 100 Mbps at which point they start backlogging like this.

So, where are the 1400 blocks of records? When I do the update, like I said, it shows them downloading headers, it increments until download/new is equal, and then disappears. What do I have to do to get them from downloaded state to the DB? Leave the computer on, but idle? Leave NB running, or turn it off?

But the posts don't show up when I try to access the group. I'm running on a dual core AMD laptop, 160 gb seagate drive. nothing special, but not a Win98 machine either.

It HAD been working ok, until I upgraded to V6. At that point, it didn't seem to find my old info, but instead loaded every newsgroup as if it was new.

Would it help to delete most of my groups, get it stable, and then re-add one at a time? PITA, but right now, nothing is happening.

Thanks

[Quade]

You don't do anything. When the number hits zero and the records are all imported, the groups should be updated to the current time. Newsbin will add the records as fast as your hardware permits. 5.XX worked completely differently. 5.XX was faster to have headers ready but, was much slower to load and search through them.

Sounds like you're downloading a whole bunch of headers which means it might be awhile before they're all imported into Newsbin.

[DThor]

It really should be painfully obvious that Newsbin is importing headers, however. If you have constant disk activity, I'd wait. It can indeed take a while if you're pulling in a lot of headers. If the disk is sitting idle, however, it's something else.

DT

[Legend]

I'm running 6.30B2 - 1695. When i am interested in a movie posted to Newshosting.com it will be posted in perhaps 5 groups of 100 files. When the headers are downloaded, some of these groups will likely be missing - typically 2 or 3 groups. Using "Search" I may be able to find them out on the Internet (in same group) and can then download them. This business of not loading all the headers apparently started in version 6 and I keep expecting it to be fixed, but every time anyone posts that they are having the problem, it is just explained away by "wait for a while until the data base finishes updating". I'm telling you - days later the posts are never displayed. This is a real pain having to go to search to find them and select them for download. I suppose it does make subscribing to Search worthwhile?

[Quade]

Your issue it not related to this one at all. I'd suggest a 1000 post or more overlap in the server options and see if that fixes it. Some server farms aren't very synchronized so if you download headers from one, and then they get downloaded from another, differences between the servers leave gaps. The whole reason I added the counter next to cache is so, you could see what Newsbin's doing. If the counter isn't visible then all posts that are going to import have imported. In the case of the OP, he had 1400 that weren't imported yet and won't see the latest till they finish.

Mind you, I'm just guessing. Using either Astra or Giganews, I don't have any missing records between header downloads. The number of groups doesn't really matter. The files that are posted as crossposts (multiple groups) are really only posted once and the crossposting is handled at the group level so, there really is only one set of files across all the groups it's posted to.

One way to test my theory, if it's newish files, is to right click "Post Storage/Use Download Age" and then set a download age that covers the file. Then download headers. If they show up complete at that point, overlap will probably fix it.

Edit: I have another idea.

[SETTINGS]
CrossLimit=N

This sets a global crosspost limit in the NBI. The default is 8. You could add this to the NBI and set it to 10 and see if it makes a difference. You'll probably have to add this field. The default used to be 5, it's possible you got hit by this too in the past. The thing is, none of the set would show up if it was crosspost filtering.