upgraded to 6.60 - very few posts available to download

[flat_beer]

I upgraded to 6.60 build 4163 yesterday. In atleast one group I'm not seeing very many posts. I just downloaded 82,000 headers and it only shows 18 items that can be downloaded.... 15 of which are single post files. The only filters I have are Hide Old and 12 Hours. If I disable all filters and sort by date I still only see this many new potential downloads. This really seems like too few. Does anyone know why more aren't showing up?

I have a screen capture of what the 82,000 headers shows up as but I don't see a way to include that.

Win XP 32-bit
Per-group display age set to 30 days.
Storage age set to 3000 days.

The group this is really showing up in is: a.b.teevee
I only monitor one other group on a daily basis and some days there is nothing posted but spam.

[Quade]

Look in the filters at the global filter. Might want to clean most of it out and see if it makes a difference.

If you look down at the "cache" line on the status bar do you see a number in parens?

[flat_beer]

Global filter has:
Minimum file size: 0KB
Maximum file size: 0KB (Does 0 mean infinite here?)
Cross-post filter: 0 (Does 0 mean infinite here?)
Age filter: none

reject if [.]exe

The cache number in parens is 0.

[Quade]

Had another guy with a similar issue. I think he solved it with a re-install. Another guy had set his storage age to 0 which tossed out all the headers.

It's pretty clear the data is there but getting filtered out at display time. The question is what's filtering it out? Are you loading up a topic or specific group? As an experiment, I might exit Newsbin and hand edit the NBI for these groups and clear out most of the setting for the group and topic to see if one of them is causing the filter.

[ALT.BINARIES.bla]
Active=1
Age=0
ColumnDir=0
ColumnSort=3
DisableAutoRAR=0
DisplayAge=0
EnableAutoMode=0
LastAccess=1437540657
MaxSize=0
MinSize=0
PAR_AutoDownload=508288
Parent=bla
ShowFilenames=0
ShowFiles=0
ShowNew=0
UseMDA=0
UseProfile=0

I might try trimming it down to just this. Kinda grasping at straws. Just don't want you to have to re-install.

[ALT.BINARIES.bla]
Active=1
Parent=Something

[flat_beer]

I had seen that post about the storage age set to 0... Mine is set to 3000.... so that's not the problem...

I tried what you recommended... cleared the NBI for the newsgroup. That didn't seem to help. The old stuff appears to all be there... for the headers downloaded in the past few days I'm only seeing small stuff (less than 150KB) and large stuff (larger than 500MB - almost everything seems to be labeled 720p or 1080p).

It seems like the missing posts aren't being stored in the spool. Just did a local search for something that should be in the spool... the 720p version is there but not the SD version. Is there any filtering that happens to the headers as they download? or as they are integrated into the spool?

[Quade]

There can be some filtering on download but it only filters based on message-id.

Your symptoms are different than everyone else then. You see posts but you don't see them all. I'd right click the group "Post Storage/Use Download Age". Then set a 20 day download age and then download headers normally. See if these posts show up. For the missing SD types, I'm sort of leaning towards a server issue. SD's are smaller so any size filters might be filtering them out too.

What kind of small files show up? Newsbin has specific filters to allow some file types like NFO and NZB's to bypass the size filter. So, even if you have a 500 meg size filter, they'll show up.

[syshog]

I'm having the same problem. I've noticed when the cache number starts to fill up around (8) NB 6.60 will start removing headers from the download list even if the downloads are not finished. The cache will move to zero (0) because NB no longer has anything to process.. My settings are close to flat_beer except I'm using windows 7.. I download headers from 224 different groups and right now 6.56 is the only thing stable for me even if it takes longer to download headers and process them.

[flat_beer]

I started typing this about 24 hours ago but didn't finish till now. I think I have it fixed now... sorta hard to tell since there are so many incompletes on astraweb over the past few days.

Tried that.... no go...

The small posts that are showing up are:
"Usenet Index Post 1436987128 yEnc (01/01)" crap
REQ: Hawaii Five-O S05E25 Please TIA
[04/20] - "New Girl - S04E16 - Oregon - mkv - by Videomann.part01.rar" - by Videomann which is 196MB
and that child porn that I wish the net police would stop.

The fix.... Quade's line about filters based on message-id got me thinking and I looked up my notes that I keep on changes I make to newsbin... (I have a directory where I make notes on various tweaks) In Dec. 2013 I added a message-id filter to try blocking some spam... It seemed to work at the time. This is the thread about the filter I had setup. http://forums.newsbin.com/viewtopic.php?f=43&t=32717

I had created a MessageIdFilter with the following lines:
tweak[-]news[.]eu
reader[.]easyusenet[.]nl

So yesterday (July 24) I deleted the MessageIdFilter completely... then did the right click the group "Post Storage/Use Download Age". Then set a 10 day download age and then download headers normally. The missing headers showed up....

So I don't know if the MessageIdFilter wasn't working correctly in version 6.56 (and other old versions) and a lot of the current posts are coming from one of those servers or if it wasn't working correctly now in 6.60... but deleting that filter seems to have fixed my missing header problem.

[Quade]

So I don't know if the MessageIdFilter wasn't working correctly in version 6.56 (and other old versions) and a lot of the current posts are coming from one of those servers or if it wasn't working correctly now in 6.60... but deleting that filter seems to have fixed my missing header problem.

The message-id filter is always dangerous. It people start posting legit data through those servers it'll be lost. On the files that didn't show up, you might check the message-id's that show up now and see if you think the filter should have applied to them.

I'll take a look on my end.

[Quade]

NB 6.60 will start removing headers from the download list even if the downloads are not finished.

Are they ending up in the failed list?

[flat_beer]

Well, atleast one posts that was missing had a message-ID of blah@reader.easyusenet.nl Which would have been nuked by the filter. So, I'm thinking that filter is what was getting rid of them for me... which means that it probably wasn't working correctly in 6.56. (I just check something I download 10 days ago before upgrading to 6.60 and it also had the reader.easyusenet.nl but was not filtered.)

Failed list? are you meaning failed files list within newsbin? Then No. They are not showing up there.

If you're meaning some data base file or text file then what is the exact file name and where should it be?

I also discovered two minor UI bugs....
1. Doing CTRL-H which toggles "Hide Old" doesn't change the label (display settings) at the top.
2. In the files list. Highlight something. Hit the delete key to remove it from the list. The stuff is deleted but nothing else is highlighted so you need to select something again to delete more. Just can't hit delete a bunch of times to clear a bunch at once. Some people may consider it a feature to not accidentally delete too much.

[Quade]

if you're talking about header downloads leaving the download list, then they won't go to the failed list. I don't see the point, you can just download them again.

Originally I thought you were talking about files not finishing.

[syshog]

The older version of newsbin 6.56 I never had to keep adding header downloads back to the download list, they would just restart the header download from where they left off.

[Quade]

You should try the 6.61 beta.

Do you actually know why the connections are failing? Connections dropping out like that will probably give you "Dirty Connections" errors too. The fundamental problem is your connections are failing. Older versions would simply retry. 6.61 retries too.

[syshog]

You should try the 6.61 beta.

Do you actually know why the connections are failing? Connections dropping out like that will probably give you "Dirty Connections" errors too. The fundamental problem is your connections are failing. Older versions would simply retry. 6.61 retries too.

They dropout but the log lists that READ Failure error I reported on a while back. I've switched providers so I'm doing a clean header download with 6.56, that will take awhile. I'll test 6.61 beta in about a week once I've made a back up of my new header set. I considered 6.60 no go with how much space the raw text files were taking up in the import folder a while. Then AW kind of took a crap header wise so.. Did you re-enable compression for the headers being stored in the import folder? It takes up too many GB when try to download a lot of headers.