newsbin.com

I haven't noticed the log message "ERROR - Unable to Insert Subject record <subject>", before, so I think it might be new (to me) in b5 or b6, but now it shows up in periods (dozens of messages/second for a few minutes). Is this due to broken XOVER data or perhaps a corrupted db3? (Or a bug??)

- mho

Don't know. I've not seen it yet. Could be any of the 3.

Quade wrote:Don't know. I've not seen it yet. Could be any of the 3.

Maybe you could add a numerical tag, or something, if the same error string is used in multiple code paths?
(perhaps with an option/config file entry to turn it on so to not confuse the innocents:-))

- mho

There's no "multiple paths". It's failing to insert, I don't know why. Your guess is as good as mine. It might not even be a real error. Maybe some spam is triggering it.

If I can't watch it happen in the debugger, or people don't report missing data, there's not much to look at.

Quade wrote:There's no "multiple paths". It's failing to insert, I don't know why. Your guess is as good as mine. It might not even be a real error. Maybe some spam is triggering it.

If I can't watch it happen in the debugger, or people don't report missing data, there's not much to look at.

I found a "suspect" group, and moved its directory aside, downloaded all, and am currently doing a ctrl-G while keeping an eye on the log. If it doesn't show up, I think it was likely that the Storage.db3 for that group had been corrupted, somehow - will report back when finished.

- mho

Quade wrote:There's no "multiple paths". It's failing to insert, I don't know why. Your guess is as good as mine. It might not even be a real error. Maybe some spam is triggering it.

Aha, I misunderstood you, or rather, maybe, mis-imagined what the code looks like:-)

If the header being processed is still associated with an article number and server at this stage, maybe this could be logged? Then one could just telnet to the server and do XOVER for the suspect article and inspect the data to eliminate/confirm one possible cause...

- mho

I could probably show the subject, article numbers are tossed as soon as the header is downloaded. Meaning it's not in the RV4 files that are processed into the DB.

Quade wrote:I could probably show the subject, article numbers are tossed as soon as the header is downloaded. Meaning it's not in the RV4 files that are processed into the DB.

The subject is already in the log message, but that doesn't help much (message ID might be better).

Anyway, the log message didn't show up when I updated all groups after having done a purge and download all for the group that I suspected.

I reverted to a copy of the old spool directory for this group, and an update now triggers the log, so it's related to the db files.

I still have a copy of the group's spool directory (7GB - at least it wasn't boneless:-/) that will trigger the log message (in the thousands!) when updating. I can put this up for ftp, if you are interested in loading it up in the debugger and reproduce.

- mho

I too have hundreds of "Unable to insert Subject record (????)"

Looking again atteh list (????) may have been part of the descriptive so a seperator may be usefull

Looking at the descriptions though probably ALL would have either been hit by my "Min SIze Filter" or less likely "User Filter"

I've seen this on occasion, seems to be related to spammy groups, for me anyway. More moderated groups never seem to have it.

DT

The key might be to wipe the headers and re-download and see if it goes away.

Quade wrote:The key might be to wipe the headers and re-download and see if it goes away.

Don't know if you read my post above, but I did this and the message went away. I still have a copy of a spool directory that will generate the message when updated if you are interested in examining this in a debugger.

- mho

Appreciate it...but there's no point. Once a DB3 is damaged, it's pretty much unusable. If you're good with the command line tools for sqlite, you might be able to recover most of it but, some data will be lost. It's one reason there's a DB3 per group. So, a single DB failure doesn't kill all of your headers.

Quade wrote:Appreciate it...but there's no point. Once a DB3 is damaged, it's pretty much unusable. If you're good with the command line tools for sqlite, you might be able to recover most of it but, some data will be lost. It's one reason there's a DB3 per group. So, a single DB failure doesn't kill all of your headers.

No problem, I already re-downloaded so I have a working db, again.
I agree that a single 100GB+ db for all headers would be a really bad idea:-)

- mho