DEBUG - Insert Failed: Shouldn't be possible: corrupt DB:

[mho]

Hi,

I got a ton of these, today, when updating headers. Likely alt.binaries.warez is the culprit.

I suspect/hope that this is an indication of corrupt/incomplete XOVER-info, rather than a corrupt db in newsbin, but perhaps you can take a look and possibly change the log message to something a bit less ominous if that's indeed the case. Thanks!

- mho

Code: Select all

[06:56:01] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: hallmark card studio 2011 deluxe-dvtiso (bx) [h33t]  [originPeterpan@gmx.de (PeterPan)
[06:56:07] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: maxprog ftp disk v1.0 incl  keygen-brdPeterpan@gmx.de (PeterPan)
[06:58:25] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: inscriber.cg for  fast 1.1aPeterpan@gmx.de (PeterPan)
[06:58:25] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: internet download manager v6 07 final build 8 incl  keygen aPeterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: keytext  2000 2.21Peterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: kyodai mahjongg  german 19.99Peterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: lastbit covertmail  2.0.384Peterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: just the  fax 2.8iPeterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: major league baseball  2k11-reloadedPeterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: k-lite pro  v2.0.0.1Peterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: light image resizer  4.0.9.5Peterpan@gmx.de (PeterPan)
[06:58:26] DEBUG   - Insert Failed: Shouldn't be possible: corrupt DB: lunascape web  browser 6.4.2Peterpan@gmx.de (PeterPan)

[mho]

Should probably mention that I get headers from astra (eu).

- mho

[mho]

I have a hunch (nothing more, so far, just by looking at task manager) that this might be a memory leak, too.
Perhaps the headers that cannot be inserted never get free()d?

- mho

[mho]

If you want to look at this, the following range seems to include a lot of PeterPan...

- mho

Code: Select all

astra-eu,Connected,a.b.warez,XOVER 388610134-388910134,224 data follows [COMPRESS=GZIP]

[Quade]

You know you can edit your own posts right? Haha.

This guys looks like a spammer and the insert is probably failing because it's a duplicate insert. Most real files don't have this issue because they're not duplicates. I thought I removed this error from the popup. Ah I see, you have it set for "DEBUG" logging level so, you see everything. If you turn your logging down, I suspect you won't see or even notice these errors. The spammers use the same subjects over and over again so, sometimes the insert fails. There's some spammer in the anime groups using the same kind of posting style.

[mho]

You know you can edit your own posts right? Haha.

Yes, but when there is some time between posts, people who have already visited the thread might miss my insightful ramblings:-)

If you turn your logging down, I suspect you won't see or even notice these errors.

What actually made me take notice was that updating headers for this group was extremely slow; took several times longer than a comparable number of headers for other groups.

Still, no biggie.

- mho

[Quade]

You'll find that many small posts are harder to insert into the DB than fewer larger ones. A 700 meg set is a couple inserts. 700 megs worth of small files is thousands of inserts. The DB is really optimized for larger file sets. The larger the better (performance wise).

[mho]

You'll find that many small posts are harder to insert into the DB than fewer larger ones.

Of course, but in this instance it was 40k headers taking a lot longer than 5M headers, etc. Yes, compaction will certainly be part of it, but it didn't seem to account for it all.
Perhaps it was just the scrolling of the log...

- mho