database.gz files

[squib308]

Ive noticed this for a while, I musta missed something - we're now gz'ing the databases or something every update?

I only notice it 'cuz I got a slow ass computer, no fancy SSD drives or anything. Db Update seems to take a loooooong time. No, scratch that, it DOES take a long time.

Also it processes the files sequentially, so if I hit a few groups as once, I have to wait and wait for the slowest one to update before the updates for the rest process.

Like I said, musta missed something. Is there a way to turn that off? I don't care about disk space really, not for headers and databases. But the time it takes to gz them..erk.

Thank you gentlemen
b

I see in the background it looks like 5 minutes have gone by since It started updating one group, and finished up with the other two. Erk!

[Quade]

The GZ files are where the headers are stored temporarily before they get imported into the DB.

Are you asking for the ability to not GZ the files? Not sure how much performance that would buy you buy, it should be doable.

[squib308]

I dunno what I'm asking for exactly, but it seems to spend an inordinate amount of time in the .gz step before it updates & can see new posts. Also processes them sequentially, so if I was dumb enough to update boneless, then 5 really small groups, I wouldn't be able to see the new posts in the small groups until after boneless had processed. (that's just an example, I don't download headers for boneless... Tried that a long time ago. Jeeesh... Not again anytime soon, maybe if I had my own cluster of systems dedicated to indexing usenet it'd be a possiblity... but since its already been done, no point. :)

Thanks for any input. Yeah, I should get a faster computer & an ssd drive or two. Right now I've got less than $40 in my account, however. Not looking hopeful anytime soon. Thank god for mac and cheese.

[Quade]

I have a change in the wings to force it to import from the currently visible post list first. So, if you select "a.b.test" tab in the display, the engine will import "a.b.test" files before any other files. I'll get that working and in the next beta. I suspect that'll solve most of your problems.