Compact Database - Automatic

This is the place to help test and discuss Version 6 Beta releases.

Compact Database - Automatic

Postby TBlack » Sun Mar 29, 2015 12:02 pm

Any thought to adding a setting somewhere, i.e. Options/Advanced, that would automatically compact the databases, in the background, during times when NB is doing nothing but waiting and listening? You could add the timeframe in Days or Hours (or maybe just Hours) or a specific time associated with the setting like you do with other categories in Options. With a high powered computer and multi-core processors, this would be a cake walk :D

What made me think of this is I am always deleting Post I don't want or need any more, so I have to manually compact the databases and Watch Lists to keep their sizes manageable. I am also not backing up dead space and unnecessary file sizes.

Also, add a time to do this, i.e. I image my boot drive daily at 6:30pm, so compacting at 5pm daily would work.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Compact Database - Automatic

Postby Quade » Sun Mar 29, 2015 1:05 pm

The databases don't grow forever. At some point they hit steady state. The space you made by deleting is used by new headers you download. Compaction can speed up the loading but it's not something you should have to do all the time. Once you've cleaned out the old records, only the new records feed in and without compaction, they'll just tuck themselves into the free space in the file.

I'm not fundamentally against the idea but, I'm not sure you realize how much data it is. My data folder is just shy of 400 gigs. It would be numerous hours to compact my data folder.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44981
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Compact Database - Automatic

Postby TBlack » Sun Mar 29, 2015 1:49 pm

The way you explained how it all works, I see your point. The most I have is 1.5G, but I always thought compacting kept them from growing and growing, thus the suggestion. I've never just let them go and observe their size.

Given your scenario, you may not choose the automatic option or if you do, you do it at night when you are asleep or you start it when you go home. Mine compacts about 1 to 1.5G in about a minute and usually gains a 25% to 35% reduction overall....400 minutes is just an overnight that's done periodically. When you come in the next morning, you check the log to see what happened or you look at the results in the Header Database Compaction Window that is there now.

I leave NB on all day for the Watch Lists and there is idle time, so I thought that would be a good time to compact when not so busy. Of course, you could prevent Header downloads while compacting.

Maybe I am wrong, but I gotta believe there some advantages to compacting for the average user, but maybe not for a database user such as yourself.

I also thought about backups, compression and file sizes, etc. I also thought about NB being a high-end program and this should be a feature of same, like high-end database tools namely MS Access, which is where some of my idea came from.

Thanks
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03

Re: Compact Database - Automatic

Postby Quade » Sun Mar 29, 2015 2:35 pm

Maybe I am wrong, but I gotta believe there some advantages to compacting for the average user, but maybe not for a database user such as yourself.


I like the idea from a theoretical perspective. It's the practical I'm worried about. For every person like you who keeps a tight reign on the size of your DB's, there's someone like me who just lets them grow. Any solution would have to work in both scenarios. So like I said. I'm not against the idea but it's not something to just whip out and hope it works.

It might have to be something happens while a screen saver is up or something.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44981
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Compact Database - Automatic

Postby TBlack » Sun Mar 29, 2015 3:17 pm

I understand what you are saying, I truly do. I used to work with a lot of non-techy/non PC literate sales guys and I brought them along, sometimes kicking and screaming :D

I have also seen you guys tell the users of changes in NB and that they need to do things a new and/or different way due to a new/updated features, like when all the databases had to be converted to new version and would not work for prior versions. You know what...they came along I bet....I know I did :D You guys have always done a great job of bringing users along during updates and changes.

Same thing here I believe. For those who have not maintained database sizes, they bite the bullet once, based on your guidance, should they choose the Automatic Feature. Bring those users along, but don't hold your advanced users back. If some don't want to use the feature, they leave it off. Just a thought :)

Quade wrote:It might have to be something happens while a screen saver is up or something.


What if a user doesn't use a Screen Saver? I didn't for many years. Suggest thinking in terms of settings that would execute the compaction at a certain time and an event frequency (3 pm Daily, Weekly, Monthly) when Headers are not being downloaded and the Post List is not being updated maybe AND prevent Header download during compaction? This would cover every option I think.

If I am looking at the NB screen and I see the Header Database Compaction Window appear at my preset time and I see the compaction occur and I see the end with the results with a short delay before the Window closing, I am okay with that and a log entry can be made as well. But then I have a very small database in comparison to some. If you want to do everything in the background, then make a log entry with the before and after results like the Header Database Compaction Window does now. I do think you need some overt indication that compaction is occurring, but maybe not.

These are just some ideas and feedback and I do appreciate your openness and discussion.

Thank you!
Tom
User avatar
TBlack
Seasoned User
Seasoned User
 
Posts: 340
Joined: Sat Mar 23, 2002 12:30 pm
Location: Indiana

Registered Newsbin User since: 04/05/03


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 2 guests