how can I clean up history

Technical support and discussion of Newsbin Version 6 series.

how can I clean up history

Postby oferlaor » Thu Jul 12, 2012 4:04 pm

SPOOL_V6 is getting quite large for me.

Is there any way I can remove history?

I have some automatic group download options (with filters), but the db3 indexes such as StorageData.db3, are 10-15GB per group.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Thu Jul 12, 2012 4:28 pm

1 - Set the storage age to something shorter.

2 - Update the groups.

3 - When the update is done and all records imported, select some or all of the group, right click and "PostStorage/Compact Database". This should get you some disk space back.

Compacting can take a long time. Might want to do it before you go to bed or something like that.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Thu Jul 12, 2012 4:51 pm

Will purge to display age help before the compacting?
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Thu Jul 12, 2012 5:32 pm

That's probably a better idea than updating the groups.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Wed Jul 25, 2012 12:18 pm

OK, I did this a couple of days ago. Still way too large.

a.b.hdtv for example has (after purge to display age and compact) two large files: StorageData.db3 (16GB) and Storage.db3data.tmp (9.5GB).

Together, the directory is about 26GB in size.

I'm going to delete the group and add it again.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Wed Jul 25, 2012 2:29 pm

It's a large group. If you shrink the "Storage Age" more. it'll get smaller. The whole question is, how many days do you want to store? For that group, 150 days seems to be about 5 gigs. My whole spool_v6 folder is about 160 gigs. That's 1400 days for the groups I track. You have complete control over disk usage. You have to trade off storage age for disk usage. Considering how cheap disks are these days. I don't think 160 gigs is that much.

I can say that Boneless over 1400 days is about 80 gigs all by itself. That's one reason I don't track boneless (I use search for that). The data in storagedata is already compressed too.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Wed Jul 25, 2012 2:36 pm

Something strange. It is set to 15 days...

I did purge to display age and then compacted...
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Wed Jul 25, 2012 2:42 pm

Right click "Show all Posts" and see if it really is purged. Sounds like it's not.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby Quade » Wed Jul 25, 2012 2:54 pm

Using a "Display Age" of 60. I right click and "Purge to Display Age". Then when it was done, I right clicked and "Show all Posts". It only showed 60 day. Then I right click "Compacted" the database and it went from 5 to 1.5 gigs.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Wed Jul 25, 2012 4:55 pm

Something very wrong with compact... It leaves the original file behind and adds a Tmp file as well.
Just compacted a 10GB group and ended up with a 19GB folder...
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Wed Jul 25, 2012 5:03 pm

The compact is failing.

How much free space do you have? You need at least twice as much and I think 3 times as much to be safe, free space as the size of the folder. So, I'm thinking you need 50ish gigs free at least.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Wed Jul 25, 2012 6:18 pm

200GB+ free...

Deleted and started redownloading the groups. Does it have to download allessages although display age is 15 days?

Something was definitely off there.
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06

Re: how can I clean up history

Postby Quade » Wed Jul 25, 2012 7:03 pm

If you delete the group, it downloads "Download Age" worth of headers unless you tell it to "Download all".
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44984
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: how can I clean up history

Postby oferlaor » Thu Jul 26, 2012 2:45 am

They are set as auto download with a filter.

After redownloading, the directories are less than 1GB each...
oferlaor
Seasoned User
Seasoned User
 
Posts: 416
Joined: Sun Oct 08, 2006 11:02 am

Registered Newsbin User since: 04/21/06


Return to V6 Technical Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests