Choose Chunk Folder Location?

Technical support and discussion of Newsbin Version 6 series.

Choose Chunk Folder Location?

Postby ppan » Thu Jan 12, 2017 6:02 am

As posted elsewhere, Newsbin becomes very slow when it tries to assembly files when the chunk folder contains 100 thousands files.
By experience I now that file operations are very slow when a folder contains 100 thousands files and the drive is formatted as NTFS. In this case, FAT32 is much faster.

I therefore tried to move the Data Folder to a FAT32 drive to test if this would eliminate the assembly bottleneck.

However, one spool file (storage.db3) in the Data Folder is more than 4 GB (4.37 GB) and this is not allowed on FAT32 drives.

Is there some way to only have the Chunk Folder on the FAT32 drive?
Alternatively, is there some way to have the spool files on an alternative (NTFS) drive?

Finally, is there some way to compact the storage.db3 file?
ppan
Seasoned User
Seasoned User
 
Posts: 245
Joined: Mon Jun 20, 2011 6:49 pm

Registered Newsbin User since: 08/24/09

Re: Choose Chunk Folder Location?

Postby Quade » Thu Jan 12, 2017 11:23 am

Finally, is there some way to compact the storage.db3 file?


Right click Post Storage/Compact Group with compact out empty space but if you don't have any empty space it won't change much. 4 gigs for a header DB isn't that big.

As for moving the folder. You might be able to map another folder to the chunks folder.

Another option is to tell Newsbin to stall the download while files are unraring. This should prevent a big buildup of chunks.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44986
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Choose Chunk Folder Location?

Postby ppan » Thu Jan 12, 2017 4:40 pm

Quade wrote:
Right click Post Storage/Compact Group with compact out empty space but if you don't have any empty space it won't change much. 4 gigs for a header DB isn't that big.

As for moving the folder. You might be able to map another folder to the chunks folder.

Another option is to tell Newsbin to stall the download while files are unraring. This should prevent a big buildup of chunks.

Unfortunately, compacting didn't reduce the file size much.

When mapping do you know if a Symbolic link or Junction would be best in this case?
Also, I am not sure if I wouldn't have the same issue with slow access [directory read] when the FAT32 directory is mapped onto the NTFS directory where Newsbin operates..?

I don't let Newsbin unrar. The chunk buildup is caused by slow file assembly.
ppan
Seasoned User
Seasoned User
 
Posts: 245
Joined: Mon Jun 20, 2011 6:49 pm

Registered Newsbin User since: 08/24/09

Re: Choose Chunk Folder Location?

Postby Quade » Fri Jan 13, 2017 4:48 am

I've never done this. It's just an idea. In windows, I believe you can map a drive to a folder.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44986
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Choose Chunk Folder Location?

Postby roywax » Sat Mar 11, 2017 9:12 am

Hello,

My chunks folder is huge, 70GB.

Can i move this folder to another folder using some settings?

Thanks

Roy
roywax
Occasional Contributor
Occasional Contributor
 
Posts: 23
Joined: Sat Feb 18, 2012 7:19 am

Registered Newsbin User since: 02/12/12

Re: Choose Chunk Folder Location?

Postby Quade » Sat Mar 11, 2017 9:40 am

My chunks folder is huge, 70GB.


Did you read the comments I already made? You might want to try disabling download during unrar/repair.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44986
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Choose Chunk Folder Location?

Postby roywax » Tue Mar 14, 2017 3:40 am

Hello,

Thanks,

My problem is not the speed or similar.

I just move all my newsbin folders to another location.

I didnt find an option to move this folder and it is messing with the windows users folder - which i dont want to use.

I think newsbin is just missing a feature for the location of the chunks folder.

Unless i missed this option.

Thanks

Roy
roywax
Occasional Contributor
Occasional Contributor
 
Posts: 23
Joined: Sat Feb 18, 2012 7:19 am

Registered Newsbin User since: 02/12/12

Re: Choose Chunk Folder Location?

Postby itimpi » Tue Mar 14, 2017 4:54 am

You can move the whole Newsbin DATA folder to another location (which includes both the CHUNKS and SPOOL folders). This cannot be done from within Newsbin itself while it is running, but the details of the method is covered in the Newsbin online documentation. basically it involves manually editing the Newsbin configuration file to point to the new DATA folder location while Newsbin is not running.

If you want to move just sub-folders of the Newsbin DATA folder then you have to get a bit fancier using the (rather obscure) Windows facility for setting up symbolic links. This works without the need to change anything in the Newsbin configuration.
The Newsbin Online documentation
The Usenettools for tutorials, useful information and links
User avatar
itimpi
Elite NewsBin User
Elite NewsBin User
 
Posts: 12607
Joined: Sat Mar 16, 2002 7:11 am
Location: UK

Registered Newsbin User since: 03/28/03

Re: Choose Chunk Folder Location?

Postby roywax » Fri Mar 24, 2017 6:35 pm

Hello,

Thanks, i checked the procedure and the data folder is already is where i want it to be.

So maybe it was old files...

I am good now, thanks.

Bye

Roy
roywax
Occasional Contributor
Occasional Contributor
 
Posts: 23
Joined: Sat Feb 18, 2012 7:19 am

Registered Newsbin User since: 02/12/12


Return to V6 Technical Support

Who is online

Users browsing this forum: Google [Bot] and 2 guests