Advice to speed up local header searches

There is no such thing as a stupid question here as long as it pertains to using NewsBin. Newbies feel free to get help getting started or asking questions that may be obvious. Regular users are asked to be gentle. :-)

Moderators: Quade, dexter

Advice to speed up local header searches

Postby rob2k77 » Sat Jan 19, 2019 1:34 pm

Was hoping for some hardware tweak suggestions to speed up locally store header searches. (I'm jealous of your online index search)

I have maybe 12 newgroups downloaded and it looks like the oldest records are 2008.

I'm trying to speed up local searches through at large number of headers. (uses about 6gb of drive space)

Using an 3-4 year old HP server with windows server 2016
I added 3 SSDs (OS, newsbin program/headers, and a data volume)
i think 2 xeon six core processors.
a bunch of memory.

Are there any switch to use more or all memory?

I've try creating a 20gb RAM-Drive and moving the newbin program and DBs to the RAM-Drive drive, but didnt see any improvement.

Any suggestion on hardware upgrades to improve local searches?

Thanks a bunch
Rob
rob2k77
n00b
n00b
 
Posts: 3
Joined: Sat Jan 05, 2019 12:30 pm

Re: Advice to speed up local header searches

Postby Quade » Sat Jan 19, 2019 2:57 pm

Compacting the databases might improve it. Reducing the number of records could improve it.

The issue is that there's no way to use a database index when doing free text searching so, every record has to be touched. The greater the number of records, the slower the search. If you didn't really need to go back to '08 you could reduce the storage age. SSD's should make it faster than spinning disks but the performance is mostly impacted by "table scanning". The part where you can't use an index.

It might be possible to write an external utility that bolt real text searching onto the database. I'd probably google search "free text searching" and Sqlite on google.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 45006
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97


Return to Newbie Forum

Who is online

Users browsing this forum: No registered users and 2 guests