Page 1 of 1

6.70 beta 5 local search finds 5x more

PostPosted: Tue Feb 23, 2016 8:30 am
by RayMark
I am running a local search on a group of groups, and the search string is "." - just a single dot. It used to find 12 million results while consuming about 16 GB of memory.
With beta 5 - my all available memory is consumed, over 30 GB, and the number of results is over 30 million, perhaps even as high as 55 million - I think I saw this once, but perhaps that also was not the final result yet.

So what happened? Were the older versions returning only a fraction of the results they were supposed to return? Or beta 5 went crazy? Or the dot now means something different?

I am looking at the version history, there are some changes that might have affected the local search:

Fixed issue where not all downloaded headers were showing up in the post list.
Sped up startup time when you have a large files list. Like > 10,000 items.
Fixed the files list search function. Works on filename, path and subject
Fixed issue where the Find in files list wasn't scrolling to the top most item.
Fixed issue with filter file sizes not applying consistently.

Re: 6.70 beta 5 local search finds 5x more

PostPosted: Tue Feb 23, 2016 10:53 am
by Quade
Why not just select the groups and "Show all Posts" on them? That's essentially what you're doing.

If I had to guess, I'd guess there's some change in how the filter profiles apply.

Re: 6.70 beta 5 local search finds 5x more

PostPosted: Tue Feb 23, 2016 1:18 pm
by RayMark
Oh, I see what happened. My mistake. And your bug?

I opened a new search tab by right-clicking on the group-of-groups in the Groups List and choosing "Search in Groups"
Then a new search tab appears and it even has the name of the group-of-groups next no "Search:"
It is natural to assume that the search will be performed only in that group-of-groups. And it was by the older NewsBin versions, even though the group-of-groups name does not show up in the appropriate drop box.
With this beta 5 is different - it searches in all the headers. When I chose this group-of-groups explicitly from the dropbox, then the result is the same. Exactly the same number.

As to "Show All Posts" - you are right. Exactly the same number and exactly the same memory usage.
I was under the impression that search is faster or easier on resources. Perhaps it was the case with some older versions.
However, I see some difference if a filter is applied. The memory usage of Search is lower (applying the filter during the search and not after?).
The memory usage with "Show All Posts" is higher but also goes down a bit after applying the filter.

In particular, with a certain group-of-groups I have 12 million posts and 16.4 GB memory usage without filters.
After applying a filter which includes only posts from certain few posters, the number of posts went down to 1.4 million and the memory usage with Search: 10 GB, with "Show All Posts": 13 GB

Re: 6.70 beta 5 local search finds 5x more

PostPosted: Tue Feb 23, 2016 1:45 pm
by Quade
With this beta 5 is different - it searches in all the headers. When I chose this group-of-groups explicitly from the dropbox, then the result is the same. Exactly the same number.


It doesn't seem to be setting the "search in" so yes, I'll call that a bug.

I was under the impression that search is faster or easier on resources. Perhaps it was the case with some older versions.


Search is only more efficient IF you actually do a search. In your case, you're doing a "Load all" even if it is a search tab because you're not actually searching for anything. "." means "match anything" so, there's no benefit over loading the entire GOG.