Search - what's up?

[gnuser12]

I suspect this is DMCA related, but until today I hadn't noticed any impact.

Typically I would find content by search only, usually once a week or so. New posts for these searches would be there, no problems, easy-peasy.

Last week, no problems.

Today, now the searches for most content are fairly sparse, and don't seem to include search results that were there last week.

My assumption is these posts are being removed, and therefore there are no search results to show, even for results that were there last week. It's not that the filenames have been hashed/obscured (although clearly thats becoming prevalent).

Something looks to have changed - at least on Giganews. I'd been happily ignorant of what was going on in the DMCA area until I went poking around today - even though they seem to be winning, something is different.

Is this across the board for providers, or might it be Giganews-specific?

[dexter]

When you say search, are you talking about our Internet Search? We index headers as they come in. If you are noticing things missing that you are used to receiving, it could be they just stopped posting. If you have some specific examples, you can PM me (don't post here) and I can look into it further.

[gnuser12]

I'm out of town and therefore away from my PC with Newsbin on it - but I'll take a closer look when I get back. I'll try to see if I actually downloaded items that don't show up any more.

My recollection was along these lines: let's say searches were for Alpha and Beta, and furthermore let's say the date was in the description as well.

So, two weeks ago (on the 10th) a search for Alpha would have shown:

Alpha.9-10-17.content.mp4 (perhaps posted just an hour ago)
Alpha.9-8-17.content.mp4 (posted a couple of days previous)
...
Alpha.6-10-17.content.mp4 (posted in June)

and so on. This has been working this way for years, good stuff. No, great stuff.

But yesterday, running a search for Alpha not only showed a dearth of posts, but (and this could be wrong) recent posts that I'd seen before also gone:

Alpha.3-10-17.content.mp4

So, items that I thought I had seen would seem to have gone away. But if content (headers) is indexed as it comes in, then that shouldn't be the case as you note.

I'll take a look again when I can.

[dexter]

When you get back and verify, if you can PM me some specific examples I can look into it further.

[steveb2954]

Like gnuser12, I too have been having problems searching for content. Items that would normally be there are not. I find only par files with long alph-numeric names. I subscribe to your Unlimited Internet Search, and I have been using this search for a long time with no problems. Am I doing somthing wrong, or is the DMCA taking everything down? Is this the end of newsgroups?

gnuser12 was using Giganews, I'm using Newshosting. So this seems to be happening all over.

Any help would be greatly appreciated.

[deanmacd]

I'm getting similar search results (or rather lack of)...

A particular search for something showed only a 77.0MB result and only has .par files.

[dexter]

The issue is with how they are posted, not with the news servers. These people are posting the .rar files with totally random filenames across multiple newsgroups so there is no way for our search engine to gather all the parts together to present the entire set. There is enough information in the .par2 files, that's why they are coming up in search, but the .rar files are missing.

Quade and I do have an idea how to support this but it is a pretty involved code change that will take a fair amount of testing to get through. We'll let everyone know when it is ready for testing.

[steveb2954]

Thanks for the update, and I wish you luck in finding a solution.

[ericeweg]

I had been getting a little frustrated over the last month or so, about the fact that I could only find (some) .par2 files and no .rar files anymore, using my unlimited subscription to Newsbin Pro Internet Search.

I figured Search might have started to filter or delete search-results and after almost 10 years of paying for it, I was planning on cancelling it all together. I am glad I decided to do a search in the forum first and am happy to see I was wrong about the reason why I couldn't find anything anymore.

Thanks for your update, Dexter. I wanted to PM you, but I guess I can't, since I had never posted anything here (I have had no real issues I can recall in 10 years: not bad! Image link not allowed for unregistered users).

I understand you can't give an E(xact)TA on when Search will be up-and-running (showing all .rar files and .par2 files again), but can you maybe give an indication in what you guys are doing? I don't mind not being able to use my subscription to Search for a bit if you guys are working on it; if you don't expect a solution in the foreseeable future however, I hope maybe you have an alternative Search method I can use, until all the coding and implementing is done (did I use those words correctly? Have no clue what I am talking about! Image link not allowed for unregistered users) and we all can start using the Search option again! Image link not allowed for unregistered users

Kind regards

[gkar]

any update on this yet?

[dexter]

Probably won't make progress on this until after 6.80 is out.

[steveb2954]

Just wondering if there has been anything new with regard to these searches?

[dexter]

Trying to get 6.80 out the door then we will concentrate on it.

[Vimes]

Trying to get 6.80 out the door then we will concentrate on it.

Seems promising. My subs for the search engine are coming up very shortly for renewal and, for the reasons you will know already, I'm finding it very difficult to think of continuing its use.
Do you have any idea if the search function can be reasonably effective once again and soon(ish).....?

~Thanks

[dexter]

I've got a working prototype that looks promising. This is a moving target so we have to build in flexibility so we can adapt as things continue to change. You will be seeing incremental improvements once 6.80 is out the door and we can dedicate more time to it.