Newsbin 6.91 RC3 - Build 5855

[Quade]

https://s3.us-east-1.amazonaws.com/down ... 3-5855.exe

It's mostly the same as RC2 except with the help of some early testers I was able to figure out why RC2 would crash from time to time.

One change I haven't talked about in 6.91 is it outright drops posts that either can't be turned into files or look like files that can never be assembled. In some groups, a majority of posts are completely unusable unless you have the NZB. They can't even form files to list.

A.B.NZB for instance is 99% unusable posts. You'll see it in the logs where Newsbin now logs how many posts are read in and how many are rejected as "not files". If you keep an eye on it, you might see that some of the groups that used to be productive now have nothing. It's not that nothing is posted to them necessarily but that the posts aren't usable by conventional readers. You might consider turning off header downloads for these groups to save bandwidth. In this way you can keep your historical group records.

I've also noticed that not all files are using proper post counters to indicate position of the post in the file but, they are including this information in the message-id. Newsbin now follows the Message-Id based numbering over the subject based if both are present.

[Stan]

Somewhere along the beta's or rc's Newsbinpro no longer supports the [SIZES] in the newsbin.nbi
I only have 2 version 6.82 and the latest rc.
6.82 supports it, but when running the rc the [SIZES] all get removed.

This is what I had to redo in the nbi for sizes, trying to get it back to what I had before.
And can only do in 6.82. Maybe in the next weeks I will try 6.90?
So until I can get 6.90 installed as a 3rd option, assuming that will work, I have to use 6.82 to keep the added sizes.
I hope to add more as I did have more but lost them all.

[SIZES]
2 GBytes
25 MBytes
250 k Bytes
250 MBytes

Went back to nb691B10-5783 and still does it.
Tried 6.90 and does not do it. It keeps the sizes fine. But any version after seems to wipe them out and only leave [SIZES} and all other info gets removed.
So may try 6.90 as I tended to really like 6.82 and see if I can stick with 6.90 or go back to 6.82.

[Quade]

I'll look thanks.

[coyote2]

With 6.91RC3b5855 I see that the "Current Progress" bar is no longer in the color of the Server being downloaded from.

I'm very unhappy with this because now I don't know when my client has to turn from my primary server (Newshosting) to my fill server's block account. The lost feature is to me essential.

(I was even happier way back when the servers were displayed on each row of filesets.)

[Quade]

It's something I'm going to fix but you can also see server activity in the server tab.

Your backup server should be disconnected most of the time. If it's connected that means it's being used.

[coyote2]

It's something I'm going to fix

Thank you very much Quade, I'm happy it's on your radar screen. (TBH this feature is so important to me that it keeps me from switching to a newsreader with RSS.)

but you can also see server activity in the server tab. Your backup server should be disconnected most of the time. If it's connected that means it's being used.

Good to know in the meantime!

[triskel]

I have a strange problem: many files appear incomplete when updating the list of headers of a group (in fact if I download them they are incomplete); the same files with the "Search" function appear (and are) complete instead...

[Quade]

PM me a group and I'll try it. I'm not seeing here right now.

Just for grins, right click the group select "Post Storage/Use Download Age" then set a 5 day "Download age" in the options and then download headers normally. It'll re-download the last 5 days of headers for that group.

[JayPea]

I noticed a bunch of headers over a period of about a week where some host in the NNTP distribution network appeared to backfilling. Even though the posts were new, they were dated about 2y previous and all were incomplete. It stopped about 3-4 days ago. I observed this across the comics & ebooks groups but who knows how widespread it was?

[meenich]

Hello! I've been happily using the 6.90 release version, but figured I'd give the new 6.91RC3 a try when I saw it posted (first time using any 6.91 version).

Installation was OK, but to my surprise when I opened the newly installed RC3 all my "Groups of groups" disappeared. All my subscribed groups remained, but are now in the "Unsorted" bucket. Has "Group of groups" support been removed in 6.91? I'd hate to have to re-sort all my existing (now unsorted) groups.

If it's of any help, I tried the following:
* Quit from RC3, and copied backed up NBI file into the RC3 data folder. Same results when restarting RC3.
* Reinstalled 6.90 release version, but with the NBI left behind by RC3 - it now also has lost all groups of groups information.
* With no Newsbin Pro active, copied backed up NBI file back into data folder, then restarted 6.90, and lo and behold all my groups of groups are back.

Any help would be greatly appreciated.

Thanks!

[itimpi]

I have been seeing an issue where Newsbin appears to give an incomplete decode of a download.

The symptoms are:
- I have Newsbin set to both download and decode to a folder named after a NZB
- I add a download via a NZB and one part is shown in the download list AFTER the main download
- Newsbin appears to download and decode the parts with no error indicated.
- When I go to the folder where the download was unpacked to all the parts for the download are still there (normally they get deleted after a successful decode).
- If I now use Winrar to repeat the decode the resultant file is significantly larger than the one unpacked by Newsbin

This does not appear to happen if I promote the single part of the download that shows in the download list to be before the main download.

[cat_man]

RC2 and RC3 download headers, but when I show posts, I get none or very few. RC1 works ok in this regard. I even tried deleting all files in Spool_v6 and redownloading headers for all groups and GOGs. Same result.

[Quade]

I've run literally terabytes of headers through RC2 and 3. Not sure what to tell you.

Are you filtering the header downloads?

In the options you can add

[SETTINGS]
SaveGZ = 1

Instead of deleting the downloaded blocks of headers, they get moved to the "Processed" folder in the data folder. You can then verify headers are downloading. You can also re-feed these header into the "Import" folder to make Newsbin process them again.

I'd exit Newbin, add this option to the NBI, then restart and try some header downloads.

RC2 had some issues with "XFeatures" , RC3 fixes if. You might experiment with disabling XFeatures in the server advanced options and see what happens. I was only able to test "Xfeatures" against usenetserver.

[cat_man]

I have no filters defined. Turned off XFeatures, no change. Added SaveGZ=1 to settings and can see all the headers downloaded in the "Processed" folder. Set Display settings to "None" and I get 2 old headers out of 40 groups. I'm using Easynews.
Guess I will have to try a fresh install with the default nbi file and add groups and see what happens.

[Quade]

Guess I will have to try a fresh install with the default nbi file and add groups and see what happens.

That's throwing out the baby with the bath water.

Right click a group and select "Show All Posts". Then wait for it to finish. To the right of "filter profile" you should see 2 numbers. The first is the visible file count, the second is the available file count. If available is large and visible is small. That's telling you, your posts are being filtered out at the display level. I'd then try unchecking the green button next to "Min Size" which disables the display filters.

See if your posts show up. They should note in the status column why they're not normally visible

[cat_man]

Tried your suggestion, had 0 total and large available. Turned filters off and display settings to None and the headers showed up. Turned filters back on and they still showed ok. It's like I toggled something and now it works. Whatever it was, it's now good. Thanks for the help

[mimauk]

Just a small query - in the last few RCs, I've noticed a Folder appearing in my Downloads Folder marked as " Screens" which contains screenshots of posts I've downloaded. I was wondering if it's NB Pro making the folder or if it's caused by some of the posts I'm downloading.
I think it's really clogging up my system as I'm downloading, as it seems to freeze while unraring files and nothing seems to be happening for minutes at a time. I'm wondering if, as a file is unraring, it is taking screenshots of the post during the process and this is causing my system to slow down. I'm just trying to think of reasons why my system seems to stop sometimes whilst unraring files, it has got me puzzled.

[Quade]

Newsbin has no mechanism for making screen shots of your desktop or its own windows. I don't believe it's Newsbin.

It sounds very suspicious.

I did a search of my download drive and there are no "Screens" folders. For your downloads to do something like that, they'd have to infect your computer, which I don't believe normal media can do.

Windows 11 has a real time screen shotting utility called "Windows Recall" but, the data it stores is supposed to be centralized somewhere. I believe it's only supposed to work on machines with NPU's too.

If you're downloading RAR files with passwords and depending on the password list, the longer the list, the slower UnRAR startup is for unraring. Since Newsbin has to find the password from, the list. I recommend keeping the password list trimmed down.

My google searches found nothing like what you're describing.

[JayPea]

Windows Recall is also opt-in. It was the opposite when it was first released to the Windows Insider program and was only ever available to people who signed up to be part of the bleeding edge 'Canary' branch. As Quade mentioned it also only runs on CoPilot+ machines. Windows Recall, and the privacy nightmare it created, has been largely what's been holding up the release of Microsoft's W11 24H2 update, as it will be included as part of, which is currently sceduled for the next Patch Tuesday (2nd Tues of every month, next one 8th Oct 2024)

[mimauk]

Thanks for the reply.

The folder actually contains thumbnails of the videos I'm downloading and not screenshots of my machine, it just happens to be named Screens. Most of my downloads are from Indexers so the password list only contains one or two items. That's why I was wondering if the thumbnails were part of the post being downloaded or were being generated as the file unrared and thus slowing down my machine.

One of my Indexers actually has a preview button by some posts whereby if you hover the mouse over the button a popup appears with a few thumbnails of the post.

Again thanks for your reply and for a great programme.

[JayPea]

Ahh. Then they are most likely thumbnails of the content you're leeching and included, at source, packaged by the uploader, as part of the archive that's being extracted to your download folder.

[Quade]

The folder actually contains thumbnails of the videos I'm downloading and not screenshots of my machine,

Yeah that sounds like something inside the RARs then.

Depending on the number of images, the slowdown might be Newsbin displaying them to the thumbnail tab.

[mimauk]

I did a little more testing by downloading one video at a time from the 3 Indexers I use and only one of them makes the Screens folder containing 4 .png thumbnail files from the video. I tried another video and the 4 .png thunbnail files were deleted and replaced with 4 different thumbnail files from the new video. I tried a couple more videos and there was no activity in the Screens folder - so only with certain posters.

thanks again.

[itimpi]

I have seen the inclusion of a ‘screens’ folder becoming increasingly common in posts.

[Quade]

You can use the "Post UnRAR filters" to kill off these files if you want. Add a filter to reject file types and Newsbin will delete them out of the set of files that came out of the RAR/7z set.

[ccarlin]

Just to help close-out on this topic - the screens sub-directory folks are seeing is part of the overall .rar file being downloaded. Further, the .nzb for the download is specific to the site/indexer you're using, not necessarily the original creator of the video/.rar file. So, as an example, site #1 will have video #1 from source #1, and when you download you only get the video file (#1) itself, with all other files stripped out. Site #2 - when you download what you believe is the exact same .nzb file will include video #1, from the exact same original creator (#1) you (may) also get an .nfo file describing the video, along with a "screens" subdirectory with sample screenshots of the video, and maybe even some other files So, depending on the content you want you select the download/index site to download from (again, generally independent of the individual/group that originally created the content). /chris

[Quade]

I fixed the progress color thing. A fix for something else broke it. I'm working on getting RC4 out soon. I think it's pretty close to release.

[triskel]

Why te vaste majority of headers are incomplete when updated, but if "Searched" are (not all, but a lot of them) complete?
And the music files are not always downloaded in a folder with the name of the record, as per the settings (the tags are all ok).

[Quade]

I'm not having that issue. Is it a specific group?

If you re-load the group do they still show incomplete?

PM me the group and I can try it.

[triskel]

Why te vaste majority of headers are incomplete when updated, but if "Searched" are (not all, but a lot of them) complete?
And the music files are not always downloaded in a folder with the name of the record, as per the settings (the tags are all ok).

Thank for you reply.
I have the problem with many a.b.sounds group: i.e. look at a.b.sound.lossless "Sagajazz" and Gregg Allman nbz (oct. 6 and 8 ). Today "Greg Allman's "I'm No Angel" is incomplete and it's not found if searched.
I use both Easynews ans Giganews
Others files are incomplete when updating headers and searching them too...

Giovanni

[Quade]

I download headers from Usenetserver. I believe the search engine uses usenetserver headers too.

I'm just seeing complete files.

You could try right clicking the group select "Post Storage/Use Download Age". Then set a download age of 5 in the main options, then download headers normally. This will make it re-download the last 5 days worth of headers for that group.

Then check completion after the headers get imported.

[triskel]

I activated a one month account with Usenetserver and downloaded the list of headers of the group a.b.lossless.jazz.
The files are all complete!
Using Easynews and Giganews the situation is as I have already described.
I sent a report to Easynews and Giganews.
What do you think?

Giovanni

[Quade]

It looked like a server problem to me.

Some servers don't completely fill in headers in order. Meaning you might be missing some headers but Newsbin has already moved on to the next group of headers because the server couldn't deliver them all.

That's why I suggested re-downloading the last 5 days. If the server is slow to fill in, maybe a glitch or something, re-downloading the last couple days worth will catch you back up.

You could delete the files from the post list (shift-delete), then re-download 5 days worth of headers from the old server and see if they fill back in complete.

There's a setting in Newsbin to always re-download some of the previous run headers. This is for cases like this. It's called "Header Overlap" in the server advanced options. At one point this issue was common and header overlap was the answer. It's rare for this to happen these days.

[triskel]

I contacted Easynews and Giganews support.
The first one told me to delete cookies and cache (!); I replied that I don't use the browser and he replied with generic explanations on how Usenet works and on the possible incompleteness of the files, also sending me a customer satisfaction questionnaire (!!!).
The second one suggested me to use a server that is not in the "official" list, but nothing has changed; I wrote again and I'm waiting for a response.
In the meantime, no problem with Usenetserver, to which I think I will subscribe regularly after the month I have already subscribed.
I tried to re-download the files and enable Headers Overlap, but had no results.
Thanks for your patience.

Giovanni

[Stan]

I can't say much for EasyNews as never used them.
But Newshosting, Usenetserver and EasyNews are all the same company.
So you would think if one has the headers, they all would.
EasyNews, I think, is more aimed at webbassed browsing of usenet. Maybe that could be a cause of header issues, I don't know.

DROP giganews you're lucky if it has 1 year of proper headers, and they are very well known in taking things down faster than any other provider.

I can 100% verify Usenetserver and newshosting have proper headers. Even if you want to look up Eweka it too has little issues, but tends to be a bit more expensive.
So look for deals and grab either usenetserver or newshosting.
Believe it or not most providers, but not all, are owned by the same company, but they treat each entity different.
Some only get access to 3,000 day retention, some even lower, some higher, the current newshosting is over 5,900 day retention, usenetserver the same.
It's odd because the same company owns Astraweb yet Astraweb just seems so inferior.

[triskel]

Thanks for the reply: I will evaluate what to do...

Giovanni

[cat_man]

Same problem as before, headers download but don't display. Different computer, RC1 works fine RC2 and 3 don't What changed? Added [SETTINGS] SaveGZ = 1. Processed folder has all of the groups with good file sizes. Load all post show 0/0 headers. Toggle filters does nothing and no filters are defined. Same nbi as other computer.

[Quade]

Pick one group. In the data folder, under "spool_v6", rename the group folder to something else. "<groupname.old> or something. Something easy to switch back. It's best if it's a fairly active group.

Pick the files in the processed folder associated with that group and move them to the "import" folder. You should see the block counter near "cache" go up and feel Newbin grinding the files.

When it's done, look in the newly created group folder and see the size of the DB3 files. If you dumped a reasonable number of files into import, they should be a decent size.

> 10 megs depending on how many and how large the files were you dumped in to import.

I don't know what the problem is at this point. The results of this test should be telling.

I don't doubt what you're seeing. I'm just not sure why.

[kenr]

OK bug in saving from watch topics.
It is saving to $GOG (Watch Topics) not to the specified older in the properties.
Might be related to the previous bug with regex, as the effected have regex searches.

[Quade]

So you load the watch up into a post list and and it's saving to the parent of the watches download folder?

Or are you using auto-download?

Once the files end up in the watch list, they don't really have a GOG anymore.

[kenr]

Its on auto, I do a header d/l and it auto d/l's it according to the watch topic.
Basically I do an update, then select the gog's I am interested in browse the list, and leave the watch list alone.

[cat_man]

I deleted all spool_v6 files, set download age to 30 and redownloaded all headers. I then renamed one group to .old and added 12 files for that group to import. I could see NB process the files from the counter next to cache. At the end did a Show all posts 0/0. I looked in the spool_v6 folder and a new folder for that group was created. The storage.db3 was 52kb before and the same for the new group created. On my other PC running RC1 the same storage.db3 file is 90MB

[Quade]

In the options page under "Setup". What's the "storage age" set to?

The storage.db3 was 52kb before and the same for the new group created. On my other PC running RC1 the same storage.db3 file is 90MB

This sure sounds like they're being filtered out during import.

My storage age is set to 10000

- First I'd try setting the storage age to 10000 then restart Newsbin and try this experiment again.

- Second. Leaving it at 10000, exit Newbin and then move or rename the filters.db3 to filters.db3.old. Then start it and repeat the experiment.

I'd verify that the date is correct on your PC too. Storage age depends on the date on the PC being correct.

When doing these tests always right click "Show all Posts".

If you can PM me the group, I'll follow along.

Edit: Open the data folder and then open the NBI file and look for "Crosslimit". This can filter out posts during import too. If it's there you might exit newsbin then delete that line. The default is 12.

[cat_man]

Date is correct, set storage age to 10000, downloaded a bunch of headers. Still 0/0 on show all. Rename group in spool_v6, put processed files (14) into import. NB loaded them but still 0/0. Renamed filters.db3 and restarted, group downloaded an additional 270,000 headers. Show all still 0/0. Size of storage.db3 group file in spool_v6 still 52KB. No crosslimit entry in the nbi file.

[Quade]

PM'd you.

[Freaky69]

I’ve got a problem with the autoshutdown function. Newsbin closes after downloading all the content but the computer doesn't shutdown. All options are marked in the setting to shutdown the pc.

[Quade]

Thanks. I'll check it out.

[frayed]

thank you for adding the .rar, .zip to load nzb

[ccarlin]

UPDATE #2: I've confirmed the majority of the failed files are actually .par files with "file retry count exceeded" error message. The data files associated with these .par files are being reconstructed correctly. I still don't know whether this all is associated with multiple sites downloading portions of the file or something else.

[Update: I don't understand Apologies, I was sloppy and didn't notice that in at least some cases it was .par files that were not being downloaded - with the .par files getting the "File retry count exceeded" message. I'll now check more carefully. So, maybe hold off on chasing this one till I have more data. [FYI, I'm seeing a lot of "Failed Files" listed, that actually have been downloaded just fine, and I don't remember this happening prior to RC3. I can't figure out the pattern of the false reports. Sometimes 50+% of the files I download (say 5 out of 10 1GB files) report as failed, but have downloaded fully/correctly. During other sessions I don't get any false reports of failures. Note I use NewsHosting as my primary download site, but have 2 others as lower priority (fill site) backups. My guess is the files marked as failed required multiple sites to complete the download (I notice the download speed stutter), but because of the RC3 color problems associated with the sites ("Current Progress" colors not correct) I can't easily tell what's going on.] Thanks. Chris.

[pdf256]

FYI, I'm seeing a lot of "Failed Files" listed, that actually have been downloaded just fine, and I don't remember this happening prior to RC3.

I am seeing this with NZBs that split the par files from one big rar file.

All the par files download, but fail as there are no rar files.

Then the one big rar file downloads and decodes just fine.

Paul