Newsbin 6.91 B5 Beta - build 5672

[Quade]

1 - Fixes a crash bug that rarely happens but needed fixing.

2 - More robust handling of passwords embedded in NZB Filenames. There was an issue with ZIP's and RARS that contain NZB's. Now the passwords priotity is:
a - Embedded in the NZB Itself
b - In the NZB Filename
c - In the filename of the ZIP or RAR that contained the NZBs


https://s3.us-east-1.amazonaws.com/downloads.newsbin.com/downloads/beta/nb691B5-5672.exe

Things I'm looking at for the next beta

1 - Report that unrar is slow. Going to compare 7z unraring to native unrar code.
2 - Encountered an old encoding for the subject that I thought was handled but apparently isn't. This is very rare. it's a 20 year old encoding that's not used anymore.
3 - Idea to speed up header display.
4 - Getting partial 7z decode working. This is needed to detect the new "encrypted RAR inside split 7z" posts.
5 - Ran into a case where a download was complete but the PAR was missing a file. It prevented automatic decoding.

[kstan12]

These aren't new issues for me but I have two problems, not sure if they are related to my config or not.

1 - Pause the Download button only seems to pause all downloads for a few seconds, then it resumes d'ling.
2 - Downloaded filenames seem to truncate at 80 chars. Can this be increased?

My dupe detector doesn't seem to work anymore either. Is there a way I can reset/fix this?

Thanks a lot.

[Quade]

1 - You must have accidentally turned on the scheduler. You can turn it off in the options.

2 - I'll have to check. Is it truncated after unrar or are you downloading files with really long filenames that aren't inside a RAR?

[kstan12]

Thank you, it was the scheduler. I've turned it off now.

These are truncated after unrar.

[stavros]

Hi, just a minor display issue, I think.

When I do an extended selection of a bunch of headers and then press ctrl + E to d/l them,
they get added to the download list and start downloading ok, but the headers do not get the queued icon
(white tick on blue circle) while they remain selected.

Their icons get updated to the queued icon when they are de-selected,
or when I scroll the headers up or down via the scroll bar on the right (while they are still selected),
or to the downloaded icon when they are downloaded (even if they are still selected).

When I select a header and double click it to d/l, a similar thing happens to the above.
This is slightly worse in effect because it is not clear if the double click actually 'took' without either deselecting or checking the d/l list.

This happens about 30% of the time, but is quite noticable when it does.

regards
Stavros.

[drez]

1 - Fixes a crash bug that rarely happens but needed fixing.

5 - Ran into a case where a download was complete but the PAR was missing a file. It prevented automatic decoding.

Can confirm this, downloaded a old file that had no pars and it wouldn't auto unrar.

Manually unraring it worked so they weren't damaged files.

[Quade]

If the files are grouped, the automatic unrar will work without PARS. If they aren't grouped, the first RAR has to be the last RAR downloaded if you want a successful decode.

The issue is non-grouped RAR downloads, when the first RAR downloaded. It simply looks like a small RAR to Newsbin so it tries to unrar it. The rest of the files aren't there yet so the unrar fails. Then the rest of the rars download. Without PARS or grouping, they're not associated with the other RAR files.

I download grouped parless rars all the time.

[Quade]

The lack of checkmarks is because the feed to the download list is now threaded it's not lockstep anymore. Everything is happening in parallel. Maybe check it before I fire off the thread.

[ccarlin]

I'm occasionally seeing an unrecoverable error - and I'm pretty sure it's always this one: ERROR Sqlite Error : Reports Unrecoverable error 1File: d:\Newsbin\DownloadMarker.db3. At minimum It's happened with both B4 and B5. Today B5 crashed when the error occurred.

In all circumstances I've received the error when quickly clicking through group names in my "unsorted" group list. Note, a few of these groups are large in size ( over a million posts)....

Related, for many years (many versions of newsbin) when I first click on a group name to get a list of posts, new posts are not listed/highlighted that first time. It's only when I click on a group name the 2nd time that new posts are displayed. So, I'm constantly clicking on group names to get updated posts .... and when I do I occasionally receive the Sqlite error.

Regards, Chris.

[Quade]

It's suggests the "Downloadmarker.db3" is damaged and needs to be deleted. Newsbin will regenerate it.

1 - In the options "Open Data Folder"
2 - Exit Newsbin.
3 - Delete "Downloadmarker.db3"

Restart.

This is the file that tracks downloads and allows you to "mark old" NZB and Search based downloads which have no local storage.

Even corrupted it shouldn't crash. I'll look into it.

[spamsilly]

Had the same issue after installing (Downloadmarker.db3 corrupted) but fortunately not crashing. Though it did get pretty noisy about it in the log window over time Image link not allowed for unregistered users

Deleting the file got rid of the issue.

Strangely enough, as I was typing this reply about all being well.. I tried loading a large group in the background and came back to "[03/02 16:39:21] ERROR Sqlite Error : Reports Unrecoverable error 1File: U:\_NEWSBIN_\DownloadMarker.db3" in the log window.

[spamsilly]

Came up with a new one.

[03/02 23:06:09] ERROR Sqlite Error : Reports Unrecoverable error 26File: U:\_NEWSBIN_\DownloadMarker.db3
[03/02 23:06:09] ERROR Sqlite Error : Reports Unrecoverable error 1File: U:\_NEWSBIN_\DownloadMarker.db3

Didn't try to fix anything since last post - Just left it running.

Maybe I have something that needs updating to work properly with the latest and greatest?

Thanks again for this wonderful project! - It's been a pleasure to use and a standard install on every main machine of mine for the past 20+ years.

[Quade]

I'd probably check your machine over pretty carefully. The DB3 files are quite robust. Even a power outage shouldn't damage them.

[spamsilly]

Thanks much Quade - Will do!

Maybe Eset is messing around.

[ccarlin]

Quade,

I deleted the "Downloadmarker.db3" file on 2/27 as you suggested and I've not had any Sqlite Error messages since. I've tried my best to make it happen and failed, so thanks on this one. Related, after deleting the file I also noticed that my second problem appeared to have gone away too, but I was wrong - it's occurrence has just decreased significantly. This second problem was new line items not showing up when updating a group (in the past I had to update the group multiple times to get all new items with 100% assurance). While it was a regular problem previously, it now appears to occur only about 5% of the time (when updating a group at least 24 hours later than previously updated), and in no predictable way that I can determine (rate of update requests, size of groups, days between update requests, etc). Just thought I'd let you know.

Regards, Chris.

[TobyTheCat]

Stumbled across this thread while searching for a solution for the embedded passwords you come across nowadays. I'm very happy with this beta, it now handles embedded passwords correctly.

Many thanks Image link not allowed for unregistered users

[kenr]

Having issues with MP3 (flac) files again.
Par files are there but the files are not put in the mp3 folder. Is it because you only look for mp3's?
Strange thing is it is not constant some are put in folders others aren't

[Quade]

I'll have to look. The issue is probably that a repair requires that all the files be in the same folder so they get moved out of their special named folders during repair.

[kenr]

There insn't some of the album in the right folder, it is some albums are right an others in in the $GOG folder
unrar unpacks the covers but usually leaves them in the $GOG as well. Again not always
these are all "loose" files with par's not rar's if that makes a difference

[Quade]

I don't believe files in a rar set are renamed using album/artist. They're left alone in whatever pathing the RAR defined. I'll have to verify though. There is a post processing step for files inside rars.

[gkar]

gald im not the only one with slow unpacking.

7z 10secs
Newsbin 1minute

spinning drive. setting pre allocate in performance makes no difference. Disabling AV makes a small difference. Ive started using another solution at this point.

[Quade]

I've looking into using 7zip to unrar. It's not reliable yet for some reason.

B4 or B5 got a fix for 7z containing many small files. It would slow down as the file count went up. That's fixed in at least B5.

[gkar]

im using the lastest beta. The file i used as a test was a RAR file.

[kenr]

Been having spped issues last few days.
A lot of files at ~400Mb ea. Download speed shows 6-7MB (my max) but nothing is being downloaded.
Pressing pause and resume seems to fix it sometimes.

But it is taking about 5min per file which should be about 1min. It is also stalling several times, stuck at 0bps.
Running 2 servers Asta and newsgroup

[Quade]

If I had to guess, I'd guess that it's the time needed to assemble 400 mb files out of 700K blocks from the chunk folder. That's 600-1500 blocks that need to read from disk to assemble the file. Download can continue at this time because Newsbin will still buffer some of the block in memory.

What kind of drive is your data folder on?

5 minutes is too long. I'd expect it to assemble in maybe 20 seconds per 400 mb file.

[Performance]
ChunkCacheSize = x


The default chunk cache is 400. As an experiment, you might try setting it to 1000 or 1500 and see if that speeds things up. On an SSD it probably won't make a difference but it could help spinning disks.

You'd need to "Open Data Folder" from the options and then edit "Newsbin.nbi". If this field isn't there you could add it. Each of these could be as larger a 1 Meg each so 1000 should be 1 GB of RAM to use as a buffer. If you got it right, the new size will be listed down near the "Cache" line at the bottom.

[kenr]

False positive, Sorry. I dropped the newsgroup server to a lower priority and off it went.
Looks like it was that that was slowing the download down. Will be dropping them anyway when my contract finishes.

[Mav99]

I've just checked the latest Beta and unfortunately had to discover that double-clicked or dropped NZBs still automatically download, no matter how the automatic download option is set. I know your explanation for this but I still really don't care for this.

*PLEASE* add an option somewhere to disable this behavior!

Or simply set the existing automatic downloads option to "On" by default and respect when users turn it off again.
All I want is that double-clicking or dropping an NZB works like it used to and how the slow, long-winded loading through the files menu still does.
That can't be too much to ask, can it?

Btw., I *NEVER* encountered an NZB that I could not download with earlier versions of NewsBin, starting the Download manually to a manually selected target folder. Never.
Worst case where obfuscated, random filenames which in the end where properly renamed by the par2 process. In one or two cases AutoPar didn't work in 9.8x and I had to use Quickpar/Multipar to do it and unpack the "repaired" files manually afterwards. No big deal and I really prefer that over automatic downloads.

[Quade]

I'm looking into it. The default will still be to the download list.

[Mav99]

Thanks. Doesn't matter what the default is, as long as there's a working option to change it.

Btw., it's not so much that downloads start automatically when dropping an NZB but that it automatically sets the target directory. I often download into the "main download folder" set in the Newsbin options but not always. And I *ALWAYS* want to be able to choose and name that target directory myself before *ANY* download of any kind. So an option to make the program ASK for the target folder for every download would be an alternative.

[kenr]

Strange issue with this beta.
Under Wine, it halts loading the screen for several seconds (>15), eventually get a quit/wait as it works.

[Quade]

It probably indicates it's doing more work than it should when it starts. Like having to load really large DB's. You might want to check the DB3's in the data folder and see if any are inordinately large. You can try moving them out of the folder one at a time to see which one causes your hang.

[kenr]

The biggest is 16.8Mb so not really large.
It started fine this morning so will just keep watching it.

[pfk505]

Had an odd issue with this beta today having just upgraded from 6.90 RC

NZB with 5 or 6 files, not rar'd or obfuscated (e.g. entire video files that had been posted)

I tried to download all but Newsbin would rename each to the NZB filename (normally a good thing for obfuscated, rar'd sets) but each file was being renamed to the NZB name and overwriting the previous one. I don't seem to recall having this kind of problem before.

[Quade]

I'm aware of that issue. Worked on it and a fix should be in the next beta.

It's renaming because the filenames are obscured. Newsbin tries to hide that from you with renaming but, as you can see it's thinking there's only one obscured file per set.

I was just about to put the beta up last week then I broke something and am in the process of getting it working again.

Thanks.