v6.50 different behavior since upgrading from v6.42

[no1butme]

Upgraded from v6.42 to v6.50 yesterday and have noticed some different behaviors:

1. Upon completed download, par2 sets aren't being named as they are posted. After downloading par2 files, they are appended with alphanumerics that aren't in the posted par2 filename. Newsbin is adding 8 alphanumerics into the par filename every single time. It never did this before and I am curious if this might be causing the issue below (item 2). If the par2 set has a "base" name of Test.archive.par2, then every single par file in that set will have something like this as a resulting filename: Test.archive.253DE84CA.par2. What gives?

2. Some people are posting files to usenet and renaming them as they are posted/uploaded. For example, the rar set may have a base name of Test.archive.rar, Test.archive.r01, Test.archive.r02, etc., but the post will show the filename as simply 2.0, 2.1, 2.2. The par2 set will be similar in nature in the naming scheme. Newsbin v6.42 would download the par2 set and somehow automatically rename the rar files as they were downloaded. Now the rars aren't being renamed. They are just sitting there with the 2.0, 2.1, 2.2 filenames. If I run Quickpar (double-click the par files), it will rename them, but this used to be done on its own for some reason - not now with the upgrade.

Should these items not have a fix, I will probably be downgrading and hoping that v6.42 works the way it once did, but I won't be surprised if it doesn't.

[no1butme]

I did a test and loaded an nzb for a group of renamed rars (did some research and this practice is called obfuscation and can be found with a simple Google search "obfuscate usenet"). Anyway, v6.50 ran all night and wasn't automatically renaming the files. My hope was that the Autopar process would fix this - but sadly, it didn't. All the files that downloaded successfully but were not manually renamed by me double clicking the par2 file were simply GONE. Not in my Recycle Bin, just GONE. It's obvious to me that something major changed and it affects the naming of the par2 files after download and it also affects the par renaming process (maybe something to do with what's in this thread, just guessing tho: http://forums.newsbin.com/viewtopic.php?f=43&t=34241)

Went back to v6.42 and all is working as it should, for the most part. I did get some sort of column error, but I can live with that for now.

I feel this autorename behavior via the par2 files shouldn't be left out in newer versions. I can provide Dexter or Quade with an NZB file, should they like to see if they are getting the same behaviors as I was with v6.50.

[Quade]

1 - It renames the files as part of the repair process. So, they rename after the download completes. This lets downloads resume without re-downloading the files that were renamed.

2 - The PAR2's get tagged with the PARID because morons are posting multiple sets with the same filenames. Sounds like what your boys are doing.

3 - If you're downloading sets that all have the same filenames and you add multiple sets to the download list at once you probably want to download each set to a unique folder so the filenames don't collide. I suspect the issue might be that repair is happening into the same folder as download and files are being renamed out from under the repair process. If you're using NZB's say, one NZB per set, you might want to add "$(NZBFILE)" to the download path. Then you can unrar to some other folder if you wish.

Please send the NZB ts@newsbin.com.

I had a report that you can't disable unrar so, maybe the files are unraring.

[no1butme]

In response:

1. v6.42 renamed each rar as soon as it was d/l, not during par repair process. v6.50 never renamed any rars without me double clicking the "main" par2 file. Granted, I didn't sit there for the entire 4 hours of downloading to see what the par repair process did. But in an attempt to at least give the par repair process a chance the rar files were simply GONE when I woke up. They should have remained in the d/l folder until I double clicked the par file (based on the behavior I was seeing during the first hour of download).

2. The par2s do have individual file names (not all the same) because when they are created, QuickPar adds .vol000+01.par2 to the "base" file name. The 3 digits after "vol" change as well as the digits after the + sign. That's standard par2 file naming that "my boys" (really? I'm trying to help here) and probably the majority of people posting to usenet typically use. Test.archive.rar, Test.archive.r00, etc. will be renamed to simply 2, 2.1, 2.2, 2.3. The par2 files follow the same renaming...2.par2, 2.vol000+01.par2, 2.vol001+02.par2. These are not the same file names at all. Each set of pars will be numbered differently - one set might be 0.par2, the next set 1.par2 and then next set 2.par2 (again, all are different file names once the volume and block count are appended).

3. All sets are downloading to their respective folder. I have Newsbin set up to download to a folder named for the NZB file. I ran all day using v6.50 with other posts that weren't using the renaming scheme and everything was fine (except for the appending of the 8 digits to the par2 file name). The repair occurred in the download folder.

Lastly, I d/l 4 or more complete posts before going to bed (well over 30GB worth of files in total) using v6.50, and I do not have Auto Unrar turned on. Newsbin never unrar'd a single archive, so I doubt it would do it for this one set that was renamed. Why does v6.42 not have these very same issues, because the posts aren't changing as I upgrade - only the methods in how the posts are handled by the software are changing, right? So I think we can remove poor posting practices from the list given that the file names aren't changing based on what version I have installed.

Normally, I never touch anything in my options after an upgrade. I simply run the upgrade and open Newsbin and load the next NZB on my list or resume whatever was downloading when I closed it out. I didn't touch a thing, ran Newsbin as I normally would. The only thing I did change was adding the old background color scheme to my .nbi file, per another thread on here. Other than when the interface was overhauled, I've never noticed such a big difference in the behavior of download results without changing a single thing in the NB options. I don't mean that as harsh criticism, just to give you an idea of the difference between 6.42 and 6.50 - nothing more.

I really appreciate your time and all of your efforts with this software as well as the time spent on the forums. I'm just frustrated because I don't know what's going on in the background and I'm not a programmer, so I am not sure how to communicate the issues in a manner in which a programmer would understand what I'm seeing EXACTLY as I'm seeing it. I only hope that the NZB file provides you with the same result as I had.

[Quade]

Lastly, I d/l 4 or more complete posts before going to bed (well over 30GB worth of files in total) using v6.50, andI do not have Auto Unrar turned on. Newsbin never unrar'd a single archive, so I doubt it would do it for this one set that was renamed. Why does v6.42 not have these very same issues, because the posts aren't changing as I upgrade - only the methods in how the posts are handled by the software are changing, right? So I think we can remove poor posting practices from the list given that the file names aren't changing based on what version I have installed.

How about putting everything back to default, make sure Autopar is enabled and unrar is enabled and try again? If autopar is disabled, there will be no renaming or unraring. I'm not 100% clear what your settings are.

When it's downloading, look at the par block counts in the download item and make sure they count up. Just sounds like it never does the repair step. Without that you don't get rename or unrar.

[no1butme]

Autopar IS enabled (auto repair) - Auto-Unrar (auto unpacking of rar archives) IS NOT enabled. Options weren't changed - ever - and they were double checked. Autopar worked every time it needed to work EXCEPT with the obfuscated files (files renamed during posting).

I have since downgraded to v6.42, so I am unable to see what is happening with v6.50 any longer. I simply tried to explain what v6.50 was doing without making a single change to my options. To reiterate, v6.50 was doing the following, which v6.42 was NOT doing:

1. Appending 8 alphanumerics to the par2 file names (all par2's, not just obfuscated ones).
2. Not auto-renaming files
3. Mis-named files were downloaded, but apparently the autopar repair process removes them (awoke to no files downloaded and empty download list - search of hard drive resulted in nothing being extracted).

I have sent "the offending" NZB file which will contain the obfuscated (renamed) posts - see if you have the same thing happen with v6.50 (at least #'s 1 & 2 - 3 will take hours). You should notice 1 and/or 2 almost immediately. Compare with the file names shown in the posts in the download list.

Thanks again!

[bobsmo]

Just FYI, I also get those strange PAR filenames.....for every single NZB I manually download and unrar....however, there are no ill effects on it.

And yes, I have Autopar disabled.

[Quade]

The renaming of pars is automatic.

[no1butme]

The renaming of pars is automatic.

By this you mean the appending of the 8 alphanumerics onto the par files that I was seeing? Or par2 renaming the rar files automatically?

[no1butme]

Just FYI, I also get those strange PAR filenames.....for every single NZB I manually download and unrar....however, there are no ill effects on it.

And yes, I have Autopar disabled.

Were you downloading files that had the original rar archive names (example: Test.archive.rar, Test.archive.r01, etc.) or had all of the posted files been renamed during posting (example: 123456-1, 123456-1.0, 123456-1.1)?

Like I've said at least twice, the main issue seems to be posts where files have been renamed (obfuscated) in that 1.) par2 files get 8 alphanumerics added to end of the base file name (test.par2 would download as test.AC14DD28.par2 instead of just test.par2). Newsbin shows the file name in the Download Description column as one thing, but saves it with the added 8 characters. 2.) The par repair process in Newsbin did not automatically rename the rar files during download nor did it rename them after all rar files had been downloaded. I had to manually start QuickPar by double clicking the par2 file THEN the rar renaming started working. Previously, in v6.42, this happened as EACH rar file got downloaded.

To add to the difficulty in finding out what is different, I wasn't able to observe the entire d/l process, only about 1/2 of it, as I went to bed. When I awoke, NONE of the files that were downloaded and NOT manually renamed were anywhere to be found. My download list was empty, the files showed up in the Files List tab (as if they'd been downloaded) but they weren't in the download folder, they weren't on the Failed Files list, nor were they in my Recycle Bin - they were just gone. Everything looked as if the files were downloaded, but physically, nothing was there OTHER than what was manually renamed when I double clicked the par2 file before going to bed. Whether the par2 repair process did this, I do not know.

I had loaded a completely different NZB to download after the renamed set and they all downloaded just fine. These weren't the renamed type though - they were straight nfo, rar, par, etc. It doesn't appear that this archive required repair because only one of the par2 files was downloaded.

Maybe a complete install of v6.50 rather than an upgrade is required - I don't know. I got frustrated with v6.50 and downgraded back to v6.42 this morning. After doing so, all of this "different behavior" stopped and reverted back to normal. I even reloaded the offending NZB with the obfuscated files and it downloaded and renamed everything without a single issue. Unfortunately, I don't have the time to sit here and babysit each NZB that I load and monitor the download and renaming process - it was automated in v6.42 and worked, so I'm fine with that.

I appreciate all of the help and input from everyone so far. I'm anxious to see the results of the NZB file I sent via email - see if they get the same behavior in v6.50 that I was getting.

[bobsmo]

Both kinds.....like I said, it happens for all NZB's....

[bobsmo]

Basically, for me, it's replacing the SxxEyy with the "PARID" mentioned above.

[Quade]

If finds the first DOT and then inserts the parid before the first dot (adding a dot). It doesn't change the filename beyond adding the number.

[no1butme]

If finds the first DOT and then inserts the parid before the first dot (adding a dot). It doesn't change the filename beyond adding the number.

And it did NOT do this in v6.42, correct? If not, no big deal. I was only pointing this out if it might affect the autopar repair or autorenaming.

Any explanation for why v6.42 will rename the obfuscated rar files as they are being downloaded versus not doing so in v6.50?

Did you or dexter receive the NZB file I sent? How did it behave?

Thanks for your patience, Quade.

[Quade]

And it did NOT do this in v6.42, correct? If not, no big deal. I was only pointing this out if it might affect the autopar repair or autorenaming.

Yes. Because of some new posting styles. Keep in mind, obfuscated posts aren't new and I've tested with plenty of them. I don't think the issue is the kind of post, so much as something unexpected with your settings. The PAR rename has nothing to do with your issue. What's not happening is the repair. These kinds of posts simply won't work if repair is disabled. Even though it seems to be enabled, it's acting like it isn't.

I'm looking at your NZB and it's in a common obfuscation format that I've tested many times.

If you start a download, let the small par download, then right click "Properties" you should see a window that looks something like quickpar showing the proper filenames for the files. If you don't. If you just see a listing of the message-id's, it means AutoPAR is disabled for some reason. When you start Newsbin look in the logging tab for information about the par DLL. I wonder if there's something wrong with the par DLL preventing repair. You should see the PAR block count count up too. Again, no count up = Autopar is disabled.

I'm letting it download. I'll report back later.

[no1butme]

Thanks again, Quade.

Other than the minor color change in the download progress bar, and the change in renaming the rar files as they downloaded, I didn't notice anything else. I did check my settings (options), and autopar was enabled (I've left it that way for years). Autopar did work on another download, and repaired the incomplete rars, but the files weren't obfuscated like those in the NZB file I sent you.

[Quade]

Sadly, it downloaded and unrared perfectly for me. I'm going to have to think about why it didn't work for you.

If you're interested, I'd try one set and actually watch the download in progress. See the PAR block counts and look at the window I mentioned above.

[nbwul62]

Up front: I am not sure if what I am writing below is the same as above.

After unRAR I have a folder with a number of .mkv and .srt files.
They all have weird names, i.e. a mixture of characters and digits. I could not figure out what was / is the first and or last file.
The folder contained a small .par2 file and a text file explaining that quickpar should be run on the .par2 file, in order to get
the original filenames back.
Whilst doing so, the option "Monitor" and "Repair" should be tagged.

I did so and indeed the original filenes were wonderfully restored.

Wonder whether this could also be done by Newsbin.

My autopar settings are are
=



=

Thanks!

[itimpi]

Newsbin does this automatically and has done for some time now.

[nbwul62]

Normally, yes, but eh .. not with me, in this case.
I installed QuickPar and did the quickpar thing manually, that is to say, all the .rar files and stuff were deleted.
The files were properly unpacked, I am talking about the renaming thing only.
=

[Quade]

PM me the subject and group and I'll check it out. It sounds like this set of files was inside the first layer of rars.

Might be something the scripting interface could be used for. If the script notes that there are pars inside the rars. It could run quickpar on it.

[nbwul62]

see pm

[Quade]

So, this is a another new format.

1 - You have a set of files with obscure filenames in the rars and pars.

obscure1.par2
obscure1.part01.rar
obscure1.part02.rar
...

2 - Newsbin downloads these files, repairs and does an unrar.

3 - What's inside the RARS though is another layer of obscure.
obscure2.par2
obscure2-1.mkv
obscure2-2.mkv
obscure2-3.mkv
....

4 - You can rename the second level obscure files with Quickpar to the real filename.

So, there's been no change in Newsbin's behavior. Instead the posters are trying to be clever. You can't see the inner PAR2 because apparently it's at the end of the set.

Seems like this would be a perfect use for the scripting interface. When you get the list of files from Newsbin, looks for an inner par2 and if you find it, run Quickpar to rename the inner files.

[nbwul62]

The steps above are correct, indeed.

Regretfully I have no knowledge of scripting, didn't even know it was there.
Was hoping it would have been some option that I had forgotten to tag.
Actually, if the textfile explaining how to proceed was not added, the entire download would have been useless as I would not have been able to figure out which parts came 1st, 2nd, 3rd... etc.
Maybe something to put on the wishlist..?

Anyway, sofar this issue has not been a regular occurrence, so I guess I leave it this way and manually run quickpar should this come up again.

Thanks again.
=

[Quade]

viewtopic.php?f=43&t=32035&hilit=scripting

Post about scripting. It's a way to post-process the files after they're unrared. For me, the clue was you has a par file after the unrar. That told me the PARs was inside the top level rar files.

[nbwul62]

Yes Quade, I already 'discovered' that post before posting my reply.
Tried to understand it, but figure one has to know a bit how things work. Really, no offence meant!
But I could not figure out how to proceed, so I 'dropped' the idea of trying to understand it. It would take too much time.
Should leave that to experts.

=