Sonarr - Completed Downloads Handling

[Alamei]

I apologize if I'm missing something obvious, but is there any way to get Sonarr to integrate with the newer versions of Newsbin for the automated "Completed Download Handling" feature (where the client passes the file info back to Sonarr for moving/renaming), or is the old DroneFactory scanning method the only current option?

[Quade]

viewtopic.php?f=43&t=38181&hilit=Sonaar

I'd suggest not setting a username and password but setting the same arbitrary API key on both ends.

Basically you setup Sonaar as if Newsbin was Sab. Newsbin emulates the Sab interface.

[Alamei]

That's why I figured I must be missing something about the setup process. I had already configured Sonarr for accessing Newsbin based on the instructions that thread, and everything has been working perfectly as far as downloading goes, but it never seems to get around to the "report back" stage, so the files just sit in the download folder indefinitely unless I configure a Drone Factory to run against that folder.

Maybe I'm misunderstanding which port Newsbin wants in the "Post Processing" options? I have that set to the address of the local Sonarr installation; is that correct?

[Quade]

Are the NZB's being fed from Sonaar to Newsbin?

If the NZB's make it over, the port should be correct. You also ought to see Sonaar reporting download progress too.

Are the unrared files ending up in folders named for the NZB Files?

[Alamei]

The NZBs are definitely being fed into Newsbin, so it sounds like that is running properly.

I'll temporarily disable DroneFactory and report back on the other questions when the next downloads run.

[Quade]

You can manually feed NZB's from Sonaar. That's how I test. Might be worth watching some downloads. Sonaar should be showing status of the downloads.

I'll temporarily disable DroneFactory and report back on the other questions when the next downloads run.

Are we talking about different things? I know Sonaar once had a different name.

[Alamei]

You can manually feed NZB's from Sonaar.

I didn't see any options for manually pushing an NZB from Sonarr. How did you accomplish this for testing?

Are we talking about different things? I know Sonaar once had a different name.

DroneFactory is just the name of the old download scraper tool built into Sonarr (which itself was previously called NzbDrone) that it used to scan the download folder to pick up completed downloads before the new "Completed Downloads Handling" feature was implemented: https://github.com/Sonarr/Sonarr/wiki/Completed-Download-Handling

[Quade]

You can go into one of your configured series and use the right most button (I think I'm trying to remember) to ask it to manually re-download the NZB to Newsbin.

[Alamei]

I tested forcing an NZB and watched what happened in both Sonarr and Newsbin. Sonarr simply puts a record in the History tab of the Activity page of the download being successfully sent to the client. Newsbin completes the download successfully and extracts it to a sub-folder with the NZB name within the folder that I specified for Sonarr downloads. After that, nothing at all happens. The file just sits there until I manually move it or enable DroneFactory in Sonarr settings.

It seems like Sonarr is just handing the NZB off to Newsbin, then forgetting about it. When you said I should be able to see the progress in Sonarr, what sort of progress is it supposed to display?

EDIT: I just figured out the issue. Sonarr said in its help documentation that the "Category" option on the download client was optional, so I left it blank, assuming that feature wasn't implemented in Newsbin. It turns out, that option has to be set to the default "tv" for things to communicate properly.

[Alamei]

Now that I've got Newsbin correctly communicating with Sonarr, I'm running into a new issue. When Sonarr adds an episode to Newsbin, Newsbin reports each of the various items that it's trying to download as part of the release back to Sonarr: Usually, the main video file, the sample video file, and a copy of the NZB file. Sonarr doesn't have a way of categorizing the downloads, so it ends up treating each of them as a separate attempt to download the episode.

When the files are complete and Sonarr receives a notification from Newsbin that they're ready for import, Sonarr correctly identifies the main video and imports it, removing one of the items from Sonarr's queue. It also seems to correctly ignore the NZB-related queue item (perhaps because it has a reported file size of 0), but it leaves the item that was associated with the sample download in the queue, which then makes another attempt to import the episode. Since the file was moved with the first import process, the remaining queue item can't find any file to import, and instead sits in Sonarr's queue with an error saying "No files found are eligible for import in <NZBPath>" until it's manually removed.

I tried blocking the download of samples, but even if a Global Subject Reject filter for samples is in place, it seems to be overridden for downloads passed by Sonarr. Is there any other way to stop Newsbin from downloading samples passed by Sonarr? If not, is there some way to stop it from reporting them back as a separate queue item so that Sonarr's queue doesn't need to be manually cleaned out from time to time?

[Quade]

Look in the the NZB options. You can assign a filter profile to imported NZB Files. That's the filter you'll want to use to clear out the files you don't want.

Nobody's actually reported this before.

Subject Filters:

[.]SFV]
[.]NZB
SAMPLE


You might want to set a similiar filter for the post unrar filtering.

File Filters:

[.]SFV]
[.]NZB
SAMPLE

You might be able to use a single filter for both because NZB Filters are subject filters and "Filename filters" are used for "Post UnRAR".

I'm kinda surprised Sonaar has a problem with these files because getting all the extra files with the NZB Is pretty common.

[Alamei]

I'm kinda surprised Sonaar has a problem with these files because getting all the extra files with the NZB Is pretty common.

I was thinking the same thing, but I've never used any of the other download clients, so I figured maybe they didn't have the granularity of Newsbin and reported all the files as a single download, instead of showing the various "parts" as separate entities.

After some preliminary testing, it looks like the NZB filter is going to do the trick. I'll monitor it for the next week or so and see if anything slips through and trips things up.

Thanks for all the help in sorting out the issue. I know it's troublesome troubleshooting communications between two systems when you only have control over the code in one of them.

[Quade]

Thanks for all the help in sorting out the issue. I know it's troublesome troubleshooting communications between two systems when you only have control over the code in one of them.

Thank you too. I want people to use it so the more experience there is with using it, the more the knowledge base increases. Like this thing with the extra files. Makes me wonder if I should build a filter in so only non-sample media files are added to the history list.

[cat_man]

I had tested Sonarr integration with NB before but then went back to Blackhole method. I have started testing again (using apikey and port 8000 on both) with 6.70B6 and am experiencing the following:

1. Sonarr passes NZB to NB
2. NB dowloads rars with par2 paused
3. Using external par2 check, files are 100% complete. After a while NB starts downloading the par2 files one at a time, pausing in between.
4. NB reports back to Sonarr that file is 100%
5. The files stay in NB download queue showing status [PAR 204(20) of 204] D:11 DL:0 Retry:0 N:0 , they are not assembled (unrared) in the download directory.
6. Since nothing happens for a while, I try to restart NB
7 . On restart NB has nothing in the download queue and rars are still listed in the file list. Rars and pars are still in the download folder. Sonarr has removed the file from the Activity queue
8. If I manually put an nzb in NB nzb watch folder it downloads properly, assembles the rars and removes it from the download queue.

[Quade]

Don't know. I'd look in the logging tab for any related messages. Once the NZB gets into the download list, it's not different than any other NZB.

If I had to guess, I'd guess the unrar folder isn't set or is incorrect in the Remote NZB Options.

[cat_man]

I agree it's confusing. Nothing I can see in the logging folder with any clues. The Main download folder, the download folder for downloads for NZB files and The remote NZB interface unrar folder are all set to the same folder as I have used for Blackhole mode. I have full access to these folders. I am running Win 10 fast ring latest version if that may have some effect. Guess it works ok for you?

[kirm]

I am running Win 10 fast ring latest version if that may have some effect

It could very well be. Running in fast ring on an older laptop and in VMWare. Have noticed some programs do not like to play correctly but I am not using anything that you are. It's not stable OS so that could be issue.

[Quade]

Guess it works ok for you?

Me and a number of other people. I'd make the Remote NZB unrar folder some other folder. I'd make sure it's got "$(NZBFILE)" on the unrar path too. Sonaar wants one folder per download.

[cat_man]

Went to my production server (Win 10) and changed it from Blackhole to NZB Remote. I did add the $(NZBFile) to the Remote NZB unrar folder (a year ago you said to use a naked path and you appended the $(NZBFile)). Turned on debug logging and exact same sequence of events. I see one error in file creation, maybe you will understand. BTW, with Blackhole I use the same download folder for all videos and Sonarr has no trouble with that.

[10:38:47] DEBUG CheckFilename: Using Path:D:\NZBdrone\Complete\
[10:38:47] DEBUG Spooler: Using Path:D:\NZBdrone\Complete\
[10:38:47] DEBUG Par File added ummy.File.Name.s02e09.Series.Name.hdtv.x264-w4f.2348D888.vol15+2.par2
[10:38:47] DEBUG CheckFilename: Using Path:D:\NZBdrone\Complete\Dummy.File.Name.s02e09.Series.Name.hdtv.x264-w4f.2348D888.vol15+2.par2
[10:38:47] DEBUG AutoPARPlugin: Good files = needed files and goodblocks = needed blocks[274504]-[FULL]-[#a.b.teevee@EFNet]-[ Dummy.File.Name.S02E09.Series.Name.HDTV.x264-W4F ]-[09/22] - "Dummy.File.Name.s02e09.Series.Name.hdtv.x264-w4f.part1.rar" yEnc
[10:38:47] DEBUG Combining BAD Complete
[10:38:47] DEBUG First Filename:D:\NZBdrone\Complete\Dummy.File.Name.s02e09.Series.Name.hdtv.x264-w4f.part1.rar
[10:38:47] DEBUG UnRAR Temp: D:\NZBdrone\Complete\Dummy.File.Name.S02E09.Series.Name.HDTV.x264-W4F.nzb\ UnRAR Filename: D:\NZBdrone\Complete\Dummy.File.Name.s02e09.Series.Name.hdtv.x264-w4f.part1.rar
[10:38:47] DEBUG CAssembleRAR UnRARPath #1:D:\NZBdrone\Complete\Dummy.File.Name.S02E09.Series.Name.HDTV.x264-W4F.nzb\
[10:38:47] DEBUG CAssembleRAR UnRARPath #2:D:\NZBdrone\Complete\Dummy.File.Name.S02E09.Series.Name.HDTV.x264-W4F.nzb\
[10:38:47] DEBUG UnRAR Error:File create error
[10:38:50] DEBUG NZB Autoload Watch: D:\NZBdrone\Incomplete Folder: D:\NZBdrone\Incomplete\
[10:38:50] DEBUG NZB Scan Folder: D:\NZBdrone\Incomplete\
[10:38:50] DEBUG NZB Scan Folder is empty: D:\NZBdrone\Incomplete\
[10:38:50] DEBUG NZB Autoload No files to process
[10:38:53] DEBUG Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
[10:38:53] DEBUG User-Agent: Sonarr/2.0.0.3953 (RestSharp/105.0.1.0; Windows/6.2)
[10:38:53] DEBUG Host: localhost:8000
[10:38:53] DEBUG Accept-Encoding: gzip, deflate
[10:38:53] DEBUG SbServer: Read Request GET /api?mode=queue&start=0&limit=0&apikey=newsbin&output=json HTTP/1.1

[Quade]

10:38:47] DEBUG UnRAR Error:File create error

Yeah, this is the core problem.

I downloaded the exact set you downloaded using an NZB I created and it downloaded and unrared. I used a tool that feeds the same interface as Sonaar.
That error is telling you it's not happy about the unrar path I believe.

My log looks like yours except my unrar succeeded.

Looks to me like you're downloading and unraring to the same path. I think that's the entire problem. In fact the problem is that the NZB File that was downloaded along with the files is the same name as the unrar folder Newsbin is trying to make so, the path creation is failing.

I think you need to either download to another folder or make the unrar happen to a different folder than the download folder.


D:\Download\
D:\Sonaar\UnRAR\ ...or something.

[cat_man]

Thanks, that indeed was the problem. But now that it unrared successfully and NB moved it to the download folder for NZB files, Sonarr has not picked it up. Sonarr logs end with Report sent to NB. I know I'm doing something wrong. My settings are
Main Download folder c:\nzbdrone\complete
Download folder from nzb files c:\nzbdrone\complete
unrar folder c:\nzbdrone\$(NZBFILE)

Remote NZB unrar folder c:\nzbdrone\$(NZBFILE)

[Quade]

You shouldn't have to do anything with the settings but if you look in the Sonaar settings for Newsbin. Check the "Advanced Setting" check box.

What do you have the "Completed" and "Failed" download setting set to?

Make sure "Drone Factory Options" aren't set to the unrar path where the files end up. Ideally it shouldn't be set at all. If you're going to set it to watch a folder, make sure it's not a folder Newsbin is using to unrar or download too.

[cat_man]

All settings are set to "Yes". Drone factory is blank, Drone factory interval is 1 and no remote path mappings. Sorry to be such a pain, I had it working in the past. Hopefully this will all help someone else too.

[Quade]

I'd try the new 6.70B8. I might delete the "DownloadsV2.db3" file too from the data folder.

Sonaar will only pick up files that it initiated the download from so if you're feeding NZB's from some other source, I don't think Sonaar will pick them up. You could check the Sonaar logs too.

[cat_man]

Using 6.70B8. Seems like everything is working. Thanks again for the help. I do the testing for fun and the experience to help others. In reality I much prefer NB over Sabnzbd for utility and flexibility, AND I really don't see a lot of benefits to the close Sabnzbd (alias NB) and Sonarr integration. Blackhole integration has worked perfectly for me and I will continue to use it. And I'll be tempted back every now and again to do some integration testing. Appreciate all your work and the best usenet downloader ever!!

[Quade]

Blackhole integration has worked perfectly for me and I will continue to use it. And I'll be tempted back every now and again to do some integration testing. Appreciate all your work and the best usenet downloader ever!!

Blackhole has what I consider to be a fatal flaw. Sonaar can't tell when files fail so, it's not going to pull different NZB's to download the files. On the other hand, integration mode allows Sonaar to watch the download and if it fails, download the same thing using a different NZB.

[cat_man]

You are correct, but of the 59 shows I download, its only occasionally I have to manually intervene for a missed or incorrect download. The plus is I can see exactly what is going on, the list of nzb's loaded (you might want to consider putting those in the "loaded" folder), standalone control of both and other benefits.

[cat_man]

[10:38:47] DEBUG UnRAR Error:File create error

This is a serious error and should show up under normal logging and not have to use debug. Could you also show the file name as part of the error message. When this happens should the download be sent to Fail? As it was, it kept downloading par files and then sat in the download queue until a restart. How about adding a unique ID (timestamp?) to the unrar folder name to prevent duplicate names(Isn't that what you do under normal unraring when no folder is set)? After Sonarr handles it, it will be deleted anyway. Course now that I know the problem it can be avoided. Thanks again for the help.

[Quade]

I changed it to be "Error" and always pop.

I unrar to a temp folder only if the unrar and download folder are the same. This is specifically a Sonaar issue because the normal $(NZBFILE) doesn't include the full NZB Filename. I could enforce that the download an unrar folders need to be different. I could delete or rename any file that gets in the way of the unrar. There are a number of things I could consider doing.