Newsbin generates from pars instead of downloading

This is the place to help test and discuss Version 6 Beta releases.

Newsbin generates from pars instead of downloading

Postby RayMark » Mon Nov 10, 2014 3:47 pm

Yes, Newsbin did exactly that! It spent more than half an hour generating two files (and using 15% of quad-core CPU) instead of downloading them in less than 2 minutes.

It happens when there is a set of separate posts and pars posted separately for all the posts together, lets say:

+Title.S03.COMPLETE.DL - Title.S03.par2
Title.S03.COMPLETE.DL - Title.S03E01.mkv
Title.S03.COMPLETE.DL - Title.S03E02.mkv
Title.S03.COMPLETE.DL - Title.S03E03.mkv
......
Title.S03.COMPLETE.DL - Title.S03E21.mkv
Title.S03.COMPLETE.DL - Title.S03E22.mkv
Title.S03.COMPLETE.DL - Title.S03E23.mkv

As you see, only the par2 post is expandable (+) - others are just single files.

A couple of those mkv files were incomplete.
So, after the S03E21 was downloaded, there was enough par2 blocks to fix the problems in the incomplete files and to create S03E22 and S03E23 files from scratch.
So NewsBin did exactly that instead of downloading S03E22 and S03E23 first.
So instead of fast download of two files and 3-block-fix, it proceeded to generate 65 blocks worth of files from par2s.
And the total size of all the files was about 40 GB. So it took a considerable time, more than 30 minutes and 15% of quad-core CPU.
After S03E22 and S03E23 were generated, only then they were also downloaded, so I got also -(0001).mkv files, thank you very much.

I'm not sure, perhaps I caused all this myself, because after the par2 header file was downloaded I forced the download of par2 blocks as well.
So not sure what happens if those par2 blocks are available but not yet downloaded.
Perhaps Newsbin thinks (very wrongly) that it is faster to use the already downloaded blocks rather than to download the remaining two files.

But I suspect that NewsBin does not even look what is next in the download queue once it has enough blocks to repair all the files covered by the par2 set.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07

Re: Newsbin generates from pars instead of downloading

Postby Quade » Mon Nov 10, 2014 6:42 pm

In 6.54, In the case of split pars, where the PAR isn't in the same set as the download, you can trigger an early repair if you have a bunch of par blocks already downloaded. It's state driven so, if it downloads and notices it has a combination of blocks and pars enough to repair the files, it initiated the repair.

6.60 works differently. This should happen less often. In fact, autopar is completely re-done for 6.60. It works much better.
User avatar
Quade
Eternal n00b
Eternal n00b
 
Posts: 44981
Joined: Sat May 19, 2001 12:41 am
Location: Virginia, US

Registered Newsbin User since: 10/24/97

Re: Newsbin generates from pars instead of downloading

Postby RayMark » Tue Nov 11, 2014 6:12 pm

Never happened to me with 6.60 yet :)

So I guess I triggered it myself by unpausing par2 blocks.
RayMark
Seasoned User
Seasoned User
 
Posts: 468
Joined: Sat Jul 21, 2007 10:40 pm

Registered Newsbin User since: 07/21/07


Return to Newsbin Version 6 Beta Support

Who is online

Users browsing this forum: No registered users and 4 guests