newsbin.com

Was just working on the par2dll.dll. It already does multi-core but, I've updated the MD5 routines (which it uses heavily) to use Intel "Performance Primitives" which means it'll take advantage of all the special instructions your CPU might have. In my testing, MD5 calculations are 25% faster than before.

I'd had an idea for awhile now for a way to speed up repair but, the code in the par2dll is pretty damn complicated so, I could never figure out how to hook in my changes. Well, I finally figured it out. What you should see when you use it is that there's far less disk IO when doing a repair. I think for some of those monster 10+ gig repairs some of you are doing, it should make a large difference.

When I release 6, I'll punch up the source for par2dll.dll. It's derived from an open source library so, we publish all the code changes I make to it. Basically, I just work around the edges, adding interface code to it so, Newsbin can use it when it needs it. For perfect files, Newsbin doesn't even use the DLL. All file checking and rar tracking is done internally using code I wrote. It's only when a repair is needed that I make use of the DLL.

I'm not sure what your code base (if that's the right term) is, but I've been experimenting with phpar2 and par2cmdline TBB (64-bit version)

phpar is currently the fastest to verify, and I think they're about equal in repair times.

If you haven't already looked at their code optimisation, it might be worth a look:

http://paulhoule.com/phpar2/index.php
http://www.chuchusoft.com/par2_tbb/

Looking at phpar2 is what gave me the idea to update the MD5 generator. Basically that's what's he's done but, using a different method. Par2dll.dll uses TBB too. Since Newsbin is commercial, I bought my own copy of both TBB and Intel performance primitives (IPP). In 6, verification is done before the file is closed, after it's initially written, in order to keep virus scanners from getting in the way.

Like I said though, most of the time the par2dll.dll isn't even used. I seldom need to repair anything. I have a test suite of broken files I generated myself because, using Giganews, I get good files, 99.9% of the time.

Quade wrote:Like I said though, most of the time the par2dll.dll isn't even used. I seldom need to repair anything. I have a test suite of broken files I generated myself because, using Giganews, I get good files, 99.9% of the time.

I used to be with GN, but now I'm a cheapskate with AstraWeb, so I get to test out par2 repair pretty frequently...

im with astraweb and dont get many incompletes

richy99 wrote:im with astraweb and dont get many incompletes

Try grabbing 500GB a week (Blu-ray) and see if you don't get any...