Page 1 of 1

Endless header leeching

PostPosted: Tue Jul 28, 2015 8:15 am
by wiggins09
Just added a new server to 6.60 b4163 [due to Astraweb having major completion issues recently, (anyone know why AW is having trouble?)]
I have allowed the new server to download headers, thinking this should update any damaged or missing headers from AW.
Having had a couple of days to 'catch up' I cannot understand why the new server is still hanging on to header downloading when the progress values are hitting numbers way above the new posts value
eg 2,148,810,000/197,000

I had assumed that the 2 values should simply match & existing headers be updated.
Given the time, bandwidth & these numbers on the same groups for several days I am wondering if there is a problem.
AW is doing what I expect it to eg 100/100; done.
Download age is set to 10
storage age to 50

Re: Endless header leeching

PostPosted: Tue Jul 28, 2015 8:35 am
by Quade
Sounds like something went wrong with the header download. I might delete the header download for that group and try again.

Re: Endless header leeching

PostPosted: Wed Jul 29, 2015 7:20 am
by wiggins09
Thanks Quade, will try it. I should have thought of that before bothering you :)

Re: Endless header leeching

PostPosted: Wed Jul 29, 2015 8:05 am
by Quade
It's one of those things that I'd like to understand but probably need to chalk it up as an internet mystery. If it's reproducible, I'd certainly like to know how.

Re: Endless header leeching

PostPosted: Thu Jul 30, 2015 9:19 am
by wiggins09
Well I deleted the headers folders & started over
Guess what? I'm back at 2,148,810,000/197,000 & counting :roll:

I am gonna turn off the header d/l for this 2nd server for now. Does this impact leeching if the only headers are from incompletes or will NBP 'discover' the needed blocks on server #2?

Re: Endless header leeching

PostPosted: Thu Jul 30, 2015 11:43 am
by Quade
If you're not seeing black blocks in the downloads (meaning missing headers) you don't need headers from the second server at all.

I might just turn off headers on the second server and save the bandwidth.

Re: Endless header leeching

PostPosted: Fri Jul 31, 2015 7:11 am
by wiggins09
Thanks for the info Quade.
I have turned it off for now. Will see how it goes.

*update*
Tried it & got a lot of empty blocks. Activated the 2nd server re-ran header leech & they went away.
Interesting but looks like I may be stuck with excess headers if I don't manually cancel the run for this server when it hits its daily totals...

Anything I can do to try & identify why it doesn't halt?

Re: Endless header leeching

PostPosted: Fri Jul 31, 2015 8:59 am
by Quade
Tried it & got a lot of empty blocks. Activated the 2nd server re-ran header leech & they went away.


Are they turning red or are they black?

Re: Endless header leeching

PostPosted: Fri Jul 31, 2015 6:21 pm
by wiggins09
They leave black blocks & the leech then stalls (if I do not manually force pars when my maths says there might just be enough). I then updated the group with the other server & they disappear once I reselect the set for download in the header pane.

Estimate the average to be about 10% per file set black blocks. Having assigned different colors to the servers it is immediately apparent where the missing parts are coming from. So thanks for making that possible ;)

Re: Endless header leeching

PostPosted: Fri Jul 31, 2015 7:14 pm
by Quade
Yeah black means missing headers. You might download headers ONLY from the second server then.

Re: Endless header leeching

PostPosted: Sat Aug 01, 2015 10:03 am
by wiggins09
yeah that is probably the best option, just gotta watch & stop it before it hoovers up 000,000,000s of old headers again :P