Restrict certain files to dl only from specific servers

[Heseltine]

Hi everyone,
I almost always have a number of files (around 10% of all files) in my dl queue that only exist on some of my newsfeeds, but have been deleted on others (430 no such article). The result of this is that my dl rate goes down from 70Mbit/s to 3-7MBit/s because onyl the connections that have the file are actually doing anything. All the other ones are retried for every single post (and of course always come up empty), but don't move on to the next file. In short: as long as the file is downloaded, I drop from many active connections to just a few, with all the other ones needlessly trying articles that aren't there.

Thus, I'd like to restrict certain downloads to only those servers that actually have a chance of retrieving parts of the file. I don't need an automatic detection of futile download tries, I can see myself when the dl rate is dropping rapidly and would like so reconfigure the download for the file so that only the servers I select take part in downloading the file, so that the rest can move on to the next item in the download queue.

Is this - or anything with a similar effect - possible, and if so, what must I do to get there?

Thanks in advance, Michael.

[itimpi]

Why are the files not on the other servers? If it is because of the age of the posts have you tried setting the Typical Retention value for each server?

[DThor]

The design is 'gimme gimme' with as little micromanaging as possible, but there are priority settings for the servers and you can tweak the retry number too. I must admit I tend to get most chunks from my primary and occasionally when there are missing chunks my fill server kicks in and it's not really noticeable. It sounds like you have a specific group that is missing from a server? Otherwise, I don't know how you can predict a given post will or won't be there. Since it pays to never assume anything, you know you can specify a server to be a 'fill' server, right?

DT

[Heseltine]

First of all: sorry for the delay, I was a little busy for the last fortnight.

To answer a your questions: no, the posts in question are not that old. It's all television episodes, none older than two weeks. The problem is that some episodes are - for whatever reason - only available on AstraWeb and/or ssl-news, both servers to which my connections only have a bandwidth in the 7-10 MBit/s range. These downloads slow down the whole queue, even though there are other episode further down the queue which could be downloaded from EasyNews or SuperNews, to which I get around six times the dl bandwidth. Until the episode is completely downloaded, every posting of every part is checked on all servers - to no avail on EasyNews and SuperNews - and nothing at all is downloaded from those two.

I'm looking for a way around that annoying waste of time; if the dl process moved on to the next show for all servers but AstraWeb and/or ssl-news, the dl time would decrease significantly.

Greetings, Michael.

[DThor]

Hmm, I understand your point. Do you have priority or fill server status set up, and if so, how? If you set astra as a fill server it should keep pressing ahead on the faster servers and only use astra when the other servers don't have a chunk.

I'm not speaking for Quade, but the entire mechanism for "getting you stuff" is wired up using connections, priority and fill status of servers, it might be troublesome having a special assign mode where you assign downloads to specific servers, since it's typically an unknown whether or not a server has a specific chunk or not. More often than not, if you have a server that regularly doesn't have posts you want, you leave it out of the mix. I understand your point about speed, though.

Have you looked into your speed issue with astra? That would be a far better solution than anything else. Have you tried alternate ports or contacted them about it?

DT

[Heseltine]

Hi,
I have tried every combination of server name/port that astraweb offers. The connections just isn't as good as to EasyNews or SuperNews I never get more than 9 MBit/s from astraweb, where EasyNews and SuperNews can't effectively max out my internet connection (around 65MBit/s). But AstraWeb (and to a certain extent ssl-news which isn't that fast either) regularly keep files (mostly TV stuff) that the other services (had to) remove(d).

Leaving out servers isn't an option either, as that would slow down the fast downloads and wouldn't help on the slow ones. Typically, I download around 30 to 50 items from the same bunch of groups. Of those, about 10% are nuked on EasyNews / SuperNews (430 no article). These 10% (3 to 5 files) account for more than half of my dl time, as the connections to Easyews and SuperNews effectively do nothing while these files are being dl'd.

I don't expect any kind of automatics (I know that that would be difficult, as NB would have to keep track of failed downloads/430s and make a decision whether continuing to dl from that server was worthwhile), it would just be nice to manually exclude certain servers from certain files. Most of the time, I see when the dl rate drops down when it happens; it would be nice to have a context-menu option to exclude certain servers to that they can move on to the next files.

Cheers, Michael.