Why do files download out of order?

[flat_beer]

Why in version 6.50 Build 3126 do files download out of order? and for that matter why do the segments inside each file download in seemingly random order?

For example a download on 2014-01-06 from a.b.teevee using ssl-us.astraweb.com servers.
The file download order is (I reversed this from the Files List):

I.have.3CD0D84F.two.balls.107.hdtv-lol.par2
I.have.two.balls.107.hdtv-lol.r03
I.have.two.balls.107.hdtv-lol.rar
I.have.two.balls.107.hdtv-lol.r01
I.have.two.balls.107.hdtv-lol.r05
I.have.two.balls.107.hdtv-lol.r00
I.have.two.balls.107.hdtv-lol.r07
I.have.two.balls.107.hdtv-lol.r04
I.have.two.balls.107.hdtv-lol.r06
I.have.two.balls.107.hdtv-lol.r08
I.have.two.balls.107.hdtv-lol.r02
I.have.two.balls.107.hdtv-lol.r09
I.have.two.balls.107.hdtv-lol.r10
I.have.two.balls.107.hdtv-lol.r11
I.have.two.balls.107.hdtv-lol.r15
I.have.two.balls.107.hdtv-lol.r13
I.have.two.balls.107.hdtv-lol.r21
I.have.two.balls.107.hdtv-lol.r16
I.have.two.balls.107.hdtv-lol.r14
I.have.two.balls.107.hdtv-lol.r17
I.have.two.balls.107.hdtv-lol.r12
I.have.two.balls.107.hdtv-lol.r19
two files from the next set in the download queue
I.have.two.balls.107.hdtv-lol.r18
one file for the next set in the download queue
I.have.two.balls.107.hdtv-lol.r20


If the files would download in order then the cache wouldn't fill up so much. I'm running on a 32-bit OS and memory is always tight.

Something related: It seems to start pre-downloading alot of stuff (50 or more items). I'm on a slow (about 2.1Mb) connection and if I'm downloading something and want to get headers from another group it doesn't even start downloading them till 5+ min after I tell it to download headers.

[Quade]

1 - The goal is to keep all the connections 100% busy. That means there's a lookahead when it's just downloading the first chunk of the set to get the filename, rar encryption status and other facts about the file. This lookahead is related to number of connections which leads to...

It seems to start pre-downloading alot of stuff (50 or more items). I'm on a slow (about 2.1Mb) connection and if I'm downloading something and want to get headers from another group it doesn't even start downloading them till 5+ min after I tell it to download headers.

2 - I'm just guessing that you're running a bunch of connections from the symptoms. If so, reduce your connections to 4-6 say. It's counter-productive to run a bunch of connections when you have a slow connection to the net. If you have a 200 Mbps connection, 50 connections might make sense but, I have 100 here and I get full speed with 20 connections. It'll solve the lookahead issue too.

3 - If you always want to be able to start a header download, you can add a copy of your server, set it to 6 connections, and make it a slave (no header downloads), then take your current server, reduce the connections to 2, set it to priority 5 and leave it a master. Then when you go to download headers, you'll always have 2 connections dedicated for header downloads. What this does is that the slave server will be used for most of the file downloads and the header server will always be available for header downloads. We both know it's the same server and the same connection pool.

4 - The file download order is "whatever downloads first". I think you'll find fewer connections = more likely to download in order. This goes back to #1, keep the connections busy. The whole ram issue is kind moot if you're running a bunch of connection. That'll dominate RAM usage.


Don't post what you're downloading. There aren't many rules in this forum but, that's one of them.

[flat_beer]

Hum. My server list:

Server...................M/S....Headers....# of connections.Priority
Astraweb ssl-us.........M........Y............2...................1
Astraweb ssl-us.........S........N............3...................1
Astraweb ssl-eu.........S........N............2...................2
Unknown free server....S........N............2...................3
teranews free...........S........N............1...................4

(Had to put dots in to make it look like a table.)

So I have 5 connections. Your saying that I should make the one that gets headers lower priority. Should it be the lowest priority?

Don't post what you're downloading. There aren't many rules in this forum but, that's one of them.

Oops. I thought it might be helpful for someone diagnosing this problem to know exactly where I was seeing it...

[Quade]

You shouldn't see much lookahead then. No more than 5 connections worth anyway. Looked to me like you were running a bunch more from the symptoms. I might see if I can simulate a slow connection somehow.

I'd set the header only server to priority 9 so, it's only used for headers. As you have it configured here, it'll be used for file downloads to so, header downloads will still stall behind the file downloads.

[DThor]

Curious if there's anything interesting in the logging tab - any timeout messages or post not found?

DT

[flat_beer]

No errors in the log.

My account is one of the astraweb accounts that is limited at 10 mbit since my net connection is only about 2.2mbit.

I just checked using a program that monitors the net... the 3 connections running now are getting some data. They are bouncing around some with the lowest speed about 40KB/s and the high 110KB/s. With the total about 239KB/s.

I'm running under Win XP Pro 32-bit... if that matters.