Stops downloading and open files.

[marcobier]

I switched from ADSL to fibber-optic.
This means my download speed is 5x faster.
But now newsbin stops downloading from time to time.
It gets stuck with updating the groups.
I solved this by doing manual updates from the groups.
But ist annoying that the automatic updating is not working any more.
I went back to version 6.34 but the problem still exits.

Another annoying thing is that over time files stay open.
These are the temporary files at the top of a directory.
I have to stop newsbin to delete these files.

[Quade]

How fast is your connection?

But now newsbin stops downloading from time to time. It gets stuck with updating the groups.

Stuck how?

[marcobier]

How fast is your connection?

But now newsbin stops downloading from time to time. It gets stuck with updating the groups.

Stuck how?

My speed is 80 Mbps.

The downloading stops sometimes when trying to update the groups.
Mostly within a day.
On top of the download list there is a list:
Downloading headers: server - a.b.group1
Downloading headers: server - a.b.group2
Downloading headers: server - a.b.group3
Downloading headers: server - a.b.group4
Downloading headers: server - a.b.group5

And nothing is downloading anymore.

[DThor]

Nothing interesting in the logging tab? It does sound like your connections hung, which could be any number of things. Which usenet service are you using?

DT

[marcobier]

Nothing interesting in the logging tab? It does sound like your connections hung, which could be any number of things. Which usenet service are you using?

DT

I cant remembering something interesting in the logging tab.
I tryd to look in the log file. But it was more than 2GB.
The service i use is http://www.eweka.nl/

I am trying to recreate the problem.
So i can look in the logging tab or log file.
I deleted the old one.
I had to upgrade to the latest 6.40 version to get the automatic updating of the headers working again.
Now i have to wait.

I use the dutch lanquage setting. But the translated texts are to long.
They overlap the controlls in the settings forms.
Minor issue.

[Quade]

I tryd to look in the log file. But it was more than 2GB.

I wonder if this could be related? I'd delete it and see if anything changes. I'm sort of at a loss right now to explain this so, I'm hoping some other people chime in.

[marcobier]

I tryd to look in the log file. But it was more than 2GB.

I wonder if this could be related? I'd delete it and see if anything changes. I'm sort of at a loss right now to explain this so, I'm hoping some other people chime in.

Newsbin did it again. Stopped downloading.


Last entry log tab:
[15:26:48] HIGH - HFC:Duplicate Detector Match: xxxxxxxxxxxxxxxxxxxx.part1.rar

Last entrys logfile:
[15:26:45] DEBUG - More than 1 par and all the current files have been processed: xxxxxxxxxxxxxxxxxxxxx
[15:26:45] DEBUG - Current files have been processed - ParChunks > nMissing chunks, assemble and lets pars take over: xxxxxxxxxxxxxxxxxxxxx
[15:26:45] DEBUG - More than 1 par and all the current files have been processed: xxxxxxxxxxxxxxxxxxxxx
[15:26:45] DEBUG - Current files have been processed - ParChunks > nMissing chunks, assemble and lets pars take over: xxxxxxxxxxxxxxxxxxxxx

There are 40+ downloads with green circles with a white arrow. In the download list.

I had to restart Newsbin again.

[Quade]

Look at the status bar. When it does this, is the "Cache" line at 0/100? If yes, there was another report of something similar.

[marcobier]

Look at the status bar. When it does this, is the "Cache" line at 0/100? If yes, there was another report of something similar.

Yes the line has Cache:0/100.

Newsbin stopped twice now in 2 hours.

[Quade]

So basically the cache is leaking away.

FileIOThread - internal crash

You don't see anything like this in the logfile do you? If you have multiple disks, it might be interesting to try downloading to another disk.

[marcobier]

So basically the cache is leaking away.

FileIOThread - internal crash

You don't see anything like this in the logfile do you? If you have multiple disks, it might be interesting to try downloading to another disk.

I think I know now what is going on.
Its a bug.
I sorted the download list from small to large.
Newsbin stopped 3 more times processing the small downloads.
Newsbin keeps running now without auto updating the headers for the group.

I think Newsbin allocates separate tasks for downloading.
And there is a maximum of the number of tasks.
If a task is done it can be assigned a new download task.
But there is something wrong if the maximum number of tasks is reached.
Then newsbin stops downloading.
Or something is wrong with a timely release of a task for a new download.
This could explain why the problem occurs now with a faster download speed.

I hope the developer fix this problem.
I could write a program that kills and restarts Newsbin if the log file is not growing for x minutes.
But its better that the developer builds in a check.

[Quade]

Yes the line has Cache:0/100.

Is the problem.

- There's a fixed number of threads.
- One thread per connection.
- Each thread owns a connection to the server.
- Each connection gets a cache item. Fills it with data and then hands it off to another thread that handles the data.

When the cache goes to zero, that means no more work can be done because there's no place to put it. So, the question is, why does the cache go and stay at zero.

That's why I was looking for log errors. That's why I suggested trying another disk.

I download at 100 Mbps from my news servers and haven't seen it leak cache items.
I download at 400 Mbps to my test servers and again, see not cache leaks.

So, I don't believe speed is the issue.

- I suggest using a different download drive.
- I suggest starting Newsbin, starting a download and not how many threads it's got in task manager. Then when it stalls re-check the thread count.
- I suggest cutting connections down to something like 10 and see if the symptoms change.
- I might uninstall any virus scanners/firewalls, reboot then try some more downloads and see if it still stalls.

If I could leak even one cache item, I could find and fix this but, downloading 100 gigs off my test server, it has yet to leak one. That's why I'm, looking for something special.

[marcobier]

Yes the line has Cache:0/100.

Is the problem.

- There's a fixed number of threads.
- One thread per connection.
- Each thread owns a connection to the server.
- Each connection gets a cache item. Fills it with data and then hands it off to another thread that handles the data.

When the cache goes to zero, that means no more work can be done because there's no place to put it. So, the question is, why does the cache go and stay at zero.

That's why I was looking for log errors. That's why I suggested trying another disk.

I download at 100 Mbps from my news servers and haven't seen it leak cache items.
I download at 400 Mbps to my test servers and again, see not cache leaks.

So, I don't believe speed is the issue.

- I suggest using a different download drive.
- I suggest starting Newsbin, starting a download and not how many threads it's got in task manager. Then when it stalls re-check the thread count.
- I suggest cutting connections down to something like 10 and see if the symptoms change.
- I might uninstall any virus scanners/firewalls, reboot then try some more downloads and see if it still stalls.

If I could leak even one cache item, I could find and fix this but, downloading 100 gigs off my test server, it has yet to leak one. That's why I'm, looking for something special.

2 days ago it stopped downloading after 20 min.
And it wasn’t downloading small files.
I checked for errors in the log file.
I found none.
The program was still responsive.
I tried to manual update the headers.
I saw them being added on top of the download list.
But nothing was starting to download.

There were 24 threads.
And there were 4 connections.

Newsbin runs on a laptop:
Toshiba Satellite L500-1ZP
Intel® Core™ i3-330M Processor 2.13 GHz
8 GB internal memory and 1 TB hard disk

Its slow. But had no troubles before the upgrade of the download speed.
So i want to try if downloading stops with my old download speed.
I tried to limit the download speed with the settings of Newsbin.
But that was not working. After a few seconds it springs back to full speed.

Yesterday it stopped downloading again.
I rebooted before starting newsbin.
It had 26 threads when started downloading and it had still 26 threads after it stopped downloading.
And there were 4 connections.
There are no errors in the log file.
The program keeps running except the thing he should do. Downloading.

- I suggest using a different download drive.
- I might uninstall any virus scanners/firewalls, reboot then try some more downloads and see if it still stalls.
I could try but newsbin was working before the download speed upgrade.
And i have no problems with other programs.

I’m starting on the program that kills and restart Newsbin.

[DThor]

A common assumption is that because an anti virus didn't cause problems in the past, it's not a problem in the present. That's actually not true, they are often updated daily(sometimes invisibly to the user), and on many occasions an update brings pain. I would completely disable anything like AV or firewall just for testing purposes. You've definitely got something very strange going on since typically an error message of some kind would show up somewhere, so I wouldn't assume anything.

DT

[Quade]

If you have another computer to run it on, I'd try running it on a different PC.

Is that 1TB disk internal or external? Sounds like it might be a USB disk.

[marcobier]

If you have another computer to run it on, I'd try running it on a different PC.

Is that 1TB disk internal or external? Sounds like it might be a USB disk.

The disk is internal.

But it looks that DThor is right.
I installed Avast antivirus and then de-installed microsoft security essentials.
Newsbin is still downloading after 2 days.
My kind of luck that the upgrade of my speed happened about the same time as the update of microsoft.
Me feeling stupid. (to trust microsoft.)
Thanks for the help.

Avast gave a warning that it couldn't monitor the SSL connection of the e-mail program Newsbin.
But is not interfering.

[Quade]

Another happy ending. It's surprising to me, because many people use MSSE with no issues.

Avast gave a warning that it couldn't monitor the SSL connection of the e-mail program Newsbin.

You might want to ask Avast to simply ignore Newsbin's talking to the news server and instead monitor the download folders. More people have had problems with Avast than MSSE.

[marcobier]

Another happy ending. It's surprising to me, because many people use MSSE with no issues.

Avast gave a warning that it couldn't monitor the SSL connection of the e-mail program Newsbin.

You might want to ask Avast to simply ignore Newsbin's talking to the news server and instead monitor the download folders. More people have had problems with Avast than MSSE.

The problem still exist.
The problem was not solved by changing the virus scanner.
Its only less frequent then before.

I still think it has to do with the slowness of my laptop.
It can take more than a half hour to correct and unpack a big file.
In that time the downloading is slowing or stops.
But normally starts again after the unpacking.

I wrote a program that kills and restarts newsbin if its stops downloading.
I’m still testing and debugging it. But it works.


Another feature i witnessed once.
Is that a bunch of NZB files were processed out of the directory where newsbin looks.
The files where moved to the ‘Done’ directory.
But were not added to the download list.
When I put them back a second time.
They were added to the download list.

[Quade]

I'd suggest you try:

- Disabling download during unrar/repair
- Disable memory buffering during download.

Both of these settings are in the performance options. Then restart Newsbin.