Version 6.00 Beta 1 Available...

[dexter]

After 6 months of development, we have a new and much improved version of Newsbin for you to play with. The User Interface has been totally re-done and significant performance improvements have been made as well.

This is an early beta and things are still in flux so have an open mind while using this version. Feel free to comment on what you like, don't like, and wish it had in this forum.

There are way too many new things to create a small summary here that would do it any justice so just go to the Beta Download Page and read the information there. Be sure to read the note about converting Version 5 spool files too. The conversion leaves the SPOOL_V1 folder intact in case you need to go back to 5.5x. It can take a long time so you might want to start it then go to bed or something. Or just do a few groups at a time.

We'd like to thank our brave Alpha testers, most of whom have been with us the entire 6 months on our IRC channel putting up with weird problems and constant changes. This beta would be nowhere near as stable as it is now without their help.

[ozzii]

I've downloaded the V6 to test the UNICODE....
The installer is not in UNICODE so the cyrillic characters are not showed.
I have an error when the installer try to download the language file.
So NBP is in english.....
I'm so sad

[KilJaden]

I've downloaded the V6 to test the UNICODE....
The installer is not in UNICODE so the cyrillic characters are not showed.
I have an error when the installer try to download the language file.
So NBP is in english.....

Because this is the first public beta the only language available at the moment is English . It will take time untill all the translation will be updated/translated and this won't happen untill there will be a final NewsBin 6 build.

As for Unicode support it has been already mentioned that Newsbin has Unicode support . Also having full Unicode support only means that like in you're case the cyrillic characters will be shown in the program , and since Usenet is still ASCII you might or not see any characters in cyrillic if they are posted on Usenet, just because of the different encoding.

[ozzii]

I know that Usenet is not UNICODE.
I just want to have the possibility to use NBP with my translation.
I've already used the translation before the final version of previous version of NBP.
So I don't think that we have to wait for the final version. Just a matter of time to "convert" the old translation "parser" into a new one.
But because the v6 alpha is 6 month old, I thought that the dev have worked for the translation parser .

[biosteel]

Yay, V6 Beta out, am very glad to be able to test it out! Thank you very much for releasing this Quade and Dexter!
The new Ui customization options and the download folding.

I've run into some issues on build 743:
XP x64, q9550, 4gb ram

-Header downloads stop after a network error like this one:

Code: Select all

[07:47:16] ERROR  news-europe.giganews.com  - NNTPServer: Compressed Headers - Connection Failed while reading Data: Unspecified Network Error

I suspect my ISP traffic shaping, now happens regardless of SSL version or port used. Weird though as Giganews and my ISP peer directly.
Using 20 connections on a 100 Mbit line, eventually all header downloads abort. 5.59 used to retry here.
Also thought v6 were splitting large header downloads up on several connections.

[spoiler]
-setting the spool download directory, the nbi setting isn't working for me, always goes to the default folder. 5.59 folder was 90gb without the 10 biggest groups, it gotten too large to keep on the App drive now. [/spoiler] -mistake on my part

-can't reorder the group list, any group inside a folder is sorted alphabetically.

-while downloading I'm unable to saturate the line with 20 connections, 5.59 seemed to only be stopped by disk activity, getting near full speed with 20 connections, need 50 for full saturation with 743, which incidentally didn't improve speeds with 5.59 much.

Some options for people to set aside gigabytes of cache for downloading files would be great. with files below a certain threshold keeping everything in ram would do wonders for unpacking and repairs.

Anyhow, I love Newsbin and don't mind the above, I know it's a Beta, reporting back in hope this helps.

[Semel]

First of all id like to thank you all , both develoeprs and testers, for the hard work done=)
i havent still installed it and started checking everything out but i hope you dont mind me saying that

Won't auto-assemble downloads with missing or bad chunks

of all the issues list should be a top 1 priority in my opinion..


Again thanx, ive been waiting for this moment for ages =)

-setting the spool download directory, the nbi setting isn't working for me

do u mean NBpro data folder?

changing it via INI settings on win xp sp3 works just fine for me

PS

1) filename options - advanced settings cant be opened. I guess its still in development?
2) No autoclean up feature after all these years? Really? =((

3) sorting group list works really fast. good work) it was pain in the ass in nbpro5

[itimpi]

-setting the spool download directory, the nbi setting isn't working for me, always goes to the default folder. 5.59 folder was 90gb without the 10 biggest groups, it gotten too large to keep on the App drive now.

This definitely works for me - I never use the default folder. Do not know why it should not work for you? It makes me wonder if youa re using the .nbi file that you think you are? I always pass mine in as a parameter on the shortcut used to launch newsbin so I can be absolutely confident that Newsbin has not reverted to a default one. It also has the useful side-effect of allowing me to keep different settings for NB5 and NB5 during testing. I give each version its own DATA folder although I share the DOWNLOAD one.

-while downloading I'm unable to saturate the line with 20 connections, 5.59 seemed to only be stopped by disk activity, getting near full speed with 20 connections, need 50 for full saturation with 743, which incidentally didn't improve speeds with 5.59 much.

This is stange - in all other tests others have done V6 has achieved far better maximum speeds than v5. What do the cache settings show in the Status bar? In v6 there is both a lot less disk activity for a given download task, and also downloaded chunks can be cached in RAM so that downloads are not slowed until you run out of free cache space. Disk I/O rates are known to be the limiting factor in download speeds which is why this area was specially optimised. I know that Quade has achieved sustained speeds of 1Gb/sec downloading from a local server - about 5 times what he could achieve with v5 on the same configuration.

Are you doing a lot header downloading or header conversions at the time that might doing a lot of heavy disk I/O? If so it might be worth seeing how download speeds are when purely downloading (and checking/unraring).

Some options for people to set aside gigabytes of cache for downloading files would be great. with files below a certain threshold keeping everything in ram would do wonders for unpacking and repairs.

You can change the amount of RAM reserved for disk caching by using an entry of the form
ChunkCacheSize=nnn
in the [Settings] part of the .nbi file. However if you are disk bound then this may not help a lot as you will simply fill up the cache with data waiting to be written to disk. I am not sure what the overheads are if you specify insanely large values for this entry so use with caution.

[Semel]

DO filters set for a specific group,work?
i set a global filter "efnet" for a group XXX, set it in the group properties and when i open it up its not activated.

BUT if i choose "enable filters" ->global it works..
im not sure if nbpro5 worked that way.. i would set global filter for a group and then filter out some stuff using filter option above..


Can i somehow make "'arrow" keys to expand the collection when i click"right" arrow over it?

I remember we talked about this on nbpro5 .. but is it still impossible to have updated posts put on top? i dunno about others but as for me its __very__ uncomfortable. Every time the group is updated which happens often i have to scroll down..
it would be nice to be able to change it... via INI for instance..

PS
by the way

when "Won't auto-assemble downloads with missing or bad chunks" is fixed we sould be able to set time interval for "download again failed files".. so that nbpro6 will automatically try to redownload failed articles after 1 min +

This is a must( time interval from 1 min and above for retry failed articles) if you have set autodownload filters since though headers arethere astraweb simetimes have no articles available still.. and the whole download fails unless you watch over it and do everything manually.. that doesnt cut it for autodownloading via filters that is

PPS is there a topic that explains INi settings? most of them are clear but not all

[richy99]

just like to point out this is a BETA, so, not everything will be working, not everything that poeople have requested will be implemented yet, yes there will be bugs, yes there will be missing/incomplete features

the 6 months alpha was to test the new internal mechanisms and interface and to get it to a point where it is stable enough for everyday general use so that more people can then hammer the crap out of the program to find issues that the alpha testers couldnt find or couldn't create as everyone works differently with usenet


semel - filters work fine, if youselect it from the gog/grouo then when you load it you have to have enable filters ticked in the post window or it will not work, v5 worked the same
right click - no

[Semel]

Sure thing, Richy.
We all get it the work is still in progress and its not gonna be all wine and roses for a while

On the bright side it works really fast and finally i can download and unrar\repair at the same time without crippling my download speed
really nice.
And opening a group is so fast lol i cant get used to it

emel - filters work fine, if youselect it from the gog/grouo then when you load it you have to have enable filters ticked in the post window or it will not work, v5 worked the same

Oh my bad then
i guess i just forgot it


PS ive yet to find a newsreader that puts new post AT THE BOTTOM when sorting by date is enabled.
this is just perverted...really..

[Kraut2003]

Hi Quade and NB Team

just downloaded the Beta - seems to work fine

its a work in progress and I for shure will test it to the limits (I have a 100 Mbit line)
looks very nice so far - still some work to be done on the optics and I have to get used to the new Interface

the biggest improvement for me at this moment is that the speed limiter now can handle more than 20 Mbit

Keep up your good work - NB was by far the best investment in software I´ve ever done since tbe beginning of my PC days

Greetings from Germany

Thomas

[Quade]

What do the cache settings show in the Status bar?

This is important. If you look at the status bar you'll see "Cache X/200" This represents the number of blocks of data in the cache. If it hits "0/200" (it's like a gas gauge). It means Newsbin has used up all of it's buffers and it waiting for disk IO to finish. That's a sign your disks are slower than your download speed. Either because they're busy, because they're just slow or because something is getting in the way of writing.

I regularly get >500 Mbps header download speeds to either Astra or Giga.

Dex and I have been fielding tech support about slow download speeds in general over the past couple days (5.59 mostly) so, I think there's something up on the 'net for some people.

About the speed limiter. It works at > 20 Mbps but, the slider isn't calibrated. That means what you set isn't exactly what you get. You might have to jerk with it some to get the speed you want.

[bootster]

I don't even see a version 6 anywhere on the beta download page. Did it get removed?

[Quade]

Scroll down, it's near the bottom.

[itimpi]

I don't even see a version 6 anywhere on the beta download page. Did it get removed?

You have to first read the release notes, and when you get to the bottom you will see the download link.

This approach is deliberately chosen to encourage beta testers to read the Release notes as they contain important information on what is known to be working/not working and also any special actions the use might need to take when using this beta.

[Semel]

If you look at the status bar you'll see "Cache X/200" T

hmm i dont see anything like that..
http://www.imagebam.com/image/2b4a01117777398

when i click RMB i see that chunck cache state is activated though

What does these numbers mean? WhenNB downloads something it shows you how much you have already downloaded of everything and when NB starts unpacking numbers change..
http://www.imagebam.com/image/044a03117777777

[richy99]

right click the status bar and see if there is a tick next to chunk state

[Semel]

I couldnt see it with 'speed progress" bar activated. when i disactivate it cache status is shown
Or if i remove anything that is on the left side from cache status

[dexter]

hmm i dont see anything like that..

Yeah, you need to make Newsbin wider before the cache status will display. Should be immediately to the left of the "Running" status. You're also missing the disk space free status. I measured and it looks like you have to have at least 1280 pixel width to see everything on the Status Bar.

[Semel]

ive tested NB6 again and i take my words back..

Im having major Issue with unpacking\downloading

core 2 duo 3Ghz\ 3.5Gb ram, win xp sp3


When NB downloads ONE item and unpacks it the improvement made in speed is significant compared to NB5 for sure

However!

2.2Gb+ 2.8Gb,to one disk, no SSL
1) 2.2 is being downloaded, its OK (10.5-11MB)
2) 2.2 is getting unpacked up to ~45% , 2.8 is being downloaded OK (cache size is getting reduced to 0)
3) 2.2 is getting unpacked after 45%, 2.8 download speed drops to 9-8-6-5-3-1.5MB and starts jumping in 1.5-5MB range, cache size is empty totally

Then it gets really pointless and i pause downloading of the second item

I didnt even use SSL connections!!

Now i switch to Usenet Explorer.
the same two files, the same drive + SSL connections!!

- 10.5-11MB
- the first item is getting unpacked.
Download speed of the second item NEVER drops below 10.3MB up the very end

I guess there is still much to be improved in this srea
Perhaps playing with rar buffer size or whatever its called..


PS I had NB6 two times just closed without popping out any error or anything when i tried to to 'download to existing folder" thing

[KilJaden]

You can increase the Cache size so it won't hurt you're downloading performance"" , just edit in the nbi file

Code: Select all

[Settings]
ChunkCacheSize=nnn

The default is 200 , but you can make as big as you like 400,600,1000 etc .

As for the fact that UE is faster, personally I never experienced that to be true . Even with the new download engine in 3.x from Ue it's still a lot SLOWER than NewsBin when it comes to downloading.

[br0adband]

Congratulations on the release. Just installed it and getting set up at the moment, first thing I noticed is the Start Menu shortcut (folder) is still named Newsbin5 soooo...

Will offer opinions and experience (and of course actual bugs) if I have issues.

Again, congratulations, it's been a long time coming and I'm sure a lot of us will miss the old consistent layout and look, but things change - it's nice to know Newsbin Pro can too.

[Semel]

but you can make as big as you like 400,600,1000 etc .

Oh i gotta try this. Thanx

Ill give it a try and reply then

Someone should create a guide for al availabe settings there.. ive just opened ini file and dont see this line.. and i would never know about it if you didnt tell me

. Even with the new download engine in 3.x from Ue it's still a lot SLOWER than NewsBin when it comes to downloading.

Define slower.
Your experience.
I dont personally have 1gbit so i can say only for 100mbit.
And its not only about spedd. UE is a lot more mature and got lots of features i would like NB6 had.. and i still do hope they will be implemented at some point. The more good newsreaders we have the merrier

[itimpi]

Note that disk I/O is still the overall bottleneck so trying to use faster disk drives (if available) and spreading the load across drives still help. You did not give any inication of what type of disks you are using, but network or USB drives will sginificantly slow things down.

One other possibility is playing around with the priority settings for the background processing. I am not sure what effect (if any) they have on multi-core systems.

One important aim of the beta program will be identify performance bottlenecks if different types of workflows so that performance tweaks can be identified and implemented. The Alpha program only stressed those used by the small numbers of alpha testers. For those users significant improvements were obtained - but that does not mean there is not more to come!

[neilt0]

You can increase the Cache size so it won't hurt you're downloading performance"" , just edit in the nbi file

Code: Select all

[Settings]
ChunkCacheSize=nnn

The default is 200 , but you can make as big as you like 400,600,1000 etc .

200 what? MB, chunks, weasels?

[Quade]

Chunks. Assume anywhere from 600K to 2 megs per chunk (on average it's 600K).

If your disk is slowish, chunks can help you ride through the slow bits. It's just a buffer though. If the difference between your download speed and how fast your disks are is too great, no amount of buffer will really help.

I don't get the SSL comment. Seems unrelated somehow. The Cache indication tells you when your disks are too slow.

- Slow download but, cache is sitting at 150 (chunks - connections) = Network
- Slow download and the cache is riding around zero = Disk.

[itimpi]

200 what? MB, chunks, weasels?

I thought you might have guessed from the name that it is Chunks!

Chunks are typically around 600KB, so a value of 1000 means Newsbin will use about 600MB of RAM when running. Whether that matters to you deends on how much RAM you have - for modern systems this sort of value is acceptable, while older ones will want a value more like the default of 200.

[itimpi]

Have you given an exclusion in your virus checker to .nb2 files? If not this definitely worth trying. It could also be worth trying to exclude the whole of the Download folder - but that may be going a step too far for normal use.

[Semel]

Ah... the format has changed.. im looking at my antivirus software and see .nb! extensions excluded from scanning
And now its .nb2

yeha ill try it right away but to be honest i really doubt it will make any noticeable difference =(

though i excluded completely data folder..where all these nb files were..so..

[itimpi]

though i excluded completely data folder..where all these nb files were..so..

These files are in the Download folder, not the Data one. Having said that excluding the Data folder as well cannot hurt.

[bootster]

Thanks. I like the new GUI and haven't seen any drawbacks so far. Thanks for putting it at the top of the Beta page as well.

[Semel]

ive already posted my specs earlier.

but here you go - core 2 duo 3Ghz, 3.5Gb RAM, 8800GTS 640mb, several disks (WD green cavier 1.5Tb) win xp sp3

[biosteel]

Thank you for the tips itimpi .

I was using "SpoolPath" instead of "DataPath". Setting Data Storage with the latter works in both versions. Am sorry for the false alert, should've double-checked.

Header downloads in v6 build 743 still drop with the Unspecified Network error log entry I posted,
very quickly.
I'll do some more tests with the cache settings and keep a look at CPU, RAM and other factors.

I tend to get a few of such Network error messages in 5.59, too, although not while downloading headers, if they drop, 5.59 appears to auto-retry, apparent by the changed maximum header size of a group visible.

Disk I/O should be ok, tested storing headers on a Velociraptor and a 7200rpm 32mb 1Tb download drive.
Data Downloads I'll test again with a higher cache setting.
I'll likely try a Ramdisk eventually, as I still like the idea of bypassing the Downloaddrive for unrar/par recovery operations on moderate download sizes, ie 1~2 gb.

It'd still be nice to be able to download a single groups' headers with multiple connections at once, seeing as my ISP limits the maximum throughput on a single connection. I repeatedly checked my RWin and other relevant TCP/IP settings but those haven't changed since my ISP introduced this limitation roughly a week after I got 100Mbit.

Anyway i'll look forward to the next versions.

[Quade]

Header downloads in v6 build 743 still drop with the Unspecified Network error log entry I posted,
very quickly.

I'd be looking closely at your system or network. The beta has seen intensive testing by the IRC testers and this was never an issue.

Cache only impacts file downloads. I have a feeling your connections are dropping out under load. Maybe the router. It's hard to say. Might try setting the server to a single connection and try a header download and see what happens. Then maybe throttle the speed down and try again. If you're on a wireless connection, I'd wire in. Usenet and Wireless don't really get along. It'll sort of work but, wireless is inherently unreliable.

[neilt0]

200 what? MB, chunks, weasels?

I thought you might have guessed from the name that it is Chunks!

Chunks are typically around 600KB, so a value of 1000 means Newsbin will use about 600MB of RAM when running. Whether that matters to you deends on how much RAM you have - for modern systems this sort of value is acceptable, while older ones will want a value more like the default of 200.

Thanks (and to Quade). I can certainly spare enough RAM for 1000 weasels. Erm... chunks.
My free weasel count dropped to about 90/200, but I didn't see it go lower. I'll keep an eye on it.

[wissertje]

First of all congrats whit the first bèta release.

- The first thing i really noticed that on high speed (100Mbit) the download is more stable.
- I don't know if this is coming in the coming beta's but is it possible that the size are showing not ik KB but in GB? Maybe automatic above 1 GB it shows in GB?
- Even the speedlimiter is changed. Thanks guys.

Further it runs really great.

[richy99]

the size where?

[wissertje]

Oops, forgot the screenshot. Sorry for that.

[Quade]

Yeah, down in the download list right?

I setup a test setup. 36 Gb of file sets on my server. The server is a linux box on the same network as my PC.

- Queued up 2 6 GB sets, a 4 GB set and a 10 GB set one right after the other.
- 10 connections to the server.
- Machine is a very powerful quad core, downloading to a 7200 RPM 2 TB WD Caviar black drive

Set #1 6 GB - needed an unrar.
Set #2 6 GB - needed a repair and an unrar.
set #3 4 GB - just download
set #4 10 DB - needed an unrar.

Set #1 downloaded, averaged 200 Mbps/ 24 MB/s
Set #1 started to unrar while set #2 started downloading. Set#2 downloaded at about 200 Mbp/s 24 MB/s. Speed dipped a little and the chunk cache dropped down to as low as 150.
Set #1 finished unrar
Set #2 finished and started to repair.
Set #3 Downloaded during the repair. Never went below 24 MB/s
Set #4 Started to download while Set #2 was still repairing. Set#3 downloaded at 24 MB/sec. If anything it dipped less than Set #2 probably be cause the files were 500 MB each.
Set #2 finished repair and started to unrar. Set #4 continued to download.
Set #2 finished the unrar.
Set #4 finished and unrared.

I don't doubt what you're seeing but, I'm pretty sure the reason you see it is that Newsbin is doing the par scan as each file downloads instead of later. Maybe it's something I can look at down the road. It's clear to me that your disk is the bottleneck, at least as far as Newsbin's current design is concerned, a different design might work better on a modestly powerful machine.

[EvilMink]

Error while importing boneless into 6.00. It happens seemingly in the same place. Corrupted rv4? 5.59 doesn't seem to have problems reading the data into program window.
[02:14:53] ERROR - WorkerThread Died: Unnamed

[Quade]

Might have to figure out which one it is and move it. If it's not too big, I wouldn't mind trying it in the debugger.

[titan_rw]

I just tried downloading two 2 gig sets. I have NB running on a VM that has gigabit connectivity. News server used was Astraweb (non-ssl) using 16 connections.

The underlying disk is a regular 7200rpm sata drive, but is shared amongst the other VM's of course.

The VM has access to 4 cores of the host computer. I have 1 gig of ram allocated.

The first set downloaded at 120Mbps. Cache status was bouncing from 80/200 down to 0/200. I interpret that as mostly disk limited.

The first set finished downloading and started unpacking as no repair was required.

During unpack, set two started downloading. This ran at 70-80 Mbps. Cache status was almost always at 0/200. At this point, I'm out of disk IO.

After unpack finished, set two sped back up to around 120Mbps, with cache bouncing up and down in the 'lower' end of it's range.


Considering VM's are not usually known for their strong underlying disk IO subsystem, I'd say this was pretty good. On the same VM, I tested downloads writing to a ram drive. I acheived 300Mbps +. This is streaming data across the internet from AW remember too. Not too bad for a VM once you remove the disk bottleneck.

[richy99]

just for shits and grins try it on a drive that isnt encrypted

[Semel]

Its only the system one and its old and more slow than WD 1.5Tb encrypted.

besides, benchmark tests i did for AES encryption didnt strike me as slow..

And also keep in mind that De-/Encryption is performend in memory and is also affected by CPU perfomance
And i dont have any problems in this department here

PS ive tried it on a system disk.. much worse..
drops to zero as well.

[wissertje]

Here goes another proof that newsbin PRo 6 beta 1 has indeed problems with simultaneous downloading\unpacking on high speed

This time its sabnzbd. Two 2 Gb files. DOwnload speed while unpacking never dropped below 10.4 MB



I didnt even have to adjust article cache size! There was no need as you can clearly see

Because, you know, when its one newsreader that works OK and another not Ok,then it might mean nothing
When its a newsreader and nzb downloader that work OK against the same newsreader which doesnt then its a pattern

Please, do something about it.
Your chunk method or whatever NB uses just doesnt cut it.. you increase cache to 1000, letting nb6 to consume 600+mb ram and it still does drop speed heavily

i was actually wondering if this exact situation(downloading\unraring of large files) was tested on high speed(up to 100 mbit) at all. .not just plain downloading which is really great, i have no complaints whatsoever about it..and of course not on SSD drive


PS You are gonna laugh your ass off lol.... Even that buggy piece of code junk- Newsleecher managed to keep steady 10.54MB while unpacking. It eventually slowed down the speed of unpacking (not like sabnzbd \Usenet explorer) but thats not the point. It passed the test anyways

I have the same problem. While decoding the download speed is about 40Mbit. When it is readdy it is back on a stable 102MB/s. Version 5.59 was more unstable download between 80 and 100Mbit but had the same issue. Newsleecher is here steady on 11.30MB/s, also during extracting.

My specs are: Intel E8500 3.18GHz, 4GM RAM, ATI HD4830, Windows 7 x64, 4 HDD's: Download HDD is a Seagate barracuda 1.5TB (a new one. I thought my other HDD was not ok. The old one was a Samsung F1 1TB), OS HDD is a WD 500GB.

Last weak i send a mail to support about this issue, but the emailadres a send isn't working anymore.

[richy99]

what address did you use?

[wissertje]

what address did you use?

My old emailadress? I can send it via pm if you want to know?

Edit: My brother in law had a different setup but also the same problem. I noticed the problem when i was switched from ISP.

[richy99]

edit : nevermind i misread it, i thought the address you were using to contact support on wasnt working

[Quade]

I'm tired of the download topic.

All that matters is that I know what you're saying Semel. I'm the only one you need to convince. Time to move on to some other topic instead of flooding this announcement topic with more performance stuff.

Left your original post, my reply and a couple other meaningful replies.

[kamakazi downloader]

Finally downloaded Beta (can't believe I actually missed the release yesterday!!) love the new look/feel/speed.....well heck, actually love it ALLL! I know there are a lot of hardcore 'high-tech' users out there who might have need of all possible features (and then some) but as for me, I'm a simple, average user with no knowledge/need of all the 'bells-n-whistles'. Even if this Beta does have a few bugs (which I'm sure will be pointed out by the more astute users) I'm happy to have this so obvious improvement over V5.59. I'm already in full swing using it, lol.
THANKS, Quade and Dexter, et al! Well done!