How big are the Newsbin database files for binary groups?

[KootchieKoo]

Hi, I haven't done much downloading from the newsgroups since 2011, back then I was struggling along with Newsleecher 4.x and Forte Agent. Currently I have only a small block account with one provider, I am thinking of getting an account with a high-retention provider with 3500+ days of binary retention. I prefer downloading headers and browsing the groups over doing searches and working with NZB files.

I am wondering how big the resulting Newsbin database file would be if I started from scratch, then downloaded all headers from a group, say a.b.sounds.lossless, from one of the providers with 3500+ days of retention. Does anyone download all headers and browse the groups anymore?

Also, if my PC lost its Internet connection or crashed, would Newsbin, upon re-connect or re-launch, be able to resume downloading headers from where the interruption occurred? How "robust" is the Newsbin mechanism for header downloads when recovering from errors? Thanks

[Quade]

Crashing, is a toss up. It might or might not be able to resume. Your PC should never crash. On the other hand, if you need to restart Newsbin, it'll continue the header download where it left off.

I'll download all headers from Lossless and see...

[Quade]

a.b.sounds.lossless with 3500 days of headers downloaded is about 2.3 gigs.

Keep in mind it's 140 million headers meaning you have to download

140+E6 * 350 bytes = 50 GB of download to get them all.

[KootchieKoo]

My PC rarely crashes/bluescreens, the latest one was a few weeks ago and related the the ESET antivirus I use. A couple more happened 1-2 years ago and were driver-related, but just updating the drivers caused them to go away.

OK thanks for running that test, I appreciate it. I am living in a house with a Comcast cable modem, I am capped at 1024 GB per month, so the 50 GB of headers for that group is a bit disappointing, but initially I think I'll grab only the oldest (500 days or so) headers from all the groups I am interested in (lossless music and a.b.dvd.*) then grab the oldest posts, to catch up. Actually I've been using my block account and some free trial accounts to grab some binaries from a.b.dvd.music that are ~3500 days old, I wanted them years ago, but back then I didn't have the disk space to grab them.

My PC may not be up to snuff, it is a Win7 Haswell system I built in 2014. Has several TB of free disk space (spinning HD's as JBOD, no NAS) - currently no SSD but I've just ordered a new Samsung 970 EVO SSD that will arrive tomorrow. It's only the 250 GB model though, I will install a fresh Win7 onto it, then start playing with Newsbin and whichever long-retention provider I choose. The group a.b.sounds.lossless is only 2.3 GB, so hopefully I can keep all the Newsbin group databases on the SSD, and initially download binaries onto the SSD, and later move completed downloads to the slower storage.

The weak link is the low RAM - the PC has only 8 GB of RAM - will that be enough when pulling 140 million headers or more?

[Quade]

Header downloads don't consume much RAM. Header display is the RAM killer. If you download headers but the tab to display them isn't open, the memory usage is minimal.

[KootchieKoo]

OK thanks for the info. I am back now, I had some serious health issues in the last half of 2018. I have an account with unlimited downloads at NewsgroupDirect (a Highwinds server with ~3500 days of retention) to play with. It's amazing how I can go now and grab posts from some of the DVD groups that were posted 10 years ago, back then I was always running out of disk space and had to be very selective about what I could download. They Newsbin app is very responsive and intuitive.

I have been using Newsbin to download from various groups that were small enough that I could download all headers of the group for the entire ~3500 days of retention. I have hit a snag though, and don't see how to accomplish something.

From your test and a simple proportionality calculation:

a.b.sounds.lossless has 140 million headers = 2.3 GB Newsbin database size and 50 GB used to download these headers

a.b.dvd has 5560 million headers = 91 GB Newsbin database size and 1986 GB used to download these headers

I would like to pull the OLDEST headers (30 days or so of the absolute oldest headers) of a.b.dvd so that I can grab some binaries I want and know are there, before they fall off the server. Is this possible with Newsbin?? - all I see is

Download Special... --> Download 10,000,000 Older Posts

but that downloads the 10,000,000 NEWEST posts which is not what I want.

[KootchieKoo]

My bad -- after just a couple minutes more of poking around I see it - Show Posts Special... with the day range option and the calendar control.

That's why I didn't see it, to me Show Posts means display posts for which headers are already downloaded, but I'm glad it's there. Image link not allowed for unregistered users

[max]

Interesting topic.