1.2 Terabyte hot swap RAID5 Tivo storage

[networker]

No bull guys, I am building an external storage unit for my HR10-250 that will have 5x 300GB disks in a fault-tolerant RAID5 configuration, 1.2 terabyte usable, 144 hours of HD. Whether you believe me or not does not matter, I will post photos when done.

Anyway, I have a dumb question (at the end)
This can be found for $2,500 today here:
http://www.firewiredirect.com/store/customer/product.php?productid=16536&cat=423&page=1

Or, $1,000 for the empty box, add your own drives. here:
http://www.firewiredirect.com/store/customer/product.php?productid=16540&cat=423&page=1

Other photo:
http://www.synetic.net/Synetic-Products/Stardoms/Stradom-6500.htm

All of these products are based on a variation of the Areca ARC-5010:
http://www.areca.us/IDEIDE.htm

with this case:
http://www.scsi4me.com/?menu=menu_scsi&pid=3283

Some actually are a black version of the ARC-5010 such as this one:
http://fwdepot.com/thestore/product_info.php/products_id/657

Now, here's the dumb question:
The ARC-5010 is $600 in qty of one, plus $39 for the LCD. How many would be in for a volume buy?

Networker.

[Jamie]

Should be interesting to see if you run into any MFS size limits. The 274GB zone size limit is well known, but it can be worked around reference (http://www.dealdatabase.com/forum/showthread.php?t=43316).

I wonder how loud it's going to be?

[Roger Dylan]

I wonder how loud it's going to be?

Problem, generally, is the fans.

My online video library (obviously exclusive of the removable drive cartridges for the individual DTivos) is on 18 250GB drives, 8 of them in 4-drive enclosures and the others in individual cases. I run simply mirrored RAID because drives are cheap (chainstore rebates). All the drives are running acoustically-managed. The noise problem is pretty much from the case fans. The 4-drive cases have been pretty quiet, I use the MAP-504F1, very nice for the price, if a little bulky. But the smaller fans in the individual cases, variety of vendors, go raucous. Sometimes just temporarily, sometimes you can give them a tap and they reseat. I've replaced four cases over the past few years because of fan noise. For short periods of time I've run without fans, with external cooling, and monitored the drive temps with only a slight increase, but I'm uncomfortable about it. I'm migrating towards more MAPs, moving forward, noise is very tolerable.

[jonbig]

Isn't the internal bus in the HD250 ATA?

[compwiz312]

Isn't the internal bus in the HD250 ATA?

Yes, it is, but something as simple as this: http://tekgems.com/Products/MJ-SATA-ADP-IDE.htm?frg could fix that incompatibility.

[networker]

> Jamie wrote:
> Should be interesting to see if you run into any MFS size limits. The
> 274GB zone size limit is well known, but it can be worked around.

I don't expect any on the HR10-250, but only trying will tell.


> I wonder how loud it's going to be?

zat is ze question. I will definitely have a fan speed regulator.


> Roger Dylan wrote:
> 4-drive cases have been pretty quiet, I use the MAP-504F1,
> very nice for the price, if a little bulky

Mind sharing a link? The price is right, but it looks that the last ones in stock are in russia...

I think I'm going for this one:
http://www.performance-pcs.com/catalog/product_info.php?cPath=66_147_218&products_id=2774

When I look at the cost of the entire thing ($2k :-(, and since it's going in the living room, might as well have a nice case.

>> jonbig wrote:
>> Isn't the internal bus in the HD250 ATA?

> compwiz312 wrote:
> Yes, it is, but something as simple as this:
> http://tekgems.com/Products/MJ-SATA-ADP-IDE.htm?frg
> could fix that incompatibility.

This would be to use a PATA drive with a SATA motherboard, but there are adapters that do it the other way or both ways as well:
http://www.satagear.com/SATA-MM-A1_SATA_Adapter.html

and
http://www.pricegrabber.com/search_techspecs_full.php/masterid=988418#description

If anything fails, the ARC-5010 (http://www.areca.us/products/html/ide-ide.htm) also has a PATA interface, but the PATA cable can be only 18 inches long, where the SATA cable can be 40 inches. I will try the SATA way first.

[Ric]

http://www.buffalotech.com/products/product-detail.php?productid=100&categoryid=19

Have you looked at the Buffalo Terastation 1.6T for $1850.00

[jonbig]

http://www.buffalotech.com/products/product-detail.php?productid=100&categoryid=19

Have you looked at the Buffalo Terastation 1.6T for $1850.00

Errr...that won't work. It's a NAS box that is only reachable by ethernet. It has no SATA or PATA interface for a host machine.

[networker]

>> Ric wrote:
>> Have you looked at the Buffalo Terastation 1.6T for $1850.00

> jonbig wrote:
> Errr...that won't work. It's a NAS box that is only reachable by
> ethernet. It has no SATA or PATA interface for a host machine.

Exact. Besides, it's a low-power PC with a software linux raid, no hot swap, only 4 disks. I can make this for $0 plus the disks, I already have a junk PC to run the host or I could drop an el-cheapo SATA card and add more disks to my existing server.

There would be an extra issue: even if someone delivered a hack that could mount an NFS volume on a remote box over IP, dinky USB 1.1 simply does not have the bandwidth to deal with three HD streams (2 record, one watch). Even if the USB ports are 2.0 I doubt the existing hardware could pump this much data over them without choking.

[rc3105]

There would be an extra issue: even if someone delivered a hack that could mount an NFS volume on a remote box over IP...
that particular util has been around so long it didn't originally support lba48



... dinky USB 1.1 simply does not have the bandwidth to deal with three HD streams (2 record, one watch)...
fair enough



Even if the USB ports are 2.0
they are



I doubt the existing hardware could pump this much data over them without choking.
true, but it can handle 2 hd streams


---


there are better ways to map extra storage into nowshowing. with a few braincells and sw available here on dd you can get the same functionality w/o buying or dedicating anything


my friends & family collectively have 50 or so tivo that share various lists and & view/burn as though recordings were local. the units don't know or care where the raw data is before it's requested. biggest annoyance is that some of the links are via upload capped dsl that can barely push one realtime sd stream

*a couple of nodes are in europe, it's nice to watch dr who in south texas as it airs :D

[cheer]

there are better ways to map extra storage into nowshowing. with a few braincells and sw available here on dd you can get the same functionality w/o buying or dedicating anything

my friends & family collectively have 50 or so tivo that share various lists and & view/burn as though recordings were local. the units don't know or care where the raw data is before it's requested. biggest annoyance is that some of the links are via upload capped dsl that can barely push one realtime sd stream
Now that's a cool idea. TivoWAN. Hmm...would be interested in how you do this. I suppose you could just use MRV over some kind of tunnelling, although by default MRV just sends advertisements to the local LAN broadcast address, right? So you'd have to UDP forward (assuming it's UDP, which it pretty much has to be). That seems unwieldy. Would also hate to try and get an IPSec client running on the Tivo.

Or do you just mean Sanderton's thing? That should work without tunneling though it doesn't exactly use Now Showing IIRC.

Hmm. Now I've got to figure this out.

And I hear you about this being a great way to expand space, but it presumes those other Tivos are available, right? If that's not an option, then other more creative methods are required. Personally I'd rather find a way to stream over the 'net than wire up an external box, if only because I'd want most programming to be available to all 4 DTivos in the house.
*a couple of nodes are in europe, it's nice to watch dr who in south texas as it airs :D
Very nice. :)

[7.1]

Now that's a cool idea. TivoWAN. Hmm...would be interested in how you do this. I suppose you could just use MRV over some kind of tunnelling, although by default MRV just sends advertisements to the local LAN broadcast address, right? So you'd have to UDP forward (assuming it's UDP, which it pretty much has to be). That seems unwieldy. Would also hate to try and get an IPSec client running on the Tivo. It's been covered: here (http://www.dealdatabase.com/forum/showthread.php?t=39034). You need a connection with a couple of Mbps for realtime transfers of even SD tivo/dtivo streams. At least until recently, mpeg4 over BT was much more practical.

Further discussion is probably inappropriate for this forum, given that this is treading near/on the show swapping taboo topic.

[rc3105]

...Or do you just mean Sanderton's thing? That should work without tunneling though it doesn't exactly use Now Showing IIRC.

Hmm. Now I've got to figure this out.

And I hear you about this being a great way to expand space, but it presumes those other Tivos are available, right? If that's not an option, then other more creative methods are required. ...

there have been numerous sharing / archive implementations ranging from bare bones extract/restore scripts all the way up to dedicated sql, php, newgroup and myth front/back end varients that speak mfs for transparent interaction. also dozens of automated & event monitoring scripts (with & without osd feedback) to interact with 'em

Tridge's vplay was one of the first utils to allow playback of video stored outside mfs and illustrates that the space allocated to mfs isn't terribly important. sanderton's mfs_ftp enhancement and tivo's HMO music streaming are also excellent examples, neither requires the source to be another tivo


as I've said before, tivo's just a goofy lil *nix box with a decent mpg chipset. the possibilities are primarily limited by lack of imagination, not the hardware

[networker]

> rc3105 wrote:
> there are better ways to map extra storage into nowshowing. with a
> few braincells and sw available here on dd you can get the same
> functionality w/o buying or dedicating anything

The reason I have a Tivo is because I like to watch whatever I want whenever I want. It does not any good to me to wait for something to download to watch. It does do any good to me to rely on someone else to provide the storage for and record what I want to watch.

Wake up. An HD stream is 15 mbit/s. An HD movie is 15GB. How many of your buddies have a) a DS3 b) a cisco 7200 router or better to handle it c) a terabyte of storage to store the stuff _you_ want d) let you access _their_ tivo to configure season passes and record stuff?

[rc3105]

the hr10-250 here has a 80 gig (yes only 80), new recordings are auto-transferred to the old athlon server in the background (couple of 400's + 2TB rw dvd changer) so they're avail to the pc's and xbox over 100bt within the house

the 80 is treated like L2 cache, it's just there to buffer recording / playback. everything on the server is avail in the 10-250's nowplaying and pulled across as needed. since a usb200m is plenty fast for a single hd stream and it's kinda hard to playback more than 1 show at at time via the 10-250 it all works rather well

Athlon64 notebook with wireless g makes a nice portable htpc, allows me to watch whatever I want whenever I want wherever I want :D


don't have to rely on other tivos for storage or recording, was just too easy to add distributed storage & wan links once the basics were implemented ;)

the same methods allow for quick add / remove / extract via usb hd & dvd. need another 50 hrs of hd? just plug another 400 into the hub or the server - doesn't get much easier than that

[cheer]

The reason I have a Tivo is because I like to watch whatever I want whenever I want. It does not any good to me to wait for something to download to watch. It does do any good to me to rely on someone else to provide the storage for and record what I want to watch.

Wake up. An HD stream is 15 mbit/s. An HD movie is 15GB. How many of your buddies have a) a DS3 b) a cisco 7200 router or better to handle it c) a terabyte of storage to store the stuff _you_ want d) let you access _their_ tivo to configure season passes and record stuff?
Well obviously if you're going to do HD streams, WAN streaming is not optimal. You're better off caching them -- auto-download to a server or whatever.

And veering a bit off-topic, I would never use a Cisco 7200 for a DS3. :)

[networker]

> cheer wrote:
> Well obviously if you're going to do HD streams,

Why else would I want a terabyte of storage? There are 24 hours in a day, most of them spent sleeping, eating, working and not watching TV. I don't need another 1,000 hour Tivo. I have only one 56" HDTV with a Dolby sound setup in the house and I don't care for watching HD on a laptop with a 500:1 contrast ratio screen. For SD I have the good old faithful DSR6000 with more storage than I need.

> WAN streaming is not optimal. You're better off caching them --
> auto-download to a server or whatever.

Then why bother with the complexity? Add the disks to the HR10-250 directly, instead of tossing in the picture software hacks, constantly nailing the server with HD streams instead of leaving it available for other tasks that really need the server, etc. It does not cost more to install the disks directly in the TiVo. Besides, I do not trust DirecTV/Tivo enough to give them a straight shot at my server, so I would have to create another subnet for the Tivo, administer the access-lists for it, possibly configure QOS, this becomes a full-time job. Direct ATA from the HR10-250 to the storage and I'm done, no need to profile bandwidth nor resources, no change management, no security risk. K.I.S.S.


> And veering a bit off-topic, I would never use a Cisco 7200 for a DS3. :)

Why is that? I'm happy with mine. What would you use?


More on-topic: Who else than weaknees has a commercial solution for making the "A" drive of the HR10-250 more than 250 Gigs?

[Jamie]

More on-topic: Who else than weaknees has a commercial solution for making the "A" drive of the HR10-250 more than 250 Gigs?PTVupgrade shows a dual 400GB option on their hr10-250 product page (http://www.ptvupgrade.com/products/hr10-250/). "mfsadd -r 4" is one way to work around the 274GB partition size limit.

[mmoore99]

the hr10-250 here has a 80 gig (yes only 80), new recordings are auto-transferred to the old athlon server in the background (couple of 400's + 2TB rw dvd changer) so they're avail to the pc's and xbox over 100bt within the house

the 80 is treated like L2 cache, it's just there to buffer recording / playback. everything on the server is avail in the 10-250's nowplaying and pulled across as needed. since a usb200m is plenty fast for a single hd stream and it's kinda hard to playback more than 1 show at at time via the 10-250 it all works rather well

Athlon64 notebook with wireless g makes a nice portable htpc, allows me to watch whatever I want whenever I want wherever I want :D


don't have to rely on other tivos for storage or recording, was just too easy to add distributed storage & wan links once the basics were implemented ;)

the same methods allow for quick add / remove / extract via usb hd & dvd. need another 50 hrs of hd? just plug another 400 into the hub or the server - doesn't get much easier than thatIs the software that you use to perform the functions in the quoted post available here on DD or any other public forum or is it your own proprietary code?

[rc3105]

Is the software that you use to perform the functions in the quoted post available here on DD or any other public forum or is it your own proprietary code?
the basic underlying components are all freely available - vserver, vplay, mfs_export, mfs_import, mfs_multicast, hmo_d, unscramble kernels/modules, text2osd, event intercept/send utils, updated usb & networking drivers, tivoweb modules, <insert forgotten category here>


a slightly more usefull question, and what I'm guessing you probably meant to ask was "are there any turnkey packages like tivoweb, extreme or instantcake?" and that's a more complicated answer. there have been quite a few, but nothing yet that's become really mainstream like tivoweb/tytool. availability depends largely on the fate of the original sources. lots of really interesting stuff has poofed due to tivocommunity censorship, aged out of the newsgroup/mailing lists or been removed due to legal issues (like tivo's takedown notices to anyone hosting images)


to answer the actual question posed: the particular configurations I use (http://www.dealdatabase.com/forum/showthread.php?t=36281) are private, mostly because I'm too busy/lazy to clean them up for general use & provide support. they're basically equivilant to a LinuxFromScratch config, not redhat/knoppix/ubunto type user friendly distros

[cheer]

to answer the actual question posed: the particular configurations I use (http://www.dealdatabase.com/forum/showthread.php?t=36281) are private, mostly because I'm too busy/lazy to clean them up for general use & provide support. they're basically equivilant to a LinuxFromScratch config, not redhat/knoppix/ubunto type user friendly distros
Is the thread you linked to still alive? In other words, has the trial balloon been punctured or is it still floating? :) Stupid poll wouldn't let me pick more than one selection...I wanted to donate cash, write the manual, and answer noob questions for all eternity...

Well, in the interim, I'll play with some of the tools myself. My original plan had been to:

Hack the 4 DTivos around the house
Batch extract shows on a nightly (or whatever) basis to a server, converting to mpg on the fly
Watch extracted shows from a hacked Xbox running XBMC

However, there's nothing like having a wife who is intolerant of complexity to put things in perspective for you. She's not nuts about XBMC. Plus, I have a couple issues with it:
It sometimes has issues with Tivo-derived mpgs -- occasionally freezes
The ff/rw function seems buggy
It is very bad with interlaced video. Since XBMC is basically running atop a modified linux, and uses mplayer (designed for PCs), it wants to deinterlace the interlaced videos and then re-interlace them on output. Ick. Plus, you have to manually turn the deinterlacing on/off -- so if I'm watching a video ripped from a DVD, I need it off, but then I need to turn it on for Tivo-derived files. PITA, and no way will the wife ever do it.

And in any case, it would mean putting Xboxes in the various rooms of the house -- many of which already have DTivos. (On a side note, can a hacked DTivo with no sat feed be used to watch shows over MRV? I have rooms where there's no need for live TV but it'd be nice to be able to watch stuff with them, and a DTivo is cheaper than an Xbox...)

I'll hafta play around with some of this stuff. I'll post results.

[rc3105]

Is the thread you linked to still alive? In other words, has the trial balloon been punctured or is it still floating?
baby on the way & bills to pay. the balloon's still afloat but my time available for non-paying projects isn't what it used to be :(


Well, in the interim, I'll play with some of the tools myself. (On a side note, can a hacked DTivo with no sat feed be used to watch shows over MRV? I have rooms where there's no need for live TV but it'd be nice to be able to watch stuff with them, and a DTivo is cheaper than an Xbox...)

I'll hafta play around with some of this stuff. I'll post results.
tivos can indeed be used for pure network playback. series 1 & 2 can use the mfs_ftp based extensions that sanderton posted, series 2 can also the mrv superpatch. if you're going that route I'd suggest running sat cables to those units and using them as additional networked tuners, $2.50/mo/tuner is cheap enough compared to the overall dtv bill

xbmp & xbmc both have the ability to ccxstream recordings (even live tv buffers) directly from tivo -> xbox, no extract / convert required. they can also handle ty/tmf on local or network storage. we have an xbox connected to the hdtv and my better half is perfectly happy using the xbox dvd remote in xbmp/xbmc.

if you still want to auto extract/convert you could look into something like etivo to save as mpg4. chopping commercials from an xvid avi only takes a few seconds with virtualdub and you can fit an entire season on a rw dvd. gf's fav shows live in her pc's music folder which is avail to the xbox via samba share (she loves music, made more sense than duplicating her library on the xbox drive)

if you're feeling particularly l33t you could load a small xbox linux distro (like x-dsl) ( http://www.google.com/search?hl=en&lr=&newwindow=1&c2coff=1&q=xdsl+xbox+damn+small+linux) and write a few scripts that use tivo enabled mplayer/mencoder to automate extract / convert entirely in the xbox when it's not otherwise occupied

[cheer]

baby on the way & bills to pay. the balloon's still afloat but my time available for non-paying projects isn't what it used to be :(
Congrats! First child? Hope you're caught up on your sleep for a while...

And I'd certainly contribute to a bounty. I suspect others would too, although I dunno if enough would to make it worth your while.
tivos can indeed be used for pure network playback. series 1 & 2 can use the mfs_ftp based extensions that sanderton posted, series 2 can also the mrv superpatch. if you're going that route I'd suggest running sat cables to those units and using them as additional networked tuners, $2.50/mo/tuner is cheap enough compared to the overall dtv bill
Yeah but my 5x8 multiswitch is full so it would mean dealing with that plus running more cable and aaarrrggghhh I'm way too lazy. :) But it's nice to know I could grab a couple of cheap HDVR2s and use 'em.
xbmp & xbmc both have the ability to ccxstream recordings (even live tv buffers) directly from tivo -> xbox, no extract / convert required. they can also handle ty/tmf on local or network storage. we have an xbox connected to the hdtv and my better half is perfectly happy using the xbox dvd remote in xbmp/xbmc.
Rightright, but if you don't have HDTV (or some kind of digital TV) you're deinterlacing->reinterlacing, which bites, plus it means you have to toggle XBMC's deinterlacing filter on and off depending on whether you are watching Tivo sources or, say, DVD rips. My wife will never put up with that. :) Plus again it would mean buying more Xboxes and putting them where I want them, which means making sure my net infrastructure is up to the task of having 3-4 people streaming vids from the server at once. Of course the infrastructure has to be there for MRV too.
if you still want to auto extract/convert you could look into something like etivo to save as mpg4. chopping commercials from an xvid avi only takes a few seconds with virtualdub and you can fit an entire season on a rw dvd. gf's fav shows live in her pc's music folder which is avail to the xbox via samba share (she loves music, made more sense than duplicating her library on the xbox drive)
Already doing something similar -- my son is a long-distance trucker and he EtiVos stuff to download to his laptop for viewing. Works quite well. And we do XBMC music/videos off of Samba shares. I guess I was just looking for a way to avoid multiple Xboxes, esp. as the Tivo interface is (A) nicer and (B) already present (rooms already have DTivos).

Alas, it doesn't seem that lgkhan's hack works with 6.2 (I spent a few hrs trying to hack it into functioning without success), so I'll have to forge ahead on my own. My difficulty is that I'm a network guy, not a coder, so it's learn-as-I-go. But the concept is there...pull files from ftp and insert. I'll probably travel down that path unless a better idea strikes me.

[rc3105]

Congrats! First child? Hope you're caught up on your sleep for a while...

not so much, that ship sailed many moons ago. sleep??? thx for the reminder, knew there was something I'd been forgetting to do...