eSATA functionality

[Narf54321]

As every Tivo Series-3 owner knows, there is an external-SATA interface port on the back of the S3 unit. I was just poking around the StageA bootup directory on my S3 (software v8.0.1c), trying to verify the kickstart codes for another post. And this is what I found on my S3:


echo "Kickstart code 6 2 - initialize E-SATA drive"
do_esata_init=1


Very interesting. I wonder if Tivo has gotten hard drive divorces built into the system software, yet. There are a few telling splash images in the /tvbin directory.

[Jamie]

I wonder if Tivo has gotten hard drive divorces built into the system software, yet. There are a few telling splash images in the /tvbin directory.It's at least partially there. Try preparing an A+B drive using mfstools then plug only the A drive into the Series 3. You'll get the "LostDrive" screen, and if you proceed it will reconfigure it as a single drive system. Of course, that's a simple case where the B drive is added, but nothing is on it yet. They may not yet be dealing with the case of a populated B drive.

Anyone with a Series3 tried to hook an esata drive up? What shows up in the kernel log?

[Robre]

Hi guys,
I'm looking at getting one of these new boxes. But gee - with HD content I'm back at the hours where my old Tivo was 4 years ago. This eSata interface might really have potential. Has anybody been inside the S3 close enough to see what eSata disk controller chip they put on the motherboard. Tell me the manufacturer and model number and I might be able to shed some light on Tivo's future storage upgrade strategy.

Also (Narf54321) wrote "I wonder if Tivo has gotten hard drive divorces built into the system software, yet. There are a few telling splash images in the /tvbin directory."

Can you tell me a bit about these splash images? What are they "saying".

[Robre]

In regards to the splash screen - Found them. Thanks.

[Narf54321]

Also (Narf54321) wrote "I wonder if Tivo has gotten hard drive divorces built into the system software, yet. There are a few telling splash images in the /tvbin directory."

Can you tell me a bit about these splash images? What are they "saying".

Tivo uses a proprietary drive format and partitions known as MFS. The older model Tivos (which were IDE-based) coud usually support two drives, which would be "married" together. The problem being that, because you couldn't "divorce" the drives, if either drive failed you pretty much lost all your recorded shows and had to restore from scratch.

With this new external eSATA connection, if it is ever activated, it presents opportunity to add or remove an external drive. Removing a "married" drive is problematic, which is why I was curious if Tivo finally added official "divorce" support when you unplug an extra drive. The splash images (for on-screen display) seem to indicate they're at least thinking hard about it.

Anyway, testing this is still on my To Do List, especially as I have a shortage of extra SATA drives.

[Robre]

I'm on the same track - they seem to "plan" to do something different and they also have to consider some FCC "rule" that you can't take content away from a "closed" Tivo (kind of a DRM). They needed to do that to not get sued out of business (remember Replay).
Short of having access to an open S3 - can you could tell me what disk controller chip they are using. If it is what I guess it is I can tell you how they are going to make this work. I need the manufacturer and the model number. Picture of the area where the drive cables are connecting to the board would help too. - Thanks in advance.

[Jamie]

Short of having access to an open S3 - can you could tell me what disk controller chip they are using. If it is what I guess it is I can tell you how they are going to make this work. I need the manufacturer and the model number. Picture of the area where the drive cables are connecting to the board would help too. - Thanks in advance.The SATA controller is onboard in the bcm7038 (http://www.broadcom.com/collateral/pb/7038-PB01-R.pdf) chip. Lots of high res photos here (http://www.tivolovers.com/Photos/Series3-Review/). This (http://www.tivolovers.com.nyud.net:8080/Photos/Series3-Review/Large/Inside-16.jpg) one may have the detail in the area you want.

[bcc]

The SATA controller is onboard in the bcm7038 (http://www.broadcom.com/collateral/pb/7038-PB01-R.pdf) chip. Lots of high res photos here (http://www.tivolovers.com/Photos/Series3-Review/). This (http://www.tivolovers.com.nyud.net:8080/Photos/Series3-Review/Large/Inside-16.jpg) one may have the detail in the area you want.The source code for the driver is even on tivo.com. It doesn't look like it supports hot-swap like the latest esata controllers do.

From those splash screens, it looks like tivo is planning pretty rudimentary support of esata drives (you can't divorce without having recordings deleted on you). For folks who already know how to upgrade internal drives, the esata support doesn't seem like much of any benefit.

[Robre]

Thanks, Jamie - the BCM 7038 is a Dual HD Video chip. I'm still scanning the images and I can't find the ESATA controller. Do you know what's under the heat sink? Is that the main processor that runs the box or ... I see the traces from the internal sata drive running toward it.

[Jamie]

Thanks, Jamie - the BCM 7038 is a Dual HD Video chip. I'm still scanning the images and I can't find the ESATA controller. Do you know what's under the heat sink? Is that the main processor that runs the box or ... I see the traces from the internal sata drive running toward it.Read the spec sheet I linked to -- the bcm7038 is an all in one chip and includes the SATA controller. The bcm7038 is what is under the heat sink and is the main processor, the mpeg2 decoder, the SATA controller, the usb2 controller, etc, etc. Again, read the datasheet.

[Robre]

Thanks - I was hoping to find a dedicated chip. In this case I can only speculate how they might try to manage the external eSata drive. Check out this site: http://www.siliconimage.com/products/product.aspx?id=108

Look at the diagram of the storage processor and then download the White Paper Capacity Expansion. Tell me what you think.

[Jamie]

Look at the diagram of the storage processor and then download the White Paper Capacity Expansion. Tell me what you think.It sounds like they've basically put logical volume management onto the controller chip.

I don't see how any of that driver virtualization stuff helps on the tivo. Whether the drive(s) appear as one aggregate virtualized drive, or individual drives doesn't really matter. Either way, the tivo needs to expand or contract it's Media File System (MFS) in response to drive additions, subtractions or size changes, and there is no way the drive controller chip is going to do that for it given that the file system is a custom proprietary TiVo file system.

I suspect it wouldn't be that hard for a hacker to get eSATA working on the S3 if they really wanted to, in the same way that dual drives currently work on the S2. It might mean a custom kernel, and/or some tivoapp patches.

[Robre]

Future will show. IF Tivo would have used the SI chip I know the drive addition would be transparent. W/o the Tivo has 250 GB space and with an "appropriate" (maybe they except any eSata) external drive the Tivo would see 1 T. With the Broadcom chip doing it all - including Sata management - we have to see. Maybe they [BC] licensed the technology. I'll be watching the development very closely. Tivo might also have changed their file system that manages file access. If they have found a better storage system that gives them size AND file locking AND keeps the networks happy they will switch to that tech.

[Narf54321]

I don't think Tivo is ready to ditch MFS anytime soon. Most likely, to appease the CableLabs folks, they'll keep the recording meta-data on the internal drive while allowing the raw data of the actual "show" to be on the external device. All the info for finding the right chunks would be stored on the internal drive partition(s).

Anyway, I believe Jamie has mentioned before the hard limit of MFS is about 2TB. The practical limit seems to be closer to around 1TB, based on some of the Tivo "RAID" projects I've read about.

Drives are going to reach 1TB this year, so anyone willing to replace the internal S3 hard drive will max out the box. At that point, worrying about the eSATA port may be moot.

[Jamie]

IF Tivo would have used the SI chip I know the drive addition would be transparent. Not without a complete redesign of their file system.

If they have found a better storage system that gives them size AND file locking AND keeps the networks happy they will switch to that tech.The have the equivalent of drive locking via the crypto chip. All A/V data written to disk is encrypted and tied to the crypto chip in a specific box. There are size limitations inherent in the current MFS file structure. The MFS limit is 2TB. Some have said that the kernel limits a single drive (logical or physical) to 1TB, but may people run dual 750GB drives on S2 tivos. Eventually the size limitations inherent in the file system will have to be addressed.

Making the content providers and distributers happy is a whole 'nother ball of wax. Has Silicon Image recieved CableLabs certification for their device?

[spike2k5]

Kickstart 62 seems to have formatted and activated my eSATA drive.
zonemaps, devlist, min. alllocation size look good.
Hours are reported correctly as well under system information.

It's tested on software v 8.1.1 on my unhacked, unmodified S3 Tivo.

[monkeyboy1010]

Engadget has picked up the story, and is running it with instructions and pictures. Nice job spike2k5!

how to use your tivos series3 esata port to add an external drive/ (http://www.engadget.com/2007/05/04/how-to-use-your-tivo-series3-esata-port-to-add-an-external-driv/)

[bcc]

Note that the kick-start support for ESATA was mentioned way back in Sept.:
http://www.dealdatabase.com/forum/showpost.php?p=265454&postcount=51
http://www.dealdatabase.com/forum/showpost.php?p=265424&postcount=48

Since nobody bothered to try it 'till now I thought there wasn't much interest for unsupported ESATA. Guess I was wrong. :)

[megazone]

Spike - I've seen you posts elsewhere, and I'm curious about the report that Kickstart 62 only works with an unmodified TiVo. Have you determined what kind of check is being performed for the unit to tell it has been updated? I also saw your mention that there is a way around this - do you have more details you can share?

[ctsshack]

If I had to guess, it would probably be the same method of doing a two drive system on a HR10-250 in that you expand the internal drive when upgrading(not use mfsadd but do a mfsbackup and then a mfsretore with the expand option) and then add the external drive. If memory serves me correctly, I believe it's due to limit of the total number of partitions (across multiple drives). Using mfsadd on the internal drive creates problems.

-ctsshack

[spike2k5]

Spike - I've seen you posts elsewhere, and I'm curious about the report that Kickstart 62 only works with an unmodified TiVo. Have you determined what kind of check is being performed for the unit to tell it has been updated?
During the process, it checks for superblock devlist and if it sees anything other than it's usual paritition list, it flags it.

"Unexpected devNamesM:/dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13 /dev/hda14 /dev/hda15"


I also saw your mention that there is a way around this - do you have more details you can share?
It's being beta tested and will let folks know when ready. It seems to work fine so far.

[megazone]

Awesome - this opens the door for a 2TB Series3. Which is the most the OS can handle, as I understand it.

I know a number of the kick start codes, but I've been looking for a complete list to see what others (like 62) I was unaware of. So far I haven't found a solid list.

[spike2k5]

I know a number of the kick start codes, but I've been looking for a complete list to see what others (like 62) I was unaware of. So far I haven't found a solid list.

Kickstart code 6 2 - initialize E-SATA drive
Kickstart code 5 1 - software upgrade
Kickstart code 5 2 - emergency reinstall
Kickstart code 5 3 - BER testing
Kickstart code 5 4 - HDD test
Kickstart code 5 5 - Tuner/AFT test
Kickstart code 5 6 - software install
Kickstart code 5 7 - force MFS check
Kickstart code 5 8 - perform MFS cleanup
Tray open code 111 - will open tray

Awesome - this opens the door for a 2TB Series3. Which is the most the OS can handle, as I understand it.

One dude is running 750GB internal & 750GB eSATA = 198HD or 1878 SD hours. :D

[Narf54321]

I know a number of the kick start codes, but I've been looking for a complete list to see what others (like 62) I was unaware of. So far I haven't found a solid list.

Well, I see spike2k5 nabbed my compiled list from TCF and posted it here in my own thread -- you sneaky guy, you. If you hack your Tivo or pull the drive the complete list shown above is from /etc/rc.d/StageA_PreKickstart/rc.Sequence_700.CheckForPanic.sh. The "6 2" entry is (obviously) the newest addition.

What'll really blow your mind is setting up your own kickstart panic code, as tivomaster showed us with stoploop (a.k.a. hackemr) (http://www.dealdatabase.com/forum/showthread.php?t=37326). Very useful if you're trying a new hack or tivoapp patch and accidentally get into a reboot loop.

[spike2k5]

Well, I see spike2k5 nabbed my compiled list from TCF and posted it here in my own thread -- you sneaky guy, you. If you hack your Tivo or pull the drive the complete list shown above is from /etc/rc.d/StageA_PreKickstart/rc.Sequence_700.CheckForPanic.sh.
Actually list is from my own post from the other forum. I know how to mount a tivo drive and use grep too you know. :) I thought I was answering a question directed at me but it's your thread so maybe I should stay away.

[Narf54321]

Actually list is from my own post from the other forum. I know how to mount a tivo drive and use grep too you know. :) I thought I was answering a question directed at me but it's your thread so maybe I should stay away.

Nah, I just thought it was amusing after all these years people are still trying to figure out "the definitive list" of kickstart codes. There was this one thread where the fella wanted to keep rebooting his tivo and try every 2-digit code, and I'm all like "look at the software, dude!"

Keep up the good work. This new rush of eSATA experimentation is fun the read.

[lrhorer]

I must admit I am not familiar with eSATA drives or especially eSATA drive enclosures. Do any of you who have already upgraded your S3 with an external enclosure have specific recommendations? Many people seem to be recommending the Seagate DB35 drives, but unless I am much mistaken, that is a bare drive, correct? I would like an economical external drive or drive + enclosure solution. I've noticed some of the external drives or drive enclosures I have seen advertise having both eSATA and USB ports. Do some of these enclosures utilize EIDE or ordinary SATA drives and have a conversion board inside? We might well be abel o come across an EIDE or SATA drive cheaper than an eSATA drive, or it might be more available or quieter.

[Narf54321]

The Seagate DB35 is indeed just the bare drive. It's billed by Seagate as a DVR drive. I think its popular for its relatively quiet operation and pretty good reliability. Some folks like the Samsung SpinPoint drives, also for quietness, although I don't know about the reliability. The Tivo Series-3 comes with a Western Digital. I think WD reliability has gotten better lately, but the drives tend to run loader and hotter.

Over at TCF the folks seem to like the Antec MX-1 eSATA case. Its more money but seems to actually perform well, unlike a lot of the cheap knockoff enclosures. I'm not thrilled about the plastic construction, but it does have a large, fairly quiet active cooling fan.

A "real" eSATA enclosure uses a SATA drive inside, and pretty much connects the external port directly to the drive's SATA connection. A good case should have an "I" connector -- and preferably include the cable. I'd avoid any case which uses EIDE (i.e. PATA) drives, and leery of the rest which advertise USB which usually means they use some sort of conversion board trickery.

[bcc]

leery of the rest which advertise USB which usually means they use some sort of conversion board trickery.These days, a good percentage of esata enclosures also throw in a usb port, seemingly without additional cost. Yes, they have a bridge board, but the bridge should just be sata->usb in this case. As long as the inside connector is sata and the external connector is esata, I wouldn't expect the sata side to be involved in any bridging.

I think folks focus too much on going cheap - if you look at reviews lots of users have had troubles with the quality of power supplies. I'd also be more concerned about cooling efficiency. For the 2 enclosures I have, I went passive cooling, with aluminum cases, but 500gb+ drives run a bit hot. I'd probably look for something like thermaltake's n0012usu enclosure if I was shopping for a new enclosure.

[lrhorer]

Over at TCF the folks seem to like the Antec MX-1 eSATA case. ...and leery of the rest which advertise USB which usually means they use some sort of conversion board trickery.
The MX-1 has a USB port, you know.

[lrhorer]

Well. since this is going in an external enclosure, I don't see that the low spin-up current is an issue, and Seagate bills their own Barracuda drives as the quietest in the industry, so I'm going to give a whack at an MX-1 enclosure with a 750G Barracuda drive in it. I just ordered both from Compusa at $60 for the MX-1 and $280 for the drive. I'll let everyone know how it goes. (In know I couldhave gotten them a bit cheaper from other vendors, but I like the fact there's a Compusa right down the block if I have any problems with the units.)

[bcc]

Well. since this is going in an external enclosure, I don't see that the low spin-up current is an issue, and Seagate bills their own Barracuda drives as the quietest in the industry,Ordinary desktop seagate sata drives are the noisiest in my experience. See also the maxtor/quantum patent on acoustic management, which used to leave seagate SOL when it comes to tuning the acoustic vs performance state of the drive.

[jonbig]

Of course Maxtor is now part of Seagate, so the quiet technology can appear across the Seagate line now.

[bcc]

Of course Maxtor is now part of Seagate, so the quiet technology can appear across the Seagate line now.
And of course, the seagate baracuda line was architected before the acquisition and so it would not be safe to assume they retrofitted quiet drive technology in the short period since then.
For the seagate 750gb, http://www.silentpcreview.com/article611-page2.html, some highlights:

"Unfortunately, AAM (Automatic Acoustic Management) is not supported...
All four of the high capacity drives were relatively loud at idle...
The implementation of Automatic Acoustic Management (AAM) to reduce the seek noise is sorely missed..."

Still noisy like the 7200.9 series it seems. Poor choice for the living room IMO.

[edentel]

I tried out the Maddog Megavault as an experiment in reuse of my IDE DB35 pulled from my HR10-250. The enclosure is advertised as having internal IDE and SATA connectors with external SATA and USB. Since I had the drive, and COMPUSA down the road carried the enclosure I figured why not....

Well after 4 or 5 shots at trying to get the TiVo to recognize the drive without luck I remembered my new pc motherboard had E-SATA connectors. Sure enough my PC wouldn't find it either. Flipped the drive caddy switch to USB and the PC finds it fine. It would appear the SATA to IDE portion doesn't work as alluded to.

Just information for anyone else who notices it on the shelf.....

[lrhorer]

Oh, this is so cool! The kickstart code worked fine and now I'm showing 132 hours of HD capacity, and it's so much less worrisome than opening the enclose, pulling the drive, putting it in an external PC, marrying the drives, and installing everything back in the TiVo. 'Much less hassle and time, too.

By the way, at least in the MX-1 housing, the drive is very quiet - certainly a lot quieter than the fan in the projector or even the air conditioning when it's running. (In Texas, it's pretty much always running.) I can certainly hear it if I sit right next to the drive in a perfectly quiet room, but 12 feet away in a room that is not sound-proofed I cannot say with any authority I can hear it at all.

[lrhorer]

Well, this is worrisome. I turned on the video system today to watch a bit of TV, and the TiVo's response to remote commands was rather sluggish. The system seemed to freeze several times while browsing the Now Playing list. Then I selected a program and deleted it, only to be greeted by the GSOD. "Oh, darn!" I said (or something like that, anyway). Of course at this point there were only really two choices: turn off the TV and do something else or sit and stare at a white on green text screen for several hours. Of course, I had no real expectations of the GSOD working, as I have never seen it actually work before, but I dithered around doing a few simple chores for about 20 minutes. Just out of fatalistic optimism, I checked the TiVo only to be greatly surprised at seeing the power up screen! The TiVo finished booting normally, and not only is the external drive still there, but all the recordings seem to be intact. I hope this isn't something systemic related to the addition of the drive, though.

Has anyone else been greeted by the GSOD one or two days after adding the external drive?

[lrhorer]

I had another GSOD a couple of days later, but none since. The symptoms were pretty much the same. The arrow key response became very sluggish. There were two programs recording, and when they both finished, the TiVo rebooted to the GSOD and recovered in about 10 minutes. It's been working fine, since.

[Jamie]

Thanks - I was hoping to find a dedicated chip. In this case I can only speculate how they might try to manage the external eSata drive. Check out this site: http://www.siliconimage.com/products/product.aspx?id=108

Look at the diagram of the storage processor and then download the White Paper Capacity Expansion. Tell me what you think.Rumor has it the new tivoHD is using a relative of this chip for its SATA controller, the SIL5723 (http://www.siliconimage.com/products/product.aspx?id=103).

[lrhorer]

TiVoHD? Is this a new offering from TiVo? Any details?

[Jamie]

TiVoHD? Is this a new offering from TiVo? Any details?Lots of discussion over at TCF. Of course, you could check tivo.com (http://www.tivo.com/) where it is featured prominently. Or the review here (http://www.tivolovers.com/a-review-of-the-tivo-hd-digital-media-recorder/), one on cnet (http://reviews.cnet.com/digital-video-recorders-dvrs/tivo-hd/4505-6474_7-32511935.html?tag=cnetfd.mt&tag=nefd.only) etc.

[megazone]

Rumor has it the new tivoHD is using a relative of this chip for its SATA controller, the SIL5723 (http://www.siliconimage.com/products/product.aspx?id=103).
Indeed it is - the BCM7401 is connected to a SiI5723, which, in turn, leads to the two SATA headers in use. (For the internal drive and the eSATA port.)