Hacking TivoHD with 2TB

[lrhorer]

I came home determined today and I believe I came to some solid conclusions...

TivoHD/Premier have a different partition table than the other Tivo's (duh).

I can't speak from experience on the Premier. Maybe Jamie or Omikron can chime in a bit more on that platform, but it is definitely not true of the TiVoHD. I don't think it is true of the Premier.

And, the tools used for other Tivo's don't work well with the HD/Premier partitions.

Again, I can't speak to the Premier, but I guarantee that is not the case with the THD. That is not to say those platforms (or even the S3) cannot make use of a scheme that is not supported under the traditional tools. Indeed, we know for a fact this is not the case, since the S3 - unhacked - can support a 2T drive, which is not possible with traditional tools. A hacked S3 can make use of a 2T drive using traditional tools, as well.

Some tools kind of work but they fail when it comes to accessing the drive.

I guarantee you that a stock THD drive, or one expanded by WinMFS or MFS_Tools can be accessed by a PC running Linux. MFS_Live should work, and has been reported to do so, but I cannot confirm personally that it works on anything other than an S3 with a stock drive, because I have never used it on anything else.

When I pdisk, fdisk, cat, dd the HD image created by jfms I get block 14 and drive doesn't exist or the partitions isn't valid errors. This is both with mfslive and PTV.

I don't know the details of what jmfs does, but a guess would be it converts the 32 bit Apple partition map into a 64 bit map. This would certainly explain the issues you are seeing. I suggest you not start with a partition map modified by jmfs. Perform your hacks first and get them all working, then try jmfs. That should work. An even better choice may be WinMFS. With a hacked kernel, it can create partitions larger than 1T

However, If I load the HD drive with WinMFS then all the above work just fine.

WinMFS does not load the drive. It merely reads the partition map and copies over the structure of each partition. It doesn't have to be able to read the contents.

The problem with this is I ended up with reboots every 20 minutes once the drive got over 300 gigs full (began using the 1.7T partition).

That makes sense.

So I tried using knoppix thinking I would be more in a linux environment.

What kernel version? Is this another live CD?

If I can just figure out a way to dd in Jamies kernel and the author file I could at least begin working from the Tivo itself. The question then becomes will busybox or wolfs all in one work with this foreign set of partitions...

Again, I suggest you try might try hacking an ordinary TiVo drive and get that working, then apply the jmfs upgrade. Unless jmfs inserts its own kernel (very unlikely), then your hacks should still work. A better solution, I think, is to use Jamie's hack that allows partitions larger than 1.1T. Then you can just use WinMFS to expand the drive. Either way, hack first, then expand.

I am considering using the 1TB drive from my S2DT knowing that a WinMFS created image will work fine so I should be able to get her up and running. I just really had my heart set on having a 2TB TivoHD but I guess my skills will have to greatly improve before I can leap that fence...

Not so much, really. I do suggest, however, that before you go too much further you consider whether you want to implement mike_s' scripted upgrades methods. Not only does his method allow for in-situ upgrades, but it also allows an easy recovery if /var gets wiped, which can happen from time to time. It also allows for a larger / and /var, making space for more applications and for backup files. Before attempting to implement his partition changes, I would say it is just about mandatory to use a regular Linux boot. I highly recommend it in any case, but trying to hand copy and update a partition map without dual windows (one showing the existing map and the other creating the map) would be an exercise in total frustration. It's not that creating a partition using pdisk is difficult, but errors are extremely easy, and without the ability to cut-and-paste... well, I certainly would not like to try transcribing all those long strings of numbers with no commas by hand. I made enough mistakes even with cut-and-paste.

My next attempt will be creating a drive using [URL="puffdaddy" http://www.dealdatabase.com/forum/showthread.php?63369-Extending-MFS-Tools-For-the-TiVo-Premiere-%28Support-Thread%29&p=308364[/url] tools to create the image and try jfms to expand it out. I don't know about all that deleting partitions and stuff but hopefully that was optional and not required. I figure using his tools would allow me to hack the drive then I can begin filling her up and see what happens.

OK, you are asking my advice, so I will give it again. I think you are headed in the wrong direction. I suggest you create a small Linux boot. I like Debian, but Ubuntu will work, and so should any other Linux distro with a 2.6.26 kernel or newer. Take a stock TiVo image and either work from it or else create a new partition map by hand using pdisk. Doing so, you can create a partition map that looks something like this:

HD_LivingRoom:/# pdisk -l /dev/hda

stat: mode = 060660, type=Block
size = 0, blocks = 0
HDIO_GETGEO: heads=255, sectors=63, cylinders=51329, start=0,  total=824600385
Partition map (with 512 byte blocks) on '/dev/hda'
 #:                type name                              length   base       ( size )
 1: Apple_partition_map Apple                                 63 @ 1          (  31.5K)
 2:                Ext2 Hack 1                            524288 @ 64         ( 256.0M)
 3:               Image Kernel 1                            8192 @ 524352     (   4.0M)
 4:                Ext2 Root 1                            524288 @ 532544     ( 256.0M)
 5:                Ext2 Hack 2                            524288 @ 1056832    ( 256.0M)
 6:               Image Kernel 2                            8192 @ 1581120    (   4.0M)
 7:                Ext2 Root 2                            524288 @ 1589312    ( 256.0M)
 8:                Swap Linux Swap                        262144 @ 2113600    ( 128.0M)
 9:                Ext2 /var                              524288 @ 2375744    ( 256.0M)
10:                 MFS MFS application region            589824 @ 2900032    ( 288.0M)
11:                 MFS MFS media region               216747008 @ 3489856    ( 103.3G)
12:                 MFS Second MFS application region     589824 @ 220236864  ( 288.0M)
13:                 MFS Second MFS media region        268617728 @ 220826688  ( 128.0G)
14:                 MFS Third MFS applicatioon region       4096 @ 489444416  (   2.0M)
15:                 MFS Third MFS media region        1465144320 @ 489448512  ( 698.6G)
16:          Apple_Free Extra                          975684336 @ 1954592832 ( 465.2G)

minus partitions 14 and 15, and of course with much more free. Note the increased sizes for partitions 2, 4, 5, 7, and 9. You can insert Jamie's kernel into both partitions 3 and 6 using dd. After creating the file systems on partitions 2, 4, 5, 7 , and 9 using mkfs, you can mount them and copy all the same data into partitions 4 and 7 to allow for easy recovery during the process using cp. If you decide to go this route - it really isn't difficult - we can guide you with help on the syntax. I might be available this weekend for phone support.

In either case, once you have all the hacks working, then you can try applying WinMFS or jmfs to expand. If you have a somewhat smaller drive - say 300G - you can get all the hacks onto it and then expand to a 2T drive using one of the expansion tools.

By the way, here is the partition map of my TiVoHD:

TiVo_HD:/# pdisk -l /dev/hda

stat: mode = 060660, type=Block
size = 0, blocks = 0
HDIO_GETGEO: heads=255, sectors=63, cylinders=56065, start=0,  total=900684225
Partition map (with 512 byte blocks) on '/dev/hda'
 #:                type name                         length   base       ( size )
 1: Apple_partition_map Apple                            63 @ 1          (  31.5K)
 2:                Ext2 Hack 1                       524288 @ 64         ( 256.0M)
 3:               Image Kernel 1                       8192 @ 524352     (   4.0M)
 4:                Ext2 Root 1                       524288 @ 532544     ( 256.0M)
 5:                Ext2 Hack 2                       524288 @ 1056832    ( 256.0M)
 6:               Image Kernel 2                       8192 @ 1581120    (   4.0M)
 7:                Ext2 Root 2                       524288 @ 1589312    ( 256.0M)
 8:                Swap Linux swap                   262144 @ 2113600    ( 128.0M)
 9:                Ext2 /var                         524288 @ 2375744    ( 256.0M)
10:                 MFS MFS application region       589824 @ 2900032    ( 288.0M)
11:                 MFS MFS media region          137630712 @ 3489856    (  65.6G)
12:                 MFS MFS application region 2     589824 @ 141120568  ( 288.0M)
13:                 MFS MFS media region 2        171919990 @ 141710392  (  81.9G)
14:                 MFS MFS App by Winmfs              2048 @ 313630382  (   1.0M)
15:                 MFS MFS Expanded by Winmfs   1639874560 @ 313632430  ( 781.9G)

[Soapm]

I have a 6 GIG drive I found. Will this work for Debian or Ubuntu? I'll have to find a place to down load it and I'm sure there will be some learning after install...

Wow this is fun!!!

PS... Where are mikes scripts? Is that where I got the code in the first post? All these threads mesh as one big blur after awhile...

[lrhorer]

I have a 6 GIG drive I found. Will this work for Debian or Ubuntu?

That's pretty thin, but if you load only the Gnome desktop and no extras, I think it would work.

I'll have to find a place to down load it

I'm not sure what you mean by that. Assuming you have a decent broadband connection, go to the Debian netinst web page, and download whichever .iso file suits your taste. Burn the .iso to a CD, insert it into the system you will be using, and boot the system. (Make sure it is connected to the internet.) Answer the questions as they come up, and when it asks for what features / packages you want pre-installed, only leave the Desktop Environment checked. Depending on your broadband connection speed, it will take something between about 30 minutes and 2 hours to complete the install. There will be a handful of questions asked during the install, but you can answer each group and then walk away for a while, coming back from time to time to answer the next batch of questions. Most of the questions can be answered with the default. Ask here if something puzzles you. When done, the system should boot into Linux with no real errors. You should have a default web browser (probably epiphany). With that, you can download all the files from this website. I suggest you open a console window as root and type

apt-get install vim

and I'm sure there will be some learning after install...

Not much, really. Not for what you are trying to accomplish.

Wow this is fun!!!

I'm glad you think so. If you are having fun right now, then I expect you will have few problems with Linux.

PS... Where are mikes scripts? Is that where I got the code in the first post?

No, that is the first of two scripts I submitted in post #39 of the "Overview of Hacking an S3" thread. His thread is here. Note I do not recommend you try to employ his methods from the outset. That would probably be too frustrating and confusing for you. I do, however, recommend you consider whether you want to at some point employ his methods. If you decide it is a good possibility, then you can create the partitions so you can do it at a later time.

[Soapm]

I got Ubuntu loaded but it freezes all the time. I've had to reload it about 5 times now and no matter what I do from opening the browser to attaching the Tivo drive and it freezes. After it freezes I no loger get the bar on the right and the mouse doesn't work so I reload it.

I went to load Debian but looks like it has to be compiled. I don't know anything about compiling software and I need this to work for all of 15 minutes. Just long enough to load the kernel and files onto the Tivo drive and that's it.

[lrhorer]

I got Ubuntu loaded but it freezes all the time. I've had to reload it about 5 times now and no matter what I do from opening the browser to attaching the Tivo drive and it freezes. After it freezes I no loger get the bar on the right and the mouse doesn't work so I reload it.

That drive may be too small. I think Ubuntu includes a lot of bells and whistles by default.

I went to load Debian but looks like it has to be compiled. I don't know anything about compiling software and I need this to work for all of 15 minutes. Just long enough to load the kernel and files onto the Tivo drive and that's it.

No, I don't know where you get that idea. All you have to do is select the correct platform to download (x86 for Intel or AMD for AMD), burn the .iso. and load the software. If any of the packages require a compile, it is done automatically and transparently to the user. Nothing is required of the user other than selecting the default options for just about everything but the root password and the user name and password for an admin user, plus selecting the default package groups to install. You downloaded the netinst for "Squeeze" (the current stable version), right? I haev a stripped down Debian system at my sister's house, and it takes 4.1G, but it started out somewhat larger.

BTW, one does not compile the whole OS in anycase. Compilig a package from source is pretty trivial, but unnecessary for a basic install in any case. Unless one has very special and specific needs, compiling the kernel is pretty much a thing of the distant past, in any case. With only a very few special exceptions, the default kernel compilation is fine for booting and almost everyting else can be loaded as a module. I haven't compiled a kernel in years,a nd it has been well over a year since I compiled any package from source, except for software I've written myself.

[Soapm]

BTW, one does not compile the whole OS in anycase. Compilig a package from source is pretty trivial, but unnecessary for a basic install in any case. Unless one has very special and specific needs, compiling the kernel is pretty much a thing of the distant past, in any case. With only a very few special exceptions, the default kernel compilation is fine for booting and almost everyting else can be loaded as a module. I haven't compiled a kernel in years,a nd it has been well over a year since I compiled any package from source, except for software I've written myself.

Sorry, I thought you had to compile it because I couldn't think of anything else you would do with 52 image files. http://cdimage.debian.org/debian-cd/6.0.3/i386/iso-cd/ Kind of intimidating to a novice at first glance.

I later stumbled across this page http://www.debian.org/CD/netinst/ which made more sense out of what I was seeing. I loaded it and so far it doesn't mount or recognize the drive either. I am now starting to suspect the image someone shared which restores via WinMFS. I am now getting invalid partition and block 14 errors even when freshly restored via WinMFS. I'm going to go begging for a different TivoHD image just to see if it makes a difference. From what I am reading and your very helpful responses there is no reason for all the trouble I am having getting this image to mount.

Of course, the problem could always be between the chair and the keyboard...

[Soapm]

You have to be kidding. This is frustrating as hell. I loaded debian but I can hardly see the screen because the picture is all jittery. I go to NVidia and downloaded the driver and have spent 3 hours now trying to install it. You have to be kidding me. All I see on the net is I need to use yast. No one seems to ever explain what yast is, where do I get yast, is it already on the system, WTF!!!

I cant believe people who use Linnux go through this just to install a driver. Installing this driver is the first thing I've attempted since install since I can't really see the screen.

Can some please help???

[Soapm]

Wow I can't believe this experience I am having.

I think I found YaST but can't install it because it says it's not trusted (important to have but not trusted. go figure).
so to fix that I found I had to run sudo
that won't run because I'm not listed as a member
tried to add myself as a member but I can't figure out how to save the file.

And all I want is to load a video driver because the screen is flickering so that I can hardly see it. Can someone please tell me there is a much easier way??? PLEASEEE someone point me to the light... This can't be the correct process just to load a video driver...

[lrhorer]

Sorry, I thought you had to compile it because I couldn't think of anything else you would do with 52 image files. http://cdimage.debian.org/debian-cd/6.0.3/i386/iso-cd/ Kind of intimidating to a novice at first glance.

No, that is the total distribution, including some 30,000+ free applications available to every user of the distro. Of course, most people don;t use anywhere nearly that many applications. I'm only using 29,999.
The distro in all its glory fits on 52 CDs or (IIRC) 8 DVDs. Most people don't worry with the entire distribution unless the PC won't have reliable broadband access. All of the applications in the distro (plus many more that are non-free or in individual downloads) are available using apt-get or one of the apt frontends like Synaptic.

I later stumbled across this page http://www.debian.org/CD/netinst/ which made more sense out of what I was seeing.

That's the link I provided just above.

I loaded it and so far it doesn't mount or recognize the drive either.

I presume you mean after running `tivopart r /dev/sdX`? I also presume you are not tring to mount the drive, correct? A raw drive can be mounted, but not a partitioned one. If the drive is partitioned, then you have to mount its partitions. In this case, one or more of /dev/sdX2, 4, 5, 7, or 9.

I am now starting to suspect the image someone shared which restores via WinMFS. I am now getting invalid partition and block 14 errors even when freshly restored via WinMFS.

You are getting this error when you try to mount one of the partitions, right?

I'm going to go begging for a different TivoHD image just to see if it makes a difference. From what I am reading and your very helpful responses there is no reason for all the trouble I am having getting this image to mount.

I have to go right now, but maybe I can help you via phone this evening.

Of course, the problem could always be between the chair and the keyboard...

That's always a possibility, but I suspect there may be something else going on here.

[lrhorer]

Can some please help???

Yeah, but give me a few hours.

[Soapm]

Thanks, I need to go blow off some steam because I'm one frustrated dude at this point.

This debian is incredibly tough to do anything on. I just want to load a video driver and I've been at it since 8 this morning. I couldn't run the tivopart for two reasons, 1. I can hardly see the screen and 2. I have no idea where to put it. And with this Sudo crap having everything locked down like a vault, I'm not sure I could run it even if I did know where it went. There should be a place I can turn off all this security just so I can get something done. I took today off work just to get this done and after an entire day I am still trying to see the screen...

thanks again...

[Soapm]

Thanks, I need to go blow off some steam because I'm one frustrated dude at this point.

This debian is incredibly tough to do anything on. I just want to load a video driver and I've been at it since 8 this morning. I couldn't run the tivopart for two reasons, 1. I can hardly see the screen and 2. I have no idea where to put it. And with this Sudo crap having everything locked down like a vault, I'm not sure I could run it even if I did know where it went. There should be a place I can turn off all this security just so I can get something done. I took today off work just to get this done and after an entire day I am still trying to see the screen...

thanks again...

[lrhorer]

Thanks, I need to go blow off some steam because I'm one frustrated dude at this point.

I understand that. Take a breath. Don't worry about where you want to go. Just concentrate on the next step in the path.

This debian is incredibly tough to do anything on.

Not really. It's much easier to get things done than in Windows. One just has to know a few basic tricks. 'Not many

I just want to load a video driver

'May not be necessary, but even if so, it's really not difficult.

and I've been at it since 8 this morning.

That's a long time. If you'll give me a few minutes, I'll walk you through it.

I couldn't run the tivopart for two reasons, 1. I can hardly see the screen

Annoying. We can either work around it or fix it.

I have no idea where to put it

It doesn't really matter. It can either be placed in the path, put anywhere you like and append the path variable to include the path where the binary is placed, put wherever you want and then create a link to the binary from a directory that is in the path, or simply supply the fully qualified file name.

And with this Sudo crap having everything locked down like a vault, I'm not sure I could run it even if I did know where it went.

Under Debian, you don't have to worry with sudo. Ubuntu does not allow logging in as root or su to root. Debian allows both.

There should be a place I can turn off all this security just so I can get something done.

There is, or just bring up a root console.

I took today off work just to get this done and after an entire day I am still trying to see the screen...

Yeah, you probably should have coordinated that. It is expected that you might run into small issues that require some more seasoned advice. Do you have a chat account, or a DMZ zone / port forwarding in your router?

[Soapm]

While trying to get the video driver running I did something that turned off the driver and has me in what looks like the 16 bit 600x800 mode. As long as it's not flickering I can deal with it.

This is what I get when I run Tivopart


root@Charity:/home/chuck/Downloads/bin# /home/chuck/Downloads/bin/tivopart.x86 r /dev/sda
(null): Bad data in block 14 from '/dev/sda'
fatal: invalid drive: /dev/sda

This data block 14 is what I get no matter what I use. It just won't let me read the partitions. I was thinking of loading winmfs on this linux machine and reimaging the drive but after spending the day just trying to load a video driver I can't imagine what it would take to try and load actual software.

[lrhorer]

Is everything working for you?