HR10-250 upgrading - can I do it twice?

[seanmcgpa]

Hey guys... I need some advice from the experts, please.

A couple years ago, I bought an HR10-250 HDTivo (with the original 250GB drive). I used the LBA48 CD to add another 250GB drive. Two years of joy.

Now I'm getting symptoms of HD failure ... random reboots, I can hear one of the HDs make a soft "click" noise occasionally. Shows are recording a minute late into a program, etc.

I purchased two 2 500GB PATA drives as replacements and need advice on how to proceed. I would love to copy my [2x250] -> [2x500] and then use the additional space, but is this possible since I've already added a drive once?

Or do I need to make a backup of just the HR10-250 file system (without recordings) to move over to the new drives? I had made a backup of the original file system two years ago, but I cannot find the tivo.bak image anywhere.

I have an old computer with a SCSI cd rom and 4 EIDE ports, so in theory I could make a backup of the original drives and save my recordings ....

Any advice on how to proceed is greatly appreciated. I am getting a lot of aggro from the wife regarding Lost (rebooting in the middle, lost half the episode), and usabillity etc.

[PlainBill]

Whoo! boy!!! I hope you have a lot of spare time. There is no fundamental barrier, but it might take a little tweaking - and a LOT of time.

Expanding the original drive to a 500 gig doesn't present a problem. The original partition table should have room for two more entriens, and you have plenty of space for a larger swap partition. The second drive should also not be a problem; it only has two partitions, plenty of room in the table for two more.

The the first problem is mfsrestore does not properly initialize a swap partition > 127 Megs, and you need a 500 Mbyte swap partition. Instructions for initializing it are here (http://www.dealdatabase.com/forum/showthread.php?p=233561).

I'd suggest using this interactive guide (http://tivo.upgrade-instructions.com/index.php) to set up the copy process. Expect it to take more than a day to copy the two drives - that's the second problem.

PlainBill

[seanmcgpa]

Thanks for the reply. That seems a lot more difficult than I expected.

Since I have an HR10-250 with two drives now and no original backup, can I make a backup of the file system (without recordings) and then expand it onto two new 500GB drives?

[ocntscha]

Whoo! boy!!! I hope you have a lot of spare time. There is no fundamental barrier, but it might take a little tweaking - and a LOT of time.It will take a lot of time but I noticed when I was using the ptvupgrade CD not long ago it had only enabled DMA transfers on 1 of the 2 drives I was working with. DMA transfers are going to make a HUGE HUGE difference in your overall copy time. Once you've booted off the ptvupgrade CD you can check whether DMA is enabled for a particular drive with the command

hdparm -d /dev/hdX

and to enable it..

hdparm -d1 /dev/hdX (X being a,b,c, or d as appropriate)

Any 250 Gig or 500 Gig drive is going to support DMA transfers. I didn't time it but as I recall it took about an hour, maybe a little more, to dd an 80Gig drive onto a larger one. That 80Gig drive was an original 5400RPM Tivo/Maxtor model too, not exactly a speed demon.

[PlainBill]

It will take a lot of time but I noticed when I was using the ptvupgrade CD not long ago it had only enabled DMA transfers on 1 of the 2 drives I was working with. DMA transfers are going to make a HUGE HUGE difference in your overall copy time. Once you've booted off the ptvupgrade CD you can check whether DMA is enabled for a particular drive with the command

hdparm -d /dev/hdX

and to enable it..

hdparm -d1 /dev/hdX (X being a,b,c, or d as appropriate)

Any 250 Gig or 500 Gig drive is going to support DMA transfers. I didn't time it but as I recall it took about an hour, maybe a little more, to dd an 80Gig drive onto a larger one. That 80Gig drive was an original 5400RPM Tivo/Maxtor model too, not exactly a speed demon.

So at a rough guess, he's looking at 6 hours, maybe less. That's a LOT better than I expected. Thanks for the information on enabling dma.

PlainBill

[seanmcgpa]

So at a rough guess, he's looking at 6 hours, maybe less. That's a LOT better than I expected. Thanks for the information on enabling dma.

PlainBill

Bill,

Thanks for replying.. but I am still not sure I can even do this. Should I try the [2x250] -> [2x500] even though I had added an addition 250 once already?

I guess I don't understand the whole file system well enough to understand the complexities of partitions under Linux, etc. and how that will effect me. I would hate to copy over 2x250 and waste 500GB of space.

[PlainBill]

Bill,

Thanks for replying.. but I am still not sure I can even do this. Should I try the [2x250] -> [2x500] even though I had added an addition 250 once already?

I guess I don't understand the whole file system well enough to understand the complexities of partitions under Linux, etc. and how that will effect me. I would hate to copy over 2x250 and waste 500GB of space.
First the basics. The flavor of Linux used on the TiVo has a partition table with room for 16 entries - in other words, you can have a maximum of 16 partitions on a drive. A 'stock' tivo drive has 13 partitions; and when you move to a larger drive it adds two MFS partitions. Adding a second drive does not affect the partition table on the first drive. Thus, if you wanted to expand a single 250 Gig drive to a 500 gig drive, it would be easy - Use mfsbackup/restore to copy the original drive to the new drive and expand to use the full capacity. Similarly, adding a second drive to a one drive system is easy - you've already done it, you know how easy it is.

Now for the warning - I've never worked with two drive sets this large.
The first problem is MFStools cannot backup / restore / expand one drive from a two drive set. There are three ways around this.

One is to work with a computer that can support five drives. Four of them are the original and new TiVo drives, the fifth is the CDROM or equivalent drive used to boot the system. You would do a backup from say the primary master/ slave set and simultaneously restore and expand to the secondary master / slave set. I am certain this will work - I did it for my DSR6000.

The next approach is more uncertain. It has been reported that mfsadd looks at the true capacity of a drive, not the information in the partition table. I have never confirmed this. If it is true, you can use the dd command to individually copy the original drives to the new drives. This would give you a pair of 500 gig drives that the TiVo would think are 250 Gig drives - as you said, not what you want to have. You would then run mfsadd to add the extra space. (It might be necessary to tweek the partition tables to show the additional space). It also results in a swap partition that is too small, which might come back to bite you.

A third approach is put the original and new 'a' drives in a computer, boot from a linux tools cd, list the partition table on the 250 Gig drive, create the same size partitions on the 500 Gig drive (except for the larger swap partition), then use dd to copy the partitions from the original to the new drive. Repeat for the 'b' drives. The boot parameters will also have to be updated, etc. Again, I am sure this will work, but it requires a lot of work. I am not sure you have the skills and patience to do it.

PlainBill

[dave868]

I have a Samsung SIR-S4120R. Over the past few years I (with the help of this forum) I upgraded to 6.2 via slices, enabled tivoweb, HMO and all the marvelous hacks I could find. I am very happy with it as well as my hughes (similarly hacked).

I was happy till I saw the great deal on the 500 gb drive for $199 in the best deal ever section. I ordered it and then realized that I can't re-upgrade the drive. At least I don't think I can. Here is my partition table from the samsung:

Partition map (with 512 byte blocks) on '/dev/hda'
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1
2: Image Bootstrap 1 1 @ 102467648
3: Image Kernel 1 8192 @ 102467649 ( 4.0M)
4: Ext2 Root 1 262144 @ 102475841 (128.0M)
5: Image Bootstrap 2 4096 @ 102737985 ( 2.0M)
6: Image Kernel 2 4096 @ 102742081 ( 2.0M)
7: Ext2 Root 2 262144 @ 102746177 (128.0M)
8: Swap Linux swap 260096 @ 103008321 (127.0M)
9: Ext2 /var 262144 @ 103268417 (128.0M)
10: MFS MFS application region 1048576 @ 103530561 (512.0M)
11: MFS MFS media region 102467584 @ 64 ( 48.9G)
12: MFS Second MFS application region 1048576 @ 104579137 (512.0M)
13: MFS Second MFS media region 128942080 @ 105627713 ( 61.5G)
14: MFS New MFS Application 1024 @ 234569793
15: MFS New MFS Media 156147712 @ 234570817 ( 74.5G)
16: Apple_Free Extra 3439 @ 390718529 ( 1.7M)

I am no expert. But the way I understand it is that I don't have enough partition entries available to grow it again. Is that correct?

Can I use mfstools with the -s switch to backup and then mfsadd after restore?

Or am I just SOL?

Thanks for your help

[PlainBill]

You are correct - no more partitions are available. You have three options. This thread (http://alt.org/forum/index.php?t=msg&goto=769&rid=0&S=7685e15b700295296be568c78ad27ee8#msg_769) explains a process for combining partitions to allow expanding onto a larger drive. Note this is fairly advanced stuff.

You can back up all your recordings to a computer, shrinking an image of the expanded drive and restoring it to the new drive, then restoring the recordings.

You can make an image of the drive including using the -s (shrink) switch, the restoring it to the new drive. This has the disadvantage that you lose your recordings.

PlainBill

[dave868]

You are correct - no more partitions are available. You have three options. This thread (http://alt.org/forum/index.php?t=msg&goto=769&rid=0&S=7685e15b700295296be568c78ad27ee8#msg_769) explains a process for combining partitions to allow expanding onto a larger drive. Note this is fairly advanced stuff.

You can back up all your recordings to a computer, shrinking an image of the expanded drive and restoring it to the new drive, then restoring the recordings.

You can make an image of the drive including using the -s (shrink) switch, the restoring it to the new drive. This has the disadvantage that you lose your recordings.

PlainBill

Thanks for the help. One more question. I thought I read somewhere that the shrink option could preserve recordings if they would fit in the factory partitions. Since my original drive was a 120 gig, if I made sure there was less than that on the drive and used the shrink option, would that free up the partitions I added when I expanded. (Or am I just showing my ignorance?)

Thanks as always.

[PlainBill]

I'd hesitate to say 'ignorance'; 'lack of experience' seems much less harsh. The -s option will preserve recordings if they ARE IN the original factory partitions. If you have even a 5 minute clip in the added partition and plenty of space in the other partitions, it won't get moved.

PlainBill

[dave868]

I'd hesitate to say 'ignorance'; 'lack of experience' seems much less harsh. The -s option will preserve recordings if they ARE IN the original factory partitions. If you have even a 5 minute clip in the added partition and plenty of space in the other partitions, it won't get moved.

PlainBill

Thank you for your kind words.

That makes sense. Is there anyway to determine what partition a recording is in? If there were recordings in the added partition would I just loose those recordings or would they all be hosed?



I also had another idea that may work. Let me know if this makes sense at all.

I have another networked tivo.
I use mfstools with the shrink option not taking any recordings.
I then restore that to the 500 gig drive and install in second tivo.
I then use HMO to move desired recordings to the newly updated tivo.

I know that is is a time consuming manual process... but it requires very little technical skill. Unless of course this hair brained idea wouldn't work at all.

[captain_video]

I recently performed a drive swap for a friend of mine that had an HR10-250 with dual 250GB drives. One of the drives was going bad so I thought I'd try a dd transfer, which I had never actually performed before to back up an entire drive to a new one. I set it up without enabling DMA and let the first drive copy overnight on an old Celereon 300A PC (overclocked to 450MHz). It was done when I checked it first thing in the morning. I copied the 2nd drive and it only took about four hours, which surprised the heck out of me since I expected it to take 2 or 3 times as long at a minimum. I tested every single recording on the HDTivo, of which there were well over 100 hours worth (mostly standard def recordings), and they all showed the correct length and played back fine so I'm sure they were all copied intact and I haven't heard otherwise from my friend.

The bottom line is that I had always thought that doing a dd copy was an extremely long process but was surprised to see that it didn't take all that long at all, but YMMV. My PC wouldn't beat a slug in a race compared to today's PCs but it works quite well for what I need it for (i.e., hacking and upgrading Tivos). I would expect enabling DMA transfers would speed things up but it didn't seem to make any difference in my case.

[PlainBill]

Thank you for your kind words.

That makes sense. Is there anyway to determine what partition a recording is in? If there were recordings in the added partition would I just loose those recordings or would they all be hosed?
I don't know if there is any way to determine that.


I also had another idea that may work. Let me know if this makes sense at all.

I have another networked tivo.
I use mfstools with the shrink option not taking any recordings.
I then restore that to the 500 gig drive and install in second tivo.
I then use HMO to move desired recordings to the newly updated tivo.

I know that is is a time consuming manual process... but it requires very little technical skill. Unless of course this hair brained idea wouldn't work at all.
HMO isn't an option on 3.1.5x. So you can't use that approach.

But your basic idea was sound. I've done something similar, and if I had known you had a second HR10-250 I would have suggested using mfs-ftp to transfer the recordings.

PlainBill

[dave868]

I don't know if there is any way to determine that.


HMO isn't an option on 3.1.5x. So you can't use that approach.

But your basic idea was sound. I've done something similar, and if I had known you had a second HR10-250 I would have suggested using mfs-ftp to transfer the recordings.

PlainBill

I think I am going to take that route. My only question I have left is about the swap. How big should it be? Based on what I have read a 500gig drive would need a 250meg swap... but isn't there an issue with a swap bigger than 127?

[PlainBill]

I think I am going to take that route. My only question I have left is about the swap. How big should it be? Based on what I have read a 500gig drive would need a 250meg swap... but isn't there an issue with a swap bigger than 127?
You are correct; a 500 Gig drive probably should have a 250 Meg swap partition. MFStools will create it but won't initialize it properly. Read this thread (http://www.dealdatabase.com/forum/showthread.php?p=233561) for instructions on initializing it.

PlainBill

[dave868]

You are correct; a 500 Gig drive probably should have a 250 Meg swap partition. MFStools will create it but won't initialize it properly. Read this thread (http://www.dealdatabase.com/forum/showthread.php?p=233561) for instructions on initializing it.

PlainBill

I just wanted to make sure. I had actually found that thread on my own, but I really appreciate the response.

[dave868]

You are correct; a 500 Gig drive probably should have a 250 Meg swap partition. MFStools will create it but won't initialize it properly. Read this thread (http://www.dealdatabase.com/forum/showthread.php?p=233561) for instructions on initializing it.

PlainBill

OK... Here's what I did

I backed up my drive with the shrink option.

mfsbackup -f 9999 -6so /mnt/dos/62base.bak /dev/hdc

I then restored to the 500 gig drive, expanding the space and creating a 250 meg swap.

mfsrestore -s 250 -xzpi /mnt/dos/62base.bak /dev/hdc

I initialize the swapfile

tpip --mkswap --swapped /dev/hdc

All seems right with the world untill I try to boot the tivo. I get to the "Almost there" screen then reboot. I then get to a green severe error screen then reboot... then almost there... then reboot..... I think you get the idea.

Could it be the swapfile? What if I just stick with the 127? Is it really a big risk? I am tempted just to try the 127 swap just to see if that is the cause.

Any suggestions are apprecitated.

[ocntscha]

I believe if you simply change this..

mfsrestore -s 250 -xzpi /mnt/dos/62base.bak /dev/hdc
to this..mfsrestore -s 250 -r4 -xzpi /mnt/dos/62base.bak /dev/hdceverything will seem right in the world again.

[PlainBill]

OK... Here's what I did

I backed up my drive with the shrink option.

mfsbackup -f 9999 -6so /mnt/dos/62base.bak /dev/hdc

I then restored to the 500 gig drive, expanding the space and creating a 250 meg swap.

mfsrestore -s 250 -xzpi /mnt/dos/62base.bak /dev/hdc

I initialize the swapfile

tpip --mkswap --swapped /dev/hdc

All seems right with the world untill I try to boot the tivo. I get to the "Almost there" screen then reboot. I then get to a green severe error screen then reboot... then almost there... then reboot..... I think you get the idea.

Could it be the swapfile? What if I just stick with the 127? Is it really a big risk? I am tempted just to try the 127 swap just to see if that is the cause.

Any suggestions are apprecitated.

In addition to ocntscha's excellent suggestion, I suggest you make a new image. While -f 9999 is valid for Series 1 systems; it does not capture all loopsets for Series 2 systems. I suggest using -l 50 instead.

PlainBill

[dave868]

In addition to ocntscha's excellent suggestion, I suggest you make a new image. While -f 9999 is valid for Series 1 systems; it does not capture all loopsets for Series 2 systems. I suggest using -l 50 instead.

PlainBill

Having trouble with the -l 50. When I use that switch the image does not shrink to the needed 39 gig (So I can re-expand to 500... which is actually the reason for this exercise.) If I lower the limit (-l 1) it does shrink to 39 gig. Do I need to keep the limit at 50?

Thanks again for all your help.

[Jamie]

Having trouble with the -l 50. When I use that switch the image does not shrink to the needed 39 gig (So I can re-expand to 500... which is actually the reason for this exercise.) If I lower the limit (-l 1) it does shrink to 39 gig. Do I need to keep the limit at 50?

Thanks again for all your help.Run "mfs_info -s" to find out the lowest possible -l and/or -f setting that will still preserve your loopset animations. mfs_info is part of mfs-utils (http://www.dealdatabase.com/forum/showthread.php?t=39487).

In my experience, -f 9999 is generally fine for stock factory drives. It runs into problems on "hacked" boxes that have had loopset slices manually installed. These manually installed loopsets can end up with high fsids, so thresholding on size rather than fsid number works better on these images. In some cases, it won't be possible to "shrink" with these images, if the loopsets were installed in the exapansion partitions.

[dave868]

Run "mfs_info -s" to find out the lowest possible -l and/or -f setting that will still preserve your loopset animations. mfs_info is part of mfs-utils (http://www.dealdatabase.com/forum/showthread.php?t=39487).

In my experience, -f 9999 is generally fine for stock factory drives. It runs into problems on "hacked" boxes that have had loopset slices manually installed. These manually installed loopsets can end up with high fsids, so thresholding on size rather than fsid number works better on these images. In some cases, it won't be possible to "shrink" with these images, if the loopsets were installed in the exapansion partitions.

I must be missing something. When I do mfsinfo with the -s flag i get an illegal seek error. Without the switch I get a report of the mfs partitions. And it claims that it can be upgraded 3 more times. I am not sure how to take that.

[cheer]

Hmmm...I just tried it on my HR10-250 and got back lots of output, including the max stream fsid (93) and max stream size (10MB). Are you running the latest mfs-utils? Check that...if that's OK, I dunno.

[Jamie]

I must be missing something. When I do mfsinfo with the -s flag i get an illegal seek error. Without the switch I get a report of the mfs partitions. And it claims that it can be upgraded 3 more times. I am not sure how to take that.mfsinfo != mfs_info. You are running the wrong command. There's an mfsinfo that's part of mfstools. I suspect you are using that rather then the program I referenced above.

[dave868]

mfsinfo != mfs_info. You are running the wrong command. There's an mfsinfo that's part of mfstools. I suspect you are using that rather then the program I referenced above.

Thanks... I download the right version and I am getting results.... however not the ones I want. I get a max fsid of 539144 and max size of 16mb... I don't think this will allow me to shrink the drive... but i will try these threshholds and let you know.

[dave868]

Thanks... I download the right version and I am getting results.... however not the ones I want. I get a max fsid of 539144 and max size of 16mb... I don't think this will allow me to shrink the drive... but i will try these threshholds and let you know.

I was able to get it to shrink with -f 539144.

I restored and expanded to the 500 gig drive.

Initialized the swap file.

And just for giggles ran mfs_tools -s against the new drive.

I get

"failed to open [/dev/hda10-I]
llseek failed"

Now I haven't tried to boot the tivo yet... but I must say that I am not optimistic.

[cheer]

Not mfs_tools; mfs_info...and how are you running it if you haven't booted the Tivo yet?

[Jamie]

I was able to get it to shrink with -f 539144.
That's one too small, IIRC. I think mfsbackup *excludes* all tystreams with fsid at or above this number, which means you'll be leaving off one of your loopset animation clips.

cheer: there's an x86 version of mfs_info.

[dave868]

Not mfs_tools; mfs_info...and how are you running it if you haven't booted the Tivo yet?

Sorry ....typo in my post.

I am running it with the drive connected to a pc and a mfstools boot disk.

I am running the right version mfs_info... it's gives me the expected results when I use the original drive. But when I run it against the new drive that I just restored to... is when I get the seek error.

I really appreciate all your help. I know I am being a pain in the ass newbie.

Thanks for your patience.

[Jamie]

I am running the right version mfs_info... it's gives me the expected results when I use the original drive. But when I run it against the new drive that I just restored to... is when I get the seek error.
If you just restored to the drive, and haven't rebooted yet, you probably need to "revalidate" the partition table with "tivopart r /dev/hda" (assuming primary master -- adjust accordingly depending on ide channel). mfsrestore just wrote a new partition table on the drive, but the kernel may not know about it yet.

[PlainBill]

Yes, you're asking a lot of questions, but you're learning. This is more than I can say for >>10% of the newbies who show up here.

Now, for the question you didn't ask: If it turns out the missing loopset is on the expanded partition and is important, this thread (http://www.dealdatabase.com/forum/showthread.php?t=42465) explains how to copy it from the old drive and insert it into the new drive.

PlainBill

[dave868]

Yes, you're asking a lot of questions, but you're learning. This is more than I can say for >>10% of the newbies who show up here.

Now, for the question you didn't ask: If it turns out the missing loopset is on the expanded partition and is important, this thread (http://www.dealdatabase.com/forum/showthread.php?t=42465) explains how to copy it from the old drive and insert it into the new drive.

PlainBill

Thank you for the kind words.... I am trying.

Here is where we stand now... I borrowed an completely stock 40 gig drive from a friend that also has a HDVR2. I did a straight disk to disk with all streams but a 250 meg swap

mfsbackup -Tao - /dev/hdc | mfsrestore -s 250 -xzpi - /dev/hda

I initialzed the swap

tpip --mkswap --swapped /dev/hda

Reinstalled in my tivo and crossed my fingers.

No luck... same reboot loop that I started with.

I forgot to use the -r4 switch that ocntscha suggested.. I will try that tomorrow.

Can you tell me what the -r4 switch does?

[Jamie]

I forgot to use the -r4 switch that ocntscha suggested.. I will try that tomorrow.

Can you tell me what the -r4 switch does?-r4 is key. It's essential when expanding by 274GB or more.

You can find a full description of the mfstools options here (http://archive2.tivocommunity.com/tivo-vb/showthread.php?p=2651877#post2651877). That description pre-dates the understanding of the 274GB partition size problem. This (http://archive2.tivocommunity.com/tivo-vb/showthread.php?p=2651665&&#post2651665) describes why -r4 addresses the large partition size problem. It was just a conjecture then, but it proved to be true.

[dave868]

The -r4 switch did it... it works great! Menus are like lightning and it reports 440 hours... all this for $199!! And now it's $10 less.

Thanks again to all of you for all your help.