PDA

View Full Version : Direct extraction from a tivo drive in a pc (not networked)



Pages : [1] 2

plaiming
03-01-2003, 09:47 AM
i need to extract some programs off my tivo because its running out of room (only 14hr) but i dont have tivonet. is there a utility that i can use to extract if i just rip out the hard drive and
attach it to my windows box? i dont see anything like that in the howtos. everything talks about having a network connection to the tivo. i also have a linux server but id prefer to do it on my windows box cause thats where my software is.

-paul


EDIT: - threads consolodated and answers added to this post

Alphawolf did a basic howto (http://www.dealdatabase.com/forum/attachment.php?s=&postid=35312) & posted it in this thread (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=8091&perpage=15&pagenumber=13) a while back

here's the 386-binaries (http://themurrays.homeip.net/downloads/tivo/for_linux/vplay-bin.tar.gz) to extract video fsids when you boot the pc from a linux utility cd or floppy (thanks to lmurray for hosting these on his site. (http://themurrays.homeip.net/downloads/tivo/for_linux/)) I think these have the 2.x schema so mfs_streams (used for listing recordings) won't work on a 3.x drive

if you install mfs_ftp (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=21915) in the tivo before moving the drive to the pc, it caches the xml & fsid numbers needed for extraction to /var/mfs_ftp/cache/ which makes the whole process much easier once the drive is in the pc

zobetron
03-04-2003, 06:10 PM
I had a similar problem just recently. No matter what I did the tivo would not boot up completely (I could get a bash prompt).

Anyway it's not too hard, if you know your way around linux. Get vplay (make sure you have the latest source).

edit mfs_streams.c
change both instances of "/Recording/NowShowing" to "/Recording/NowShowingByClassic"

to compile (on a pc running linux):
make proto
make i386

I then copied the i386 folder to a pc running windows, say c:\i386.

I attach my tivo drives to my pc thus:

primary tivo -> primary slave
secondary tivo -> secondary slave

I use Kazymyr's Bootable Utility CD. Got it here: http://www.9thtee.com/tbdv2_6i.iso and burn it to a cd.

I boot up off the cd and then mount the partition (on the primary master) that contains the i386 folder thus:

mount /dev/hda1 /mnt/c

then you can cd into /mnt/c/i386 and run the programs therein.
before you do so however you need to have a correct MFS_DEVLIST (which is the tricky part):

ie.,
export MFS_DEVLIST="/dev/hdb10 /dev/hdb11 etc... /dev/hdd2 ..."

to find the correct devices to add to the above statement you need to type this on the command line:

export MFS_DEVLIST="/dev/hdb /dev/hdd"

and then run:

./mfs_info
it will warn you about some errors or somesuch but in it you will find the needed /dev/hd... strings.

then you can create the correct MFS_DEVLIST statement. For example mine reads:

export MFS_DEV="/dev/hdb10 /dev/hdb11 /dev/hdb12 /dev/hdb13 /dev/hdd2 /dev/hdd3"

type it on the command line.


Then you type:

./mfs_streams

which will return a list of the file parts (fsids) for each show.

edit...
actually it only returns the first part's fsid. So what I do is:
./mfs_ls "/Recording/NowShowingByTitle"
which returns a object fsids for each recording, then I do:
./mfs_dumpobj <fsid of the show I want to extract>
which returns a bunch of stuff. I then make note of the fsid's for each RecordingPart.
end edit...

Then you can extract the ty (to a directory on your c drive, eg, ty_files).

./mfs_export <fsid> /mnt/c/ty_files/recording_partX.ty

I know this seems convoluted but I am no expert. However, it works for me. YMMV.

Z

jamesq97
04-10-2003, 01:37 AM
I have the same issue but with a Philips unit that has the kernel patch for extracting. The unit was used for video extraction and now it is dead, the drives appear good and are mountable. I just have not been able to find a step by stop to do what Alphawolf seems to have been doing before (place drive in PC, pull video, put drive back in tivo).

Please help me/us if you can.

Thanks in advance
-J

was running a unscrambled Philips DTivo Xtreme 2.5 with net card....

osetivo
04-10-2003, 11:21 AM
Originally posted by bhuddleston
I have a Sony Tivo which I've added a second drive to...really loved it. -Brian

Not an expert, but since they haven't jumped in, I take a stab.

Assuming the problem is not the harddrives. (big assumption) Is your Sony a Standalone (aka SA) or a DirectTV-Tivo Combo (aka DTivo)? If it's a Combo then you must have already installed noscramble.o. Extraction/processing of scrambled DTivo streams is not possible (at this time?). I suspect that noscramble.o was NOT installed.

If it's a standalone "SA" then take a look here for some info.

<edit>dead link removed - the extract process is identical</edit>

Good luck and hope it helps.

EDIT: Do you know anyone with the same type of machine that would let you insert you HD so you could at least "Save to VCR"?

AlphaWolf
04-10-2003, 12:09 PM
Well, basicaly it works like this, you take the hard drive out of your tivo, get ahold of (or compile via tridges sources) an i386 version of the vplay libraries. You need to identify the FSID of the specific showing you want to extract, then use tridges mfs_export on it. You can normaly do this with mfs_stream, but for some reason its broken for newer software revisions. Unless somebody fixes it, you aren't going to be able to identify the FSID of the shows you want to extract.

wadevb
05-30-2003, 06:17 PM
Ok, I had a 100Gig drive die on me. Well, mostly die. Tivo went into a neverending reboot cycle. Drive diagnostics shows it needs replaced. However, I can still mount to it, and see partitions with dylan's boot disk.
I don't want or need to recover all the data, just 5 or 6 key episodes.
It was a two drive tivo, with 100gig on A, with a 60 gig on B. I still have the B drive untouched (presumably ok). I have since restored to prior image, onto a new single A drive.
Is there some threads, or posts that might give me enough info to track down the files on the mfs partitions, and get them out as ty's?
I have resolved to myself that they are lost, but I just have to try before I destroy data on the B drive.

Thanks in advance.

lmurray
05-31-2003, 06:20 AM
AlphaWolf's info on this type of extraction can be found here:

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=8091&perpage=15&pagenumber=13


Then there's zobetron info which is here:

http://www.dealdatabase.com/forum/showthread.php?s=&postid=87426&highlight=vplay#post87426

I've cached both of these articles on my website here:

http://themurrays.homeip.net/downloads/tivo/


hope this helps,
-lloyd-

wadevb
06-02-2003, 08:05 PM
Riley, and Lmurray:
Thanks for the info.
I have not attempted ms_ftp, as I have allready got my tivo up and running. Also, I am pretty sure it is not getting very far.

I have looked at the other methods, by compiling mfs_ ... tools. After fighting some lib issues, I got it too work. But, I get a similar error as to when I tried to use mfstools 2.0 to back up the drive.
./mfs_info
Found 4 MFS partitions
Not a TiVo super block! (magic=0xefbeadde)

Do I have any other options? Any ideas on how to trick it into accepting the partitions?

I may go dig through the code, and try some dangerous stuff. See if I can bypass the partition checking.

Thanks

holnet
07-27-2003, 01:24 AM
OK... I've had my Tivo for several years now, and it has accrued quite a bit of stuff on it I would like to preserve. I don't have a TivoNet card ($70 seems expensive for a network card.... though I might get one eventually), but I would like to just take the HDs (it is a Philips HDR612 dual drive 60 hour Series one) out and put them into my Linux PC and just extract locally to another HD. I believe that if I set MFS_DEVLIST to the Tivo drives, then the mfs tools will work on them. Assuming I'm right so far, I have to get the FSIDs for the pieces. Everything I've read explains how to do this with TivoWeb... I don't know how to do this without it. Also, I'm pretty confused about the Tystream stuff.... If I've understood right, the files I get from mfs_export will be 'wrapped' as a tystream, and I have to use tyconvert to get them to a mpeg file. OK. So I guess my questions are, am I right so far and how do I get the FSIDs without TivoWeb. I apologize if the answer is already on this forum (I'm sure it is somewhere), but searches have been fruitless, and I already have 23 internet explorer windows open right now :). Thank you.

David

holnet
07-27-2003, 06:52 AM
OK. Well, been up all night. But so far I've made a backup image from my drives. Then I started playing around with the partitions. I have to be careful, as I really don't want anything to happen to those recordings.... Anyway, I'm becoming acquainted with the mfs but as I feared, I can't get the FSIDs. Been using mfs_export to grab some of the files in the /Recording folder thinking the FSIDs were in there. I gather that the FSIDs given by mfs_ls and inside the files dumped by mfs_export are simply the FSIDs of the DB for that recording, not the recording itself.... Anyway, this was just an update so you know I'm still reading and trying on my own.... Don't want to be a load :).

David

Generiq
07-27-2003, 08:58 PM
Someones got Turbonet cards for $59 shipped in the for sale forum (in case that sounds a bit better). Extraction and Tivoweb alone make it well worth the $$$.

holnet
07-27-2003, 09:27 PM
Yeah, that does sound a little better. I'm probably going to do that as soon as I get a little extra money. But for the time being I'm going to continue what I've started here (as soon as my girlfriend is done watching Sex and the City :mad: ). The last problem I was having was getting the FSIDs. I had read the attachment in that sticky before, but was getting tripped up when it said to run ShowList.tcl on your Tivo.... Seems odd since the post was for those without a TivoNet or serial connection. Using mfs_dumpobj seems to be what I need to do, but it keeps coming up with an error (ERROR: could not find FSID=0 or something like that...) I get the FSID for the DB by using mfs_ls and looking in the Recording/NowShowingByTitle or NowShowingByClassic, but it doesn't seem to matter what FSID I give mfs_dumpobj. If I make up a fake FSID, it gives some other error, so I guess the problem is with mfs_dumpobj. I'm going to download the version from the CVS, compile it, and give it a whirl. The binary I was running was precompiled and was possibly an old version or something. Anyway, I'll keep trying. And thanks for the information.

David

chuck_333
08-07-2003, 02:31 PM
My TiVo is hosed and I want to rebuild the system from a backup. However, I want to save all of the recordings. Is there a way to extract the recordings without having the TiVo running (like hooked up to a PC)? Then after I restore my backup, how do I put the recordings back?

Thanks!

Chuck

phrend
08-13-2003, 09:15 AM
If you can't access the MFS then I don't see how you would be able to decode/extract anything useful... Any super hackers know the answer to this question?

chuck_333
08-13-2003, 09:39 AM
The MFS partition may well be screwed up to the point where the show directory is unusable, but there also might be a chance it's just a configuration issue. The TiVo just reboots over and over and then eventually locks up. It all started when it froze during the weekend and was stuck on a picture of live TV. I rebooted it and then it went through the GSOD rebuild. After that it worked for a few hours and then locked up, this time never to reboot again.

It sounds like the TiVo hardware itself may be bad, so that's why I want to use a PC to extract the data at least. I have another unit but it's hard at work recording stuff. I just ordered DirecTV and a new DirecTiVo, so I'll be able to screw with the good one and the broken one as much as I want when it's installed. I just need guidance on what to do.

Thank you in advance!

gtwrek
08-22-2003, 06:56 PM
I'm interested in trying to do the same thing. Just had my DSR6000 ripped apart because of a bad power supply. Wanted to extract a few of the things from the
drives.

Read the FAQ but was wondering about how to get those FSIDs. So holnet, any more luck?

holnet
08-23-2003, 07:20 AM
Unfortunately, not anything that will help you.... I got the FSIDs by setting up TivoWeb and accessing it over the serial connection. But that might not be your only problem. After getting the FSIDs I found that the new recordings extracted flawlessly (burned them onto DVDs). But my older recordings would have a few errors during the conversion, and there was a considerable amount of missing video (several minutes). I never did figure out (or get a response here about) how to resolve this. So I finally gave in and bought a Turbonet card. It will arrive on Monday :). I assumed the problem was with the older files I was using to do the extraction. So hopefully the newer tools available to extract video over the network will fix that problem. Anyway, another thing you might need to know is that to mount the Tivo drives in your PC, you need to modify the linux kernel source and recompile. I found the diff file somewhere, but it was for an old version of the kernel and the code in question is in a different source file now. Of course, grep can help you find it and you can manually apply the patch. Or use MFSTools Boot CD. That works well. Anyway, I wish you luck in what you are trying to do. I guess you could say I gave up :( . If you need anymore help, don't hesitate to post here or e-mail me. I'll do what I can.

David

Fugg
08-23-2003, 05:04 PM
"The last problem I was having was getting the FSIDs. I had read the attachment in that sticky before, but was getting tripped up when it said to run ShowList.tcl on your Tivo.... Seems odd since the post was for those without a TivoNet or serial connection."

If you do have a serial connection:

Before you power down your tivo and take your drive out, transfer (using your existing serial connection/bash prompt) NowShowing.tcl (http://www.dealdatabase.com/forum/attachment.php?s=&postid=54445) to a directory in /var, set it executable and run:

./NowShowing.tcl > fsids.txt

when it completes (it can take a few min if you have a full tivo), you will have a file called "fsids.txt" in that directory containing a list of what's on the tivo with the fsids!

Good Luck!!!

blasted bill
08-25-2003, 09:31 AM
hey all,
i have a tivo drive which will not boot, so i need to wipe it and restore from the orginal tivo drive, but 1st i want recover the stored streams i have. i read...
http://www.dealdatabase.com/forum/attachment.php?s=&postid=77812

to try to get the streams extracted. the problem is, when i put my drive on SM of my linux box, only hdc shows up. there's no 4, 7, or 10. if i boot from the floppy or cd rom the show up just fine. why doesn't it not show up in linux? btw, it's mandrake.

next, i've read several times that mfs_streams is broken past 1.3. has there been a fixed version for 3.0? i can't exactly put my HD back in my tivo and run nowshowing.tcl to get the fsids. thanks

blasted bill
08-26-2003, 09:23 PM
ghagha. how can i run the binaries from the cdrom if i have to complie them on my linux box? is there a way to complie them so they'll run on the cdrom boot cd?

blasted bill
08-26-2003, 09:48 PM
ah, i re read what zobetron posted. appeartnly his version of linux is compatible with the bootable cd rom. when i complie the binars on mandrake 8.1 then boot from the cd rom and try to run the binaries, it says.....

./mfs_streams: /lib/libc.so.6: version 'GLIBC_2.1.3' not found (required by ./mfs_streams)

and of course there's no way to replace the libc.so.d when the cdrom is running. so how can i complie them so they will work on the cdrom boot up?

lmurray
08-27-2003, 09:11 PM
I have the vplay biniaries here:

http://themurrays.homeip.net/downloads/tivo/for_linux/

look for the vplay-bin* file.

Not sure if this will help you or not. They were compiled under a RedHat 7.3 (i think) box.

If these work, let me know.


In the post above you were talking about using 2 different libc.so.6 files. Copy the new one to the ramdisc, but call it libc.so.X. Then edit your biniaries (YES, EDIT). Find all occurances of libc.so.6, and replace them w. libc.so.X. (Yes, I've done this before.) It works fine w/ vi BTW. I did this when I compiled stuff on my PowerPC that was running Linux.

hope something in the post helps you...
-lloyd-

blasted bill
08-28-2003, 09:52 AM
Thanks for the tips lmurry! no longer nessecary as i got frustrated with trying to compile the binaries and went back to trouble shooting the orginal problem(tivo wouldn't boot) and went through the rc.sysinint line by line until i found the file it was trying to call that was missing. it works now now, but the network does not work ): posting about that in the apporate forum. thanks.

TheWickedPriest
10-17-2003, 11:21 PM
...but I see only references to using mfs_export for the final stage. That works, but it only pulls the stream out in pieces, which can't be simply joined; and they sometimes have added junk in them as well.

Using "vplay" instead of "mfs_export" produces files that are slightly smaller (?!), but the same problems remain.

tserver, mfs_ftp, etc. don't work because they're dependent on tivosh (regular TCL won't do).

However, I found a simple way to get full streams, all parts cleanly joined, and no extras, from a locally-connected Tivo drive: a combination of vserver and mplayer-tivo on the same machine. It works exactly as it would if vserver were running on the Tivo -- and when used with the "-dumpstream" option, mplayer becomes an extraction tool instead of a player.

The first steps are the same as outlined elsewhere -- set up your MFS_DEVLIST and MFS_DEV environment variables. Then start vserver. From another shell, start up your Tivo-patched version of mplayer:

mplayer-tivo tivo://localhost/list

See an FSID you like? Then just:

mplayer-tivo -dumpstream -dumpfile whatever.ty tivo://localhost/12345

I've mentioned this method in a couple of threads elsewhere, but I thought it might help someone here. Alternatively, maybe someone here will point me to the easier solution that I've overlooked. (Actually, this is pretty easy; but maybe there's something faster?)

jdiner
10-17-2003, 11:28 PM
TheWickedPriest:

Grab mfs_stream rather than mfs_export. It is the same general thing. But you can then use mulitple FSIDs in 1 call.

You will still get some some junk. That code has always produced some junk. But VSplit on the PC/Unix is fully aware of this and corrects for and removes the junk.

Using mfs_stream no join is needed after the fact.

--jdiner

TheWickedPriest
10-17-2003, 11:32 PM
OK, thanks. Well, the vserver/mplayer solution is free of junk (at least on playback in mplayer, and on editing with TyNix), so perhaps it's the better solution after all. But I'll try it with mfs_stream if I ever have to do this again.

rc3105
10-17-2003, 11:53 PM
with either vplayer or mfs_export & join (with cat or dos copy) the tys are clean and can be inserted with mfs_ftp even if you didn't get the xml info (edit to whatever you want with tivoweb)

TheWickedPriest
10-18-2003, 12:10 AM
I can't agree that mfs_export + cat yields clean tys. Maybe mfs_ftp can insert them cleanly, but they're not clean in mplayer.

How does one extract with vplayer?

rc3105
10-18-2003, 02:13 AM
mplayer trips over the ocassional junk in the cracks - the difference between space allocated & actually used at the end of the fsid (which has chunks from old recordings that weren't zero'd)

mfs_export + cat == 1:1 copy of the fsids, not necesarily equal to the video IN the fsids

vplay has a "play to file" option

TheWickedPriest
10-18-2003, 02:29 AM
Originally posted by rc3105
mplayer trips over the ocassional junk in the cracksExactly. Hence, "not clean".


the difference between space allocated & actually used at the end of the fsid (which has chunks from old recordings that weren't zero'd)Yep, I noticed that. :-)


mfs_export + cat == 1:1 copy of the fsids, not necesarily equal to the video IN the fsidsRight. So, fine for mfs_ftp insert, but not as nice as the vserver/mplayer method, AFAICT; and plain unacceptable for later playback in mplayer. Plus it uses (slightly) more space. Are there any advantages?


vplay has a "play to file" optionI know... but you said "vplayer" above. There are actually two separate programs, "vplay" and "vplayer" in the Tridge tools package.

rc3105
10-18-2003, 01:47 PM
by clean I meant uncorrupted - not degraded during extraction

from tridges readme

use vplay to "play" with that fsid to a file or device

use vplayer to play files remotely
vplay default usage msg

/var# vplay
optind=1 argc=1

vplay <fsid> <vidfile>
OR vplay -p <fsid>

options:
-s <start> starting chunk
-c <count> number of chunks to play
-v be verbose

use the 2nd form for direct playback on a TiVo

vplay takes a fsid for stream files. See liststreams
so a little experimenting will generate something like this

vplay 1400984 temp1.ty
mfs_export 1400984 temp2.ty
mfs_stream 1400984 temp3.ty

/var# ll *ty
-rw-r--r-- 1 0 0 7208960 Oct 18 07:43 temp1.ty
-rw-r--r-- 1 0 0 7340032 Oct 18 07:43 temp2.ty
-rw-r--r-- 1 0 0 7340032 Oct 18 07:43 temp3.ty

vplay grabs just video, export & stream get the entire fsid

ice5
10-22-2003, 02:40 PM
How can one get the FSIDs without a serial connection?


Originally posted by Fugg
If you do have a serial connection:

Before you power down your tivo and take your drive out, transfer (using your existing serial connection/bash prompt) NowShowing.tcl (http://www.dealdatabase.com/forum/attachment.php?s=&postid=54445) to a directory in /var, set it executable and run:

./NowShowing.tcl > fsids.txt

when it completes (it can take a few min if you have a full tivo), you will have a file called "fsids.txt" in that directory containing a list of what's on the tivo with the fsids!

Good Luck!!!

FredThompson
10-29-2003, 02:59 AM
I've just found myself in a situation similar to that which blasted bill mentions earlier in this thread. One of the 2 drives which were installed in my T60 with xtreme 2.5.2 and xupgrade started acting up and the TiVo won't boot with them. You know the screen, critical error, connect the phone cord, wait 24 hours... Yeah, well, that would probably screw everything up.

The drives are removed and I'm wondering how to extract the data given there's no way to get the fsids using the TiVo itself.

It certainly seems part of the data on the drives is toast but, considering they WILL spin and seem to act ok for now, it should be possible to pull from them.

Would someone please give me guidance on how to do this?

rc3105
10-29-2003, 03:43 AM
if you were running mfs_ftp prior to the crash the fsid numbers will be in /dev/hda9 /mfs_ftp/cache/recname.pts

if not, your next best bet is to connect both drives to the pc & make a divorced mfstools 2 backup, restore it to a third drive & if the restore will boot, let it run through the gsod repair (without the phone line attached). if it recovers you can use mfs_ftp or nowshowing.tcl (or whatever) to get the fsid numbers you need. lotta if's there, but SO much easier than...

if those fail, follow the mls & dumpobj steps outlined in tridge's readme & AW's post


--
Riley

FredThompson
10-29-2003, 03:57 AM
no mfs_ftp, just vserver but not on boot, only by demand. Don't remember if it was running when things crapped out.

yes, a lot of "ifs" but it's certainly worth a shot to try your first suggestion. I just printed the first page from this thread and was reading AlphaWolf's post about using the tridge binaries. zobetron's post is interesting, also.

Thanks for the suggestion. I think you're absolutely correct, best to try MFSTools to combine the 2 "old" drives onto a single "new" drive and boot the TiVo with it only to see how things go.

Sure would be nice if it "heals" the problems during the last recording and the other stuff is fine.

TheWickedPriest
10-29-2003, 02:07 PM
vserver on a PC will sometimes (as in my case) work with a drive that the Tivo can't boot from. I was getting the green screen of death, but I was able to extract nearly all my recordings once I put the drive in my PC.

In this case, you can get the FSIDs as described above:

mplayer-tivo tivo://localhost/list

and to extract:

mplayer-tivo -dumpstream -dumpfile whatever.ty tivo://localhost/12345

IMHO much easier (and faster) than trying to recover the drives; not that you shouldn't do that, too.

FredThompson
10-29-2003, 02:17 PM
Unfortunately, I'm not running a Linux box so this isn't an option. Sure would be nice, though. It would allow the TiVo to do it's thing while I mess with the drives.

Oh, what about booting a Linux CD then running? That just might work.

(Sure wish Linux and NTFS got along or Windows could run Linux binaries. It would make things a heck of a lot easier.)

In this particular case, the drives aren't critical, there are already new drives in the TiVo. I'd far prefer to pull the stored content and low-level the drives to see if they fail. Also, I guess there's a chance of corruption but wouldn't it be a good idea to combine the 2 source dirves onto one drive via MFSTools before anything? It would certainly reduce the complexity.

moe1766
10-29-2003, 03:56 PM
I'm trying to extract some streams off a SVR2000. Its been upgraded to a dual drive 110 hour unit. The unit is NOT networked. I'm selling it and I'd like to retrieve the streams without added cost.

Are the vplay tools from tivo.samba.org working or just outdated. I find that after setting my MFS_DEVLIST to "/dev/hda /dev/hdb" and trying any of the mfs_* or vplay programs I get an warning that the superblock and actual sizes don't match. It doesn't look like its reading or mapping the second drive.

Has anyone else had this problem?

Using mfs_ls or mfs_streams I can get the directory information to see the names and fsids of the streams listed in /Recordings/NowPlayingByTitle. Extracting the streams gets part way through and exits with an error about not finding the next part of the stream (presumably on the scond drive).

Thanks.

gobsmack
11-02-2003, 12:14 PM
EDITED:

Ok, take notes on your drive partitions when they boot and follow Z's DEVLIST command thematically, adding additional /dev/hdxN partitions as needed.

In my case, I have gotten close, but now have an error: the MFS filesystem size doesn't match the expected size (difference is 1024).

I already know the FSIDs I need -- is there a way to supress the errors and grab the fsids directly?

Thanks

keithc
11-09-2003, 05:51 PM
I get the same errors as gobsmack and moe1766 - it does seem to be ignoring the MFS partitions on the second drive.

The mfs_export that comes with mfs_ftp must come from a different source but I can't find it anywhere.

I came up with the following patch to the CVS code and can now mfs_export FSIDs that previously errored. Just reinserting the ty now to test (airnet's a bit slow) but if anyone wants to try it out...

btw my disks are from a UK TiVo.

edit:

attachment deleted as it doesn't work properly.

keithc
11-10-2003, 05:56 AM
Hold fire on the patch as although it extracted and inserted, it didn't play back properly but I think I'm close.

I don't think this is a compiler version issue.

Running on tivo, the list of mfs partitions is taken straight from the superblock on $MFS_DEVICE.

On i386, you have to set $MFS_DEVLIST and it parses the partition tables looking for MFS types. The CVS code only scans the first device.

keithc
11-12-2003, 06:18 AM
Found the problem.

The attached patch now produces streams identical to those extracted using mfs_ftp.

rich2940
11-22-2003, 04:44 PM
I would like to try this procedure. But, I would like to duplicate my tivo drive before starting. Would backing up my Tivo dirve with "Backup up your TiVo drive(s) with Mfs Tools", give me a drvie I can extract video files from?

rc3105
11-22-2003, 08:37 PM
yes, if you use the -a backup option to include recordings

KalTorak
12-14-2003, 06:32 AM
Problem: The file that I extracted expecting a .Ty stream isn't loading into TyStudio Editor.

My hardware: S2 SA, Tivo-branded, originally 40 hours, upgraded with a 200GB disk as described at Hinsdale, running system software 4.0.1 (when the recordings were made) and 4.0.1a now.

--------------------------------------------------
I used one of the MFS Tools boot disks and the vplay binaries from lmurray's server to boot up with my TiVo drive attached, exported MFS_DEVLIST="/dev/hda", and ran mfs_info to get MFS_DEVLIST set properly (I wanna say it came back with /dev/hda10 through 15)

Used mfs_ls to get the filenames and FSIDs in /Recording/NowShowingByClassic, and found that the FSID for the stream I wanted was 39593.

Ran mfs_dumpobj to get a list of FSIDs for the parts of the stream 39593; had lots of parts - RecordingParts, Showings, and I think a Recording at the end. It seemed that only the RecordingParts had FSIDs associated with 'em, and they ran from 39540 to 39559.

I finally used mfs_export to send the first RecordingPart (FSID = 39540) to a DOS (er, Win95 FAT32 LBA) partition that was mounted, and it came out around 500MB (kinda reasonable, the whole stream's 4:20 at best quality). I renamed it 39540.ty, detached my Tivo drive, and rebooted into Windows.

Started up TyStudio Editor, opened the local 39540.ty file, and... nothing. Says "failed to load the selected Ty file", and that's that.

So... Did I, in fact, grab a Ty file? Do I need to somehow connect the pieces before I use TyStudio to see them? Could I have a scrambling Tivo (how do I know??)

Help - thanks!

eastwind
12-14-2003, 02:51 PM
Originally posted by KalTorak


Started up TyStudio Editor, opened the local 39540.ty file, and... nothing. Says "failed to load the selected Ty file", and that's that.

So... Did I, in fact, grab a Ty file? Do I need to somehow connect the pieces before I use TyStudio to see them? Could I have a scrambling Tivo (how do I know??)

Help - thanks!

I could be wrong, but that sounds like the error you would get if you recorded the program scrambled. Did you in fact have the scrambling turned off (with a hack)?

ew

KalTorak
12-14-2003, 03:10 PM
Nope; I've only thought scrambling applied to the DirecTiVos.

CRAP.

eastwind
12-14-2003, 06:21 PM
IIRC, all the series 2 Tivos use scrambling. Just something I remember reading, I don't own one.

ew

rc3105
12-14-2003, 10:56 PM
KalTorak:

Scrambling Issues: Archiving, Restore & Mpeg conversion (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=30547) has info on what you can currently do with those scrambled recordings.

s1 streams are decryptable now, the s2 util is still in development. if you back them up as tmf you'll be able to restore-decrypt when the s2 version works

NutKase
12-21-2003, 02:50 PM
Originally posted by rc3105
Scrambling Issues: Archiving, Restore & Mpeg conversion (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=29819)

What happened to the thread you mentioned above rc?

More TiVo 'intervention' ?

NutKase


[EDIT] Here's the link to the newly rebuilt Scrambling Issues: Archiving, Restore & Mpeg conversion (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=30547) :)

trouper
12-23-2003, 05:34 PM
I have a series 1 stand-alone (not networked) tivo. It is a 30 hour unit with an additional 80Gig upgrade. Appears to be running system 3.0-01-1-000. After reading and searching through the forum I decided to extract a saved recording. I partitioned my computer and installed RedHat 7.1 and the gcc compiler and vplay. Then I hooked up both tivo drives as slaves and did the following operations:

From within the i386 folder

#export MFS_DEVLIST="/dev/hdb /dev/hdd"

#export MFS_DEVICE="/dev/hda10 /dev/hda11 /dev/hdb2 /dev/hdb3"

#./mfs_ls "/Recording/NowShowingByTitle"

RESULT
Found 2 MFS partitions
WARNING: total sectors doesn't match (total=57699328 sb=217780224)
dir:fsid = 1189998 count=33

fsid type name
______________________________
2268406 tyDb 1:SECRETS:87600:31601:2268406

(it showed a big list but that's the file I want to get)


#./mfs_export 2268406 secret.ty


RESULT
Found 2 MFS partitions
WARNING: total sectors doesn't match (total=57699328 sb=217780224)

exporting FSID 2268406 of size 956 to secret.ty
starting at 0 for 956 bytes


___________
Just for fun I tried to open the file once I got it back into winxp with TyShow but it won't go. I'm thinking it should at least a few hundred megs and not 956. So where did I mess up or what do I need to extract this file?

rc3105
12-23-2003, 07:14 PM
that's the fsid of the recording object (title, description, etc) you need the fsids of the recording parts

easiest way to do that will be connect a serial cable to the tivo & run a script like nowshowing that will dump them, or dig through the info you dumped with a hex editor as described earlier in the thread

rc3105
12-24-2003, 01:20 AM
allmost exactly the same topic as Direct extraction from a tivo drive in a pc (not networked) (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=23192)

use mfs_export, mfs_stream, extractsream, whatever floats your boat to extract a recording locally. unless there's a network mount (nfs/samba) mapped into the tivo filesystem you need enough non-mfs space to save the recording (oversize var, repartitioned drive, another drive (http://www.dealdatabase.com/forum/showthread.php?s=&postid=135743#post135743))


here's a little utility - unscramble.tcl (http://www.dealdatabase.com/forum/attachment.php?s=&postid=135039) - originally written for something else that has a list option to generate fsid numbers usable for extraction - including deleted but recoverable

jonbig
12-25-2003, 06:35 PM
If you back them up as tmf you'll be able to restore-decrypt when the s2 version works

So is there any way to back up to TMF format from a tivo drive directly attached to a PC? I can extract as .ty, but since my drives won't boot, I can't run mfs_ftp.

Alternatively, is there an easy conversion back from .ty to .tmf? I recognize I'd lose the title, description & such and have to replace them with generic stuff.

jonbig
12-25-2003, 06:45 PM
So is there any way to back up to TMF format from a tivo drive directly attached to a PC? I can extract as .ty, but since my drives won't boot, I can't run mfs_ftp.

Never mind. I did a bit more searching, and found info that mfs_ftp *can* reinsert a .ty file back to a TiVo, it just loses the detailed show information.

rc3105
12-26-2003, 02:35 PM
Never mind. I did a bit more searching, and found info that mfs_ftp *can* reinsert a .ty file back to a TiVo, it just loses the detailed show information.
keep reading. you can create ty+ easily via direct extraction, & they do contain all the detailed info. you could create tmf, it'd just be a bit more work

jonbig
12-26-2003, 09:43 PM
keep reading. you can create ty+ easily via direct extraction, & they do contain all the detailed info. you could create tmf, it'd just be a bit more work

Hmmm. There weren't too many hits searching on "ty+", and most of them were with mfs_ftp, and none with direct extraction. Mfs_ftp doesn't help me working with an unbootable TiVo drive.

AVD
12-27-2003, 07:15 PM
I'm not going to pretend that I know anything about the tivo filesystem, but you post about attached storage has got me thinking. Can you add some sort of removable drive to the second ide connector on a two drive tivo (a big drive, some sort of dockable HDD) and extract video from the MFS filesystem to a FAT32 (or some other) partition, then dock this drive into another TiVo or a PC?

AVD
12-27-2003, 07:55 PM
According to this thread i can mount a TiVo drive in a PC and have access to the MFS filesystem.

Now, can I run some version of MFS_FTP on the linix PC so I can transfer video to another TiVo? If this TiVo was built on a MFStool restore of the image of the drive in the PC (backed up without video) would help with the insertion of shows?

I want to juice up my 30 hour SA S1 TiVo, but don't want to wait to transfer video using mfstools.

rc3105
12-27-2003, 11:41 PM
include "removable" & "sled" in your search paramaters, oughta turn up lots of ways folks have figured to do it

usb hdd is the easiest so far (well, after running a dvd burner that is)

das Monkey
12-29-2003, 02:14 AM
The edit to the first post indicates that the binaries will not work with 3.1. I wandered through the steps to this process as best as I could piece them together with a 2-drive 3.1 setup (booted into MFSTools with "vmlnodma hda=bswap hdb=bswap") and got as far as the mfs_streams command, which replied "Cannot find fsid=1". I'm not sure if that error is related to 3.1 or a flaw elsewhere in the process, but I very much need to access some recordings on a drive pair that will not boot up for whatever reason (boot cycles at "Updating Database"). Is there another method I should use for a 3.1 setup? I have a T60. I do not have a linux machine, but I am familiar with Knoppix.

I would very much appreciate some assistance with this. I am willing to try most anything. Thank you.

Oh, I did have mfs_ftp running, but for the life of me, I cannot find that cache directory.

das

jonbig
01-02-2004, 09:55 PM
I have just finished hacking together bits and pieces from various toolsets to create a new tool that extracts recordings in .tmf format directly from a TiVo drive attached to a PC. It's based on mfs_tarstream (used by mfs_ftp) and the vserver-1.2 distribution. It contains C code that generates the needed showing.xml part of a .tmf file directly from the MFS partitions on the drive.

The nice thing about this tool is that it works for extracting scrambled recordings that you can then upload back to your TiVo via mfs_ftp. At least it worked for the first recording that I just extracted from my non-bootable TiVo drive.

Give me a couple of more days to test and clean up the source and I'll release it.

I guess I are now a certifiable TiVo hacker! :)

FredThompson
01-02-2004, 10:19 PM
Are you saying this is an all-in-one package to recover stored recordings from "dead" drives or drive sets?

Linux? Windows? Mac? FAT? FAT32? NTFS?

jonbig
01-02-2004, 10:36 PM
Are you saying this is an all-in-one package to recover stored recordings from "dead" drives or drive sets?

Linux? Windows? Mac? FAT? FAT32? NTFS?

It runs on Linux currently and accesses the drives directly, so you don't need any hacks to be able to understand the TiVo disk partitioning. I'm not familiar with running the tools on MacOS X, but if vserver runs there, it probably would too.

You can store the extracted stream anywhere you can access from Linux, as long as the filesystem supports files the size of the stream you are extracting. So FAT32 will fail for streams > 2GB.

I will be releasing the source, since the sources I worked with are GPL'd.

AlphaWolf
01-02-2004, 10:53 PM
I have just finished hacking together bits and pieces from various toolsets to create a new tool that extracts recordings in .tmf format directly from a TiVo drive attached to a PC. It's based on mfs_tarstream (used by mfs_ftp) and the vserver-1.2 distribution. It contains C code that generates the needed showing.xml part of a .tmf file directly from the MFS partitions on the drive.


Excellent, this is what I have been trying to push for for a very long time, but lack the knowledge of the C language to do myself.

I hate to just request features out of nowhere, but here are a few things that you might consider adding later on down the line if you don't already do them (no other TMF/tar utilities do it thus far):

Detect if the tystream is scrambled by checking the first four bytes of each tystream part. If its unscrambled, the first four bytes will be: 0xF5467ABD. Anything else will be a scrambled stream. The purpose of doing this is that you don't need to include the CommercialSkipOffset attribute, that way when you insert it into another tivo, it will play it back whether the tivo is in noscramble mode or not. (including CSO makes the stream fail in scramble mode)

Also, if you want to keep scrambled recordings workable from a dead tivo hard disk, you might want to include a means of getting the /State/Media.DiskConfiguration attribute (you can include it in the TMF's XML data outside of the normal object tags, whatever reads the TMF can decide what it wants to do with it from there, be that to ignore it or whatever else.)

FredThompson
01-02-2004, 11:47 PM
Yup, this does look promising.

Does it support a 2 drive pair or just 1 drive?

How will it be packaged? May I suggest 2 ways; Linux executable (and source) and an ISO with a minimum install of a recent Linux build to auto-load for people who aren't Linux saavy? (You know, similar to how people use MFS Tools to enlarge their drive space. It protects Linuxophiliacs from hurting themselves.)

AlphaWolf
01-02-2004, 11:55 PM
I would just have sleeper include it with his bootable ISO.

FredThompson
01-03-2004, 12:08 AM
Good idea.

jonbig
01-03-2004, 01:06 AM
Detect if the tystream is scrambled by checking the first four bytes of each tystream part. If its unscrambled, the first four bytes will be: 0xF5467ABD. Anything else will be a scrambled stream. The purpose of doing this is that you don't need to include the CommercialSkipOffset attribute, that way when you insert it into another tivo, it will play it back whether the tivo is in noscramble mode or not. (including CSO makes the stream fail in scramble mode)

Also, if you want to keep scrambled recordings workable from a dead tivo hard disk, you might want to include a means of getting the /State/Media.DiskConfiguration attribute (you can include it in the TMF's XML data outside of the normal object tags, whatever reads the TMF can decide what it wants to do with it from there, be that to ignore it or whatever else.)

I'll add the unscramble detection to my TODO list.

I just used the mfs_dumpobj tool to get the DiskConfiguration attribute from my broken drive and hand edited it into XML format. That's pretty trivial, so the need for a tool is minimal.

jonbig
01-03-2004, 01:11 AM
Yup, this does look promising.

Does it support a 2 drive pair or just 1 drive?

How will it be packaged? May I suggest 2 ways; Linux executable (and source) and an ISO with a minimum install of a recent Linux build to auto-load for people who aren't Linux saavy? (You know, similar to how people use MFS Tools to enlarge their drive space. It protects Linuxophiliacs from hurting themselves.)

I have a patch to the mfs tools that handles 2 drives properly. I wrote my own before I discovered somone else wrote one too. :)

I just set MFS_DEVLIST to '/dev/hdc /dev/hdd' and everything works great.

I will talk to Sleeper eventually about including it. I had to make a couple of other patches to the vserver-1.2 distribution mfs source as well, so maybe I can coordinate to get a new release of that package to include my tool.

FredThompson
01-03-2004, 01:15 AM
I have a patch to the mfs tools that handles 2 drives properly.There are 2 S1 drives sitting on a shelf here which are now very, very excited. Uh...so am I ;)

jonbig
01-04-2004, 07:33 PM
Testing on my new tool is going well, but I'm finding various recordings are missing different MFS object attributes, (most make sense, but a few seem odd), so I'm having to go back and update the source to handle the non-existence of these attributes.

Does anyone have a good pointer to documentation on the MFS object schema and .tmf that might indicate what attributes are *absolutely* necessary vs which ones could be left out without messing up the extracted .tmf?

rc3105
01-07-2004, 10:24 AM
title, episode, description & duration are really the only biggies. the first 3 for convenience & the last so the playback bar is accurate

the DC key isn't really the sort of thing you want embedded in each recording, a seperate util/script to extract it would be simpler all around

AlphaWolf
01-07-2004, 10:54 AM
the DC key isn't really the sort of thing you want embedded in each recording


Why not? It wouldn't hurt anything. Just only add it if the stream is known to be scrambled, much like how CSO keys should be handled.

rc3105
01-07-2004, 11:52 AM
Why not? It wouldn't hurt anything. Just only add it if the stream is known to be scrambled, much like how CSO keys should be handled.
that's great if you only have 1 scrambled stream, or all your scrambles have the same dc key. if the insert util restores a key from a scramble then anything else on that drive scrambled with a different key just quit working... (and did it backup the old key? did it know which streams were associated with the old key?)

for more than one key or a few streams what you're proposing requires a full blown caching system to keep track of which key goes with which stream, then substituting it in when tivoapp tries to play it AND a reboot for it to take effect. alternatly you can tweak noscramble to directly provide tivoapp with mfs_scramble_stream style keys looked up from a cache

any way you slice it it's a headache. got utils to do it either way, but releasing that tarbaby was not one of my new years resolutions...

ronnythunder
01-07-2004, 01:00 PM
ok, sorry if this is a dumb question, but there's actually three parts to the scrambling keys, right? (1) cso keys (per part) (2) dc keys (per box) and (3) crypto chip key(s) (per box).

so, even with the cso and dc keys, we still can't move streams to another box, right? for example, i have a s1 dtivo that's dead (mobo probs of some kind) and i have scrambled shows on there that i'd like to liberate. i'm sol because of the crypto chip keys, aren't i?

ronny

rc3105
01-07-2004, 01:49 PM
ok, sorry if this is a dumb question, but there's actually three parts to the scrambling keys, right? (1) cso keys (per part) (2) dc keys (per box) and (3) crypto chip key(s) (per box).

so, even with the cso and dc keys, we still can't move streams to another box, right? for example, i have a s1 dtivo that's dead (mobo probs of some kind) and i have scrambled shows on there that i'd like to liberate. i'm sol because of the crypto chip keys, aren't i?

ronny
theoretically no. practically, maybe. I've retrieved scrambled streams from a few fried boxes by moving the crypto to a working motherboard

anybody with the soldering skills & a guinea pig dtivo should be able to decrypt your recordings if you can get them your crypto (without destroying it during removal) & a dvd full of tmf scrambles along with the dc key from that drive

ronnythunder
01-07-2004, 02:09 PM
if i had a huge pr0n stash on there or something, i might be willing to do that :)

most of the stuff on my dead machine is just old, forgotten hockey games, so no worries.

ronny

StanSimmons
01-07-2004, 03:20 PM
if i had a huge pr0n stash on there or something, i might be willing to do that :)

most of the stuff on my dead machine is just old, forgotten hockey games, so no worries.

ronny

Maybe KRaveN could move the chip for you. If you can find a live donor box...

kevinallein
01-07-2004, 05:04 PM
Hi Jonbig,
today my primary Tivo drive broke. The drive has developed a HW bug in the area of parition 9 (MFS applications). I have a dual drive SA1 Tivo (primary 120G, secondary 30G). Any chance your tool will allow me to save my recordings ? I have some stuff on the Tivo I really would like to have.

I was planing to save of the recordings through turbonet for a long while, now I am not able to boot anymore (any access to hda9 hangs up the Tivo with error messages. I tried to dd hda9 from a MFS backup image, but the HW errors appear hear as well).

Thanks
Kevin

AlphaWolf
01-07-2004, 05:22 PM
that's great if you only have 1 scrambled stream, or all your scrambles have the same dc key. if the insert util restores a key from a scramble then anything else on that drive scrambled with a different key just quit working... (and did it backup the old key? did it know which streams were associated with the old key?)


Thats why you don't do that; you store the DC key outside of the normal XML tags as I described, since it is reserved for a special purpose. You only apply the DC key to the tivo if the user specifically requests it. You DO NOT apply it for a casual reinsertion under any circumstances.

The DC key is only a few hundred bytes and it is very relavent for scrambled streams. I don't see any reason to not include and handle it in the manner I described.

jonbig
01-07-2004, 06:14 PM
Hi Jonbig,
today my primary Tivo drive broke. The drive has developed a HW bug in the area of parition 9 (MFS applications). I have a dual drive SA1 Tivo (primary 120G, secondary 30G). Any chance your tool will allow me to save my recordings ? I have some stuff on the Tivo I really would like to have.

It may work, but I have a couple of more things I need to clean up before I'm ready to release it. Perhaps by this weekend.

I had to hack my copy of the mfs library to extract stuff from my dead drive, because the MFS superblock got trashed. I'll be releasing it in source form, since the code I used is GPL'd, but you may have to add some code to work around the particular damage in your MFS filesystem. Luckily, the recordings aren't in the partition that you have damage on.

kevinallein
01-07-2004, 06:20 PM
It may work, but I have a couple of more things I need to clean up before I'm ready to release it. Perhaps by this weekend.

I had to hack my copy of the mfs library to extract stuff from my dead drive, because the MFS superblock got trashed. I'll be releasing it in source form, since the code I used is GPL'd, but you may have to add some code to work around the particular damage in your MFS filesystem. Luckily, the recordings aren't in the partition that you have damage on.

Thanks, I will hang tight !
Kevin

kevinallein
01-08-2004, 03:12 AM
One more question for my preparation: I am using RedHat 9.0 on my linux box. I asume I need a kernel with swapping turned on for your tool ?
So far I never modified the kernel on my main box for swapping, but just booted from a floppy for the Tivo modifications. Since I want to move 150G now I need to get connected to the net when the tivo drive is mounted and change my method here.

Thanks
Kevin

jonbig
01-08-2004, 03:26 AM
[QUOTE=kevinallein]One more question for my preparation: I am using RedHat 9.0 on my linux box. I asume I need a kernel with swapping turned on for your tool ?
So far I never modified the kernel on my main box for swapping, but just booted from a floppy for the Tivo modifications. Since I want to move 150G now I need to get connected to the net when the tivo drive is mounted and change my method here./QUOTE]

I'm not particularly familiar with S1 boxes, but I don't think you need to do anything special. The mfs utility functions I am using can access the disk directly through the raw /dev/hd? devices, and appear to have detection and handling of byte-swapping built in.

But then again, since I don't have an S1 box to play with, you'll have to find out definitively for yourself.

kevinallein
01-08-2004, 07:13 PM
quote:
I'm not particularly familiar with S1 boxes, but I don't think you need to do anything special. The mfs utility functions I am using can access the disk directly through the raw /dev/hd? devices, and appear to have detection and handling of byte-swapping built in.

But then again, since I don't have an S1 box to play with, you'll have to find out definitively for yourself.[/QUOTE]

Usually both Mips (S2) and PowerPC(S1) run in Big Endian mode (although both have little endian options for WindowsNT support. So if it works for a S2 it will also work for S1. Do you use a stock kernel from a distribution or are you using something done for Tivo (i.e. something based on Dylan's boot disk or so ?)

Kevin

jonbig
01-08-2004, 07:25 PM
Do you use a stock kernel from a distribution or are you using something done for Tivo (i.e. something based on Dylan's boot disk or so ?)

I've been running my new tool on a stock RedHat 9 kernel with no problems.

AlphaWolf
01-09-2004, 05:46 AM
Tridges mfs code automatically handles byteswapping on S1 drives. No special kernel is required.

BTW: I think the mfs_tmfexport thread was best left separated from this thread, not only because its a separate development effort, but because if vplay (which this is based off of) is any indication, this will probably be able to compile for the tivo architecture (be it S1 or S2,) and will be useful for live TMF extraction, hence reducing the need for buggy/bloatish/slow tivosh.

IMO, merging it with this thread just worsens the thread bloat that so many people complain about.

rc3105
01-09-2004, 10:40 PM
BTW: I think the mfs_tmfexport thread was best left separated from this thread, not only because its a separate development effort, but because if vplay (which this is based off of) is any indication, this will probably be able to compile for the tivo architecture (be it S1 or S2,) and will be useful for live TMF extraction, hence reducing the need for buggy/bloatish/slow tivosh.

IMO, merging it with this thread just worsens the thread bloat that so many people complain about.
bloated threads, "censorship" from deleting junk, info spread too thin to be usefull... choose the least of the necesary evils :(

once mfs_tmfexport is more than vaporware it probably will rate it's own thread - still falls squarely in the direct extraction category

fwiw: olaf's tyserver allready retrieves title/episode/etc from mfs. can be tweaked to produce ty+ easily (hardly more than cut & paste) - tmf is a bit more involved but not much, just borrow a bit of tar code from mfs_tarstream. PII-400 running tyindex/tyserver indexes SO much faster than the tivo, edit/extract/convert on the fly across 100 or tcp/firewire is allso pretty groovy :p

jonbig
01-10-2004, 12:50 AM
once mfs_tmfexport is more than vaporware it probably will rate it's own thread - still falls squarely in the direct extraction category

Yeah, yeah, yeah. I've got a life. :)

I'm almost done with the changes I wanted to make, so you'll all see it soon.

rc3105
01-10-2004, 12:17 PM
Yeah, yeah, yeah. I've got a life. :)

boy howdy, know whatcha mean :eek: make sure the readme is airtight or you'll spend 100x the time answering dumb questions as writing the util in the first place...

iaw4
01-10-2004, 04:19 PM
tivo series 2, software 4.x, pulled tivo's 80gb harddrive, and connected it into an external usb enclosure, which mounts as /dev/sda under mandrake 9.2. (works great! allows connecting and disconnecting of the drive!)

first, thanks for all the info and software. I downloaded the latest vplay sources (Sep 2002, I think), fixed the line-spanning usage strings in the .c source for gcc, recompiled the source, and exported the two MFS_* variables.

I can now obtain fsid numbers via
$ mgs_ls /Recording/NowShowingByBucketTitle/
and all my recordings show up by name, with a nice fsid in the first column. (I cannot interpret all the other fields.)

Alas,
$ msg_export fsid destination.ty
seems to give me files of around 1K in size, only. now, I know that my shows are bigger than that!! ;-). mfs_streams does not work, but I read somewhere that it is supposed to be broken.

Am I doing something wrong? How do I get the full show into one .ty file? does someone have a wrapper that queries show by show and does the rest of the extraction for each show automatically?

regards,

/ivo

jonbig
01-13-2004, 06:45 PM
I know I sort of promised that I'd finish and release my mfs_tmfstream tool this last weekend, but my Linux machine spent most of the weekend in pieces, which makes it rather hard to edit or compile code.

Hang in there, this weekend should be less, um, busy. :)

kevinallein
01-13-2004, 07:58 PM
I know I sort of promised that I'd finish and release my mfs_tmfstream tool this last weekend, but my Linux machine spent most of the weekend in pieces, which makes it rather hard to edit or compile code.

Hang in there, this weekend should be less, um, busy. :)

Hi Jonbig,
I guess most of us have been there. ... only a small fix ... and than we spend 2 weeks fixing something we did not want to mess with in the first place. Good luck !

BTUx9
01-13-2004, 09:17 PM
Alas,
$ msg_export fsid destination.ty
seems to give me files of around 1K in size, only. now, I know that my shows are bigger than that!! ;-).
It looks like you are exporting the show fsid... you need to use the fsids for the recorded chunks.
(I also assume you meant mfs_export and mfs_ls)

ronnythunder
01-15-2004, 12:10 PM
ok, sorry if this seems dumb, but..

i'm trying to extract from a failing single drive from a 3.1 s2 dtivo. which of the mfs_* variables is supposed to have the drive name, and which is supposed to have the partitions? all i get is the "not a tivo superblock (0x35313537)" (numbers may not be exact, but all four bytes are 3x something).

ideas?

ronny

BTUx9
01-16-2004, 12:28 AM
i'm trying to extract from a failing single drive from a 3.1 s2 dtivo. which of the mfs_* variables is supposed to have the drive name, and which is supposed to have the partitions? all i get is the "not a tivo superblock (0x35313537)" (numbers may not be exact, but all four bytes are 3x something).

ideas?

ronny

Are you using tivopart to correctly load the apple partition, or a patched linux kernel, or are you using a tivo-specific boot CD? If you don't use one of those methods, the partition info isn't read properly by linux (because tivo doesn't use a standard apple partition). A good test is to try to mount your root partition. If that succeeds, linux is probably seeing the partitions correctly.

Hope this helps.

jonbig
01-16-2004, 12:34 AM
Are you using tivopart to correctly load the apple partition, or a patched linux kernel, or are you using a tivo-specific boot CD? If you don't use one of those methods, the partition info isn't read properly by linux (because tivo doesn't use a standard apple partition). A good test is to try to mount your root partition. If that succeeds, linux is probably seeing the partitions correctly.

Hope this helps.

Actually, I had the same problem when my twin 120GB drives died in my HDVR2. One of them developed back blocks, and when the TiVo hung, it corrupted the superblock, writing ascii digits over the top of just the magic number. I had to hack the code in mfs.c to work around the problem while I was developing my mfs_tmfstream tool.

I was able to extract most of the recordings on those disks and upload them back to my HDVR2 with different drives.

BTUx9
01-16-2004, 12:45 AM
RonnyThunder: Sorry, I didn't answer your actual question.
MFS_DEVLIST is the env. variable that holds the list of MFS partitions.

ronnythunder
01-16-2004, 12:45 AM
thanks jonbig, i'll give that a try. i'm using a knockoff of the mfstools 2.0 disk that has a lba48 kernel (my bad drive is a 160gb actually being used as a 160, i.e. tivo has a lba48 kernel).

ronny

ronnythunder
01-17-2004, 01:05 AM
woo-hoo! great news, it's working. i tried to copy the disk to another 160gb, but it was spewing errors on every sector once i got about 2/3 of the way into the disk. i gave up on the copy and just started in with the mfs_ls, dumpobj and export, and it's working great so far.

ronny

DPlatt
01-18-2004, 11:07 PM
if you were running mfs_ftp prior to the crash the fsid numbers will be in /dev/hda9 /mfs_ftp/cache/recname.pts


I'm having trouble with this-- I'm attempting to recover some shows from a 3.1 DirecTiVo Series 1 that crapped out, and I was running mfs_ftp. With the drives in the PC, I navigate to my mfs_ftp/cache directory, and it's empty. Aside from the vplay method, is there any other way to obtain FSIDs?

jonbig
01-19-2004, 01:46 AM
I'm having trouble with this-- I'm attempting to recover some shows from a 3.1 DirecTiVo Series 1 that crapped out, and I was running mfs_ftp. With the drives in the PC, I navigate to my mfs_ftp/cache directory, and it's empty. Aside from the vplay method, is there any other way to obtain FSIDs?

This worked for me:

mfs_ls /Recording/NowShowingByTitle

ronnythunder
01-19-2004, 02:01 AM
This worked for me:

mfs_ls /Recording/NowShowingByTitlethis will give you the shows in alphabetical order; if you do "mfs_ls /Recording/NowShowingByClassic", you can get them in date-recorded order, but there's no titles. with a little crude scripting, you can get the titles in date-recorded order; here's what i used iirc:

for i in `./mfs_ls /Recording/NowShowingByClassic | cut -b 1-12`
do
./mfs_ls /Recording/NowShowingByTitle 2> /dev/null | grep $i
done

it's cheesy, but it works.

ronny

kevinallein
01-28-2004, 02:10 AM
I am investigating my failing drive and there is still a ton of valid data in the MFS Media Region but it seems, that most of the MFS application region is dead. Is there any way to extract shows without the MFS application region ? Or is this like repairing an old FAT file system with the FAT being destroyed, that you have all the individual blocks, but no other hint of what belongs together and in what sequence ?

Looking for help
Kevin

ronnythunder
01-28-2004, 03:31 PM
kevin, have you tried the techniques in this thread? what errors did you get?

ronny

kevinallein
01-29-2004, 02:48 AM
kevin, have you tried the techniques in this thread? what errors did you get?

ronny

So far I was trying to copy the content of the failing drive to another drive. I received my warranty exchange drive two days ago. I used a MFS tools backup from my original drive to create a working Tivo disk on the new drive with the same structure as the old drive and than I thought I would copy all the MFS partitions (not the other ones like Boot, Root, Bootstrap, var) from the old drive to the new drive. I am using dd_rescue (a special dd version, that does not abort on errors). Unfortunately there is a ton of error, if I access the MFS application region.
Am I wasting my time here on this drive or is there hope to recover, although the MFS application region partition is severly damaged.

Kevin

jonbig
01-29-2004, 03:10 AM
So far I was trying to copy the content of the failing drive to another drive. I received my warranty exchange drive two days ago. I used a MFS tools backup from my original drive to create a working Tivo disk on the new drive with the same structure as the old drive and than I thought I would copy all the MFS partitions (not the other ones like Boot, Root, Bootstrap, var) from the old drive to the new drive. I am using dd_rescue (a special dd version, that does not abort on errors). Unfortunately there is a ton of error, if I access the MFS application region.
Am I wasting my time here on this drive or is there hope to recover, although the MFS application region partition is severly damaged.

Kevin

Kevin, I've PM'd you with the location of my mfs_tmfstream tool that I've currently got out for beta testing. You can use it to extract shows in .tmf format that you can reinsert into a new drive on a TiVo that has been hacked to run mfs_ftp.

kevinallein
01-29-2004, 05:50 PM
Kevin, I've PM'd you with the location of my mfs_tmfstream tool that I've currently got out for beta testing. You can use it to extract shows in .tmf format that you can reinsert into a new drive on a TiVo that has been hacked to run mfs_ftp.

Hi Jonbig,
thanks a lot - I will have a go at it over the weekend. Am I right in the asumption, that it gets its info from the MFS application partition ? Or is there any way, that it can actally pieces from the MFS media partition ?

Kevin

jonbig
01-29-2004, 07:38 PM
Hi Jonbig,
thanks a lot - I will have a go at it over the weekend. Am I right in the asumption, that it gets its info from the MFS application partition ? Or is there any way, that it can actally pieces from the MFS media partition ?

Kevin

I'm not a master on how MFS is laid out. My disks that died also had problems in the MFS application partition, but I was still able to use my tool to find and extract the recordings I needed. So give it a whirl and report back.

kevinallein
02-02-2004, 07:21 PM
Hi guys,
this drive is really giving me a hard time. The partition map was corrupted and I tried to restore it. It looked O.K. with Dylan's boot disk, but when I tried to use the MFS tools, it complained about a bad tivo magic number. Is that magic number in the actual partition or somewhere in the map ? How can I reconstruct that ?
Please help, since I have to send back my RMA drive with the recordings.

Thanks
Kevin

BTUx9
02-03-2004, 03:10 PM
Hi guys,
this drive is really giving me a hard time. The partition map was corrupted and I tried to restore it. It looked O.K. with Dylan's boot disk, but when I tried to use the MFS tools, it complained about a bad tivo magic number. Is that magic number in the actual partition or somewhere in the map ? How can I reconstruct that ?
Please help, since I have to send back my RMA drive with the recordings.

Thanks
Kevin
The magic number is in the actual partition, and if it's not there, there's a good chance that you didn't rebuild the partition map correctly. Can you mount var and the root partitions (4,7,9)? If not, then theres a VERY good chance the partition info is wrong.

kevinallein
02-03-2004, 05:51 PM
The magic number is in the actual partition, and if it's not there, there's a good chance that you didn't rebuild the partition map correctly. Can you mount var and the root partitions (4,7,9)? If not, then theres a VERY good chance the partition info is wrong.

I had a MFS backup of my drive (without recordings). I restored that to the replacement harddrive and than tried to copy the MFS partitions with dd_rescue (special dd, that does not abort on errors) from the original drive. Somehow I messed up the partition map. I had noted down the restored map before and used pdisk under Dylan's bootdisk to recreate the partition map. Now I get that magic error. The MFS partitions had a few errors, but I believe not in the first few sectors. I am a bit lost now, how I could still save some of the recordings. In my ultimate stupidity I ruined the rest of the original drive yesterday by doing a MFS restore to it (I wanted to start the whole process again by restoring to the replacement drive and hooked up the wrong drive). Am I right, that MFS restore has completely ruined the chance to restore anything from the original drive ? I do not think it does anything to the media region, but I assume it completely wipes the application region ?

jonbig
02-04-2004, 03:36 PM
In my ultimate stupidity I ruined the rest of the original drive yesterday by doing a MFS restore to it (I wanted to start the whole process again by restoring to the replacement drive and hooked up the wrong drive). Am I right, that MFS restore has completely ruined the chance to restore anything from the original drive ? I do not think it does anything to the media region, but I assume it completely wipes the application region ?

It's dead, Jim.

kevinallein
02-04-2004, 05:11 PM
I am still hoping to extract something from the RMA drive. Is there a describtion of how MFS partitions look like or is the source of the MFS tools the only describtion ?

Kevin

ehf
02-07-2004, 06:56 PM
I'm trying to extract direcly from a good tivo HD directly to my computer w/o networking. I hooked up the tivo drive as primary master, and FAT32 drive as primary slave, and boot using JohnnyDeath's CD. using mfs_info, I set my MFS_DEVLIST to parts /dev/hda10 - 15. I then do a
mfs_ls "/Recording/NowShowingByTitle", which returns the list of programs on the drive. It looks like the last number on the line is the FIRST PART FSID of the program, so I take that FSID number and do
./mfs_dumpobj <fsid>
which returns a whole bunch of junk which I don't really understand (it's a lot of UNKNOWN=<some number>), but I try to extract from the info as many things that look like FSIDs as possible, and then do a
./mfs_export <fsid> <myfile>

So, usually I get one file that is really huge, and the other smaller files around 1-2k. I then boot back into windows and run TyTool to try and generate keyfiles for the streams, but none of the files seem to be valid tystreams for tytools to use. It complains that it "failed to get the first initial 10 chunks"

I've also tried concatenating all of the parts into one .ty file, and that does not seem to work either.

All the programs on the tivo were unscrambled, using the bash hack, but an update came down and trashed everything, so I took out the HD and I'm trying to extract those unscrambled recordings now.

Any hints, tips or tricks will be greatly appreciated.

NutKase
02-14-2004, 09:57 AM
All the programs on the tivo were unscrambled, using the bash hack, but an update came down and trashed everything, so I took out the HD and I'm trying to extract those unscrambled recordings now.

All of your recordings are still scrambled.

NutKase

ehf
02-14-2004, 05:23 PM
All of your recordings are still scrambled.

By the update downloading, it rescrambled all of my recordings that I previously could grab off the tivo through tytools over the network and play fine?

ronnythunder
02-15-2004, 01:42 AM
./mfs_export <fsid> <myfile>

So, usually I get one file that is really huge, and the other smaller files around 1-2k. if you're getting the right number as fsid, the parts you get should be a max of exactly 536870912 bytes (meaning 512mbyte parts; some systems seem to have 256mbyte parts, in which case the magic number is 268435456)

so, if you have a show that is greater than 512mbytes, you'll get some number of parts of 536870912 plus (likely) a final part that's smaller.

if you're getting anything bigger than 536870912, you're not getting the right fsids. if you have anything smaller than 131072, it's not a valid fsid; that's the minimum size of a chunk.

ronny

carrotz
03-05-2004, 03:01 AM
Jonbig is your mfs_tmfstream tool still being worked on?

jonbig
03-05-2004, 03:31 PM
Jonbig is your mfs_tmfstream tool still being worked on?

I haven't had much time to work on it recently, but it seems pretty solid. I need to find some time to research how I ought to release it generally.

rc3105
03-06-2004, 02:06 AM
imho... start a thread on alt.org & post the source there - that's pretty much the 'official' repository of progress when it comes to mfs code ;)

carrotz
03-07-2004, 12:48 AM
Jonbig, do you need one last beta tester? I have a tivo that keeps rebooting every 5 minutes and won't let me get programs off by any other means.

jonbig
03-07-2004, 06:49 PM
imho... start a thread on alt.org & post the source there - that's pretty much the 'official' repository of progress when it comes to mfs code ;)

Riley, my main question is how to avoid stepping on Tridge's toes, since my stuff is based on his code with some patches.

BTUx9
03-07-2004, 06:55 PM
Riley, my main question is how to avoid stepping on Tridge's toes, since my stuff is based on his code with some patches.
EVERYBODY's stuff it based on Tridge's with some patches. It's GPL, and I think that's what he wants.

rc3105
03-07-2004, 08:27 PM
Riley, my main question is how to avoid stepping on Tridge's toes, since my stuff is based on his code with some patches.

long as you don't intentionally violate the "gpl spirit" you're good. if he didn't want folks to be able to benefit/enhance that code it would be under a different license. that prolly means if you're going to post binaries (here on dd for instance) either include the source (allways good) or a link to the source somewhere easily accessable like AO

search here is a nightmare & many things will probably get overlooked in the transfer to the new layout ( lots of groovy utils dissapear that way :eek: ) AO & the cvs / sourceforge it links to are about as close as we'll get to an official API for the tivo-verse

one other thing to consider, a post/thread here will attract lots of noob questions & support requests. if you want to be the poster-boy for retrieving recordings from toasted drives go for it. if you just want to provide a util that sleeper or whoever can incorporate into a knoppix extract boot cd (& let him write a howto etc) then ao's the place to post

TheWickedPriest
03-08-2004, 04:05 AM
If your code is based on GPL'd code, then you must release it under the GPL, or not at all. There is no other legal choice.

BAHitman
03-16-2004, 12:27 PM
Is there any update on when this will be available as I would like to test it out. I would be happy to add to the beta testing going on.

contemplativo
03-31-2004, 08:32 AM
Wouldn't connecting Tivo drives to a PC be the preferred extraction strategy for archiving the entire contents of full healthy drives, not just damaged drives? Downloading hundreds of gigs of files over a network connection seems relatively straightforward, but very slow. Wouldn't it be much faster to extract from drives directly connected to a PC (especially if one had removable drive sleds and bays set up on the Tivo and PC)?

BAHitman
03-31-2004, 09:12 PM
Well, sure, you could use direct extraction for archiving, however, you have to power down your Tivo, and then put the drives in the PC, you would also have to make sure you didn't miss any TV While the thing were beeing archived...

contemplativo
04-01-2004, 05:00 AM
You're right, BAHitman, swapping drives out could typically be more trouble than it's worth, at least when archiving small amounts of material at a time.

In my case, I've got tons to archive and getting drives out of the Tivo and into the PC is no problem. I've got a stack of full 120 gig drives sitting around in removable drive trays, which I can slide into and out of an external drive bay connected to my Tivo. I just slide one of the disengaged drives into my PC's drive bay, and I'm ready to extract, without interrupting viewing or recording.

I'm wondering about the speed of the rest of the procedure, compared to networked extraction, when extracting 100+ gig of files at a time.

BTUx9
04-01-2004, 05:02 PM
I'm wondering about the speed of the rest of the procedure, compared to networked extraction, when extracting 100+ gig of files at a time.
It's quite fast.

contemplativo
04-05-2004, 03:39 PM
jonbig, I'm tempted to try to make a tool like yours, probably starting this week, but I would take no pride in reinventing your wheel and would love to be spared the effort. Any chance you would include me in your testing group? It would be much appreciated.

jonbig
04-05-2004, 04:31 PM
jonbig, I'm tempted to try to make a tool like yours, probably starting this week, but I would take no pride in reinventing your wheel and would love to be spared the effort. Any chance you would include me in your testing group? It would be much appreciated.

I'm just about ready to release for beta-test. Nag me by the weekend if I haven't posted the tool by then.

Quadropheniac
04-07-2004, 11:33 PM
When I run mfs_* i get an error
"llseek failed"

I have run a search query and only two answers returned, all were irrelevant
I have also not had any luck finding instructions for installing gcc which I believe may be the problem (not sure) any help would be appreciated, unless your name is David Bought. I've already experienced that flame and ask that you just ignore this if you can't refrain from being cynical

I did do my homework. My HDVR2 was monted,unscrambled, wirelessly networked, and running MFS_FTP.

This "DEMO Mode" has decided to reboot my tivo and delete everything so I am trying to get my movies off before it does

thx to all who helped me get so far I truly appreciate it

taycrowe
04-10-2004, 04:01 PM
i was playing with XplusZ in Xtreme2.52(series1 DSR-6000) and while i was playing around with DELETE and UNDELETING my shows, when i saw that the FSID was listed onscreen when i used the "display deleted program info" command #0#10 in the call waiting prefix screen of the menu(the standard place to launch your command codes for XplusZ).....
so if you delete the programs you want to extract, then do the #0#10 display command, it will print out your FSIDs on the screen....write them down, then go and undelete those programs........
that may be cheating, but for me it seems a lot easier for a novice to take on as a first step in extracting.....
later,
taycrowe

Quadropheniac
04-11-2004, 01:17 PM
Thanks for the help I wish I could try that but I'm running sleepers 1.02 and the latest patches/fixes for that distro

carrotz
04-11-2004, 04:12 PM
Jonbig - Will you be posting your mfs_tmfstream beta soon? Anxious to try it out.

Dave

jonbig
04-12-2004, 02:26 AM
Jonbig - Will you be posting your mfs_tmfstream beta soon? Anxious to try it out.

Yes, I posted a beta version yesterday in the Extraction development forum.

alphaclown
04-12-2004, 11:57 PM
Thanks jonbig - mfs_tmfstream allowed me to extract the shows on my virgin S2 SA drive without any problems..

Now I have 20 gigs of .tmf files I'd like to transfer to my hacked tivo drive, directly connected. What's the best way to do this w/o using mfs_ftp?

BTUx9
04-13-2004, 12:05 AM
Now I have 20 gigs of .tmf files I'd like to transfer to my hacked tivo drive, directly connected. What's the best way to do this w/o using mfs_ftp?
Umm... good luck?

To the best of my knowledge, there aren't any publicly available tools out there that'll let you insert recordings to a tivo drive that isn't in a tivo.

BTUx9
04-13-2004, 02:48 PM
Here's a link to the mfs_tmfstream beta thread:
http://www.dealdatabase.com/forum/showthread.php?t=34027

ursapooh
06-19-2004, 04:43 PM
I have looked all around for a way around my problem. I have a SAT-T60 upgraded to 2 120GB hard drives which is in a GSOD cycle. The drives mount when I boot from the MfsTools cd but TiVo will not boot. All of the extraction options seem to require a Linux box (I donít have that only a bootable Linux cd) or a bootable TiVo environment with Ethernet access (I donít have that either). Give my set of circumstances, just two loose TiVo drives, a bootable Linux cd, and a Windows based PC, are there any options for me?

rc3105
06-19-2004, 10:46 PM
the mfstools cd boots your pc into a linux invironment, connect drives like this

primary master == fat32 drive with space to save recordings
primary slave == cd/dvd to boot mfs tools cd from
secondary master == tivo a drive
secondary slave == tivo b drive

and re-read this thread till it makes sense. the instructions, links to required files & related threads are all right here

contemplativo
06-20-2004, 03:39 AM
Using the mfs_tmfstream extraction utility (see jonbig's related thread), I also had luck booting from Sleeper's Knoppix-Lite.

stevel
06-20-2004, 09:47 AM
All of the extraction options seem to require a Linux box (I donít have that only a bootable Linux cd) Voila! That and a regular PC gives you a "Linux box".

ursapooh
06-20-2004, 11:35 AM
I have read this thread over and over and though still confusing I have understood enough to have issues. 1) Some of these replies mention compiling code on the Linux box. I can't do that with just a bootable Linux boot cd or floppy. 2) Some of these replies mention that I would have had to have had the foresight to run unscramble on my SAT-T60 before the crash to be able to extract after the crash. I didn't. 3) Some of these replies mention adding a program to TiVo. My TiVo won't boot. 4) Some of these replies mention TivoNet and TivoWeb. My TiVo isn't networked. 5) Some of these replies mention extraction from Series 2. My TiVo is Series 1.

So noting all that I am going to start with jonbig's thread "mfs_tmfstream tool BETA release" and use rec3105's suggestion for mounting my drives. Knowing my issues is there anything I need to be aware of in following jonbig's thread?

BTUx9
06-20-2004, 02:42 PM
I have read this thread over and over and though still confusing I have understood enough to have issues. 1) Some of these replies mention compiling code on the Linux box. I can't do that with just a bootable Linux boot cd or floppy. 2) Some of these replies mention that I would have had to have had the foresight to run unscramble on my SAT-T60 before the crash to be able to extract after the crash. I didn't. 3) Some of these replies mention adding a program to TiVo. My TiVo won't boot. 4) Some of these replies mention TivoNet and TivoWeb. My TiVo isn't networked. 5) Some of these replies mention extraction from Series 2. My TiVo is Series 1.

So noting all that I am going to start with jonbig's thread "mfs_tmfstream tool BETA release" and use rec3105's suggestion for mounting my drives. Knowing my issues is there anything I need to be aware of in following jonbig's thread?
I'm afraid you're in big trouble.

Without having your tivo networked, even if you recover your shows with mfs_tmfstream, you won't be able to re-insert them without major effort and knowledge. Even with the know-how, you'd need an extra drive with enough space to hold your shows, either temporarily or to be your new tivo drive.

Of course, if you could find and fix the error in MFS that's giving the GSOD, then you'd be golden, but if the guys who built tivo and wrote the s/w can't write a utility to fix it, I imagine it'd take a pretty talented hacker to do so.

rc3105
06-20-2004, 03:25 PM
edit rc.sysinit to start tivosh (as far as switcherstart -m) but not actually boot the tivo. this will let you run scripts that need mfs access. use get_DC_key.tcl to extract the drive configuration key which is required later for restoring recordings

if you can't do that (not enough knowldedge, mfs really toasted, whatever) you'll have to get/write a util to extract the DC_key from the drive while connected to the pc

contemplativo
06-21-2004, 04:54 AM
Without having your tivo networked, even if you recover your shows with mfs_tmfstream, you won't be able to re-insert them without major effort and knowledge. Even with the know-how, you'd need an extra drive with enough space to hold your shows, either temporarily or to be your new tivo drive.

edit rc.sysinit to start tivosh (as far as switcherstart -m) but not actually boot the tivo. this will let you run scripts that need mfs access. use get_DC_key.tcl to extract the drive configuration key which is required later for restoring recordings

if you can't do that (not enough knowldedge, mfs really toasted, whatever) you'll have to get/write a util to extract the DC_key from the drive while connected to the pc.

Riley and BTUx9 point towards a reinsertion strategy that might be called "direct (non-networked) reinsertion," for which no widely used newbie-friendly utility is available.

An alternative might be to install networking hardware on your tivo, offload recordings onto a temporary storage drive using mfs_tmfstream or equivalent, create a new tivo drive or set of drives using mfstools or equivalent along with the tivo drive image appropriate for your unit, install drivers for turbonet or whatever on that new tivo drive, then reinsert the shows onto the tivo from the temporary storage drive using mfs_ftp. All this could be done with user-friendly utilities.

A further alternative might be to offload the recordings using mfs_tmfstream or the like without reinserting. You could watch them as *.ty's on your computer using Mplayer or convert them to *.mpg's (using tytools or tystudio) and burn them to DVD's (e.g. with the help of tytools).

ursapooh
06-21-2004, 07:51 AM
At this point re-insertion is a mere dream. Extraction to any viewable state is my goal. But is extraction possible on a SAT-T60 without having no scramble?

BTUx9
06-21-2004, 01:55 PM
A further alternative might be to offload the recordings using mfs_tmfstream or the like without reinserting. You could watch them as *.ty's on your computer using Mplayer or convert them to *.mpg's (using tytools or tystudio) and burn them to DVD's (e.g. with the help of tytools).
Umm... I don't think so... the streams are still encrypted.


At this point re-insertion is a mere dream. Extraction to any viewable state is my goal. But is extraction possible on a SAT-T60 without having no scramble?
If there is any way out there to view scrambled streams without using the tivo itself to unscramble them, I haven't seen it.
It's also pretty unlikely because part of the decryption key exists within the crypto chip on the tivo m/b.
This is why encrypted streams need to be played on the tivo they came from (unless you physically move the crypto chip to a different m/b... not for the faint of heart).

contemplativo
06-21-2004, 02:09 PM
Right, re-reading your second post, I see you mentioned that your recordings are scrambled. As far as I know, unscrambling these recordings for the purpose of off-tivo viewing, using unscramble.o, would involve a few seconds of playback on your unit. So back to options 1a or 1b: non-networked reinsertion or networked reinsertion.

ursapooh
06-21-2004, 08:51 PM
So then non-networked reinsertion it is. What now?

ursapooh
06-26-2004, 10:24 AM
An alternative might be to install networking hardware on your tivo,

I have an unused TurboNet card. But I am not sure how to make the DirecTiVo network aware. My TiVo gets the GSOD so if I have to boot TiVo then this won't work. I understand that I can edit a file on my TiVo that makes it boot some but not all of the TiVo programs.


offload recordings onto a temporary storage drive using mfs_tmfstream or equivalent,

I have an empty hard drive. But I am not sure how to initialize it for storage of linux files or make it network aware. All I have to work with is a linux boot cd.


create a new tivo drive or set of drives using mfstools or equivalent along with the tivo drive image appropriate for your unit,

I have the drives plus other working DirecTiVos so I could probably create an empty TiVo installation.


install drivers for turbonet or whatever on that new tivo drive,

Same process as above once I understand how to make the TiVo network aware..


then reinsert the shows onto the tivo from the temporary storage drive using mfs_ftp.


I can get mfs_ftp.


All this could be done with user-friendly utilities.


Sounds great. Can you help me get started by looking at my issues above?

stew1165
06-30-2004, 07:43 AM
. You need to identify the FSID of the specific showing you want to extract, then use tridges mfs_export on it. You can normaly do this with mfs_stream, but for some reason its broken for newer software revisions. Unless somebody fixes it, you aren't going to be able to identify the FSID of the shows you want to extract.

I have drives with programs recorded before hacking, that I have removed from my DTivo. I have replaced the drives with new ones and have hacked the Dtivo so future recordings will be "extractable". I have printed out AlphaWolf's hard drive extraction instructions but I don't know what any of the FSID's are for the programs on the pre-hacked drives. Does anyone know of a way to determine what the FSID's are so I can begin the extraction process? Or is there a way to just extract all of the programs! Ultimately, I will be extracting them all anyway. Thank you in advance for your assistance. :)

contemplativo
06-30-2004, 02:06 PM
I have an unused TurboNet card. But I am not sure how to make the DirecTiVo network aware. My TiVo gets the GSOD so if I have to boot TiVo then this won't work. I understand that I can edit a file on my TiVo that makes it boot some but not all of the TiVo programs.

OK, I haven't done some of these things for a long time, but I thought that the only thing necessary to make the unit network aware is to put the card in and install the turbonet driver by attaching the drive to your pc and using a linux boot cd with the right tools. This, of course, only after you turn your empty drive into a Tivo drive using mfstools or the equivalent. Anyhow, I seem to recall that all I did was follow the instructions supplied at the turbonet website, so I suggest you try the same. By the way, did you do a search on GSOD? It's not supposed to be difficult to resolve, just let your Tivo sit and fix itself over a period of 15 minutes to 2 days, even without the phone line.



I have an empty hard drive. But I am not sure how to initialize it for storage of linux files or make it network aware. All I have to work with is a linux boot cd.

Get mfstools (2.0, I believe) and follow the instructions on the readme. Turning the drive into a tivo drive is to turn it into a linux box of a certain sort.


I have the drives plus other working DirecTiVos so I could probably create an empty TiVo installation.

Good, as long as they are the same model.

BTUx9
06-30-2004, 02:44 PM
I have drives with programs recorded before hacking, that I have removed from my DTivo. I have replaced the drives with new ones and have hacked the Dtivo so future recordings will be "extractable". I have printed out AlphaWolf's hard drive extraction instructions but I don't know what any of the FSID's are for the programs on the pre-hacked drives. Does anyone know of a way to determine what the FSID's are so I can begin the extraction process? Or is there a way to just extract all of the programs! Ultimately, I will be extracting them all anyway. Thank you in advance for your assistance. :)
Look for a thread on tmfstream by jonbig. That tool makes things much easier. There's also a small support program by me in the thread that lets you get a list of all your recordings, edit it to remove those you aren't interested in, and then run tmfstream for each recording remaining in the file.

cagey
07-08-2004, 12:34 AM
I'm having a problem getting vplay to work using Mandrake 10.0. I have followed AlphaWolf's instructions from the beginning of the thread. I get the error below. I am NOT Linux inclined, so this is very alien to me, but I need to get some videos off of a bad drive. What does the error mean? I'm wondering if I need to insall gcc compiler. I looked into it, but Linux is just so different from Windows I'm having problems understanding how to install stuff in Linux. Any help would be appreciated.

[dg132@localhost dg132]$ cvs -d :pserver:cvs@pserver.samba.org:/cvsroot login
Logging in to :pserver:cvs@pserver.samba.org:2401/cvsroot
CVS password:
cvs login: warning: failed to open /home/dg132/.cvspass for reading: No such fil
e or directory
[dg132@localhost dg132]$ cvs -z5 -d :pserver:cvs@pserver.samba.org:/cvsroot co tivo
cvs checkout: Updating tivo
cvs checkout: Updating tivo/ethernet
U tivo/ethernet/3c503.h
U tivo/ethernet/8390.c
U tivo/ethernet/8390.h
U tivo/ethernet/Makefile
U tivo/ethernet/README
U tivo/ethernet/TIVOISA.ISP
U tivo/ethernet/TIVOISA.JED
U tivo/ethernet/dcr.c
U tivo/ethernet/dumpdrc.c
U tivo/ethernet/isr.c
U tivo/ethernet/probe.c
U tivo/ethernet/reset.c
U tivo/ethernet/tcd8390.c
U tivo/ethernet/tivoether.c
U tivo/ethernet/tivoisa.h
U tivo/ethernet/tivone.c
cvs checkout: Updating tivo/ethernet/ispProgs
cvs checkout: Updating tivo/guidekit
U tivo/guidekit/AusTVGuide.pm
U tivo/guidekit/FoxTVGuide.pm
U tivo/guidekit/Makefile
U tivo/guidekit/TivoGuide.pm
U tivo/guidekit/common.c
U tivo/guidekit/daily.cron
U tivo/guidekit/db.pl
U tivo/guidekit/readguide.c
U tivo/guidekit/schema.txt
U tivo/guidekit/slice.h
U tivo/guidekit/weekly.cron
U tivo/guidekit/writeguide.c
cvs checkout: Updating tivo/palkit
U tivo/palkit/Makefile
U tivo/palkit/README
U tivo/palkit/palmod.c
U tivo/palkit/ukchan-2.5.5.c
U tivo/palkit/ukchan-2.5.5.o
U tivo/palkit/ukchan.c
U tivo/palkit/ukchan.o
U tivo/palkit/ukchan_grant.c
U tivo/palkit/ukchan_grant.o
U tivo/palkit/ustuner-2.5.5.c
U tivo/palkit/ustuner3.c
cvs checkout: Updating tivo/vplay
U tivo/vplay/.cvsignore
U tivo/vplay/Makefile
U tivo/vplay/README
U tivo/vplay/bitmap.c
U tivo/vplay/crc.c
U tivo/vplay/export.c
U tivo/vplay/io.c
U tivo/vplay/make-preload-schema.pl
U tivo/vplay/mfs.c
U tivo/vplay/mfs.h
U tivo/vplay/mfs_bitmap.c
U tivo/vplay/mfs_dump.c
U tivo/vplay/mfs_dumpobj.c
U tivo/vplay/mfs_dumpschema.c
U tivo/vplay/mfs_export.c
U tivo/vplay/mfs_getslice.c
U tivo/vplay/mfs_info.c
U tivo/vplay/mfs_ls.c
U tivo/vplay/mfs_poke.c
U tivo/vplay/mfs_purge.c
U tivo/vplay/mfs_streams.c
U tivo/vplay/mkproto.awk
U tivo/vplay/object.c
U tivo/vplay/partition.c
U tivo/vplay/preload_schema.h
U tivo/vplay/proto.h
U tivo/vplay/query.c
U tivo/vplay/schema.c
U tivo/vplay/schema.txt
U tivo/vplay/util.c
U tivo/vplay/vplay.c
U tivo/vplay/vplayer.c
U tivo/vplay/vserver.c
U tivo/vplay/vsplit.c
[dg132@localhost dg132]$ cd tivo/vplay
[dg132@localhost vplay]$ make host
gcc -Wall -g -c mfs.c -o i386/mfs.o
make: gcc: Command not found
make: *** [i386/mfs.o] Error 127

BTUx9
07-08-2004, 03:01 AM
cagey: you might want to try mfs_tmfstream (look for its thread). It's a good tool for rescuing recordings.

cagey
07-08-2004, 10:54 AM
cagey: you might want to try mfs_tmfstream (look for its thread). It's a good tool for rescuing recordings.

Can it be done to a dead drive. One that Tivo is on an endless bootup loop?

NutKase
07-08-2004, 11:06 AM
Can it be done to a dead drive. One that Tivo is on an endless bootup loop?

So do you have a 'dead' drive or just a tivo in a bootup loop?

A bootup loop is a separate issue.


NutKase

cagey
07-08-2004, 09:18 PM
The drive is in a bootup loop. And I ran Maxtor's Power Max Utility and it came back with an error code. So, I will be returning the drive for a new one. I can also see the partions on the drive, so I'm trying to extract a few things before returning it.

BTUx9
07-08-2004, 11:09 PM
Can it be done to a dead drive. One that Tivo is on an endless bootup loop?
mfs_tmfstream uses the same functions as vserver, for the most part, so if one of them works, there's a good chance the other will (tmfstream has the added bonuses of working with encrypted streams, restoring some show info, and I've written a trivial program that automates it somewhat)

cagey
07-09-2004, 01:23 AM
mfs_tmfstream uses the same functions as vserver, for the most part, so if one of them works, there's a good chance the other will (tmfstream has the added bonuses of working with encrypted streams, restoring some show info, and I've written a trivial program that automates it somewhat)


I am going to read more on the mfs_tmfstreams and I will give it a try.

I re-read the instructions posted by AlphaWolf, and he mentioned making sure all of the developement packages were installed. So I reinstalled Mandrake 10.0 making sure I installed the developement programs, but I got this error message.

[dg132@localhost dg132]$ cd tivo/vplay
[dg132@localhost vplay]$ make host
gcc -Wall -g -c mfs.c -o i386/mfs.o
gcc -Wall -g -c object.c -o i386/object.o
gcc -Wall -g -c schema.c -o i386/schema.o
gcc -Wall -g -c query.c -o i386/query.o
query.c:106: error: conflicting types for `query_object'
proto.h:29: error: previous declaration of `query_object'
query.c:28: warning: `memdup' defined but not used
make: *** [i386/query.o] Error 1

What would cause this error and how can I fix it. Is there a better copy of Linux to use for this? I have Fedora2, but my keyboard and mouse wouldn't work with it (very frustrating!).

cagey
07-14-2004, 02:33 AM
I'm trying to extract shows from a s1 dtivo. I have sucessfully extracted a 1kb file using mfs export. But I read that if I got only a 1kb file I used the wrong fsid number wich was the main number.

I tried ./mfs_dumpobj <fsid> and I got a bunch of stuff. I tring to determine what fsid numbers I should be using. I read that I should look at the "RecordingPart." but I'm still confussed.

I thought I got it going because when I extracted a file, it would have a percentage countdown and then there wouldn't be a *.ty file in my destination directory.

Can someone please explain to me what fsid numbers I should be looking at?

Thanks,

cagey
07-20-2004, 06:24 PM
Is this a dead thread? Can anyone please give me some ideas as to what I might be doing wrong with these fsid numbers.

Thanks,

ibsleepn
08-01-2004, 12:54 PM
I'm trying to pull streams off a drive that is stuck in a reboot loop. I was able to use the mfstools boot floppy to mount partitions and see the contents of the drive and try to backup the drive (which failed btw). I have since got a hold of knoppix 3.3 mini to run the mfs_tmfstream files. I get "llseek failed" when ever i run mfs_ls vplay in order to get a list fsid's. I am trying to figure out if the drive is totaly hosed now or what.

Series 1 DTivo 3.01b dual WD 120's
Knoppix 3.3 mini boot cd
export MFS_DEVICE=/dev/hdc10
export MFS_DEVLIST=/dev/hdc
with the tivo drives in Secondary Master and Slave.

Knoppix would't auto mount the 2 tivo drives but it does see them.

Ran this command.. "vplayer -l" and recieved "llseek failed"

Ran this also "mls_ls /Recordings/NowShowingByTitle" and recieved "llseek failed"

and.. "mfs_info" with "llseek failed" response.

Been doing a lot of reading and did a search on llseek and was only two items with only 1 pertinant and he didn't get a response on his question about it.

Any help??

Ib

btw ran wd utilities and it returned error code of a failing drive. Already have the replacement from them but still have 20 days to return the rma drive.

Ivan
08-01-2004, 09:39 PM
The llseek error probably means you're not root.

Ivan
08-02-2004, 12:21 AM
I just spent the better part of a weekend repairing my Tivo from a dead drive, and Since I see other unanswered questions in this thread about it, and because it was hell getting all the info from various places I figured I'd collect my experiences with this together into a post. Hopefully somebody will find this helpful.

First a little background. I have a DVR7000 that I had upgraded to 120GB. I dnd no other modifications. It's been running great for over a year (The S1 sitting underneath it has been going great for 4.5 years with 2 disks. Damn you samsung!) but recently it didn't come back from a brief power outage. I popped the backup drive in and everything worked fine, so I figured it was the disk. A quick scan turned up the dreaded error 0x40 on several sectors... So with a new 120GB drive in hand (seagate this time. 5 year warranty, .2db louder than the samsung) I loaded my backup, expanded the image, forced a call to get to 3.1.1c and felt very empty... Most of the old disk was good, so I was able to get almost every show off my old drive and import it to the new one. As an added bonus I'm now mfs_ftp and tivoweb enabled. Here's how it worked:

This all assumes you have a working linux box. If you don't, get a debian nework installer, and run 'apt-get install build-essential' after you're up and running. That will give you everything you need, and you'll like it.

First you need a way to run commands on your tivo. That means you want a shell of some sort. I opted for bash through telnet. I picked up a linksys USB200M ($29.95 at CompUSA) and set about getting it to work. You'll want to set up two kernel monte to load your 2.4.4-tivo kernel with no initrd. I did this manually, though I pilfered files off of Sleeper's ISO that I couldn't find elsewhere. I recommend the instructions by Cobelli at http://generationtivo.com/monteguide.html. There are other guides, but this one will give you the best idea of what's going on, and why. I recommend running tivoftpd by hand at the telnet prompt instead of in the rc.sysinit.author though for a variety of reasons. If your network doesn't come up, running tivoftpd will reboot your tivo, if you run it by hand it will exit when you exit your shell and you won't forget you have a gaping security hole on your hands, and there's also no point in leaving stuff you're not using running and eating up resources.

Tips you'll be interested in while following that guide:


There's a 3.1U5 kernel and Filesystem image on Sleeper's ISO in /cdrom/tivoutils/kernel

You can use 'dd' to copy that filesystem image onto your root partition ('D' partition in the guide) just like you did with the kernel. Ignore what it says about tar.

You don't have to make a romfs. Use the one in /cdrom/romfs on Sleeper's ISO. It's the same as what you'd make by hand.

Get the networking modules from the first post in this thread (http://dealdatabase.com/forum/showthread.php?t=24626).

Once you can telnet in you can put the screws in and be done with the drive swapping. Everything else you'll ever need to do can be done with the bash prompt.

Telnet in and run tivoftpd, ftp in and upload the mfs_ftp tarball from the first post in this thread (http://dealdatabase.com/forum/showthread.php?t=21915). You'll need tar to uncompress it, and it contains file types you can't transfer over ftp, so you *must* untar it on the running tivo. There is no 'tar' on the tivo by default. (Or 'ls'. Grrr.) You should upload the 'busybox' binary from the tivoutils/busybox directory on Sleeper's ISO into /bin on the tivo. Then you can run 'ln -s /bin/busybox /bin/tar' to have a working 'tar' command. If you'd like you can repeat that command with 'ls' and 'ps' instead of tar for your own sanity.

Now that you have tar, and the mfs_ftp tarball you need to do the following:


'cd /var/local'
'tar xvf /path/to/mfs_ftp.tar'


Now with your ftp client upload s2bins.tar from the first post in this thread (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=29459) and do the same thing you just did with mfs_ftp.tar.

Finally, unzip the other archive from that post, unzip, and upload the enclosed mfs_extract and mfs_streams into the newly created /var/local/mfs_ftp directory. (Yes this is getting rediculous. It's still faster than building a cross compiler. Trust me.)

Type 'cd /var/local/mfs_ftp; mfs_ftp.tcl' and you're running. Everything else happens on your linux box in the extraction of shows and uploading.

The rest of the instructions will be in the next post down. This one is getting too long.

Ivan
08-02-2004, 12:45 AM
... continued from the previous post.

Now that you've got remote access to your tivo, and a way to insert shows via ftp, you need to extract the shows from the old drive. With 3.x you'll need the version of tridge's vplay that has been modified and posted in this thread (http://dealdatabase.com/forum/showthread.php?t=34027). I got the source and built it ('make clean all' in the uncompressed directory if you have a linux system configured as I described above), but the binaries should work just fine.

You also need your linux box to be able to see the tivo partition table. The tivo partition table is a mac partition table with a different magic number and a fixed block size. You could modify fs/partitions/mac.c in your kernel tree and build a new kernel, or you could just grab one of the bootable hack discs out there and use the kernel from that. A list of mfs partitions will get printed out at boot, and you'll need to put those into an environment variable in order to run the mfs_* commands. You'll want to do 'export MFS_DEVICE="/dev/hdX"' where X is the letter that corresponds to your tivo drive, and then 'export MFS_DEVLIST="/dev/hdx10 /dev/hdx11..."' but with the list of all your mfs partitions. This is conviently documented in the readme that came with the vplay code.

Now for the least explained part of the entire process. To extract shows with mfs_tmfstream you need "fsids". The way to get these correctly from your unhacked and dying drive is to run './mfs_ls /Recording/NowShowingByTitle'. This will give you a list of shows. The first column is the fsid you want for the extraction. mfs_streams will give you better show details, but the fsids it prints are *incorrect*.

Now that you've got the fsids you can extract and upload. Extract with './mfs_tmfstream -o show.tmf <fsid>' (you'll use just about 512MB for each half hour). Upload them by ftping to port 3105 on your tivo ('ftp <tivo ipaddress> 3105'), and typing 'cd /tmf', then 'put show.tmf'.

You can automate this, but since your drive is dying, this will likely break when it gets to the parts of the drive that are bad. I used a hacked copy of Roderick Schertler's 'ftp-upload' perl script (http://search.cpan.org/~rosch/ftp-upload-1.4/) with the login section commented out and the port number changed in a bash for loop. To do this, make a text file with one fsid per line and then run the following as one command:

'for i in `cat fsids.txt`; do mfs_tmfstream -o $i.tmf $i; ftp-upload -h <tivo ip-address> -d /tmf $i.tmf; rm $i.tmf; done'

Then wait a long time... I've been seeing about 2MB/sec transfers. It's a 120GB drive...

Hopefully this helps somebody... Good luck!

flynmoose
08-31-2004, 08:50 PM
You can automate this, but since your drive is dying, this will likely break when it gets to the parts of the drive that are bad. I used a hacked copy of Roderick Schertler's 'ftp-upload' perl script (http://search.cpan.org/~rosch/ftp-upload-1.4/) with the login section commented out and the port number changed in a bash for loop. To do this, make a text file with one fsid per line and then run the following as one command:

'for i in `cat fsids.txt`; do mfs_tmfstream -o $i.tmf $i; ftp-upload -h <tivo ip-address> -d /tmf $i.tmf; rm $i.tmf; done'


Ivan,
I could not get this working to save my life. What exactly did you hack in the perlscript (I'm not an PERL expert but I fiddled around commenting out lines related to password)? I'm not sure why it shouldn't be sending USERNAME and PASSWD since when I use the command line FTP it asks for same. Anyway, neither commandline FTP or ftp-upload work for me. Commandline will start but always creates a file of size 0 and I've already mentioned my problem with ftp-upload.

In frustration I looked around for another batch FTP program and came up with NcFTP (http://www.ncftp.com/download/). Works like champ.

Here is my version of your comand "FOR" loop.


for i in `cat fsids.txt` ; do ./mfs_tmfstream -o $i.tmf $i; ./ncftpput -u anyname -p anypass -P 3105 -DD -F 192.168.2.30 "/tmf" "$i.tmf" ; done

Even barfs out stats as it is transfering.

Also,



Then wait a long time... I've been seeing about 2MB/sec transfers. It's a 120GB drive...


Is that really what you are getting for throughput? :eek: I am seeing about 450 kB/s using WS-FTP from WINDOZE and about 560 kB/s using the above method.

I just started the command line and am trying to transfer files from a failing MAXTOR drive. We'll see where it craps out.

Moose

flynmoose
09-01-2004, 12:56 AM
Well after a faf with having to create an EXT3 partition to temporarily hold the file before ncftp took over (I had some movies that were over 2.0G which a FAT32 partition should be able to handle but wasn't when mounted from Knoppix), I'm having great success and should have full transfer sometime tomorrow evening if the old MAXTOR doesn't crap out by then.

Seeing averages of almost 620kB/s - sure would be nice to get that higher - must go do some more reading on mfs_ftp. Using a Linky USB200M with the 2.0 driver (ax8817-something).

Regarding what happens when it hits a bad spot on the drive (twice now after about 8 hours) - mfs_tmfstream craps, ncftp transfers whatever was outputed and the FOR loop keeps running. Haven't yet had a fatal disk error but I imagine that will cause the loop to just transfer a bunch of files FSID.TMF that will need to be deleted later.

jonbig
09-02-2004, 04:53 PM
I had some movies that were over 2.0G which a FAT32 partition should be able to handle but wasn't when mounted from Knoppix

Actually, no. FAT32 won't handle files larger than 2GB. I've learned this the hard way when writing some custom backup software.

rc3105
09-02-2004, 09:21 PM
funny, I have a couple of hundred files here > 2 gig stored in fat32

it's a 4 gig limit, allthough a lot of windows programs choke at 2 due to poor design

flynmoose
09-03-2004, 02:14 PM
it's a 4 gig limit, allthough a lot of windows programs choke at 2 due to poor design


That's what it says in the spec and I also have some files that are greater than 2GB that work fine under WINDOZE.

Evidently this is either a Knoppix issue or possibly a mfs_tmfstream issue because everything works fine if I have mfs_tmfstream output to a file on an EXT3 mount but using the same shell/config, it craps out if I output to a FAT32 mount.

I'm using Knoppix because the other Tivo Boot disks out there have the wrong GLIBC.so.6 library and the MFS utilities won't run.


Transfer went well - all but 2 movies out of >70 hours of video transfered over to my Western Digital drive. Over that entire transfer, I averaged a little over 600 KB/s. Took a lot longer than I thought!

WD drive working like a champ but it is considerably louder than the MAXTOR - lots of clunking that reverberates inside the media cabinet. I've got the drive in a drive bay on the outside of the case, so I'm going to put a mouse pad under and see if that helps.

lifetap
10-06-2004, 09:15 PM
I am about to run out of space on my series 2 tivo. I upgraded it to a 120gb drive long ago. I was hoping for an easy way to get files off without having to hack the tivo but it looks like thats not the case. I'm not technical and to be honest I'm having some difficulty with the various HOW-TO's out there.

I have another 120gb drive, the exact same model as the installed drive. Can I connect both, boot from a linux cd, and DD the tivo drive to the free drive? If that will make an identical backup, then I could attempt to hack one of the drives without fear of losing everything.

Also, if I have the second drive and am successful at modifying the first, can I transfer the streams across the network to the tivo to decrypt them? Or would it be better to use one of hte other methods?

Can anyone tell me if that will work before I take apart the tivo?

thx

TheWickedPriest
10-07-2004, 12:13 AM
If you can expand your system to a larger drive, it's not that much harder to do the more serious hacks. Yes, you can dd the drives. But you can forget about extracting your existing streams, sorry -- you have to have disabled encryption before the stream is recorded.

jonbig
10-07-2004, 05:26 PM
Clarification: If your TiVo streams are encrypted, you *can* extract them and then reinsert them back on the same TiVo successfully. You just can't view them to another TiVo or use a non-TiVo player to view them.

lifetap
10-07-2004, 05:34 PM
Thx guys or gals....

I'll do a DD backup of my drive tonight using the mfstools cd. I have an old Dell pIII 500 with no HD and a cdrom drive that should do nicely.

Right now I'm not too concerned with being able to play my recordings elsewhere, I'd just like to be able to pull some off to make room for the rest of the series. I normally just play back and record using a capture card. Unfortunatly 88 episodes in 10 days doesn't leave me a lot of time to clear off the recordings.

It's a SA S2 running 4.01b-02-2-230 and I'd like to try out some of the hacks, web access and ftp would be nice. I think I'll give it a go witht he backup drive.

thanks again.

Comedian
10-07-2004, 05:50 PM
Hi-
I'm trying to extract video from a "dead" HR10-250 drive. (The TiVo entered a boot loop, and wouldn't stop looping even after removing rc.sysinit.author and restoring kernel to 3.1.5d.)
Some of the videos on the TiVo are encrypted, but most or not. I compiled the vplay / mfs binaries under Debian Woody using libc 2.2.5. Booting from the lba48 MFSTools Boot CD, with my HR10-250 drive as Secondary Slave (HDD), and setting MFS_DEVLIST="/dev/hdd", I get the following error when I try to run mfs_info:

Found 4 MFS partitions
crc mismatch len=512 0xffffffff 0xe57ca975
sector wrong in zone (217140224 -1)

Any ideas? Am I on crack?

Comedian
11-02-2004, 02:46 AM
So... that was "on crack" then?

jbuehl
11-02-2004, 01:24 PM
Try running

# pdisk -l /dev/hdd

Partition map (with 512 byte blocks) on '/dev/hdd'
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1
2: Image Bootstrap 1 4096 @ 77261888 ( 2.0M)
3: Image Kernel 1 4096 @ 77265984 ( 2.0M)
4: Ext2 Root 1 262144 @ 77270080 (128.0M)
5: Image Bootstrap 2 1 @ 77532224
6: Image Kernel 2 8192 @ 77532225 ( 4.0M)
7: Ext2 Root 2 262144 @ 77540417 (128.0M)
8: Swap Linux swap 260096 @ 77802561 (127.0M)
9: Ext2 /var 262144 @ 78062657 (128.0M)
10: MFS MFS application region 1048576 @ 78324801 (512.0M)
11: MFS MFS media region 33100800 @ 44161088 ( 15.8G)
12: MFS Second MFS application region 1048576 @ 79373377 (512.0M)
13: MFS Second MFS media region 44161024 @ 64 ( 21.1G)
14: MFS New MFS Application 1024 @ 80421953
15: MFS New MFS Media 154017792 @ 80422977 ( 73.4G)
16: Apple_Free Extra 879 @ 234440769

This will list all the partitions. You need to set MFS_DEVLIST to include all the mfs partitions:

# export MFS_DEVLIST="/dev/hdc10 /dev/hdc11 /dev/hdc12 /dev/hdc13 /dev/hdc14 /dev/hdc15 /dev/hdc16"

If you try to run mfs_streams and get an empty list, it's probably because there is a version around that is broken. Jamie is in the process if fixing it.

Comedian
11-02-2004, 02:28 PM
Cool thanks! I will give that a shot.

chuck_333
11-06-2004, 10:48 PM
Wow... I thought I was kind of smart but after being out of the Linux game for eight years, I'm having a bad time getting a system up and running that can read my drives. I've been scouring this forum for weeks and it doesn't seem to be getting any clearer. I hear people say that they spent 18 hours getting a system up and running, but I'm guessing they must be pretty familiar with Linux to begin with.

I downloaded the Debian install as Ivan had described and got "build-essential" installed. I also installed a few other packages to get some familiar tools. However, PDISK does not seem to be on the standard install and I don't see it on the Debian package list. Also, I tried the Sleeper Knoppix Boot CD and it just bombs when I try to start up. I'm just running a plain-jane Dell with no special hardware, so I can't see that as a problem.

Is there any newer image that I can boot from that has everything I need to pull data off a drive pair from a DSR7000 and put it on a drive for later transfer back to the TiVo?

Additionally, I find all sorts of stuff that says don't use Sleeper's tools but people seem to refer to them all of the time. Is there a new bible for prepping a DSR7000 image with all of the latest and greatest stuff?

Oh and another thing... I have a Philips S1 TiVo that's been offline for a year that I would love to grab the content from. Is there anything special about doing that? It would be nice to burn some of it to DVD+RW and watch it or maybe transfer it to my S2 DirecTiVo to watch. What's my best bet?

Thanks in advance for the help! And sorry for being so dense.

Chuck

ace351
11-09-2004, 05:31 AM
i got a hard drive and did the insane (everything) install of redhat 8.0 (wanted to be as close to the orginal os as i could for the instructions at the start of this thread) and i am having an issue getting vplay to compile keep getting the error

make host
gcc -Wall -g -c mfs.c -o i386/mfs.o
gcc -Wall -g -c object.c -o i386/object.o
gcc -Wall -g -c schema.c -o i386/schema.o
gcc -Wall -g -c query.c -o i386/query.o
query.c:106: conflicting types for `query_object'
proto.h:29: previous declaration of `query_object'
query.c:28: warning: `memdup' defined but not used
make: *** [i386/query.o] Error 1
[root@localhost vplay]#

the amount of coding i can do wont fit on a pin head, so i have no clue, but i have about 70 hours of shows i would like to save, but cant seem to get passed this step. and needless to say im out of space and have 4 hrs of shows i want to save tommorrow.

any help would be greatly appreciated!!!!!

ace351
11-12-2004, 05:00 PM
ok i got a copy of the source from the murrys site and was able to use mfs_export to get a stream, but they are all the same size, both a 2 hr movie and a 1 hour tv show are the same size. am i doing something wrong?

./mfs_export 1906151 /root/vid/sa_suspicion.ty
is the command line that i used

my export looks like this (tivo HD is second master drive)

export MFS_DEVICE="/dev/hdc10"
export MFS_DEVLIST="/dev/hdc"

and mfs_info then shows

./mfs_info
Found 6 MFS partitions
Super:
state=0 magic=abbafeed
devlist=/dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13 /dev/hda14 /dev/hda15
zonemap_ptr=1121 total_secs=159901696 next_fsid=132
backup_zonemap_ptr=ffffe zonemap_size=1
/dev/hdc has 160835584 sectors

and my end product looks like

ls -pal
total 1574420
drwxr-xr-x 2 root root 4096 Nov 12 14:27 ./
drwxr-x--- 22 root root 4096 Nov 12 14:21 ../
-rw-r--r-- 1 root root 536870912 Nov 12 14:26 heavy_metal.ty
-rw-r--r-- 1 root root 536870912 Nov 12 14:22 pump_up_the_volume.ty
-rw-r--r-- 1 root root 536870912 Nov 12 14:28 sa_suspicion.ty



?????????

any help would be appreciated

jbuehl
11-13-2004, 11:50 AM
I downloaded the Debian install as Ivan had described and got "build-essential" installed. I also installed a few other packages to get some familiar tools. However, PDISK does not seem to be on the standard install and I don't see it on the Debian package list. Also, I tried the Sleeper Knoppix Boot CD and it just bombs when I try to start up. I'm just running a plain-jane Dell with no special hardware, so I can't see that as a problem.

Is there any newer image that I can boot from that has everything I need to pull data off a drive pair from a DSR7000 and put it on a drive for later transfer back to the TiVo?

Additionally, I find all sorts of stuff that says don't use Sleeper's tools but people seem to refer to them all of the time. Is there a new bible for prepping a DSR7000 image with all of the latest and greatest stuff?

Oh and another thing... I have a Philips S1 TiVo that's been offline for a year that I would love to grab the content from. Is there anything special about doing that? It would be nice to burn some of it to DVD+RW and watch it or maybe transfer it to my S2 DirecTiVo to watch. What's my best bet?


I was able to use a standard Fedora installation to do what you are trying to do. You need to get tivopart, which is available from this forum (sorry I don't have the link handy).

# tivopart r /dev/hd* < where * = c or d , etc.>

will allow you to access the tivo disk(s). pdisk is also a utility you can get from this forum.

I have been using the mfs utilities (in vplay) and mfs_ftp to pull shows off my Phillips S1 and put them on my S2 D-tivo. Unfortunately there are copies of the mfs utilities going around that are broken. Jamie has been fixing them: http://www.dealdatabase.com/forum/showthread.php?t=38654

jbuehl
11-13-2004, 11:59 AM
ok i got a copy of the source from the murrys site and was able to use mfs_export to get a stream, but they are all the same size, both a 2 hr movie and a 1 hour tv show are the same size. am i doing something wrong?


You should use mfs_tmfstream to extract an entire show. A show consists of an fsid that contains the information about the show (title, recording time, etc.) which points to one or more fsids that contain the actual video. mfs_export will only move a single fsid, which are always the same size and will just contain a chunk of video, and not the show information. The documentation on these utilities isn't too clear, so it's hard to know which fsid to use.

ace351
11-15-2004, 02:51 AM
You should use mfs_tmfstream to extract an entire show. A show consists of an fsid that contains the information about the show (title, recording time, etc.) which points to one or more fsids that contain the actual video. mfs_export will only move a single fsid, which are always the same size and will just contain a chunk of video, and not the show information. The documentation on these utilities isn't too clear, so it's hard to know which fsid to use.

documentation is defiantly a little ruff, i gave up on this and ordered the cache/network card and hope to have it tuesday, but the install docs on the stardust forum and barren for a newbie, and i thought i had done well eXtreeming a drive last year. i hope to revisit this project in a few weeks for the other s1 tivo i have, have a few shows i want off of it but im not in as much of a time crunch for it.

danss
11-18-2004, 12:59 PM
i try and extract an FSID 256909 via mfs_extract 256909 /mnt/c and it says it completed but the file is only 988 bytes, any idea on how i can fix it?

-Dan



./mfs_export 265909 /mnt/c/test.ty

Found 6 mfs partitions
exporting fsid 256909 of size 988 to /mnt/c/test.ty
starting at 0 for 988 bytes

then it goes back to the prompt

Jamie
11-18-2004, 01:22 PM
i try and extract an FSID 256909 via mfs_extract 256909 /mnt/c and it says it completed but the file is only 988 bytes, any idea on how i can fix it?
I gave a detailed reply here (http://www.dealdatabase.com/forum/showthread.php?p=194611#post194611).

snoots
11-19-2004, 10:54 AM
I have been able to follow the information posted for USB 2.0 on my HR10-250. I have an external USB drive with a Linux partition set up. I have been able to pull single FSID's using mfs_export, last night I tried mfs_tmfstream and pulled a whole HD show. I used explore2fs to move it to the pc. My problem is that winrar gives me errors extracting the ty files from the tmf. Is there a simple way with the USB mounted on the tivo to pull the whole show with one command without having to piece the individual FSID's together ? My thoughts were that the USB drive would be faster than using my network and the usual HDtytools. Any one have better idea ? Using the network to extract seems to be very slow. I do realize that the shows are HUGE. Just trying to speed things up a bit. One more question. I partitiioned the drive with 2 80 gig partitions, one linux one windows fat32. The tivo can mount the windows partition but can I write files bigger than 2 gig to it ? Any help would be appreciated, Thanks !

Jamie
11-19-2004, 12:07 PM
I have been able to follow the information posted for USB 2.0 on my HR10-250. I have an external USB drive with a Linux partition set up. I have been able to pull single FSID's using mfs_export, last night I tried mfs_tmfstream and pulled a whole HD show. I used explore2fs to move it to the pc. My problem is that winrar gives me errors extracting the ty files from the tmf. Is there a simple way with the USB mounted on the tivo to pull the whole show with one command without having to piece the individual FSID's together ? My thoughts were that the USB drive would be faster than using my network and the usual HDtytools. Any one have better idea ? Using the network to extract seems to be very slow. I do realize that the shows are HUGE. Just trying to speed things up a bit. One more question. I partitiioned the drive with 2 80 gig partitions, one linux one windows fat32. The tivo can mount the windows partition but can I write files bigger than 2 gig to it ? Any help would be appreciated, Thanks !
Look at the link to the detailed reply in my previous post. It has the command you need to dump multiple fsid's in one command.

You're doing better than me with usb-storage. I've tried similar things to what you're doing, but it seems to hang my tivo regularly. Same usb drive works fine on my x86 linux system.

I think you are right that you'll run into a 2GB (or is it 4GB?) file size limit with fat32.

snoots
11-19-2004, 12:29 PM
I have more problems with stuttering live tv using tytools to extract than pulling using mfs_tmfstreams. My problem seems to be with the tmf file as winrar doesn't like it. So I was looking for a way to get the ty direct without having to manually type in all the FSID's. It looked to me like the only way was tmfstream which winrar choked on.

Jamie
11-19-2004, 12:47 PM
So I was looking for a way to get the ty direct without having to manually type in all the FSID's.
Cut and paste? Run mfs_streams and grep the results to get the show you want, then cut/paste the "part" fsid's into your mfs_stream command line.

I'll have the option of extracting ty+ from a Recording FSID in the next snapshot.

snoots
11-19-2004, 01:01 PM
I will give that a try, I overlooked the obvious for a lazy solution. I am extracting CSI NY over the network now, will compare it to the same show USB'd to the drive using the cut and paste of the FSID's and see how it works out time wise. Thanks for the quick help,

Snoots

snoots
11-24-2004, 10:10 PM
I am using a USB drive hooked to my HR10-250. I can copy using the mfs_uberexport command to the mounted drive which is formatted as ext2 linux. Doing a "ls" gives the above error. Trying to "rm" the file gives the above error. I have googled on this and it appears to be a characteristic of files larger than 2 gig. Does anybody know how to correct this? Would this be a kernel file system problem with the usb/scsi storage drivers ? I have also tried a different extenal usb drive and also tried ext3 filesystem.

Jamie
11-25-2004, 12:50 AM
I am using a USB drive hooked to my HR10-250. I can copy using the mfs_uberexport command to the mounted drive which is formatted as ext2 linux. Doing a "ls" gives the above error. Trying to "rm" the file gives the above error. I have googled on this and it appears to be a characteristic of files larger than 2 gig. Does anybody know how to correct this? Would this be a kernel file system problem with the usb/scsi storage drivers ? I have also tried a different extenal usb drive and also tried ext3 filesystem.Sounds like the rm and ls commands you are using aren't built with large file support. It wasn't clear from your message whether you are running these commands on the TiVo or on your PC. If it's on the TiVO, busybox needs to built with the CONFIG_LFS option. If it's on your PC, probably the explore2fs package you are using (an ext2 file system plug in for windows, as I understand it) doesn't support files larger than 2 GB.

snoots
11-25-2004, 12:34 PM
Sorry I was not clear enough. The error message is on the TIVO while telneted in. The files can be exported using the windows extfsexplore program but it does not support writing to the linux partition so you can not delete the files after copying to the pc. When you ls or try to delete the files with the rm command you get the error message on the LINUX box.

snoots
11-25-2004, 12:46 PM
I think you are probably correct with the ls and rm commands being an older s2 version. I am currently not running busybox so I am downloading it now to see if it corrects the problem. Thanks for the quick help,

Snoots

snoots
11-25-2004, 01:52 PM
Jamie was correct with the solotuion to my problem. See below listing. Installed the AIO utils and using the busybox ls I am good to go. My probably stupid question is what is the best way to remove the obsolete commands. I don't want to kill or remove something the tivo needs from /bin which is where my original s2bins were installed. If I move the /tivo-bin ahead of /bin in the path environment variable will that do the trick ?



bash-2.02# cd /
bash-2.02# mount /dev/sda2 /usbdrive
bash-2.02# ls usbdrive
csi.ty dwarf.ty lost+found
bash-2.02# ls -al usbdrive
ls: usbdrive/csi.ty: Value too large for defined data type
ls: usbdrive/dwarf.ty: Value too large for defined data type
total 53
drwxr-xr-x 3 0 0 4096 Nov 25 01:58 .
drwxr-xr-x 21 1048576 24244387 1024 Nov 25 16:57 ..
drwxr-xr-x 2 0 0 49152 Nov 24 01:48 lost+found
bash-2.02# /tivo-bin/busybox ls -al /usbdrive
drwxr-xr-x 3 0 0 4096 Nov 25 01:58 .
drwxr-xr-x 21 1048576 -1870528 1024 Nov 25 16:57 ..
-rw-r--r-- 1 0 0 7547650048 Nov 24 04:18 csi.ty
-rw-r--r-- 1 0 0 7230980096 Nov 25 04:02 dwarf.ty
drwxr-xr-x 2 0 0 49152 Nov 24 01:48 lost+found
bash-2.02#

Jamie
11-25-2004, 01:57 PM
Jamie was correct with the solotuion to my problem. See below listing. Installed the AIO utils and using the busybox ls I am good to go. My probably stupid question is what is the best way to remove the obsolete commands. I don't want to kill or remove something the tivo needs from /bin which is where my original s2bins were installed. If I move the /tivo-bin ahead of /bin in the path environment variable will that do the trick ?

Yes, just make sure hour hacks directory is in front of /bin:/sbin etc on your PATH.

snoots
11-25-2004, 02:28 PM
OK,

Now I can view the files but busybox does not contain an applet for the rm command. Is there a version floating around for s2's that will work ? I did a google search for rm for tivo's, large file support for tivo etc etc. No luck.

bash-2.02# ls -al /usbdrive
drwxr-xr-x 3 0 0 4096 Nov 25 01:58 .
drwxr-xr-x 21 1048576 -1870528 1024 Nov 25 16:57 ..
-rw-r--r-- 1 0 0 7547650048 Nov 24 04:18 csi.ty
-rw-r--r-- 1 0 0 7230980096 Nov 25 04:02 dwarf.ty
drwxr-xr-x 2 0 0 49152 Nov 24 01:48 lost+found
bash-2.02# rm /usbdrive/csi.ty

rm: /usbdrive/csi.ty: Value too large for defined data type
bash-2.02#

Jamie
11-25-2004, 03:01 PM
OK,

Now I can view the files but busybox does not contain an applet for the rm command.

AlphaWolf may have left it out of his busybox compile, but it is included in the busybox sources. The config options are CONFIG_RM and CONFIG_RMDIR.

{Edit: A LFS rm is now in the latest AW's AIO package.}

snoots
11-25-2004, 10:15 PM
I downloaded the sources and was going to work on trying to compile a set on my own either on my PC or on the tivo. Got back from turkey day at Moms' and boom AlphaWolf had already added it at Jamies request ! Thanks a bunch guys my USB drive is now rid of the huge ty files without me having to partitiion it on the PC. Thanks again !!

Snoots

jasonb
12-05-2004, 10:34 PM
I was recently asked to upgrade a s1 tivo (circa 2000) with the original 30gig quantum to a new bigger drive. so happens that frys had seagate 120's on sale for 30 bucks. (supposed to be quiet too i hear). booted up from cdrom and the drive was unlocked for me. copied everything over. good so far.

next i was asked to pull a show off the quantum. so i stuck it in my suse 9.1 box as slave and the kernel automatically unlocked it and i was able to pull the show off using the tools linked from the first post in this thread (vplay-bin.tar.gz).

I left dma turned on and set these:
MFS_DEVICE=/dev/hdb
MFS_DEVLIST=/dev/hdb

mfs_ls -lR /Recording/NowShowingByTitle
pick a fsid and mfs_export <fsid> <filename>

stock kernel 2.6.4 with dma turned on works perfectly.

cheers
jason

PA_Riledo
12-19-2004, 04:10 AM
Just read this thread and hope someone can clear things up for me.

S2 Dtivo, RID unit (Hughes sddvr40, upgraded to a 120gb months ago, now full). Scramble has been on. Can I direct extract or not? If so, some pointers would be appreciated.

Thanks.

Lester123
12-29-2004, 11:46 PM
I've followed the direction (as far as I can tell) and I have 2 questions.

1. in the mfs_dumpobj output where are the fsid's in the listing?
It starts off with fsid 0 is not a directory and then it list lots of lines with UNKNOWN=XXXXXX (various numbers)
2. If I try to mfs_export the fsid I got from the mfs_ls command (thought I could at least get the first part to see it it works) all I get are little <1K files.

Another post suggests that the fsid's that I'm getting ate the fsid's of the description not the recording parts.

What am I missing. Probably something super obvious :)

Jamie
12-30-2004, 12:42 AM
The methods described in this thread are sound, but I believe they pre-date the unified mfs_* package I released. That package incorporates JonBig's package for PC exporting of tmf files.

So here's what I'd suggest:

Grab the mfs_bin.Linux-i686 package here (http://www.dealdatabase.com/forum/showthread.php?t=39487).

Boot linux, unpack the package, and make sure it is on your path.

Get tivopart (http://www.dealdatabase.com/forum/showthread.php?t=25219&highlight=tivopart) and run "tivopart r /dev/hDX" for each tivo disk. If you have the kernel patch for tivo partition tables, you don't need to do this.

Export this environment variable to set the device where your tivo disk is installed:


export MFS_DEVICE=/dev/hdX

hdX should be hda for primary master, hdb for primary slave, hdc for secondary master, hdd for secondary slave. I'm assuming a single disk setup, but you can do dual disks too (export MFS_DEVICE="/dev/hdX /dev/hdY").



Run mfs_streams to list all the streams available. The left most number is the FSID of the recording object for each stream. The title string is next. Make a note of the FSID's you want.

For each FSID you want to extract, do one of the following:

For .tmf format:
mfs_uberexport -txv -R FSID -o NAME.tmf
For .ty+ format:
mfs_uberexport -xv -R FSID -o NAME.ty+
For .ty format (no metadata):
mfs_uberexport -v -R FSID -o NAME.ty
where FSID is the desired FSID and NAME is name of the file you want to dump it in.

I suggest .tmf if you plan on reinserting these shows with mfs_ftp.

Lester123
12-30-2004, 12:03 PM
Thanks for the quick response. I'll give it a try tonight.

Lester123
12-31-2004, 12:02 AM
It worked perfectly. My wife thanks you :)

One other tip for those as linux chalanged as I am. if you do a

mfs_streams > ShowList.txt
Then when you get back to windows you can print your fsid's for easier reference.

oldbastivo
02-26-2005, 03:42 PM
I want to pull a bunch of shows off a S2 DTivo unit that is unhacked (scrambled recordings) and plan to follow Jamie's post. [when I'm done, I'm going to blow it away and upgrade to 4.0, as I've done w/ another s2dtivo].

Unfortunately my tivo has two drives and I can't find any info about how to set my linux environment to recognize two married drives.

Does anyone know what modification to these steps I'll have to take?

[edit!]

I almost forgot! Can I do this w/ my regular Fedora Core 3 install, or do I need to use a Hinsdale, etc boot disk?

Thanks

Jamie
02-26-2005, 03:48 PM
I want to pull a bunch of shows off a S2 DTivo unit that is unhacked (scrambled recordings) and plan to follow Jamie's post. [when I'm done, I'm going to blow it away and upgrade to 4.0, as I've done w/ another s2dtivo].

Unfortunately my tivo has two drives and I can't find any info about how to set my linux environment to recognize two married drives.

Does anyone know what modification to these steps I'll have to take?Short answer "MFS_DEVICE="/dev/hdA /dev/hdB" where A and B are the correct letters for your A and B drive positions on your IDE channels.

Longer answer: see the README.MFS_DEVICE in the mfs_* (http://www.dealdatabase.com/forum/showthread.php?t=39487) package. Here's an excerpt:

1) You have an A and possibly a B TiVo disk connected to your linux PC.

AAA is the block device file for the A disk (e.g. /dev/hdc or /dev/sda).
BBB is the block device file for the B disk (optional).

export MFS_DEVICE="AAA BBB"

The code will look for an MFS superblock on partition 10 of device
AAA and use the devlist there, substituting AAA for the A drive and
BBB for the B drive.

Your linux kernel must recognize the TiVo partition tables on your
disks for this option to work. The tivopart program, available on
DDB, can help with this. Alternatively, use a linux kernel that
has been modified to recognize the tivo partition tables. Most
linux ISO images customized for use with TiVo disks should have
this modification.


[edit!]


I almost forgot! Can I do this w/ my regular Fedora Core 3 install, or do I need to use a Hinsdale, etc boot disk?FC3 should be fine (that's what I run), but you'll need to run "tivopart r" on each tivo disk to inform the kernel of the linux partition tables.

oldbastivo
02-26-2005, 03:56 PM
Thanks for the immediate reply! I was just feeling guilty about the post, as I only just found this thread:
http://www.dealdatabase.com/forum/showthread.php?t=40846&page=2&pp=15&highlight=mfs_devlist
in which you walk someone through the use of their fedora install for FSID discovery.

Thanks again for the replay ... fortunately, I expect a lot of the pitfalls in the other thread, since I not only have fc3 installed, I actually use it :-P

You rock!

Jamie
02-26-2005, 04:20 PM
Thanks again for the replay ... fortunately, I expect a lot of the pitfalls in the other thread, since I not only have fc3 installed, I actually use it :-P
I should point out that scrambled shows aren't going to be very useful after you extract them. You might want to look at the s2_unscramble stuff to unscramble them on the tivo before you extract.

oldbastivo
02-26-2005, 04:39 PM
I had hoped there would be some way to unscramble them w/ an x86 port of the unscramble util, but I suppose I should have looked into that a bit more...

Well, if I have to comprimise the box to unscramble there, I might as well populate the FSIDs there too. I'll still pull the drive for the extraction to save copy time.

Thanks for the heads up

Jamie
02-26-2005, 04:51 PM
I had hoped there would be some way to unscramble them w/ an x86 port of the unscramble util, but I suppose I should have looked into that a bit more...
If you can get a PC unscramble solution to work, please tell us about it :-)

Note that if the end goal is to get 4.0.1b installed and save your recordings, it may be possible to upgrade via slices to do that. Search around a bit. Not my particular expertice, but I know there is info about it.

mikeaco
03-09-2005, 01:23 PM
I have an old DSR6000 that I have 2.5extreme on with xPlusZ loaded on it. It has two drives...the original 40GB drive and a secondary 80GB drive. I have some old hockey games, recorded scrambled, that I want to do a one-time extraction of. I do not have a ethernet card, so I will be attempting a PC side extraction.

I've figured out the unscramble part of the process and have tested the extraction of a small test .TY file through the serial cable (i.e. ran unscramble.o, primed the show I wanted to extract, use mfs_uberexport to create a small ~50MB .TY file on the /var partion, and then sz'd (slow) it to my PC). I have tested the resultant .TY file in TyTools and everything came out OK.

Now I'm ready to remove the drives and attach it to my PC, boot linux and extract using Jamie's instructions HERE (http://www.dealdatabase.com/forum/showpost.php?p=201494&postcount=210) as a base for how to accomplish the extraction.

I have two drives, and will probably install those as TiVo_A = Secondary Master and TiVo_B = Secondary Slave. So, when I boot to Linux, I'll need to:

export MFS_DEVICE="/dev/hdc /dev/hdd"
correct? I'll also need to set up MFS_DEVLIST. However, the current TiVo disks don't have pdisk on it. So, how can I tell what I need to enter in for:

export MFS_DEVLIST="/???/??? /???/??? /???/??? /???/???"
Will *any* standard TiVo Linux boot ISO have pdisk on it so that I can run the command and determine the DEVLIST partions? What is currently the best ISO image boot disk to use for old (non-LBA48) S1 drives? I think that I used the original MFS_Tools_2.0 disk and the 2.5extreme image.

The only other thing that I'm concerned with is the disk that I will be copying the files over to. Since the extracted hockey games are probably at least 3+ hours, can I expect them to be more that 2GB (even if they are Standard Def)? If so, will I run into any problems if the "extract to" disk is formatted FAT? You can't mount NTFS, right (or is that just for mfsbackup)? Any help with making sure I won't have problems with large extraction files would help!

Other than that, I think I ready to give it a shot! Thanks for any help you can provide.

psxboy
03-09-2005, 01:41 PM
Couple of things... you only need MFS_DEVICE or MFS_DEVLIST, not both. If you're extracting directly from your Tivo HD, you use MFS_DEVICE. If you have dd'ed the mfs partitions out to files, you would use MFS_DEVLIST.

Regarding the FAT FS and the 2Gig limit... yeah, I ran into that problem too. You can either extract the parts individually or do what I did... use an ext2 filesystem to store the extracted recordings. It's mountable, writable, and doesn't have the 2Gig limit. (I think the actual file-size limit on ext2 is somewhere in the neighborhood of 4 Terabytes?)

Also, I had great success using the knoppix live CD to do the extraction. It was able to mount all of my FS types (even the Tivo, with tivopart), has networking enabled, you can layer your console sessions on top of each other in X-Windows instead of constantly flipping through them in text mode, and it seemed to do the extraction a lot faster than the PTVUpgrade LBA-48 CD I was using previously.

Since you have an S1 however, I'm not sure how the byte-swap issue will affect you if you use a non-byte-swap-aware distro. Seems to me that the new mfs_* toolset handles that internally now, so the worst outcome would be that you wouldn't be able to mount the non-mfs partitions, but for simple extraction that's a moot point.
-psxboy

mikeaco
03-09-2005, 01:57 PM
Thanks, psxboy, for giving me some pointers in the right direction. I'll have to look, but hopefully I can create the ext2 filesystem (i.e. format the destination drive) using the Knoppix live CD. I don't have a Linux system to boot the drive to beforehand.

Almost there! I can't wait to have the Red Wings / Avs playoff series from 2002 archived to DVD...especially games 6 and 7. :D

psxboy
03-09-2005, 02:25 PM
I'll have to look, but hopefully I can create the ext2 filesystem (i.e. format the destination drive) using the Knoppix live CD. I don't have a Linux system to boot the drive to beforehand.

mke2fs [device]
(ie. mke2fs /dev/hdc1)

-psxboy

mikeaco
03-10-2005, 09:03 PM
Thanks psxboy! It took a little while, but I finally got things going.

First, I only had a spare HDD that was being used as a "data" only disk in a Windoze system. It was NTFS formated, but had a lot of free space on it. I tried to repartition it in Linux, but had problems becuase the NTFS partition took up the whole drive. I didn't spend much time figuring out if some Linux utility could access and resize the partition. So, I booted back into Windoze and used Partition Magic to reduce the NTFS partition and add a Linux ext2 partition. I was then able to boot up Linux and mount the newley created partition (hopefully preserving the NTFS partition and data).

Next, I put the two TiVo drives in the PC and booted. I tried several Linux boot disks (Knoppix Live, Knoppix Lite, PTV-LBA48-4.0), but could not get pdisk to recognize the TiVo drive partion tables. After reading up a bit, I surmized that the disks were booting up without swap enabled. It took a while to figure out how to configure bswap at boot. I actually ended up using the PTV-large-ISO that has a vmlinux or vmlnodma kernel boot options (does the Knoppix boot disks have this option? I know the PTV-LBA48-4.0 does not).

Finally, I was able to see the TiVo disk partition tables and use the mfs_* utilities to extract the FSID's I wanted. There was one small hitch to that...I somehow mistakenly initially used the ppc mfs_* binaries and not the i386 ones. So, that took a bit to figure out.

Well, I'm extracting now. Hopefully, the resulting files will not have any problems. Thanks for the help!

joepistritto
03-20-2005, 02:25 PM
So a few months back a few posters have said that direct insertion was not possible. Is that still the case today? If so, what needs to be done to get direct insertion to happen? In my case, I want to use re-insert the streams back to a new drive that I am replacing for my tivo so I don't really care about being able to view the streams in a different tivo unit.

I did look at mfs_import but it looks like it still requires something running on the tivo itself to create an fsid? That makes it non-direct.......... Thanx!

Jamie
03-20-2005, 02:31 PM
So a few months back a few posters have said that direct insertion was not possible. Is that still the case today? If so, what needs to be done to get direct insertion to happen? In my case, I want to use re-insert the streams back to a new drive that I am replacing for my tivo so I don't really care about being able to view the streams in a different tivo unit.
Yes, insertion currently depends on tivosh code to allocate streams and manipulate MFS tyDB objects describing the recordings. There is a very limited write capability in the tridge mfs library, but it isn't sufficient to do these things.

It should be possible to produce a tivosh script to do all the allocation on the tivo, then move the disk back to a PC to do the actual stream insertion there. Don't know of anyone that does this.

PC-linux to tivo transfers with mfs_ftp over a usb2 host-to-host cable are pretty fast (7MB/sec). That's the fastest way I know of to do bulk transfers of shows.

jerrymc
04-02-2005, 11:56 PM
I have a SAT-T60 that started rebooting continuously and I want to save some of the shows by extracting them on my RedHat 8 system for reinsertion later on the new drive. The old drive runs xPlusz on v3.0 and all shows are descrambled. I probably have close to 200 hours of shows on this puppy that I'd really like to save.


The methods described in this thread are sound, but I believe they pre-date the unified mfs_* package I released. That package incorporates JonBig's package for PC exporting of tmf files.

So here's what I'd suggest:

Grab the mfs_bin.i386 package here (http://www.dealdatabase.com/forum/showthread.php?t=39487).

Boot linux, unpack the package, and make sure it is on your path.

Export this environment variable to set the device where your tivo disk is installed:


export MFS_DEVICE=/dev/hdX

hdX should be hda for primary master, hdb for primary slave, hdc for secondary master, hdd for secondary slave. I'm assuming a single disk setup, but you can do dual disks too (export MFS_DEVICE="/dev/hdX /dev/hdY").

Run mfs_streams to list all the streams available. The left most number is the FSID of the recording object for each stream. The title string is next. Make a note of the FSID's you want.

For each FSID you want to extract, do one of the following:

For .tmf format:
mfs_uberexport -txv -R FSID -o NAME.tmf
For .ty+ format:
mfs_uberexport -xv -R FSID -o NAME.ty+
For .ty format (no metadata):
mfs_uberexport -v -R FSID -o NAME.ty
where FSID is the desired FSID and NAME is name of the file you want to dump it in.

I suggest .tmf if you plan on reinserting these shows with mfs_ftp.


I followed Jaimie's instructions and read most of this thread looking for all the info I needed, but I clearly missed something. I loaded the mfs_bin package (3-26-05 version) onto my Linux box, and also tivopart. I ran "tivopart r /dev/hdc", then tried to run mfs_streams to get a list of the FSIDs with the following result:


> mfs_streams /dev/hdc
crc mismatch len=232 0x3ef3aebf 0x7c656b11
Failed to byte swap correctly

What am I missing? I currently have Tiger's mfstools2 loaded on this very machine and use them routinely without any problems. Any help or pointers is greatly appreciated!

Regards,
Jerry

mikeaco
04-04-2005, 11:36 AM
Jerry,

I would just find a tivo-based linux boot disc (CD) that supports byte-swapping. You won't have to mess with tivopart that way. Make sure that you have a partition setup on your RedHat setup so that, after you mount to it, you can copy the shows over using the mfs_* utilities.

Mike

jerrymc
04-05-2005, 11:18 AM
Jerry,

I would just find a tivo-based linux boot disc (CD) that supports byte-swapping. You won't have to mess with tivopart that way. Make sure that you have a partition setup on your RedHat setup so that, after you mount to it, you can copy the shows over using the mfs_* utilities.

Mike

Mike,

Thanks for the reply. The problem with the CD solution is that I have a second EIDE controller card in my PC and most of the boot CD's that I've run accross don't support it. I only have hdc and hdd available on the main controller and large disks don't cohabitate on the same cable happily. Something about the DMA settings on this old (Pentium-233MMX) PC makes a simple mfsinfo take 20 minutes. Never did figure out how to fix that so I just put the drives on different cables.

Will the latest PTVUpgrade LBA48 CD do the trick and support the extra controller? BTW, the emergency is over. A "clear guide data and to-do list" fixed my reboot issue. Now I just want to learn something new since I've started researching it anyway. :)

Regards,
Jerry

oakwcj
04-13-2005, 10:28 PM
That's the error message reported by mfs_uberexport when I tried to extract recordings from my SD-DVR40, which had been running 6.2 until yesterday when tivosh "strayed" and suffered a fatal error while moviesort.tcl was running. Mfs_streams provided me with a nice list of FSIDs, but it appears that the recordings are unrecoverable. If anyone can provide me any documentation with respect to the error message, I'd appreciate it.

Jamie
04-13-2005, 11:23 PM
That's the error message reported by mfs_uberexport when I tried to extract recordings from my SD-DVR40, which had been running 6.2 until yesterday when tivosh "strayed" and suffered a fatal error while moviesort.tcl was running. Mfs_streams provided me with a nice list of FSIDs, but it appears that the recordings are unrecoverable. If anyone can provide me any documentation with respect to the error message, I'd appreciate it.This error occurs when generating the showing.xml file for ty+ or tmf format output. It's trying to add the tivo software version to the xml. It probably shouldn't be a fatal error if it can't find it. The code is looking for the Name attribute in the /SwSystem/ACTIVE TyDB object. What does 'mfs_dumpobj /SwSystem/ACTIVE' tell you?

In the meantime, if you extract in ty format (no xml), you shouldn't get the error.

{Edit: I've attached a version that doesn't exit if it can't find the tivo software version, returning "UNKNOWN" instead. I'll include this in the next published build too.}

oakwcj
04-14-2005, 01:05 AM
Thanks, Jamie. The new version of mfs_uberexport still returns the error, but it then does its job.

The output from 'mfs_dumpobj /SwSystem/ACTIVE is NOT 6.2-01-2-351 or any other intelligible response. it's a right brace } . I'm thinking that's not good.

BTW, I was getting the error while attempting .ty exports. I don't want to try insertion with 6.2, given the reports of problems, so I'm just going to burn these recordings to DVD with TyTool.

Jamie
04-14-2005, 08:50 AM
The output from 'mfs_dumpobj /SwSystem/ACTIVE is NOT 6.2-01-2-351 or any other intelligible response. it's a right brace } . I'm thinking that's not good.Hum. 'mfs_ls /SwSystem' might tell us something. I wonder if the ACTIVE entry was lost for some reason.
BTW, I was getting the error while attempting .ty exports.I guess it goes through the xml generation code even for .ty exports.

oakwcj
04-14-2005, 09:49 AM
Originally posted by Jamie:


Hum. 'mfs_ls /SwSystem' might tell us something. I wonder if the
ACTIVE entry was lost for some reason.

SwSystem has entries for 6.2 and 3.1. [I was running 4.0.1 before 6.2. During a daily call, 3.1 was downloaded. It was marked as an unsafe upgrade, though, because of the database change. And I do have the "uprgradesoftware=false" boot parameter.] Based on a post from ADH, I even dbloaded the swsystem slice for 6.2. I couldn't set it to active, though, because tivosh errors out and causes a reboot every time. My uneducated guess is that moviesort.tcl is very processor-intensive and was hogging the processor when the TiVo wanted to perform another operation. I didn't think to set it to low priority before I ran it. Then, again, I'd never had a problem with it before, although I've since seen other posts reporting non-fatal GSODs.

fredfillis
08-15-2005, 01:36 PM
I have an unhacked S1 Dtivo (DSR6000) and am trying to extract recordings for insertion on my now hacked DSR7000.

Have tried to follow AW's quickie how too here (http://www.dealdatabase.com/forum/showpost.php?p=35312&postcount=184) and also zobetron's post here (http://www.dealdatabase.com/forum/showpost.php?p=87426&postcount=2). As far as vplay binaries, I have Jamies unified MFS tools from here (http://www.dealdatabase.com/forum/showpost.php?p=195327&postcount=1) on my fat32 partition (bin.linux.i686). I am using the PTV universal boot disk and have also tried the "kazymyr" boot disk that zobetron mentions. I have my tivo disk hooked up to my PC as Primary slave. I have been able to determine from MFStools-2.0 on the boot disk that my MFS partitions are hdb10 through hdb13.

If I follow AW's how-to and run mfs_info I get an error that basically says "4 MFS Partitions" next line says CRC Mismatch (detail omitted) and next "Sector in Wrong Zone (33197056 0).

I have previously had a wrong magic number 4d50 in partition 0 but I don't recall the context.

This S1 is a totally clean unit that was working perfectly for 3+ years.

Can anyone point me in the right direction?

Jamie
08-15-2005, 01:45 PM
If I follow AW's how-to and run mfs_info I get an error that basically says "4 MFS Partitions" next line says CRC Mismatch (detail omitted) and next "Sector in Wrong Zone (33197056 0). AW's instructions pre-date the current tools. I'd recommend following the instructions here (http://www.dealdatabase.com/forum/showpost.php?p=201494&postcount=210).

If you are getting crc errors, I'd guess that either your PC is getting errors reading from the ide device (check dmesg for errors), or your MFS file system is corrupted. There are backup inodes, and the tivo will use them when it encounters an MFS inode with a bad crc. The mfs-utils don't do that, though it could be added if you are so inclined.

So, one option might be to put the drive back in the tivo and force a GSOD and hope that cleans up any bad crcs. Another option might be to enhance mfs-utils to go to the backup inode on crc failiures. A third option might be to backup and restore the disk with mfstools. I don't know for sure if that would work, but I remember seeing some backup inode handling in the mfstools source.

fredfillis
08-15-2005, 02:02 PM
So, one option might be to put the drive back in the tivo and force a GSOD and hope that cleans up any bad crcs.

Thanks for the tip Jamie. I believe the basic process I have followed to date is pretty close to what you referred me to so it seems like I have some "corruption" going on.

I think I'll do a backup first and try the forced GSOD route with a restore as my Plan B.

In terms of forcing GSOD, I have not done that before but I found the following information.


While booting the power light will change from green to yellow as the line 'checkpanic' in the rc.sysinit gets executed. This happens a few seconds before the screen changes from the startup image to "almost there" and it's your cue to hold down a button (use pause) on the remote to trigger a panic. When the record light changes yellow, key in one of the following sequences:

Snip ...

End Snip ...

57 - mfs check
58 - perform mfs cleanup
-- both of these will cause the green screen and various mfs checks


Is that still good info for Dtivos?

Jamie
08-15-2005, 02:14 PM
Is that still good info for Dtivos?I assume so, but I don't really know.

Another thing you can try is running a ppc mfs_info on the tivo itself. If that *doesn't* get the crc errors, than it tells you that MFS is ok and is something about the PC setup that's messing things up.

There's also a remote possibility that the mfs-utsils tools are broken for series 1 disks. I don't have a series 1, so I can't test anything on that platform myself.

fredfillis
08-15-2005, 03:46 PM
Jamie, thanks for your help.

Since I was doing a backup anyway I figured I would just go ahead and restore. That fixed my CRC issues and I was able to get a list of FSIDs via mfs_streams and have successfully uberexported to my PC. :D

Jamie
08-15-2005, 05:15 PM
Jamie, thanks for your help.

Since I was doing a backup anyway I figured I would just go ahead and restore. That fixed my CRC issues and I was able to get a list of FSIDs via mfs_streams and have successfully uberexported to my PC. :DGood. Eventually I should look at dealing with the backup inodes, I guess. It's nice to know that a backup/restore will fix it though.

fredfillis
08-20-2005, 03:37 PM
Now that I can theoretically insert to my s2 I need to tackle the next part.

The drive from my virgin S1 Dtivo is hooked up to my PC. I can extract recordings from the S1 drive to my PC. However, these recordings are scrambled. I have run mfs_dumpobj and got the output in the form of a text file. Based on reading up on MFS_FTP unscrambles, my DiskConfigurationKey information needs to be in XML format. I can probably figure that out. However, I'm not sure that inserting my S1 recordings to my hacked S2 unit is going to be successful. Based on posts I have read, there are three things required.


Limitations: currently 3 things must match to play or decrypt a scrambled recording

1) CommercialSkipOffset keys (stored in ty+ or tmf archives)

2) tivo hard drive DiskConfigurationKey

3) tivo motherboard crypto chip

if the DiskConfigurationKey has changed for any reason (re-imaged the drive - sw update - clear & delete) the correct DiskConfigurationKey (DC_key.xml) must be restored

Firstly, I'm a little unclear about the first item, CSO. I have seen what I think is conflicting information about CSO keys, do they need to be in or out?

Secondly, am I dead in the water due to the crypto chip issue?

Lastly, I've also read about "compability problems" when transferring S1 ty files to S2. ty1to2 is available here (http://www.dealdatabase.com/forum/showpost.php?p=210473%26postcount=49). Do I need to run that or have unified MFS tools brought us beyond that issue?

I'm not interested in hacking my S1, I would just like to get the shows off it and onto my S2. Is this even possible?

Jamie
08-20-2005, 04:09 PM
Firstly, I'm a little unclear about the first item, CSO. I have seen what I think is conflicting information about CSO keys, do they need to be in or out?CSO keys are required to playback encrypted shows. You'll get them if you export the xml meta data along with the show (-x option to mfs_uberexport).

Secondly, am I dead in the water due to the crypto chip issue?Pretty much. Your only option is to unscramble them on the S1 before you extract, but you say you have no interest in hacking the S1
Lastly, I've also read about "compability problems" when transferring S1 ty files to S2. ty1to2 is available here (http://www.dealdatabase.com/forum/showpost.php?p=210473%26postcount=49). Do I need to run that or have unified MFS tools brought us beyond that issue?I'm not an expert on this (never had an S1), but the unified MFS tools do nothing to convert between S1 and S2 formats, so I assume it is still necessary.
I'm not interested in hacking my S1, I would just like to get the shows off it and onto my S2. Is this even possible?You'll have to hack your S1 to unscramble the recordings before extracting them.

fredfillis
08-20-2005, 06:51 PM
You'll have to hack your S1 to unscramble the recordings before extracting them.

Since I have nothing to lose I tried putting noscramble.o on my S1 and put the drive back in the unit. After it booted up I attempted to play each show in the list for 10sec or more. Only two would play, the others gave me a "delete now" followed by a "no video" error. Anyhow, I then pulled the drive and uberexported as ty+ to my PC. Attempted to run ty1to2 but appears not to have worked.

When I looked at my S1 drive again I could not find the noscramble.o nor could I find the rc.sysinit.author I put on there. I guess my S1 decided to deep six both of those. I'm trying to insert the one show I extracted but I'm pretty sure it won't work.

By not interested in hacking my S1 I really mean I don't want to spend any dollars on getting a network connection on that machine. This is more a learning exercise for me rather than a matter of life and death.

*** Update ***
The show I transferred as described above caused my S2 to reboot when I hit play. Oh well ... as expected!

Jamie
08-20-2005, 07:51 PM
Since I have nothing to lose I tried putting noscramble.o on my S1 and put the drive back in the unit.Again, I know little about S1's, but I think it is unscramble.o, not noscramble.o, that you want. See this (http://www.dealdatabase.com/forum/showthread.php?p=135106#post135106) post, and in fact the whole thread. It's an old thread, so some things may have changed a little, but I think the concepts are still relevant for unscrambling S1 shows.


When I looked at my S1 drive again I could not find the noscramble.o nor could I find the rc.sysinit.author I put on there. I guess my S1 decided to deep six both of those. I'm trying to insert the one show I extracted but I'm pretty sure it won't work.That's certainly what happens on S2's when you run with a stock kernel/initrd. That's what killhdinitrd is all about avoiding. I don't know anything about S1 hacking, but I'll bet you need to bypass the initrd in some way.

fredfillis
08-21-2005, 05:13 PM
Again, I know little about S1's, but I think it is unscramble.o, not noscramble.o, that you want. See this (http://www.dealdatabase.com/forum/showthread.php?p=135106#post135106) post, and in fact the whole thread. It's an old thread, so some things may have changed a little, but I think the concepts are still relevant for unscrambling S1 shows.

That's certainly what happens on S2's when you run with a stock kernel/initrd. That's what killhdinitrd is all about avoiding. I don't know anything about S1 hacking, but I'll bet you need to bypass the initrd in some way.

Don't know why I put noscramble on that thang, I did already read the thread you pointed me to and had intended to use unscramble.o. Brain fart I guess.

Because I can't help myself I restored the drive and then dropped in a kernel from my various PTV boot disks. I tried several and kept getting stuffed at the Welcome screen. Eventually I got it right using a kernel from
this boot disk (http://www.ptvupgrade.com/products/software/lba48/lba_4.0_license.html) and then ran tivoflash. Got booted up ok this time. About half the shows would play and the others gave me the "no video" error. Pulled the drive, exported one "working" show as a ty and another as tmf and dumped them on my s2 with mfs_ftp. Neither worked.

Anyway, because I apparently have too much free time I'm going to try a few more moves to see if I can finesse something that works.

artdent002
09-13-2005, 11:10 PM
I'm trying to recover shows on a drive that is trapped in a GSOD... currently the wife is happy with the backup drive, but when she realizes she lost last week's "So you think you can dance?", I could be in trouble...

The drive is an upgraded (via Hinsdale) drive for a DTivo 2, orig 40g, now 120. Was running 6.2 with 3.1.5 from PTV, but I don't care about recovering the hacks, etc.... I just want the shows.

I have my TiVo drive on HDC (secondary master). I've tried using the i686 binaries from Jamie (here (http://www.dealdatabase.com/forum/showthread.php?t=39487) , and I consistently get a "Not a TiVo super block!" error. I've tried all combinations of setting MFS_DEVICE to /dev/hdc, /dev/hdc10, /dev/hdc11, etc. and setting MFS_DEVLIST to the entire device (/dev/hdc) as well as one or more of the 6 MFS partitions ("/dev/hdc10 /dev/hdc11 ..."). No matter what I try I get the same error - the only thing that changes is the magic number depending on which partition is is the MFS_DEVLIST variable.

I've also checked the partition table with the pdisk from Knoppix-lite and from the PTVUpgrade-LBA48 boot disks... everything looks ok, and it matches the partition table from my working backup exactly. BTW, the backup is on a 200G drive that was MFS backup and restored from this one to save the shows... yes, there is a wasted 80g on that one, but I've already read up on how to address that issue...

Here's what pdisk shows:
Partition map (with 512 byte blocks) on '/dev/hdc'
#: type name length base ( size )
1: Apple_partition_map Apple 63 @ 1
2: Image Bootstrap 1 1 @ 77261888
3: Image Kernel 1 8192 @ 77261889 ( 4.0M)
4: Ext2 Root 1 262144 @ 77270081 (128.0M)
5: Image Bootstrap 2 4096 @ 77532225 ( 2.0M)
6: Image Kernel 2 4096 @ 77536321 ( 2.0M)
7: Ext2 Root 2 262144 @ 77540417 (128.0M)
8: Swap Linux swap 260096 @ 77802561 (127.0M)
9: Ext2 /var 262144 @ 78062657 (128.0M)
10: MFS MFS application region 1048576 @ 78324801 (512.0M)
11: MFS MFS media region 33100800 @ 44161088 ( 15.8G)
12: MFS Second MFS application region 1048576 @ 79373377 (512.0M)
13: MFS Second MFS media region 44161024 @ 64 ( 21.1G)
14: MFS New MFS Application 1024 @ 80421953
15: MFS New MFS Media 159694848 @ 80422977 ( 76.1G)
16: Apple_Free Extra 3903 @ 240117825 ( 1.9M)


When I try MFS_INFO from the PTV disk or Tiger's MFS Tools 2.0, I get slightly different results:

Primary volume header corrupt, trying backup.

Secondary volume header corrupt, giving up.

mfs_load_volume_header: Bad checksum.

Segmentation fault

I've searched and read all the posts that contain the words: MFS super block, and all the posts that contain "volume header corrupt". Hopefully I'm just doing something stupid here... but it might be something stupid that hasn't been posted before.

Any ideas?

Jamie
09-13-2005, 11:29 PM
Any ideas?The superblock is the first block in partition ten. There's a magic number that is expected to be there in bytes 3-7: 0xabbafeed. It should look something like this:
% hexdump -C sb.dat
00000000 00 00 00 00 ab ba fe ed 30 0d 52 e5 00 00 00 10 |........0.R.....|
00000010 00 00 00 01 00 00 00 40 00 00 02 00 00 43 aa e4 |.......@.....C..|
00000020 00 00 00 01 2f 64 65 76 2f 68 64 61 31 30 20 2f |..../dev/hda10 /|
00000030 64 65 76 2f 68 64 61 31 31 20 2f 64 65 76 2f 68 |dev/hda11 /dev/h|
00000040 64 61 31 32 20 2f 64 65 76 2f 68 64 61 31 33 20 |da12 /dev/hda13 |
00000050 2f 64 65 76 2f 68 64 61 31 34 20 2f 64 65 76 2f |/dev/hda14 /dev/|
00000060 68 64 61 31 35 00 00 00 00 00 00 00 00 00 00 00 |hda15...........|
00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
...Notice the aforementioned 0xabbafeed in the hex dump and the device list in the ascii.

If your superblock doesn't look like this, something is seriously wrong and you might just be out of luck.

bcc
09-13-2005, 11:58 PM
This may be relevant, but I just ran into the same problem, and I had to hack mfs.c to fix it. Sorry if this was addressed already, as I was running the 20041129 build. I had my s1 disk mounted on a linux box with dma enabled (ie no kernel byte swapping to correct the byte order discrepancy). mfs_info fails with:
% mfs_info
Assuming MFS_DEVICE=/dev/hda
crc mismatch len=232 0xf0c87f81 0x2de3bed0
Failed to byte swap correctly
% sync
And the fix was:
% diff -c mfs.c~ mfs.c
*** mfs.c~ 2004-11-29 11:06:10.000000000 -0800
--- mfs.c 2005-09-10 20:14:53.000000000 -0700
***************
*** 161,166 ****
--- 161,167 ----
fprintf(stderr, "Warning: filesystem is inconsistent. Run fsfix and mfscheck ASAP\n");
}

+ mfs_read_partial(&super, 0, sizeof(super));
check_crc((void *)&super, sizeof(super), &super.crc);
byte_swap(&super, "i9 b128 i17");
The code already has a mfs_read_partial() call earlier to try and re-read the super block, but that code is not executed. byte swapping gets turned on at the 0xfeedabba case statement, and so the above changes allows the superblock to be re-read with the byte order fixed.

Jamie
09-14-2005, 10:39 AM
This may be relevant, but I just ran into the same problem, and I had to hack mfs.c to fix it.
...I'll take a look. I don't have a series 1, which means the S1 specific stuff (e.g. byte swapping) doesn't tend to get tested.

It doesn't seem like this would be artdent002 problem, since he has a series 2.

jonbig
09-14-2005, 02:30 PM
I've searched and read all the posts that contain the words: MFS super block, and all the posts that contain "volume header corrupt". Hopefully I'm just doing something stupid here... but it might be something stupid that hasn't been posted before.

Any ideas?

I've had this problem before, and my solution was to hack the mfstools source code to bypass the specific checks that were failing. I was able to get most of the shows off the disk, but a few were trashed. You're basically in uncharted territory, since the likelyhood that your disk was trashed in the same way as mine is small.

bcc
09-14-2005, 03:32 PM
I'll take a look. I don't have a series 1, which means the S1 specific stuff (e.g. byte swapping) doesn't tend to get tested.Looks to me like the problem is still outstanding with your 20050604 release. Course the problem doesn't occur if you're running with nodma+bswap set in the kernel.
Also this problem doesn't exist in the older mfs code in the cvs repository at pserver.samba.org (ie it has the extra super block re-read after byte order detection).
It doesn't seem like this would be artdent002 problem, since he has a series 2.Doh, missed that.