PDA

View Full Version : Recovery of video from clobbered disk



jbuehl
10-25-2004, 02:01 PM
I apparently clobbered the database on my Dtivo while installing 4.0, and it won't boot. I have given up on fixing it and I'm going to just do a reinstall, but there are a couple of shows that I really want to save, if possible. I can access the disk on a linux box, but all the extraction methods I have found assume that the database is intact, such as mfs_tmfstream, and one can find out the FSIDs of the shows, but in my case, I have nothing except the (presumably) valid bits of video out there on the disk somewhere. Of course if the video has been written over there's no hope, but more likely the pointers are not there.

I'm about to start looking at modifying source code (either mfstools or vstream) to do something like looking through all the FSIDs for tystream headers, and copying them. Basically a video recovery utility. Then I will be able to figure out how to identify the content that I want and restore it. I wanted to make sure this hasn't been done before and I'm not about to reinvent another wheel.

alldeadhomiez
10-25-2004, 02:09 PM
I apparently clobbered the database on my Dtivo while installing 4.0, and it won't boot. I have given up on fixing it and I'm going to just do a reinstall, but there are a couple of shows that I really want to save, if possible. I can access the disk on a linux box, but all the extraction methods I have found assume that the database is intact, such as mfs_tmfstream, and one can find out the FSIDs of the shows, but in my case, I have nothing except the (presumably) valid bits of video out there on the disk somewhere. Of course if the video has been written over there's no hope, but more likely the pointers are not there.

Nobody's going to be able to help you if you continue to describe the filesystem corruption in such vague terms.

With regard to the tridge tools, what works and what does not work? What do they say when they fail? What filesystem structures have you examined by hand and verified to be "clobbered"?

jbuehl
10-25-2004, 02:53 PM
Nobody's going to be able to help you if you continue to describe the filesystem corruption in such vague terms.

I was deliberately vague about the problem because I was specifically asking if anyone knew of any existing tools to poke around at a low level in the mfs filesystem, and not for general help in fixing the problem. I posted the details yesterday in another forum http://www.dealdatabase.com/forum/showthread.php?t=38747 and concluded that I wanted to try to recover the video rather than trying to repair the database.


With regard to the tridge tools, what works and what does not work? What do they say when they fail? What filesystem structures have you examined by hand and verified to be "clobbered"?

Running mfs_streams results in an empty list:

[root@localhost vserver]# ./mfs_streams
Listing streams in /Recording/Complete
[root@localhost vserver]#

AlphaWolf's extraction writeup describes copying video, but only if you already know the fsid.

I haven't looked at any filesystem structures because I don't know how to do that. If I knew how, I wouldn't have asked the question.

alldeadhomiez
10-25-2004, 03:43 PM
You're going to have to start doing some poking around (possibly in the vplay sources, but hopefully not). For starters:

mfs_info
mfs_ls /
mfs_ls /Recording
(etc.)
mfs_dumpobj <fsid_of_tyDb_object>
mfs_dump 0 1

It will help to have a good image for reference.

AlphaWolf
10-25-2004, 05:38 PM
mfs_streams is broken BTW. I have started a general petition for somebody with C skills to fix it.

http://www.dealdatabase.com/forum/showthread.php?t=38654

Jamie
10-25-2004, 05:47 PM
mfs_streams is broken BTW. I have started a general petition for somebody with C skills to fix it.

http://www.dealdatabase.com/forum/showthread.php?t=38654

Working on it. I'm close, but prefer to risk my MFS before I ask others too.

I've tried to unify all the various vplay derivatives and improvements I could find into a single package, but I need to contact all the contributers (ADH, jdiner, embeem, ...) to make sure they're okay with a redistribution of their work.

ronnythunder
10-25-2004, 06:06 PM
jamie, that would be awesome. i've been doing a little testing with adh's util.c that he posted a while back that uses the series 2-style ioctl-based low level disk i/o; no hard numbers yet, but it does work and seems quite a bit faster. i'm also thinking about enhancing it to read up to four "sectors" (which, in this context, is a 131072 bytes entity) at a time, which is how the tivo software does it when it's playing a show. same amount of data in one system call instead of four; that's got to be a win.

ronny

jbuehl
10-25-2004, 06:17 PM
mfs_streams is broken BTW. I have started a general petition for somebody with C skills to fix it.

http://www.dealdatabase.com/forum/showthread.php?t=38654

I saw that post and I volunteered to help with S1 a little while ago.

I did have luck with using mfs_ls and mfs_dumpobj to find the video. For anyone who is interested,

mfs_ls /Recording/NowShowingByBucketTitle

Lists the recorded shows with recognizable names. Apparently it also includes deleted shows too.

mfs_dumpobj <fsid>

dumps out the show data, and the section

RecordingPart 111228/13 {
CommercialSkipOffset[20]=34 591602296 1098507569 -980958763 -1636064301 982474432 -1463397165 -413974067 -887763917 30764
File[18]=124818
Begin[16]=0
End[17]=1799479
}

points to the fsid that contains the video, in this case 124818. mfs_export created a file about the right size, so I think it probably worked. It's encrypted, so I'll need to figure out how to play it before I know for sure.

Thanks for the help. The documentation for the mfs_* tools is pretty thin.

Jamie
10-25-2004, 06:27 PM
jamie, that would be awesome. i've been doing a little testing with adh's util.c that he posted a while back that uses the series 2-style ioctl-based low level disk i/o; no hard numbers yet, but it does work and seems quite a bit faster. i'm also thinking about enhancing it to read up to four "sectors" (which, in this context, is a 131072 bytes entity) at a time, which is how the tivo software does it when it's playing a show. same amount of data in one system call instead of four; that's got to be a win.
The ADH util.c is in my tree. I also have some performance improvements I added that reduce unnecessary buffering (mallocs and data copies) in the mfs_fsid_pread function that might help extraction. embeem already did something similar for insertion, but it was specific to mfs_import (aka mfs_stdinsert). I'll look at increasing the chunk size for reading/writing too.

I'm interested in seeing if the sendfile linux syscall could be used to avoid unnecesary transfers to and from user space when copying data from MFS to a socket. Probably won't get to that in the first round though. I'm not sure the series1 kernels are new enough to have that call.

Sorry for taking this thread off topic. Mods: feel free to move.

AlphaWolf
10-25-2004, 06:40 PM
Outstanding. BTW if it isn't much trouble see if you can do that stuff with mfs_tmfstream as well.

Jamie
10-25-2004, 06:46 PM
Outstanding. BTW if it isn't much trouble see if you can do that stuff with mfs_tmfstream as well.
mfs_tarstream? mfs_ftp uses that for tmf format. There isn't an mfs_tmfstream.c in my source tree.

[Edit: found it here (http://www.dealdatabase.com/forum/showthread.php?t=34027&highlight=mfs_tmfstream). One more contributor and one more source tree to merge in....]