PDA

View Full Version : Building MFS from scratch on an S2



alldeadhomiez
11-25-2003, 10:14 PM
A long time ago, d18c7db (evil_alex) posted instructions on how to use the fsmake and load-all.tcl utilities bundled with SA software v1.3 to construct an arbitrarily small (or large) MFS partition from scratch.

Times have changed, the TiVo software distribution has changed, and exporting/importing tystreams is no longer the hard part of the process. I am posting some utilities that may allow an intermediate or advanced S2 user to experiment with this himself.

This is for development purposes only; I am not attempting to provide a complete solution. No warranty, no assurance that this will work at all, yadda yadda. WORKSFORME.

The prerequisities for this project were:

1) A method of extracting and re-importing the following parts of the MFS hierarchy:

/Resource/LoopSet
/Resource/VideoClip (seems to be contained entirely within the LoopSet)
/Genre
/Component

I have modified mbm's dumpobj-tcl script to do this by traversing the filesystem and invoking mfs_export when appropriate.

If you notice any parts of the hierarchy I am missing, let me know.

2) Slice files for the software version you are trying to install. They can be captured during a phone call or possibly from a satellite download.

3) An image of a blank MFS filesystem, freshly initialized with fsmake. Included.

4) A copy of a working TiVo root filesystem and kernel image. 3.1u5 makes the job easier, and since you aren't starting myworld there's a good chance it will work on the newer DTiVo boards.

5) Another drive or another S2 TiVo, in working order, which has been compromised with BASH_ENV or a PROM hack, from which you can extract the needed resources.

6) A tar binary for the TiVo.

The basic steps I took were:

0) Read the original d18c7db document several times. It is included in the attachment.

1) Extract the resources from a working TiVo setup.

Boot the working TiVo and run resource_dump.tcl. It will create a tarball called resources.tgz which contains everything needed to reconstruct the necessary resources. Copy this tarball to your PC or something.

2a) Create a blank MFS filesystem.

I have done this and included it in the archive. I followed the d18c7db instructions up to "fsmake" on my S1, then messed with mfstools until I got the byte order right in the backup.

2b) mfsrestore the blank MFS filesystem to an empty drive

Use -x if you wish, or expand it later. It will fit on very small drives, which was the intent of this project. It will also fit on the stock 40GB drive and allow you to make larger root/swap partitions for hacking purposes, at the expense of recording space. I don't know of any reason why it wouldn't work with, say, a 200GB drive, if you are equipped for LBA48. This has not been tested.

3) Populate the non-MFS partitions.

This backup image ONLY contains hda10 and hda11; all other partitions are blank. Therefore you must populate hda6/hda7 with the kernel/root, mkswap on hda8, and write a valid bootpage to hda. Format /var while you're at it, since rc.sysinit isn't going to be running quite yet. Although my configuration was significantly more complex, you should be able to get away with a basic BASH_ENV hack, as long as you do not allow the virgin rc.sysinit to start myworld (don't fork bash into the background).

mke2fs -O none /dev/hda7
mke2fs -O none /dev/hda9
mkswap -v0 /dev/hda8
dd if=31u5kern.img of=/dev/hda6
bootpage ... blah blah

4) Start up the TiVo and get MFS running.

Power up the TiVo. Get MFS running. Something like:

export TIVO_ROOT=""
export MFS_DEVICE=/dev/hda10
mount -n /dev/hda9 /var
/tvbin/switcherstart -m

Other environment variables might be necessary. Maybe not. You'll find out.

5) Check to make sure MFS is okay.

Start up tivosh and mls /. Ensure that MFS is empty and accessible. If you're paranoid you might want to run fsfix - see rc.sysinit for the syntax.

6) Install the resources.

Transfer resources.tgz to this unit, unpack, and install:

cd /tmp
tar zxvf resources.tgz
export TIVOSH_POOLSIZE=16777216
cd work
tivosh resource_create.tcl

You may safely ignore any warnings about the poolsize being a few hundred bytes too big.

There's probably a way to use cpio to unpack the archive if you don't have tar. I don't remember it offhand.

<split here to make the forum software happy>

alldeadhomiez
11-25-2003, 10:17 PM
7) Insert the TiVo software into MFS.

Transfer the slices to this drive. The slices I used for 3.1.1b were:

GZbin-16099016-1.slice GZetc-16099018-1.slice GZkernel-16099020-1.slice GZlib-16099022-1.slice GZprom-16099024-1.slice GZsbin-16099026-1.slice GZtvbin-16099028-1.slice GZtvlib-16099030-1.slice swsystem-16099034-1.slice utils-16099032-1.slice

gunzip if necessary.

Use the included dbload.tcl script to load these slices into MFS:

cd /tmp
work/dbload.tcl *.slice

I got several errDbNotFound errors which did not happen on my normal 3.1U5 box, however they did not cause any problems (yet?).

9) Start installation.

Use /tvbin/installSw.tcl <version> to install the new software. My normal (clumsy) procedure for this is to copy it to /tmp, comment out the reboot line, and then fix up my bootpage after it runs so that the new software does not lock me out. Do whatever best suits your method of compromise. Also, make sure the software doesn't misdetect your TiVo root filesystem and clobber the root that you are running from. If it does... have a backup handy. Since most of the process executes from the utils slice, you will need to unpack that slice if you want to step through it by hand or change things (like commenting out bootpage references).

The version number can be found by doing an mls on /SwSystem after the slices have been loaded. In my case it was 3.1.1b-02-2-151. If /SwSystem is empty you are probably missing some really important resources.

10) Reboot or switcherstart -k and restart the TiVo software.

Obviously you will need to reboot if you are installing a software version which uses an incompatible kernel, for instance if you are upgrading from 3.1 to 4.0. At this point the process should be complete. If you're using monte, BASH_ENV, etc. do any necessary fixups now.

I am currenting running with a 1GB MFS media region, on a very small drive. My status screen confirms that my SW version is 3.1.1b and my recording capacity is 0 hours.

Nagging questions which I haven't figured out yet:

1) How do you adjust the "used" tystream size? mfs_stdinsert does not do this so I need to use mfsbackup -T to correctly back up the inserted streams. Edit: this appears to be part of the mfs inode structure, but nobody knows of a safe way to change it.

2) Why do I get an error 0xffff when I try to delete MFS objects? This has got to be something obvious I'm overlooking. Edit: dbobj $obj markasrubbish, or MfsRubbishTree.

3) How do I tell the difference between a tyfile and a tystream object? Edit: mfs streamsize

4) How do I initialize a tcl array, which may or may not exist, to zero elements? set then unset is very ugly.

5) Are there going to be problems in the long term by using the v1.3 fsmake binary to generate an MFS filesystem that a 3.x or 4.x box uses?

6) Is there anything else I'm leaving out or installing incorrectly that will impact the setup's stability?

Any useful feedback would be appreciated.

Everything in the archive worked for me at one point but should be considered untested at the moment.

Edit: This has been tested successfully with 3.1.1b and 4.0 so far. Hint: do not run a 0-hour box; it will not work for long. Use mfsadd or something to expand it.

Edit 2: I have run into some trouble when the number of "used" chunks in the loopset stream is too large (i.e. does not match the actual length of the loopset data). This seems to happen most often on 4.x and 5.x. If you are seeing weird tverr log entries or reboots associated with the rendering of loopsets, this may be causing your problem.

Edit 3: Scroll down to Jamie's post to grab the latest versions of the resource_dump stuff.

rc3105
12-27-2003, 11:29 PM
1) How do you adjust the "used" tystream size? mfs_stdinsert does not do this so I need to use mfsbackup -T to correctly back up the inserted streams. Edit: this appears to be part of the mfs inode structure, but nobody knows of a safe way to change it.

change line 5 of support.tcl from

set fsid [mfs streamcreate 131072 $y]

to

set fsid [mfs streamfill 131072 $y]

and used size will equal allocated size


cool util, makes the process SO much easier :D

n4zmz
01-03-2004, 03:02 PM
alldeadhomiez - your pm folder is full.

It is hard to reproduce your work without fsmake and the slices. Care to share?

alldeadhomiez
01-03-2004, 03:41 PM
alldeadhomiez - your pm folder is full.

It is hard to reproduce your work without fsmake and the slices. Care to share?

fsmake came from TiVo software v1.3. You don't need it because I included a blank mfsbackup image in the archive - just byteswap accordingly for S1 or S2.

The slices are the standard TiVo software slices which you can get from the usual sources. Names and such will be listed in your tclient.log.

AlphaWolf
01-03-2004, 05:15 PM
you know what would be trick, is to create stream files (e.g. tivosh's mfs streamcreate) in MFS without the use of tivosh.

alldeadhomiez
01-03-2004, 06:48 PM
you know what would be trick, is to create stream files (e.g. tivosh's mfs streamcreate) in MFS without the use of tivosh.

I believe the following items would be necessary or helpful in pulling this off:

descriptions of some of the undocumented "fill" fields in the mfs inode
information on the structure of the zone map data
information on how buddy maps work
information on what the first 1000-odd sectors in mfs, between the superblock and the inode zonemap, are used for

It seems like your intent would be to completely remove tcl from the process, so we will also have to figure out what is involved in creating and linking tyDb objects into the database, indexing them, and adding dirents. With any luck, fsfix and mfscheck will noisily correct our mistakes and tell us what we are looking for.

It would appear that several of the "old school" TiVo hackers have access to some of this information but have not published it.

AlphaWolf
01-03-2004, 07:32 PM
I believe the following items would be necessary or helpful in pulling this off:

descriptions of some of the undocumented "fill" fields in the mfs inode
information on the structure of the zone map data
information on how buddy maps work
information on what the first 1000-odd sectors in mfs, between the superblock and the inode zonemap, are used for

It seems like your intent would be to completely remove tcl from the process, so we will also have to figure out what is involved in creating and linking tyDb objects into the database, indexing them, and adding dirents. With any luck, fsfix and mfscheck will noisily correct our mistakes and tell us what we are looking for.

It would appear that several of the "old school" TiVo hackers have access to some of this information but have not published it.

If anybody could do it, tridge could...but he hasn't been around since the pre-ddb days :)

But yeah, tivosh gets on my nerves.

Juppers
08-23-2004, 05:45 AM
I'm trying to move some loopsets from one box to another, I can't seem to get it to work. Everything looks like it should be working. It just doesn't create the LoopSet tyDb for anything to be inserted into. Here is a snippet of the resourse_create.tcl file and what it returns. Any ideas what could be going wrong?


#!/tvbin/tivosh
source "support.tcl"
set db [dbopen]
puts "Creating object: /Resource/LoopSet/GreenBasic_3"
if [catch {
RetryTransaction {
set LS_0 [db $db create LoopSet]
set id_LS_0 [list [dbobj $LS_0 fsid] [dbobj $LS_0 subobjid]]
dbobj $LS_0 set ServerVersion 2
dbobj $LS_0 set Name {GreenBasic_3}
set LSC_0 [db $db createsub LoopSetClip $LS_0]
set id_LSC_0 [list [dbobj $LSC_0 fsid] [dbobj $LSC_0 subobjid]]
dbobj $LSC_0 set Entrance 1000
dbobj $LSC_0 set Exit 1001
set VC_0 [db $db create VideoClip]
set id_VC_0 [list [dbobj $VC_0 fsid] [dbobj $VC_0 subobjid]]
dbobj $VC_0 set ServerVersion 2
set fsid [tycreate "GreenBasicLoop.mpv"]
lappend inserts "$fsid GreenBasicLoop.mpv"
dbobj $VC_0 set File $fsid
dbobj $VC_0 set Name {GreenBasicLoop.mpv}
dbobj $VC_0 set ServerId {13245010}
dbobj $LSC_0 set VideoClip $VC_0
dbobj $LS_0 set State $LSC_0
set LSC_1 [db $db createsub LoopSetClip $LS_0]
set id_LSC_1 [list [dbobj $LSC_1 fsid] [dbobj $LSC_1 subobjid]]
dbobj $LSC_1 set Entrance 5
dbobj $LSC_1 set Exit 1000
set VC_1 [db $db create VideoClip]
set id_VC_1 [list [dbobj $VC_1 fsid] [dbobj $VC_1 subobjid]]
dbobj $VC_1 set ServerVersion 2
set fsid [tycreate "GreenBasicDrawOn.mpv"]
lappend inserts "$fsid GreenBasicDrawOn.mpv"
dbobj $VC_1 set File $fsid
dbobj $VC_1 set Name {GreenBasicDrawOn.mpv}
dbobj $VC_1 set ServerId {13245012}
dbobj $LSC_1 set VideoClip $VC_1
dbobj $LS_0 set Trans $LSC_1
dbobj $LS_0 set ServerId {13840307}
dbobj $LS_0 set Port 5
}
}] {
puts "Creation failed"
}
dbclose $db
tyinsert $inserts
puts "Finished."


[/var/dump/work]: tivosh test.tcl
Creating object: /Resource/LoopSet/GreenBasic_3
creating stream of size 8: fsid 398925
creating stream of size 8: fsid 398927
Creation failed
inserting stream GreenBasicLoop.mpv into fsid 398925
ERROR: Didn't find fsid=398925!
while executing
"exec ./mfs_stdinsert $fsid $fname"
(procedure "tyinsert" line 6)
invoked from within
"tyinsert $inserts"
(file "test.tcl" line 46)

ozTiVoEd
08-31-2004, 11:25 AM
Hi,

Love the toolkit. Great work! This has worked perfectly for us doing series 1 (and even a bit of series 2) stuff in Australia. I've been trying to do an image with a 128MB partition 2 (for dedicated hacks).

I've created the mfs with fsmake, but I can't get the resulting mfstools backup down anywhere near the size of yours. all partitions dd'd with /dev/zero except mfs.

Any pointers?

Cheers,

Ed

alldeadhomiez
08-31-2004, 12:44 PM
Love the toolkit. Great work! This has worked perfectly for us doing series 1 (and even a bit of series 2) stuff in Australia. I've been trying to do an image with a 128MB partition 2 (for dedicated hacks).

I've created the mfs with fsmake, but I can't get the resulting mfstools backup down anywhere near the size of yours. all partitions dd'd with /dev/zero except mfs.

Try zeroing the entire drive (or all mfs partitions) before you run fsmake. mfstools is probably backing up leftover "junk" that fsmake did not erase; as you can see, the uncompressed image size is still very large despite that fact that it is an empty filesystem.

compwiz312
10-29-2004, 08:03 PM
ADH, can't thank you enough for your method. Unfortunetly when trying to run the resource dump script on my 4.1b SA unit, I get the following errors:


invalid attribute: 0x2f0013
while executing
"dbobj $obj get $attr"
(procedure "dump" line 43)
invoked from within
"dump $f $obj "{no parent}" 10 " ""
("uplevel" body line 19)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set obj [db $db open $x]
set fsid [dbobj $obj fsid]
set subid [dbobj $obj subobjid]

# this check is redundant but ..."
("foreach" body line 2)
invoked from within
"foreach x $list {
RetryTransaction {
set obj [db $db open $x]
set fsid [dbobj $obj fsid]
set subid [dbobj $obj subobjid]

# this che..."
(file "resource_dump.tcl" line 155)

Any insight you can provide would be highly helpful.

Thanks in advance,
Justin

ronnythunder
12-28-2004, 05:13 PM
ok, here are some additional notes from my experience with this. i have a 0 hour hdvr2 running 3.1.1c (i've got all of the 3.1.1e slices except tvlib, grrrrrr) that seems to come up fine, though i haven't tried any live buffering or anything with it (doesn't even have sat cables attached). one of the cool things about this config (if it isn't obvious) is that it only uses one mfs pair. of course, as adh said, a 0 hour tivo isn't going to be good for much, so you'll have to make one expansion, but you get make that expansion as you wish. for example, if you had a 160 gb drive, you could load a 0 hour backup, expand, and have a 160gb tivo that only uses two pairs. you could then, at a later date, get a 400gb drive, and expand to a 400gb a drive without running out of partitions.

based on adh's step numbers, here are the extra things that i did (and/or should have done):

1) for some reason, my mfs_export was bitching with crc errors when extracting the loopsets. after much consternation, i realized that tcl will bomb if an exec'd process puts *any* output on stderr, even if the return code is zero. to work around this, i nulled the first byte of the "crc mismatch" text in the mfs_export binary. the resources.tgz was created ok after that, and the loopsets looked fine.

3) i assume adh just overlooked this when trying to "dumb down" his config for this post, but... you gotta have the contents of a root fs after the mke2fs in order for anything to happen on the tivo! i just used the u5fs.img that most folks have laying around from the original two partition monte days. i renamed the stock rc.sysinit and put a brief one based on adh's step 4, adding the bash execution:
/bin/bash </dev/ttyS2 &>/dev/ttyS2 &at the end. i also put:
export PATH=/sbin:/bin:/tvbin:.:/etc/rc.d at the beginning.

also, this is a good time to mount the /var partition and place the slice files and resources.tgz onto it. you should put the tar binary in /bin of your chosen root partition as well. since you have /var mounted, create the following dirs under it: packages, run and tmp. you'll need those later, trust me.

i used a killhdinitrd'd u5 kernel, nothing fancy

4) after a false start with the missing /var/run and /var/tmp dirs, the eventswitcher started ok. however, i think, in retrospect, i should have set the date and time on the box here; a lot of stuff in mfs has the 1970 date, but it seems ok.

9) after the install script completed (don't forget to remove/comment the reboot statement!), i used dd to copy the killhdinitrd'd kernel from the current to the new kernel partition, then appended the bash statement onto the rc.sysinit in the new root.

i also installed the fakecall.tcl script to get it past the guided setup. i rebooted, came to the gs screen, ran the script, rebooted again, and it came up to directv central.

at this point, i'll take a mfstools backup and probably give it to, ahem, someone who has a lot of backup images so that it can be tried by others. of course, thanks to the lovely (not) license with killhdinitrd, any image that gets onto a public server will be kernel-less or have a stock u5 kernel that will blow away the rc.sysinit changes upon boot.

many thanks to adh and the others who toiled before him to bring this together. the process is really not difficult, and between adh's instructions and my notes, it's darned near turnkey.

ronny

rc3105
12-28-2004, 06:15 PM
of course, thanks to the lovely (not) license with killhdinitrd...

feel free to donate your own exploit to the ebay leeches

really, how hard is it to type one additional dd command? :rolleyes:

alldeadhomiez
12-28-2004, 06:25 PM
i assume adh just overlooked this when trying to "dumb down" his config for this post, but... you gotta have the contents of a root fs after the mke2fs in order for anything to happen on the tivo!

That was included in "blah blah." ;)

Back in the BASH_ENV days, several extra steps were needed, which were well-documented elsewhere on the forum.


at this point, i'll take a mfstools backup and probably give it to, ahem, someone who has a lot of backup images so that it can be tried by others. of course, thanks to the lovely (not) license with killhdinitrd, any image that gets onto a public server will be kernel-less or have a stock u5 kernel that will blow away the rc.sysinit changes upon boot.

The rc.sysinit changes shouldn't really be necessary, if you have already completed the process. At this point you can restore a virgin 3.1.x root filesystem and kernel image, and distribute the 0-hour image.

BTW: I'm not sure if I mentioned it, but on 4.0+ you will want to use known good slices to insert the loopsets, or you will need to find a way to adjust the "used blocks" field in the inode.

ronnythunder
12-30-2004, 04:58 AM
someone asked that i post the mfs_export that i'd hacked to not show the "crc mismatch" errors; here it is:

ronny

Jamie
12-30-2004, 11:47 AM
someone asked that i post the mfs_export that i'd hacked to not show the "crc mismatch" errors; here it is:

ronny
I'm not keen on silently ignoring errors. Might it not be better to just redirect stderr in the tcl script if you don't care about it, rather than changing mfs_export? Something like adding 2>>/tmp/mfs_export.stderr at the end of the exec line would do it. Then you could at least examine the errors afterwards.

A more important question is why you were seeing crc errors at all. Is there something broken in the crc checks in the tridge mfs library? Or was there something wrong with your loopsets?

davidblack
12-30-2004, 06:59 PM
I've just tried to run the both the original and the modified mfs_export and i'm seeing the errors below.

Any suggestions?

Cheers
David

Dumping object: 695/12
Skipping visited object: 679/12
Skipping visited object: 688/12
Skipping visited object: 493/12
invalid attribute: 0x2f0013
while executing
"dbobj $obj get $attr"
(procedure "dump" line 43)
invoked from within
"dump $f $obj "{no parent}" 10 " ""
("uplevel" body line 19)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set obj [db $db open $x]
set fsid [dbobj $obj fsid]
set subid [dbobj $obj subobjid]

# this check is redundant but ..."
("foreach" body line 2)
invoked from within
"foreach x $list {
RetryTransaction {
set obj [db $db open $x]
set fsid [dbobj $obj fsid]
set subid [dbobj $obj subobjid]

# this che..."
(file "resource_dump.tcl" line 155)
bash-2.02#
bash-2.02#

alldeadhomiez
12-30-2004, 07:33 PM
I've just tried to run the both the original and the modified mfs_export and i'm seeing the errors below.

Why would that be an mfs_export issue?

Jamie
12-30-2004, 07:34 PM
I've just tried to run the both the original and the modified mfs_export and i'm seeing the errors below.

Any suggestions?
I saw this too. The issue seems to be an attribute not in the schema.

Tivoweb:

Component 943/12 {
ServerVersion = 147
Code = 539/-1
Name = STARCAST
(attribute 0x2f0013 not in schema)
Type = 2
ServerId = 371781
Version = 1
IndexPath = /Component/Type/2/STARCAST /Server/371781
}mfs_dumpobj:

bash-2.02# mfs_dumpobj 943
Component 943/12 PRIMARY {
ServerVersion[9]=147
Code[18]=539/-1
Name[16]=STARCAST
UNKNOWN[19]=1
Type[17]=2
ServerId[8]=371781
Version[1]=1
IndexPath[4]=/Component/Type/2/STARCAST /Server/371781
}
I wrapped a catch around the attr loop body in the dump proc to allow it to continue in spite of the bad attribute. Haven't yet tried to restore though.

compwiz312
12-30-2004, 08:52 PM
I wrapped a catch around the attr loop body in the dump proc to allow it to continue in spite of the bad attribute. Haven't yet tried to restore though.

Would you mind posting this modified version, this is the problem I encountered a while back as well and I had received no fix/response. Your help is appreciated.

Thanks,
Justin

Jamie
12-30-2004, 10:23 PM
Would you mind posting this modified version, this is the problem I encountered a while back as well and I had received no fix/response. Your help is appreciated.
Here's the deal: the fix was only tested as far as making sure the resource_script.tcl script will run. I have no idea if the resources.tgz is produces is still any good in the presence of these "caught" errors. There may be more work to do to be able to recreate these objects properly.

It was just a two line change. Rather than post the whole script, here's a "diff -u". You could apply this with patch, or just edit those two line changes in yourself. The body of the catch if should probably do something rather than be a noop as it is now. Note that catch will also catch the explict continues in the body of the loop in addition to the errors.


--- resource_dump.tcl 2003-11-25 19:10:14.000000000 -0700
+++ resource_dump-jamie.tcl 2004-12-30 19:16:58.966536450 -0700
@@ -65,7 +65,7 @@

puts $out "${prefix}set id_$myobj \[list \[dbobj \$$myobj fsid\] \[dbobj \$$myobj subobjid\]\]"

- foreach attr [dbobj $obj attrs] {
+ foreach attr [dbobj $obj attrs] { if [catch {
set attrs [dbobj $obj get $attr]
set atype [dbobj $obj attrtype $attr]
set aindex 0
@@ -107,7 +107,7 @@
}
incr aindex
}
- }
+ }] {} }
return $myobj
}

ronnythunder
12-30-2004, 11:01 PM
I'm not keen on silently ignoring errors.neither am i, but i wanted to get past the damned error, and i'm very handy with hexedit. :)

i, too, wonder why there were errors; the loopsets all played fine with mplayer on my pc, and the tivo that i put them in (and took them from) are fine.

ronny

ronnythunder
01-06-2005, 03:49 AM
i don't know how many others are playing in this somewhat rarified air, but...

i've test-built these zero hour configs probably a total of 10 times now. one very interesting and perhaps crucial issue that i've run into: the finished product, be it the bone stock system right after the reboot following installsw.itcl or after any and all enhancements, cannot be backed up with mfstools without using "-Tao". well, specifically, you can backup, but the backup won't work.

i've tried "-f" values all the way up to 29999 with no luck. in every case, an assertion is thrown upon mounting mfs and the machine reboots. upon reboot, fsfix "fixes" the database, but the cure is worse than the disease, and if it works at all after that, the loopsets are gone and who knows what else doesn't work.

my first attempt with "Tao" seems to be working, with one oddity: while booting, the console displays this:
Checking new software installation
Could not check software installation: can't open object (errDbNotFound)not sure if that's a big deal, and i can't recall if the original said that as well. it does get far enough along to display live tv, but as adh said, it doesn't last long; after some number of minutes, it segfaults badly and reboots. i put it on channel 490 and it seems ok for now.

so, if "Tao" is the only way to backup such a config, this would bode ill for a machine that's actually been expanded and has recordings. it would seem to mean that you can't make a working backup at that point. interesting.

ronny

alldeadhomiez
01-06-2005, 12:13 PM
Checking new software installation
Could not check software installation: can't open object (errDbNotFound)

That's because /SwSystem/ACTIVE doesn't exist in an freshly initialized filesystem. It's normal.


so, if "Tao" is the only way to backup such a config, this would bode ill for a machine that's actually been expanded and has recordings. it would seem to mean that you can't make a working backup at that point. interesting.

You shouldn't be using "-f" anyway. Maybe "-l 32" or something, but not "-f". MFS noobs need to stick with "-Ta". I would suspect this is an mfstools issue, since your top fsid is bounded by some number, but it is irrelevant because "-f" should not be used. If you want to back up your settings, make a script that dumps them to a file.

Also, the used_size fields on the tyStream inodes are not initialized properly, which could explains why things might break without "-T".

packrat
01-16-2005, 05:23 PM
I made a mistake and issued a global delete on a mask and now I need tivoapp for 4.0.1b and i have the slice files and am wondering how to unpack a slice file to get this file back on my tivo

Thanks,

-PCP

ozTiVoEd
01-25-2005, 04:56 AM
I made a mistake and issued a global delete on a mask and now I need tivoapp for 4.0.1b and i have the slice files and am wondering how to unpack a slice file to get this file back on my tivo

Thanks,

-PCP

gzip -d <filename>.slice.gz
readguide -f <filename>.slice
# with series 2 stuff the filename often comes out as "1"
# if its not "1" then use appropriate filename for following steps
mv 1 1.gz
gzip -d 1.gz
cpio -id < 1 # this extracts the cpio archive into dirs.

AlphaWolf
03-20-2005, 08:22 PM
If you try this with a seed tivo sw version 5.x and above (including 4.x?) in your rootfs, you wont be able to start the mfs daemon with switcherstart -m. TvLauncher now does that instead, but it is a real pain in the ass to get started manually (there are even more variables and other crap that need to be set for yet even more programs to run before MFS can be started.) The easiest way is to remove stages e, f, and g from the rc.sysinit script, and then just run the rc.sysinit normally.

Thinkdiff
03-27-2005, 07:05 AM
hey, thanks a lot for the tool kit. I've been working on this for a couple days now (BASH_ENV was impossible to get working, so I just put my own rc.sysinit on there and make it only load ethernet and telnet. worked great). I built my first 0hour image last night, used 4.0.1b slices and made an image just doing the normal -6so. Came out to 76MB! very cool!. So I reimaged it onto my 40GB HD and it boots great and works fine on my HDVR2. But When I restored the image onto my DVR40 (and did RID hacks), I have no backgrounds! I did the resource_create script on a HDVR2.. does that matter? I'm currently trying to reinsert the 4.0 loopset slices on the dvr40, and if that doesn't work, i'll install it onto the other partition and see if that works.

compwiz312
04-05-2005, 12:21 AM
You probably don't have the backgrounds because they weren't backed up. -6so won't backup the backgrounds from my experience.

When backing up a freshly created 0 hour image, I find the following setting to work weel and backup everything (in a small image)


mfsbackup -9svao output.mfs /dev/hdX

TivoWare
07-22-2005, 05:22 PM
I'm try to use these scripts to insert on object. The object is created but the insert of the file fails and I have treid everything I know.

Tivo:/var/work> ./resource_create.tcl
Creating object: /Resource/Image/aande-s1-p1
creating stream of size 1: fsid 79424
inserting stream aande-s1-p1 into fsid 79424
importing fsid 79424 of size 4194304 from aande-s1-p1
Done

crc mismatch len=33792 0x68bb8e3e 0x1ac898b9
while executing
"exec ./mfs_stdinsert $fsid $fname"
(procedure "tyinsert" line 6)
invoked from within
"tyinsert $inserts"
(file "./resource_create.tcl" line 6885)


I tried the insert by havd and got the same error.

Tivo:/var/work> ./mfs_stdinsert 79424 aande-s1-p1
crc mismatch len=33792 0x68bb8e3e 0x1ac898b9
importing fsid 79424 of size 4194304 from aande-s1-p1
Done

I tried using a new mfs_import, but it does not look like I am using it correctly.

Tivo:/var/work> mfs_import 79424 aande-s1-p1
recv error, falling back to read loop. : Socket operation on non-socket
input stream truncated early: 0 out of 4194304 bytes received


Any Ideas?

TivoWare
07-22-2005, 05:46 PM
Nevermind my last post.. I think I found another way to do it from looking at the bufferhack. Any comments are welcome.


#!/tvbin/tivosh
set db [dbopen]
RetryTransaction {
set name "aande-s1-p1"
set logo [db $db open /Resource/Image/$name]
set logoid [dbobj $logo get File]
set newlogo [ToMfs $name]
puts "Loading $name into FSID $logoid"
dbobj $logo set File $newlogo
}
dbclose $db

Jamie
07-22-2005, 06:58 PM
..
I tried using a new mfs_import, but it does not look like I am using it correctly.

Tivo:/var/work> mfs_import 79424 aande-s1-p1
recv error, falling back to read loop. : Socket operation on non-socket
input stream truncated early: 0 out of 4194304 bytes received


Any Ideas?
The 'recv error,..." message is not truly an error. You're going to see it anytime you are reading from a file rather than a socket. I special cased the socket case because I found that using recv was faster than read since it can read a larger chunk in a single system call (read returns no more than the MTU size in each call). The process still exits with a zero exit status, but it does still write to stderr. I should probably just take that message out, but in the meantime, redirect stderr to a file or /dev/null so that tcl won't cough.

The truncated early message implies that your source file is a zero length file.

Note that I'm currently in the middle of reworking the resource_dump stuff to get it to work with 7.1. Turns out that the 7.1 tivosh does not seem to allow more than 60 open objects in a transaction, and this breaks both resource_dump.tcl and the resource_create.tcl it produces. The Sports Genre seems to be the first place where it runs into that limit. I have workarounds for that, but I haven't completed testing yet.

By-Tor
10-10-2006, 06:07 PM
Tried the resource_dump.tcl (w/Jamie's mod) on a HR10-250 (3.1.5f) and root filled up (script bombed). 256MB root. After experimenting with doing part of the collection at a time it looked like the problem was fitting all the uncompressed resources, plus the resource_create.tcl script, plus the archive. So I wiped the alt root, created a blank fs with mke2fs, mounted it, and hacked the script to write the archive and resource_create script there. Worked fine.

Jamie did you ever get anywhere with getting this going on 7.1?

Jamie
10-11-2006, 03:50 AM
Jamie did you ever get anywhere with getting this going on 7.1?Yeah, here's what I have. I did this sometime back, but it works with all 6.x and 7.x versions, AFAIK. Here's the Changelog entries:
2005-07-23 <Jamie@DDB>

* Use smaller transactions to work around limits in 7.x on the number of objects open in a transaction.

* Use latest mfs-utils mfs_uberexport and mfs_import binaries.

* Replace the bootpage in the freshly_initialized_mfs_s2.mfs image so it is usable with newer PROM's (remove netboot image name.)

* Reorder dumped directories to try to get low fsid's for tystreams

* Ignore attributes missing from schema.

imaloserbaby
02-11-2007, 08:12 PM
Jamie (or anyone else), I noticed that your resource_dump-20050723.tgz file did not include this tweak from Riley:


change line 5 of support.tcl from

set fsid [mfs streamcreate 131072 $y]

to

set fsid [mfs streamfill 131072 $y]

and used size will equal allocated size

Was that intentional? Is it a recommended change, or is there a downside to this? Just wondered.

tivo4mevo
04-11-2007, 11:31 AM
Using the great information in this thread, I've successfully built several images from scratch for both SA and combo units (from 4.0.1b to 8.1a), but am having trouble successfully building a 6.3c image. I'm able to build a stock image using my own extracted resources and using [what I'm fairly sure are known, valid] resource slices. In both instances, the resultant image boots fine on a dsr704, but crashes on an HR10-250.

On the hr10-250, tivoapp crashes with a "Activity TvRecorderActivity <256> strayed!" error, and I've attached a relevant snippet of the bootlog.

Just wondering if anyone had run into this before, as my searches only turned up FOX OTA audio issues (which I thought were ironed out with 6.3c).

Some more (possibly) relevant information:
- happen to be using a 7.1b root image (with rc.sysinit neutered to halt at sysinit stage D) to build mfs.
- needed to export SerialNumber (as '/tvbin/crypto -gsn') to satiate updateroot
- resultant image has no hacks--completely stock, kernel and all.
- no usb adapter attached
- no sat or OTA inputs connected

Any input is appreciated. Other than varying the root image used to build mfs, attempting to build a precursor 3.1.5 image, and tracing through tivoapp a bit; I'm out of ideas.

drez
04-11-2007, 01:57 PM
I have a hunch that a 7.2 root image might work.

6.3x seems to be more closely related to 7.2. (The Tivo-brand Wireless G driver in 6.3x.)

There's a 7.2.2 mfs image here: http://www.dealdatabase.com/forum/showpost.php?p=270112&postcount=917


(6.3x is probably based on 7.2 or 7.2.1, because the Linksys USB200M v2 driver that was added in 7.2.2 is not in 6.3x...)

Jamie
04-11-2007, 02:30 PM
It has been a long time since I've looked at this, but it may be that my "ignore attributes missing from schema" change, together with an incompatible base OS version could be leaving things out of MFS that are needed on the hr10-250 hardware.

tivo4mevo
04-12-2007, 10:49 AM
Thanks for the helpful ideas, I believe that not expanding MFS was what caused the hr10-250 to crash before it fully booted.

ADH had warned not to run a zero hour unit. I never had trouble (with an SD, zero hour unit) so long as I tuned live TV to blank channels, but it appears necessary to obey that warning for the hr10-250 (and perhaps for a S3 as well?).

drez, I think you're correct about the closest common SA ancestor of 6.3c. 6.3c generates inapplicable upgrade messages named "721_..", another piece of evidence that they forked 6.3 from the SA line at 7.2.1.

drez
04-15-2007, 08:14 PM
Since you got it working, how about uploading your 6.3c image to www.mediafire.com and sharing it in the Backup Image and Slice thread (http://dealdatabase.com/forum/showthread.php?t=31863)? :)

(No registration required to download or upload to mediafire and the file will stay up forever. Unlike rapidshare, there's no wait to download the file and the downloads are faster than rapidshare too.)

unitron
05-10-2010, 11:50 PM
A long time ago, d18c7db (evil_alex) posted instructions on how to use the fsmake and load-all.tcl utilities bundled with SA software v1.3 to construct an arbitrarily small (or large) MFS partition from scratch...

Apparently my google-fu isn't strong or sophisticated enough for something that obscure. Would you happen to have a link to those instructions?

ScanMan
05-11-2010, 10:10 AM
Apparently my google-fu isn't strong or sophisticated enough for something that obscure. Would you happen to have a link to those instructions?Here (http://www.dealdatabase.com/forum/showthread.php?t=2519) is a reference to an older post. The referenced link to http://pvrhack.sonnik.com/tivo/d18c7db/TiVo-Restore.htm is broken but the wayback machine has it: http://web.archive.org/web/20021212143824/http://pvrhack.sonnik.com/tivo/d18c7db/TiVo-Restore.htm

jkozee
01-13-2013, 05:35 PM
I've been playing around with this script on 11.0k, and found that the subobject attribute is not handled, so I patched it accordingly.

I also added code to identify and handle handle both tystreams and tyfiles separately. This includes importing them only once, and then just referencing the file as needed.

In addation, I believe the solution to "Nagging question" number one:

1) How do you adjust the "used" tystream size? mfs_stdinsert does not do this so I need to use mfsbackup -T to correctly back up the inserted streams. Edit: this appears to be part of the mfs inode structure, but nobody knows of a safe way to change it.

Is to export the loopsets as slices, using the modified version of jt1134's dump_loopsets.tcl posted here (http://www.dealdatabase.com/forum/showthread.php?61004-Load-THD-slices-and-loopsets-into-a-THDXL-image-to-convert-it-to-a-true-THD-image&p=316777#post316777).