PDA

View Full Version : Proven Concept: Tivo Stream Importing



AlphaWolf
08-16-2002, 11:59 PM
First let me say: yes, it finaly has been done without the use of ele2pestripple.

We can export a tystream from one tivo, and import it directly into another, and it works perfectly.

But before I provide details, the forum link I am providing will not now, nor ever, get you anything that you can use for this, that stuff will be posted here when its deemed ready for release. The forum I am about to link to is primarily for developers.

Does that mean you need to be able to write any software? No, god knows that I can't even do that. That forum is only for developmental posts, meaning information that is useful. Questions about "I cant get this to work", "how do I do this", or statements about what you want to do with it, etc, are not allowed there, and will be deleted. You either ask those questions here, or on AVS, when the software is available (which will no doubt be a matter of only a few days at most).

Now getting down to the point, I don't have any software that can do this yet, I can only tell you that its been done, (I know this first hand, and I don't think that I have ever lied to any of you) and that the theory I described has been proven to work. We need some of you talented developers out there to help come up with some code that the end users can use. :)

http://alt.org/forum/index.php?t=thread&frm_id=2&rid=23&S=1c3b9bf76e23317ac22a2a5d74c62ad1

AlphaWolf
08-19-2002, 11:55 AM
Just a quick heads up: If you want to save any tystreams for future use on another tivo, do not extract them using any current tystream extraction tools. The reason why is because the versions of mfs_stream that are going around currently are known to mangle the tystreams to where tivo can't read them. Sometimes some extra garbage gets put into the tystream through stderr, and also the "master chunk" of the tystream (which contains important information about the video) is completely ommited.

The best extraction tool I have seen so far was the one from edpuffedmonster, which simply was mfs_export that queued each tystream part into one big file, without ommiting anything or adding anything, but it seems to have been deleted from the sticky thread.

captain_video
08-19-2002, 01:56 PM
I just added a link for mfs_export in the HowTos thread in the File Downloads post. I'm not sure if it's the same one and it doesn't have a readme file with it but there it is. Run a search in the forum to get more info.

AlphaWolf
09-14-2002, 01:27 AM
Another heads up, those of you who have extracted scrambled tystreams in the hope that they can be descrambled later are a bit out of luck. Embeem, dtype, and a few others have pointed out to me that for each scrambled tystream is a key thats associated with it, but that key is not stored in the tystream itself. When you extract the stream, you are leaving that key behind.

When you reinsert the stream, its believed that tivo sees no key associated with it and assumes that its a stream that was recorded in 2.0 where streams were not scrambled (the 2.5 upgrade does NOT scramble old recordings, they remain unscrambled), and hence doesn't try to descramble it, nor could it if it even tried since it doesn't have the key (get it?). I have done a few tests to confirm this, so AFAIK, extracted scrambled tystreams are basicaly useless, unless somebody wants to write some code to brute force descramble them. Tystreams that you haven't extracted but are still scrambled aren't useless of course, but as of yet I haven't seen any developer incentive to work on this, nor do I see any reason for anybody to want to work on it, as most of us just assume record the show the next time its on again :D

psperry
11-11-2002, 06:39 PM
ok, probably missed something obvious, but here it is anyway...

set up my nfs server on my xp machine ... works great - have it mounted to /var/tmp/contents.

downloaded rcinsert.tcl and mfs_insert to /var/insert and did chmod +x on both files .

added /var/insert to my path as suggested

while in /var/insert I get the following:


declare -x DEBUG_BOARD="false"
declare -x HDA_ID="354029229304"
declare -x HDB_ID="Unknown"
declare -x HOME="/"
declare -x HOSTNAME="(none)"
declare -x HOSTTYPE="i686"
declare -x IrdSerialNumber="532F0898"
declare -x MACHTYPE="i686-pc-linux-gnu"
declare -x MFS_DEVICE="/dev/hda10"
declare -x OSTYPE="linux-gnu"
declare -x PATH="/bin:/sbin:/tvbin:/devbin"
declare -x PROMVERSION="

TiVoProm Monitor, release version 2.05 (ntsc)
"
declare -x PWD="/var/tmp"
declare -x SHELL="/bin/sh"
declare -x SHLVL="2"
declare -x SerialNumber="0010000202F0898"
declare -x TERM="linux"
declare -x TIVO_REMOTE="TIVO"
declare -x TIVO_ROOT=""
declare -x TIVO_SVR_ADDR="192.168.50.1:80"
declare -x TV_STD="NTSC"
declare -x _="/bin/bash"
declare -x chmem="0x806029C0"
declare -x prom="r"
declare -x root="/dev/hda7"
declare -x runideturbo="false"
declare -x varpartition="/dev/hda9"
bash-2.02# cd /var/insert
bash-2.02# ls
dtv02.ty mfs_insert rcinsert.tcl usage.txt
dtv02h.ty rcinsert.tcl
bash-2.02# rcinsert.tcl /var/tmp/contents 'test' 'test'
bash: rcinsert.tcl: command not found
bash-2.02#

I haven't gotten an error like this since I was a wee newbie; I am still a pretty big newbie so can someone please tell me where I have gone wrong?

kshep
11-11-2002, 11:39 PM
Well it works for me on my first single part test file! (SAT-T60, 2.5)

Only hitch was that rcinsert.tcl calls mfs_insert with the argument "dtvh02.ty", but the file as distributed is called "dtv02h.ty".

If you don't rename the file, you'll get a "inserting a dummy header block in fsid #0" error when it gets to the first mfs_insert call.

I scratched my head over that one for a while until I removed the devnull output redirect from the mfs_insert call and saw that it was causing a file not found error.

AlphaWolf
11-11-2002, 11:47 PM
as a side note though, I am glad to see that interest in this has picked back up.

kshep
11-12-2002, 12:17 AM
Well I really just stumbled into this as I was trying to figure out how I could exchange tystreams with a friend who just got a SAT-T60. I'll admit it's a little tricky following who's working on what, what with all the different threads on a couple different boards.

This seems like the simplest solution around at the moment, and it seems like your further ahead than anybody else, Riley.

I've checked out the alt.org/forum thread, but it doesn't really seem like there's a concerted effort to develop a full blown tool to do this. Am I just missing something?

AlphaWolf
11-12-2002, 12:33 AM
It was being worked on, but some of us had other things get in the way. We intend on building a tivoweb module where you can basicaly do a one step extraction or insertion via your web browser, and it automaticaly copies over all of the MFS objects/attributes (the guide data relavent to the stream) and the stream into one single file so that when you reinsert it, its exactly as it was when you exported it.

On a side note, there are also other tools sorta like this under development, I heard mbm and lightn talking about adding a feature to make it possible to back up all of your configurations/settings/thumbs data/season passes, etc etc etc in one step from tivoweb, so well that you could actualy do a clear and delete everything, then put it all back on, and it would be exactly as you left it. Pretty cool IMO as this will leave the possibility of doing all kinds of other things in the future.

Tivoweb definitely has a lot more work going into it. All of this will be in the official tivoweb distributions when they are completed. Don't associate the tivoweb developers with the AVS guys that easily, they have a lot more interests in doing the more "cool" things with tivos than you may think, in fact had it not been for them, our stream insertion project wouldn't be anywhere near where it is today.

keith721
11-12-2002, 10:02 AM
I had kinda wondered :confused: if all the discussion had moved completely to the IRC channel, since the alt.org/forum message threads hadn't been updated :( in a couple of weeks. It's nice :p to hear you guys are having success, and moving forward. Thanks !! :D

kshep
11-12-2002, 11:52 AM
Thanks for this background info.

I might be able to lend a hand either on the full blown tivoweb module side or, if it's not a significant duplication of effort, on a sort of tivoweb module wrapper to your script, Riley. I'm picturing something that scans the source directory looking for .ty files with names in the TyTools "Get Part" format, then presents the user with a list of files to choose from. It could extract the title and episode info from the file, then iterate though the parts, even splitting the program into multiple Now Showing entries if the program has more than six parts. Perhaps before building this as a tivoweb module it'd make sense to do it as a command line tool, then make both versions available.

I'm new to tivo development, but not to Tcl or web development in general. Perhaps building this simple version of an insert module would be good preparation to help in the full blown effort...if you folks could use a hand, that is.

Torg
11-12-2002, 03:01 PM
It was being worked on, but some of us had other things get in the way. We intend on building a tivoweb
module where you can basicaly do a one step extraction or insertion via your web browser, and it
automaticaly copies over all of the MFS objects/attributes (the guide data relavent to the stream) and the
stream into one single file so that when you reinsert it, its exactly as it was when you exported it.


This has already been done. I am attaching the ui.itcl I use as well as the program it uses. Be advised this is for tivoweb 1.9.3, but it can be easily added or adopted to newer verstions.

The reason it needs to be in ui.itcl is it has to display links with the "Now Showing" parts of the script.

It is essentialy one line added to put in the export link. Three procs to monitor and do the export. And finaly the last line to register it as a part of the menu.

pfl2000
11-13-2002, 02:23 AM
This is great, I backed up this weeks Sopranos (great episode BTW) to my Linux box via NFS, and then restored it back using rcinsert. It worked flawlessly.

I've not tried any of the automation scripts, but would certainly like to see all of the show's information come out with the export and then be restored with the insert.

I've got some free time, if there are any parts to this project that have yet to be tackled let me know and I can give it a try. I know enough TCL to be dangerous and have my cross compiler setup from my mods to elseed and yac.

AlphaWolf
11-13-2002, 02:27 AM
If any of you are good at TCL and/or C, we need more assistance in the project.

Torg: We were working on having a file format thats basicaly a tarball containing each individual tystream part, as well as a plaintext file containing an XML dump of the streams guide data. Just one file that contains all of the info, much easier to deal with that way. See alt.org for details.

drewhallward
11-18-2002, 10:06 PM
I am getting a file not found error on rcinsert.tcl too, I chmod +x it, it is in my path, I am still getting the error however. Do I need to set the source $tcl_library path to anywhere, I noticed on the second line of the script. I have looked for a while on the error and came up empty, any help is appreciated.

drewhallward
11-18-2002, 11:51 PM
Still no go with the ./rcinsert,
here is my ls -al

-rwxrwxrwx 1 0 0 11849 Nov 19 02:01 rcinsert.tcl

I was really liberal with the permissions.
Have a Phillips SA with 3.0 if that helps any.

el jefe
11-19-2002, 02:17 AM
Originally posted by drewhallward
Still no go with the ./rcinsert,


I've had this sort of problem with other scripts before (haven't tried this one), don't really know why, but I can usually get them to work by running it like:

bash-2.02# tivosh rcinsert

el jefe
11-19-2002, 02:34 AM
Originally posted by rc3105
you have to include the .tcl extension

./rcinsert.tcl options...

--
Riley

I've had the problem with files that have the .tcl there.
Actually, now that I think about it I think it was a DOS ^M thing (for me at least). Maybe run dos2unix on it if adding the .tcl doesn't work.

Edit:
Ok, wanted to try it so I transferred over the rcinsert.tcl to my tivo, and the size went from 11,515 bytes to 11,849 bytes (which is the same as yours). This version didn't work. (not found when I tried to run it).
Then I renamed the rcinsert.tcl to rcinsert on the Windows computer and transferred it again, it stayed at 11,515 bytes and it ran fine.
It seems that for some reason Windows is adding ^M's during the transfer. Maybe it has something to do with the fact that I have .tcl files automatically open in a test editor when double-clicked?

drewhallward
11-19-2002, 08:46 AM
It does seem to be the whole ^M thing. I am wondering if me unzipping it in solaris has anything to do with it, if anything I thought it would help, dos2unix'ed it and it runs. Now i just have to do it with the rest of the files I guess. Thanks for the help.

Torg
11-19-2002, 11:31 AM
Sorry guys I should have posted this earlier. What you ran into was the VERY common file ending error. Essentaily DOS (windows) uses CR while Unix uses CRLF. Most of us have gotten this error from time to time when you transfer files between Unix and Windows boxes. As a note, it is not just scripts that get screwed up. It plays havoc with C files as well.

Well how do you fix this.

One way is a program called dos2unix (unix2dos exists too). The program uses the input as a file and will convert it. Type "dos2unix --help" for help with the program.


(as always this is a tarball not a text file. MODS PLEASE FIX THE DAMN FILE TYPES!!!!!!!!!!!!!)

Kythorn
11-19-2002, 06:43 PM
Scenario: Already have DVDs full of tystreams because I ran out of storage.

If I recall, an FSID maxes out at 512 megs, I think.

Since I currently have the full tystreams (didn't use get parts, doh!), what's my recourse?

Can I use tyfilesplit to break the files up into chunks that I can feed back into the tivo? If so, anyone know offhand how many chunks I can get into 512 megs?

That'd be kind of painful I suppose, because then I'd have to copy the whole thing off the DVD first, but if it's the only way to do it, so be it.

Kythorn
11-19-2002, 07:59 PM
Thanks, I'll give the batch file a whirl in a little bit.

el jefe
11-21-2002, 11:32 AM
Originally posted by rc3105
the first line of this script


#!/tvbin/tivosh


causes this script to run via tivosh automatically.



...but "#!/tvbin/tivosh^M" doesn't.
So after a file is screwed up by Windows, this is the line it fails on. But it seems that tivosh can deal with the ^M's at the end of the lines so when you run it explicitly it works.

Torg
11-21-2002, 01:49 PM
This is not a Tivo problem, it is a file type problem. ALL Unix varients have this problem, so do the Macs. Gee if everything by windws has a problem with it shoudlnt it be a Windows problem.

Yes Windows mangles text files. Read Microsft can not follow any conventions. Wether its a text file or crypto.

If you look above I have already attached a fprogam to convert these.

el jefe
11-21-2002, 02:33 PM
Originally posted by Torg
This is not a Tivo problem, it is a file type problem.

I understand what the problem is and how to fix it, I was just explaining why the #!/tvlib/tivosh wasn't working in this case.

But an actual question is when I ran this to check, I inserted 3 entries in Now Showing that are blank (I didn't have any extracted video or anything, so I just wanted to see if it would run). Now I can't delete them, is there a way?

kshep
11-21-2002, 02:35 PM
Just to throw another tool into the mix here if it's not been mentioned before...Anybody using named pipes (aka "fifo"s) to route files between mfs_export, mfs_insert, nc, etc?

If you type:

% mkfifo testpipe

You'll get a "file" named testpipe that sort of queues up whatever you write to it for another file to read. For example, you can run:

% ls > testpipe &

to start an ls process in the background, then

% cat testpipe

will print the output of the ls command, passed through the pipe.

I'm inclined to use take this approach to my tcl scripts that pull/push tystreams to/from the tivo so I don't have to set aside space for .ty files.

TiVolkan
11-23-2002, 06:27 PM
I got insertion working on a Dtivo 2.5.2! For a newbie like me, thats quite an accomplishment, but I have a few questions for the gods:

1. My first insertion was a 2 FSID segment (not a complete show - At recording time, I pressed record midshow, then stopped recording mid-show) of these filesizes:

11/22/2002 08:52p 159,383,552 Name-1.ty
11/22/2002 08:28p 268,435,456 Name-.ty

but the Tivo seems to get confused about the lenght of the show.
I compared it to the original clip:

Original: 14 minutes of a 3 hour block program on MTV.
Says 14 minutes in the Show Description screen. According to the Tivo green status graph at bottom during playback, the clip started at 2:24 and ended at 2:38.

Inserted Clip: Says 30 minutes in the description screen, but the green status graph reads correctly.

If I let the inserted clip go all the way to the end, the Tivo will reboot.

Not sure if you can do anything about this now, but thought you may find the bug report useful for future revs.

As this was only my first one I'll try a more 'Normal' clip now - 1/2 hour or 1 hour.

2. Are all the FSID's somehow married together? In other words, If I only extract one part of a multi-part clip, can I reinsert the one part back in as a complete program? Or as another example, If I extract all parts, can I just reinsert one or two part without messing up the Tivo OS?

Thanks again for your hard work!

TiVolkan
11-24-2002, 04:00 AM
Now how do I dice off my header chunk? I'm assuming the chunks are fixed-length? Is there a Windows utility? And how do you guage what chunk a commercial is at?

....So many questions, so little time.....

TiVolkan
11-24-2002, 03:23 PM
Thanks for the info - I 'll experiment with the chunks.

On another note, Some of the errors I'm getting with rcinsert.tcl are consistent with not enough free disk space, or some sort of timing issue, when allocating space for a part insertion. I'm trying to really put the script through the ringer (working in QA for a large enterprise software company) , and trying to insert at the same time while recording through the Tivo UI. I've seen error occurs when allocatiing for part 0, and then on another try with the same clip, when allocating for part 1, then the script aborts. I've saved the errors and call-stack if interested, but at the risk of making the post too long, I won't post now.

I'm leaning toward more of a free-space issue, and assuming that the script does not call the cleanup routines to clear programs sitting on the bubble if space is needed, like the recording function on the Tivo, is this correct?

After clearing some programs manually, The insertion proceeds without error.

TiVolkan
11-24-2002, 04:49 PM
Just when I thought I had the behavoir pegged, It's doing things I dont understand when inserting. Is there an Error code lookup for FsStream object? as in FsStream::Create failed (0x0001102f). I usually see the 1102f code (when allocating part 0 or part 1 on different tries), but now I'm seeing 11006 as well (after a cold-reboot). I have cleared ample free-space, and now I cannot insert anything - Here's the call-stack if you have a minute:


bash-2.02# ./rcinsert.tcl /nfs 'name' 'NameDesc'
check to see if part 0 exists
# allocate space & create part 0
check to see if part 1 exists
# allocate space & create part 1
FsStream::Create failed (0x0001102f)
while executing
"mfs streamfill 131072 [expr $streamsize / 131072]"
("uplevel" body line 38)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
#puts " # create the recording"
set recording [db $db create Recording]
dbobj $recording set BitRate 0
dbobj $..."
(file "./rcinsert.tcl" line 73)
bash-2.02#

OvitivO
11-29-2002, 12:40 AM
rc3105: very nice.

Is there a console script out there that can simply list the now showing fsids (as opposed to the video blocks)?

(i.e., I don't use TivoWeb, and I'm looking for methods to do all of my extraction/insertion via shell.)

OvitivO
11-29-2002, 12:51 AM
Stop looking over my shoulder when typing. ;)

NowShowing.tcl is what I originally used, but I had an error with using what I thought was the proper fsid.

I'll generate the error again and post.

Well, it wasn't exactly an "error," but more of an oddity that will likely not work for re-insertion.

Everytime I use the script I *always* get px.ty file sizes of 1310720 bytes.

# /tvbin/tivosh rcxtract-with-info.tcl 1054287
madedir /var/bols/{Edited Title}/{Edited Episode}
created /var/bols/{Edited Title}/{Edited Episode}/nsinfo.dbt
{Edited Title}
{Edited Episode}

3600
12011
12012
AutoSyncNo
original air date 11-20-2002
1060329
536870912
1060331
536870912
1060333
343932928

# ls -al /var/bols/{Edited Title}/{Edited Episode}
total 3864
drwxr-xr-x 2 65534 65534 4096 Nov 29 05:12 .
drwxr-xr-x 3 65534 65534 4096 Nov 29 05:12 ..
-rw-r--r-- 1 65534 65534 160 Nov 29 05:13 nsinfo.dbt
-rw-r--r-- 1 65534 65534 1310720 Nov 29 05:12 p1.ty
-rw-r--r-- 1 65534 65534 1310720 Nov 29 05:12 p2.ty
-rw-r--r-- 1 65534 65534 1310720 Nov 29 05:13 p3.ty

Unless you've reversed entropy, I don't see how those 3 tystreams are going to provide the data necessary to re-insert a recording back into a Tivo. :)

compwiz312
11-30-2002, 08:47 PM
Hi, I have just been reading all of this and I was wondering, is there any way for me to convert MPEG2 files (such as DVD or SVCD encodes) to Tivo streams so that I could import them successfully to my Tivo?

Thanks for your help,
Justin

AlphaWolf
12-01-2002, 04:30 PM
I think we just need to finnish the tivoweb interface

EDIT:

Speaking of which, have you used tivodvlprs mfs_tarexport yet? It already extracts shows perfectly in our tar format that we are going to use. May want to start working on code that will take that tarfile and rebuild its contents (xml file typarts and all) back into MFS instead.

OvitivO
12-02-2002, 06:34 AM
An extraction with this script will take roughly 2 hours for every 512MB segment, however a full insertion will take roughly 15 minutes per 512MB segment.

What could possibly be causing this discrepancy?

OvitivO
12-04-2002, 04:06 PM
Note the timestamps:

-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 02:59 p1.ty
-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 04:55 p2.ty
-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 06:48 p3.ty
-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 08:41 p4.ty
-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 10:34 p5.ty
-rw-r--r-- 1 nfsnobody nfsnobody 512M Dec 4 12:27 p6.ty

OvitivO
12-04-2002, 04:57 PM
Beefy Linux Box <-> Tivo (i.e., no XP)

I'll check with some additional mounts to see if I have the same problem with writing, but that would seem odd if that were the case. (btw, I remembered that mfs_stream by itself does not have this problem.)

One other issue I have (which may be related) is that during an extraction, if I have the Tivo turned "off," then I am not able to power the unit back up until the extraction is finished.

In addition, when I telnet to the Tivo (during an extraction) and attempt to run something like tivoftp, I get a fork error.



# /var/hack/tivoftpd
tivoftpd: error forking; Resource temporarily unavailable


The only other thing I've noticed is that if I kill the extraction before it is finished (e.g., ^C), then the Tivo will reboot. :)

keith721
12-04-2002, 06:49 PM
mine is mounted using 8K buffers, and seems to perform reasonably well (Sony SVR-2000 Standalone TiVo)

bash-2.02# mount -t nfs -o rsize=8192,wsize=8192 192.168.1.10:/g/ /mnt/nfs/g

OvitivO
12-04-2002, 07:40 PM
Good point, my buffer sizes were set to 4096. I'll test again w/ 8192.

EDIT: 8192 definitely improved the performance

AlphaWolf
12-10-2002, 12:33 AM
Just want to pass word around that we are open for suggestions as to what we should name the extension for our tarballed tystream format.

In case anybody doesn't know how it works yet, basicaly its a unix .tar (Tape ARchive) file that contains each tystream part as an individual file, along with an XML'ed dump of all of the information in your tivo thats relavent to that specific recording that you have extracted. This was origionaly designed with re-insertion in mind, but it also has other uses. Namely, it could help automate the title information when you move the tystream to another medium - such as DVDs. This is a very clean, neat format. Much so, that anybody who wants to tinker with it can use an everyday application such as winrar, winzip, or a number of mac/unix applications, to alter its contents. Not only that, but every time you extract a tystream, you get all of this in just one neat little file, no directories or a mess of specialy named files that make a more complicated way of accomplishing the same thing, which makes dealing with tystreams a lot easier for the user. And yes, for once you can have peace of mind that files extracted into this format are perfect and can be used for anything we can forseeably do with tystreams later.

Naturaly, I also (passively :D) suggest that anybody who writes any software that processes tystreams in any way support our new format once it becomes finalized (probably already is finalized actualy, it is a pretty simple format when you think about it, just needs a name). The full source codes to everything we have done so far can be found on alt.org.

FreydNot
12-10-2002, 02:42 AM
How about .TarVo ? It kind of rolls off the toungue :)

olaf_sc
12-10-2002, 03:17 AM
Do you know if there is a lib that has functions to parse a tar file? I would be pretty nice if we could read the tystreams directly in the tar files. It would aviod the unpacking and I could e.g. demux directly from the tar file.

Cheers Olaf

PS: Yes it's open etc but I rather not code the parser if I can avoid it :)

jkeegan
12-10-2002, 10:39 AM
Hmm..

.tya (TYstream Archive)
.tyb (TYstream Bundle)

(I think I like tya better)

..Jeff

kshep
12-10-2002, 01:06 PM
I'll nominate:

.tva

...which you could read as TiVo Video Archive or just TiVo Archive.

I can imagine a "tva" file would have a required XML data file up front that includes not only program information but could also specify the format(s) of video segments to follow. These could be tystream chunks, "joined" tystream files ala TyTool, or mpeg files. In fact, a TiVo Archive (tva) file could contain multiple programs...in mixed formats, even...so the format could be used to archive not just a single program but an entire TiVo.

You might even generalize it a bit to something like ".var" (ala ".tar" and ".jar") and submit it as a W3C standard for video archives.

BTW, I'd be happy to start using this format in code I write...if someone would publish a draft DTD.

AlphaWolf
12-10-2002, 01:10 PM
Well, bear in mind its also not realy intended as an archive. The intent is that one file holds all of the information relavent to one recording. We could have just kinda globbed it all together in a single file, but felt a tarball would be a nice way to separate each peice of the recording.

You can see what we have so far by downloading tivodvlprs tarexport server from alt.org, it will extract a tystream in our format through tivoweb.

AlphaWolf
12-10-2002, 01:15 PM
Originally posted by olaf_sc
Do you know if there is a lib that has functions to parse a tar file? I would be pretty nice if we could read the tystreams directly in the tar files. It would aviod the unpacking and I could e.g. demux directly from the tar file.

Cheers Olaf

PS: Yes it's open etc but I rather not code the parser if I can avoid it :)

Understood completely :) I don't know myself, might ask mbm, or look on gnu.org. I anticipate that your 0.5.0 release could use this as part of the DVD features though.

kshep
12-10-2002, 01:26 PM
Originally posted by AlphaWolf
Well, bear in mind its also not realy intended as an archive. The intent is that one file holds all of the information relavent to one recording...

Gotcha. But even if it's just one program I still think you could (and many folks will) call it an archive, particularly when one common use will be for people to archive shows off their TiVos. I honestly don't mean to argue symantics, I just think you've come up with a format that could be pretty conventient for a number of extraction/insertion/archiving tasks and you might want to position it that way.

Besides, it's a tar file. I suspect many people will be using tar to create and extract from the file, so it might be a good idea to carry along that "heritage" in the naming and choice of extension (ala .jar)

AlphaWolf
12-10-2002, 05:03 PM
For any dev'ers looking at this, I might add that there are also some not so obvious specifics to this format, the most important being that the XML file must come first in the tarball, then each tystream part comes one after the other in their origional order as stored on the tivo. The reason the file order is important is for re-inserting on the fly without the use of temporary files.

sting0r
12-12-2002, 10:01 AM
Hi --

Anyone willing to help a newbie out. I see there is so many version of scripts etc... Anyone have a step by step or would be willing to point me in the right direction to get insertion working? I am interested in inserting media I have on my PC for watching on the Tivo, instead of burning to a dvd and watching on a dvd player. Any help would be appreciated..

Sting0r

AlphaWolf
12-12-2002, 12:39 PM
Originally posted by kshep
Just to clarify...

By "multi-part downloading" are you talking about taking chunks from different tivos and combining them into a single program? I assume you're not suggesting a tarball created by combining the results of a "Get Parts" extration using TyTool couldn't be downloaded/inserted.

They *possibly* can, all of the streams I have exported with tytool seem to have a mangled master chunk, I thus haven't tried reinserting them because its my expectation that they wont work. Since I haven't tried it, I can't say. If they do work however, that just means there are a lot of extra steps involved. Anything you have extracted using the classic mfs_export should work fine, just need to separate the tystream parts is all.

SR712
12-12-2002, 01:14 PM
As far as a name goes.... how about... TyBall ? :D

AlphaWolf
12-14-2002, 03:59 PM
Well, if you have a lot of time, you could write software that runs on the tivo which on command would basicaly duplicate a tystream, only the new copy has no commercials.

OvitivO
12-17-2002, 03:31 AM
I have noticed that rc-insert is relatively "polite" when it comes to resources on the Tivo, and I have yet to notice any major problems with using the Tivo during an insertion, but rc-xtract is quite the opposite. rc-xtract will basically use up enough resources to effectively stop any additional interaction with the Tivo during the extraction process.

Would anyone have any suggestions or is this an inherent side effect of pulling the streams out of the Tivo?

(I'm using v1.10a of the insert script and the v.1.0 of the extract script. 1.10a adds a few features that help out with my set up.)

mpauley
01-18-2003, 04:37 PM
RC3105,

Do you know why I would have an issue downloading a file over 3.99GB? The file I am currently trying a show that is 4.6GB and it just hangs @ 3.99GB. I have Windows XP Build 5600 extracting to a 128G drive formated to NTFS.

Thanks for any insight....

--mp

compwiz312
01-24-2003, 01:04 AM
If you are running FAT32, there is a 4GB limit. Switch to NTFS, that should solve your problems. If not, don't shoot the messenger.

AlphaWolf
02-02-2003, 09:39 PM
Finalizing the standard...

http://alt.org/forum/index.php?t=msg&th=18&rid=23&S=c0ed5148508db135667a68e91857e4a4&pl_view=&start=140#msg_num_10

wheelbarrow
02-06-2003, 10:45 AM
Hi Guys,

I've been reading though the thread and have a simple question. My upgraded uk Tivo is now playing up and i need to get some programs from it.

I don't need to watch the then now, and can wait for code to be relased before trying to put them back on to the machine when i have put a new disk in.

All i need to know is which tool and what version i should use to get the shows out in such a format that i will be able to put them back at a later date.


keep up all the great work.

Thanks

Paul