Mfs_Ftp: extract, archive, restore & transfer recordings

[rc3105]

Originally posted by bjarne
Now for my question: In the past I have tried with success to insert tmf files, and also transfer directly from one of my DTiVos to the other, not that I needed to, but it was fun !!!

When I did the insertion, the show would show up in Now Playing after transferring a little bit, and I could start watching almost immediately. When I do the insertion with the new ty format, it does not show up untill the whole file has been transferred. Is this a limitation of the new format, where the xml data is at the end? Bjarne

if you watch the nowshowing list as you insert the recording will appear with the name of the file actually being uploaded, then when it reaches the xml at the end it will change to the correct info. you can still start watching (streamview) as before

[tards]

OK, I think I got this installed and I can run it. I also got smart_ftp and can ftp to the tivo. Now what? Can somebody post some "newbie" or "for dummies" instructions on USING this program?

Example questions:
1) where are the program files (tmfs?) located on the Tivo (I probably should know this but I don't).
2) What's all this command line ftp stuff? Can't I just drag/drop?
3) how to download and re-upload using smart-ftp...in layman's terms.

I've been able to install and run every DTivo app I've tried without asking any questions. But this one gives me some problems. And how do I stop the program?

[SirTrini]

In MFS_FTP if you were to use an FTP prog, then all you would do is "click" on the Shutdown folder and that woudl stop MFS_FTP.

In DOS, I would assume if you were to change dir's to "Shutdown" then that would stop the program.

I think in the simpliest of terms, the shutdown folder is "like" to a function that excutes the shutdown.

You can drag and drop, I would suggest download SmartFTP and use that as it seems to work the best.

The TMF's are an "older" format that I think RC3105 will stop using. TY's now include the show info so that is all that is needed. Once again:

1. Launch SmartFTP
2. FTP into your Tivo on the port that you setup MFS_FTP on (newer releases has this as 3105 as a default) - anonymous logon is fine
3. you'll get a list of folders double click on the TY folder
4. you'll see a list of shows drag it to your HD and it will start to download
5. to shutdown MFS_FTP double click on the folder "shutdown" - I don't do this I just disconnect from the Tivo. I want to be able to reconnect without having to restart the program.

I hope this helps!

[tards]

Whoah, it works! I was not using port 3105 in my ftp. Now it works great!! Thanks...this is a NICE app!!

[johnmacd]

OK since I am a newbie this may sound like a dumb question.

The TMF's are an "older" format that I think RC3105 will stop using. TY's now include the show info so that is all that is needed.

Wha is the difference between TY and TMF? I think there were a few other folders that I saw as well (I'm not where I can access my Tivo right to to look), what are those formats? I was using the TMF's and it worked great.

John

[SirTrini]

TMF's work great for insertation back into the Tivo, but other conversion apps: TYStudio/TYTool etc... don't use this format yet (not sure if they will either).

To get a TMF into a TY you would decompress it with a program that support RAR, join the pieces back together and discard the XML information. RC3105 did develop a program to do that in one step and it can be found on the main page to this thread.

RC3105 updated the original TY format to include the TMF info, so now TY's can be inserted back into the Tivo, and run through regular conversion programs. IMHO it's the best of both worlds and allows the most flexability.

TYX is a developing format - I don't know much about it, the other folders TXT is a text file of the show info, XML is an XML file of the show info, shuthdown is to stop the server process and I think phoenix, is shutdown and restart (not 100% sure).

[rc3105]

Originally posted by SirTrini
To get a TMF into a TY you would decompress it with a program that support RAR, join the pieces back together and discard the XML information. RC3105 did develop a program to do that in one step and it can be found on the main page to this thread.

don't discard the xml!!! append it to the end of the tyfile so it can be restored!!!

tmf is a tarball with the the recording parts encapsulated. building tmf's from ty or ty+ is easy enough, but why not just insert a ty+? (easier to build with cat)

the real advantage tmf has over ty+ is that the xml is at the beginning of the file instead of the end. this allows the info to be restored before the insert starts. I was hesitant to release ty insert at all since there are about a zillion corrupted tytool extracts out there & I REALLY didn't want to waste any more time on support questions because of someone else's error.

considering cwingert's view on parsing tmf's in mplayer,

Originally posted by cwingert
...Parsing this in the context of chunks (at least for my application) is somewhat of a nightmare.

that anybody who hasn't heard me mention some tytool extracts are corrupt by now is brain dead, and that I really have no desire to crawl in & add tmf support to mythtivo, I figured it was time time to release ty insert.

once tivoweb adds insert capability it would be trivial to add ty support, but I can certanly understand not wanting to open that can of worms

[osetivo]

NM:

EDIT: Being a happy camper I haven't been following MFS_FTP closely. Figured I'd better catch up and see what's going on.

Originally posted by rc3105
recordings can now be extracted, archived, restored & watched as ty+ w/o any drawbacks so go ahead & convert your tmf library to ty+ (I am)[/B]

OK, Riley, I know you're not converting 1 file at a time. Do your progs do Batch processing? Your TyDeTar that I use doesn't. (I could just be stupid)

I tried tmf2ty once but IIRC I had 2 episodes with the same name, Seinfeld: The Trip, without any Part 1 or Part 2) and it overwrote the first so I stuck with tydetar which throws in the FSID. Has tmf2ty take over from a developement standpoint?

I've got 200GB+ of TMFs. "IF" you get the time, how about a bone?

TIA

[johnmacd]

To get a TMF into a TY you would decompress it with a program that support RAR, join the pieces back together

So lets see if I understand this correctly. Since you can get a TY out of a TMF, one could simply store the TMF on your PC and then later get a TY out of it for use in other programs (like TyTools). There's no need to duplicate TY's and TMF's that are stored on a PC.

its more than likely that when tivoweb adds extraction/reinsertion features, TMF is the only file format it will support.

When this module comes out its gonna kick butt.


John

[rc3105]

osetivo:

tydetar was just for extracting the ty prior to conversion to mpeg. tydetar doesn't append the xml info to the end so those ty's aren't really going to work for insertion.

the dos command

FOR %a IN (*.tmf) DO tmf2ty_cfg.exe %a

will sequentially launch tmf2ty_cfg.exe for every tmf in the current directory.

if you want to call it from a batch file use

FOR %%a IN (*.tmf) DO tmf2ty_cfg.exe %%a

Originally posted by johnmacd
...There's no need to duplicate TY's and TMF's that are stored on a PC...

right, BUT, tmf2ty tyfiles are equivilant to tmf's in that they contain all the same info. they can be converted back to tmf. once you convert the tmf to ty+ you can just delete the tmf.

the point is that mythtivo / mplayer / tystudio / tytool can't read tmf's, so if you want to be able to watch / convert a file allready on the pc w/o the nuisance of an extra conversion step just store it as a ty+

load the notebook hd up with ty+ to watch on the plane. give the kids a dvd of ty+ to watch on the xbox... whatever

[deebo]

I am able to extract/import .tmf files succesfully, However I have read that .tmf's aren't the best way to archive shows anymore. Should you instead archive the shows in the ty folder, or the tyx folder? Are ty files the same as ty+ files?
-David

[rc3105]

for PURE archiving, p2p transfer or streamview tmf is the best format. (easier to repair a corrupted file or access the xml) if you never ever use mplayer or convert an extracted recording to mpg stick with tmf.

mplayer, tystudio & tytool can't read tmf files. if you use those utilities then extract files as ty

ty from mfs_ftp is really ty+. the difference between regular mfsstream ty & ty+ is that in ty+ the xml data is appended to the end. (the xml stores the actor list, original air date, all the extended info)

ty+ contains all the same info as a tmf but you don't have to convert the file before you can use mplayer / tystudio / tytool / ty2vob or whatever

use whichever format best suits what you'll be doing

[rc3105]

the fifo in /var/mfs_ftp is only used for resume on extracts

the cache directory contains 2 types of files, xml & pts. a .pts file is just a text file listing each fsid in the recording. the xml is the xml text that's included in a ty or tmf.

there's a few reasons for the cache dir:

1) by caching the xml data to an ext2 partition /dev/hda9 (var) the number of mfs db accesses is drastically reduced and things run smoother all around

2) when a recording is inserted not all the xml data is restored to the db (yet) the entire xml from an insert is saved to the cache dir, then on extract it's retrieved from there. this allows you to insert a tmf & extract an identical tmf (maintaining the actor list 'n so forth) if you put a 2.5 ty/tmf into a 3.1 box it plays, then when you extract it again you get your original 2.5 ty/tmf back

3) with the xml & pts info on an easily accessable partition you can automate the process of extracting recordings by connecting the hd to a pc. this is usefull for retrieving recordings from a drive that's gone GSOD or doing REALLY fast drive to drive extracts. see this thread for a howto

[larray]

Originally posted by wkozun
Am I doing something wrong? Is this how things work? Is there any way to change it so that I can see progress.

Nope, your doing it right.

The reason you don't get status is because the Tivo is actually sending it, not SmartFTP. Once the process is initiated, everything moves between the Tivos and not through SmartFTP. (I believe)

[blahman]

Originally posted by Stephen
Ok I will admit I did not read all 51 pages of this thread but DID read the read me's and instructions.. I did get the program running properly a quick word to those of you that are running regular FTP CHANGE THE FTP PORT!!!! or else it will look like a regular ftp client

General Question I didnt see in the readme.. Which files Do I need to download? I see the files listed in ty ty+ xml txt tmf ect.... If I want to import them back into the tivo Im guessing ty and xml?

To re-import later, either the TMF or the TY+ will work. Both have the data that is stored in the XML embedded within them.

TY+ is compatible with the TY file editting tools as it appendeds the XML data to the end of the file so as not to interfere with the editors.