Page 1 of 99 1231151 ... LastLast
Results 1 to 15 of 1476

Thread: Mfs_Ftp: extract, archive, restore & transfer recordings

  1. #1
    Join Date
    Mar 2002
    Posts
    1,339

    Post mfs_ftp: easy extract / insert via ftp protocols

    here's a little tivo side ftp server for moving recordings from tivo -> pc, pc -> tivo or tivo <-> tivo with a regular browser or ftp client. video can be moved between different versions of tivo hardware / software and streamviewed on a recieving tivo

    mfs_ftp.tar.zip - current version of the complete program dir including s1 support binaries
    (attached to this post)

    installation:

    shutdown any previous version of mfs_ftp running in the tivo

    unzip the mfs_ftp.tar.zip to mfs_ftp.tar on your pc

    open mfs_ftp.tar & extract readme.txt - READ IT!

    transfer mfs_ftp.tar to /var in the tivo

    backup the old mfs_ftp dir: "mv /var/mfs_ftp /var/mfs_ftp.bak"

    from the "/var " prompt unpack with

    "tar -xvf mfs_ftp.tar" or "cpio -i -H tar -d -F mfs_ftp.tar"

    start from bash "/var/mfs_ftp/mfs_ftp.tcl" or load from a startup script


    mfs_ftp 1.2.9 MAY NOT BE REDISTRIBUTED. PERIOD! (read the license and terms of use)

    sadly many are offering mfs_ftp installed on units for sale, since the legality of the license is unlikely to deter the unscrupulous here's the deal. if you want to include mfs_ftp on a tivo your selling (presumably as an inducement to buy) send your contact info with a $40 paypal to rc3105@hotmail.com I'll provide a transferrable license tied to the tivo itself & permission to install / distribute mfs_ftp on that particular tivo. this provides a legit means for techs to install sw for the clueless & specified damages that can be recovered (in court, etc) from those distributing in violation of copyright

    my paypal is rc3105@hotmail.com and donations are allways appreciated. mfs_ftp is FREE for personal non-commercial use if you install it yourself, so enjoy


    Usefull things to have / know about

    THIS THREAD IS THE FAQ! (and moderated to stay that way)

    please read readme.txt & this ENTIRE thread BEFORE posting a mfs_ftp related question - off topic posts, obsolete info & chitchat more than a month or so old gets nuked

    please ask any questions re: installation / usage in this support thread instead of cluttering the forum starting new ones


    ALL required S1 support files are included in the zip

    here's links to usefull alternate threads

    MFS_FTP on Series 2 Units contains S2 support binaries - substitute s2 binaries after installation but before starting mfs_ftp

    Mfs_Ftp: suggestions, feedback, etc

    mfs ftp issues in the newbie forum

    tmf / ty+ tivoweb extraction module: extract tmf or ty+ via tivoweb

    tmf2ty_cfg.exe is a windows binary that extracts ty from tmf. these ty are insertable with mfs_ftp & compatable with all ty utilities. if savepath.txt is in the directory with tmf2ty_cfg.exe (or in the dir with the tmf under some versions of XP) the first line of the file is used as the output path. these are tcl paths so use forward slashes - "x:/extracts/" "d:/my_recordings/" "v:/video_files/" etc

    Commonly asked questions (and answers)
    Last edited by rc3105; 04-24-2004 at 03:37 PM.
    ---
    Give a man a fish and he will eat for a day. Teach a man to fish and he will sit in a boat all day and drink beer

  2. #2
    Join Date
    Jan 2003
    Posts
    1
    Nice one Guys, this is the best TVIO app i have got. Can you please pass thanks onto all the clever people who made this possible.

    Ok, I'm having a "Blonde" moment, I;ve copied the 3 files to /mfs-ftp, made them exce with chmod. but when i do ./mfs_ftp.tcl i get "No Such File or Directory" ?

    I have missed somthing here, i can run the app by doing /tvbin/tivosh msf_ftp.tcl, but it stop when i try to download as it cant run the other file.

    Please call me stupid as i'm sure it a very silly mistake that i have made.

    Regards

    Paul

  3. #3
    Join Date
    Mar 2002
    Posts
    1,339
    set your ftp client for binary mode whenever transferring via tivoftpd.

    a tmf is just a tarfile. winrar & some other utils can only see files in the first 2 gig of a tar. winzip works fine

    many ftp clients will only download the first 2 gig even if you have fat32 (4gig limit) or hpfs, ntfs, ext2/3 (no limit) please make sure your ftp client isn't broken
    Last edited by rc3105; 10-04-2003 at 12:15 AM.

  4. #4
    Join Date
    Feb 2002
    Location
    Natick, MA
    Posts
    34
    So after doing some testing with the tool, results were as I expected. I can ftp a show off my standalone and ftp it to my DirecTivo and it works beautifully (although for a half hour show, the Tivo thinks it's an hour, however the green in the progress bar stops at 30 mins like it should). Also as I expected I cannot ftp stuff off of my DirecTivo and get it to play on my standalone. I had suspected this would be the case from earlier work I was doing with insertion. Overall, things look great.

    What I'd love to be able to have is a tool that I could give an AVI, MPEG, etc along with some details about it, and could then covert that to the TMF format for insertion onto the Tivo. i.e. Download a movie trailer from the net then insert it on the Tivo.

    Keep up the great work...

    Derek aka ][ronMan aka DSmooth

  5. #5
    Join Date
    Mar 2002
    Posts
    1,339

    TMF specs

    ok, couple of people have asked, so here's the info.

    tmf stands for tivo media format. the NAME was AlphaWolf's idea :-p the original code to generate it was done by tivodvlpr & mbm.

    edit: sorry, how did I forget to mention dtype? duh@me...

    a .tmf file is a regular tar file that contains the following, in this order.

    showing.xml
    part00.ty
    part01.ty
    ...
    part04.ty


    showing.xm is a xml file with the info from the nowshowing database stored as regular text. title, episode, description, everything. the .ty parts each represend an individual fsid as it's stored in the tivo.

    pretty simple really. the xml is a bit hairy, but easy enough to parse out the title / description etc and it contains all the info the tivo stores about a recording.

    --
    Riley
    Last edited by rc3105; 02-18-2003 at 10:56 PM.

  6. #6
    Join Date
    Feb 2002
    Location
    Natick, MA
    Posts
    34

    Thumbs up

    Originally posted by ][ronMan
    So after doing some testing with the tool, results were as I expected. I can ftp a show off my standalone and ftp it to my DirecTivo and it works beautifully (although for a half hour show, the Tivo thinks it's an hour, however the green in the progress bar stops at 30 mins like it should). Also as I expected I cannot ftp stuff off of my DirecTivo and get it to play on my standalone.
    Correction... With the 1.1.1 version I am also able to ftp a show off of my DirecTivo and ftp it onto my series 1 standalone. This is really starting to rock!

    Derek aka ][ronMan aka DSmooth

  7. #7
    Join Date
    Feb 2002
    Location
    Natick, MA
    Posts
    34
    Originally posted by AlphaWolf
    DT wouldn't work on an SA because of the fact that its very non-standard. Again, doing this would involve being able to generate tystream archives.

    Also another comment on the binary optimizations: It would be better to do it anyways, b/c you could further minimize the chance of pausing or any other errors.
    See my last post... This is working OK with 1.1.1 and greater...

    Derek aka ][ronMan aka DSmooth

  8. #8
    Join Date
    Feb 2002
    Posts
    285
    Originally posted by rc3105
    PASV is most likely the problem it's still a bit iffy. there it was trying to connect to itself. use regular mode instead.
    Sure enough, as you were posting this I had just tried turning off passive mode, and that fixed the problem. My guess is that's what the majority of the client-specific problems are (I can't be sure of course, but it seems likely).

    Would it be possible to add a short "how-to" to the readme.txt file? IMO, it should include the following:
    [list=1][*]command-line server configuration[*]turn passive mode OFF[*]filenames (i.e., command-line clients need only use the FSID listed as the filename to get)[*]importing vs. exporting[*]deleting programs (and how to prevent it (?) )[*]other[/list=1]
    Might save a lot of repetitive questions. Btw, fantastic work!!! This blows away my old export-via-nfs method! (For some reason nfs is really slow on my Tivo, even with rsize=8192&wsize=8192.)

    - Stealth Dave

  9. #9
    Join Date
    Feb 2002
    Location
    Natick, MA
    Posts
    34

    Re: Advice... things not working

    Originally posted by alanjudy
    The other question is, while I have seen that someone has gone from SA to DTivo, has any sucessfully gone from DTivo (2.5.2) to SA (3.0.1)

    Thanks

    =alanjudy
    alanjudy,
    This is my exact setup, and as you see from my post above, with versions 1.1.1 and beyond it works in bot directions...

    Derek aka ][ronMan aka DSmooth

  10. #10
    Join Date
    May 2002
    Location
    West Hartford CT
    Posts
    620

    SA2dtivo - dtivo2SA - working!

    Well; I'm xfering a .tmf originally from my dtivo to my SA. It's the same one that created my 0Byte file, undeletable listing. I found out that my SA is MUCH slower to respond and transfer to. Both tivos are 3.1 - I just didn't WAIT long enough for it to allocate space .. It is a 2hour movie, so it appears it;s going to take some time. I'll report back on results.

    here is a dump of the transfer sofar...

    echo to verify: PORT 192,168,1,49,7,156
    ip is 192.168.1.49 port is 1948
    200 PORT command successful.
    echo to verify: STOR Hackers .tmf
    150 Opening BINARY mode data connection for "Hackers"
    entering dumpxml
    segment is showing.xml
    leaving dumpxml
    entering parsexml
    title is "Hackers" - episode "unknown episode"
    leaving parsexml
    attempting to create recording - success
    attempting to set recordingfsid - success
    attempting to set bitrate & such - success
    attempting to create a dummy station - success
    attempting to create a showing - success - trying to set showing info - success
    trying to create the program & set the info - success
    attempting to showing set program - success
    created Hackers : unknown episode : fsid# 2842851
    entering tmf2fsid
    starting segment part00.ty
    entering AddPart
    trying to open recordingfsid
    success
    createing recording ? success
    trying to add part - success
    trying to mfs streamfill allocate space
    success - created newfsid 2842854
    trying to set file to fsidtwo - success
    leaving AddPart
    ~Hi8

    (4) Hughes SD DVR40
    (1) Hughes HR10-250
    (2) xbmc XBOX X2 & xbit

  11. #11
    Join Date
    Mar 2002
    Posts
    1,339
    the initial space allocation can take bit, especially if the db is busy & it has to retry a few times.

    I'm working on that issue

    set your ftp client timeout to at least a minute, 2 ought to be worst case.

    --
    Riley

  12. #12
    Join Date
    May 2002
    Location
    West Hartford CT
    Posts
    620

    Re: SA2dtivo - dtivo2SA - working!

    Originally posted by Hi8
    I'll report back on results.

    It works perfectly! Plays and looks as good if not BETTER than anything ever recorded on my SA, well, there you have it!

    the down side is xfer speed. it took about 5hours to xfer a 1h45m movie to my SA. What is surprising is how fast it downloaded from my dtivo, less than 25mins.

    I noticed the time difference the listing info reported a 1hr recording yet the green progbar reported correctly at 1:45.

    Great Job RC!

    here is the ftp sessionlog info:

    sending Hackers .tmf as Hackers .tmf (1 of 1)
    PORT 192,168,1,49,7,156
    200 PORT command successful.
    STOR Hackers .tmf
    150 Opening BINARY mode data connection for "Hackers"
    Transmitted 1920999936 bytes in 17739.4 secs, (1.04 Mbps), transfer succeeded
    226 File transfer complete
    PWD
    257 "/" is current directory.
    TYPE A
    200 Type set to A.
    PORT 192,168,1,49,8,109
    200 PORT command successful.
    LIST
    150 Opening data connection for file list.
    Received 1068 bytes in 0.1 secs, (100.00 Kbps), transfer succeeded
    226 Transfer complete.
    QUIT
    shutting down & exiting


    **** please note the last entry ****

    on my SA it shuts the server down at the end of a session, on my dtivo it just returns a 221 call 'server closing control connection' the former is abit bit of a disadavantage as I have to go in and manually restart it for each session. On my dtivo it starts on bootup, and runs along side my tivoftpd only on port 24.

    using the same client WS_FTP Pro, and both my tivos are running 3.1, and mfs_ftp 1.1.2
    Last edited by Hi8; 02-20-2003 at 02:07 AM.
    ~Hi8

    (4) Hughes SD DVR40
    (1) Hughes HR10-250
    (2) xbmc XBOX X2 & xbit

  13. #13
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,823
    ok heres an explanation of the file formats

    .ty is a plain binary dump of the individual tystream parts combined into one file. This is the legacy method of extracting tystreams, and its my personal recommendation that nobody dumps/extracts/archives to this format anymore.

    .tmf is a tar archive of the above, only a few additions. Each part of the tystream is stored as an individual file inside of this tar archive. On top of that, there is an XML dump of the recordings guide data. This provides a lot of the flexability needed for insertion, and can also be used to help automate dvd authoring.

    That said, converting between the two is simple. To convert from .tmf to .ty, you can just untar the archive (winrar does this), discard the showing.xml file, and merge each of the .ty parts into one file. To go from .ty to .tmf, you would need to break apart the tystream parts at their boundaries and into individual files, and generate the xml data by manualy filling in the title information, description, genre, etc etc etc, as this is not stored in a tystream dump.

    There is one exception to this however. Any tystreams that were extracted using jdiners tytool will have issues with the master chunk. These streams may be converted to .tmf, but they probably won't be able to be reinserted into a tivo properly, because they were not extracted properly to begin with. This is an issue with the way tserver exports tystreams.
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  14. #14
    Join Date
    Mar 2002
    Posts
    1,339
    Originally posted by laserfan
    I'm still confused about why/how tar_server (tivoweb module) is used in all this. Why is this needed? It's mentioned in your comments/notes RC but there's no discussion of what module is used for what. Is Tivoweb supposed to be running when you do an ftp transfer to-or-from?
    Sorry, but I can't sort out how this thing works.
    mfs_ftp uses the same xml dumping routine (tcl) and the same mfs_tarstream (binary) as tivodvlpr's tivoweb tarserver module, the idea being that files produced are identical and interchangable with the (eventual) tivoweb insert module

    tivoweb mfsstream produces regular .ty files.

    mfs_ftp does not need tivoweb running.
    Last edited by rc3105; 09-30-2003 at 03:56 AM.

  15. #15
    Join Date
    Jun 2001
    Posts
    10

    Problem decompressing large .tmf file? Nope it was winrar.

    For what its worth,

    I just downloaded a 5.59 GB (6,008,352,768 bytes on disk) to my ntfs formatted drive. When I opened it up with winrar, it (winrar) said the file was corrupt. I got a similar error with winzip.

    So I tried a couple of other windows based decompressors and they worked just fine. (Specifically 7-Zip File Manager from source forge (free) and WinTAR-SCSI (trial)).

    Z

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •