Mfs_Ftp: extract, archive, restore & transfer recordings

[mpauley]

Originally posted by kangolo
I'm running a stock Philips 20hr SA (no hd upgrades) with a turbo net card - anyone else got this setup?

I'm getting 500k/sec - which is fine, but I'm seeing others get 6x that speed.

Is my hub the limiting factor (compared to a switch), or the hard drive?

Riley - this is a truly great app - takes me 11 minutes for a half hour show at basic quality - takes about a minute in tytool 6r3 to convert it to a m2v which I can watch through media player.

Time to start taking tv shows to work on the ipod to watch at lunchtime

kangolo,

I have a 14 hr Philips SA with a TiVonet card and Im getting 800k/sec ... with a Turbonet on my other TiVo, I get about 1.5M/sec.

Mickey

Share your shows at LookMa-NoHands.Com!

[kangolo]

950k/sec on another machine .

So,
600k/sec from AMD K6 400 running XP with a realtek card
950k/sec from a P2 400 running XP with a 3com card.

Both with loads of ram, and maxtor 40gb's as the data drive, os on a seperate one

Time to replace those crappy realteks

[kangolo]

Originally posted by Neutronflux
Hey Riley -- I am having trouble with tystudio extraction -- and I'm dying to get mfsftp running. I've tried several programs, all with the same result. [smart ftp, filezilla, etc]

I can login to the tivo and it lists all directories and recorded shows [FAST] -- however, when I try to transfer over a recording to my pc, the time counter just rises and eventually times out, creating a 0kb file. Has anyone else seen this? I would love to get this program running so I can extract files for use with tystudio.

Thanks!
Kev

Have you got ftp set to use passive (PASV) connections?

Do you have a firewall?

Commands and listings are sent over a seperate TCP port to data - so one may be being blocked while the command conncetion works ok

[FreydNot]

Originally posted by Neutronflux
I can login to the tivo and it lists all directories and recorded shows [FAST] -- however, when I try to transfer over a recording to my pc, the time counter just rises and eventually times out, creating a 0kb file. Has anyone else seen this? I would love to get this program running so I can extract files for use with tystudio.

That sounds exactly like what happens if you have a DirectTivo with out the noscramble module/patch. That's not the problem is it?

[osetivo]

Bingo Doc!

Take a look here.
http://www.dealdatabase.com/forum/sh...ght=noscramble

[Neutronflux]

Riley - thanks so much for your great programs and all your help, it's really appreciated. People on this board really do rock!

This is the readout from my telnet into my tivo box:

/var/hack/mfsftp# ./mfs_ftp.tcl 2000
00:00:00 - entering init_mfs_ftp
00:00:00 - entering background
/var/hack/mfsftp# 00:00:00 - entering init_mfs_ftp
00:00:00 - entering background
00:00:00 - ping - mfs_ftp running on 127,0,0,1 : 2000
00:00:00 - mfs_ftp version 1.2.5.1 - checking tivo system sw
00:00:01 - version 3.1.0-01-1-011
00:00:01 - setting mfspath to /Recording/NowShowingByClassic
00:00:01 - insert throttle is 0, == 0 ms
00:00:01 - extract throttle is 0, == 0 ms
00:00:01 - entering reset_mfs_ftp
00:00:01 - port 2000 control socket initialized - ready for connections
00:00:01 - entering buil_dir_LIST
00:00:01 - entering update_rec_fsids
00:00:01 - getting current rec_fsids list
00:00:02 - rec_fsids doesn't match cached_rec_fsids, deleting cached info
00:00:02 - entering build_rec_list
00:00:02 - from mfs "/Recording/NowShowingByClassic"
........................................
00:00:15 - build_rec_LIST complete

****

as you can see -- here I selected to use port 2000... the mfs_ftp is using the 127.0.0.1 loopback address [dhcp] is this my problem? Should I specify the Tivo's IP address [192.168.1.101] manually and if so, what's the syntax for that?

[rc3105]

ports below 1024 are reserved for unix services. 21 is the port assigned to ftp. if you use a port other than 21 it should be > 1024. the default ftp data port is portnum-1. so for control port 21, 20 is used for data. for 3105, 3104 is used for data

passive mode & custom port usage is in the help/readme for your particular ftp client

the default ip is set to 127.0.0.1 on startup, the first time a ftp client connects it auto-detects the tivo & client ips

[n4zmz]

When moving a stream from a DTiVo to a Standalone, a couple of inconsistancies were found.

The mfs utilities were all on an NFS mount on both machines. 1.2.5.1 was used in the test.

Rating not set. (PG-13)
Advisory not set. (AC, AL)
Classifications missing (movie)
Genre missing (action/adventure)
Attributes missing like CC
Save until not set. (until I delete)
Time was off by 1 hour. (4:30 displayed instead of 5:30)

The tzos was not set correctly. It was set for central and I'm in eastern. But that should not have changed anything.

Normally I would have fixed this myself, but finding the sources which were used has been a real task. (One which I haven't completed yet... pointers/source code would be helpful)

The only real pain is the standalone is using a tivonet card (5+ hours to inserta 2 hour show).

[rc3105]

not all info is restored. never has been, I may or may not chase down the details. there are differences in the info between 1.x-3.x sw. the "universal" info (and what I look at) is restored. nearly everything is included in the xml and available for archiving / catalogging like tmf2ty does. (side note, the few attributes that aren't extracted were left out to maintain compatability with tmf 1.0 as implemented by mbm/tivodvlpr)

the compiled binaries only do the actual extract/insert of video data. unless you really know the layout of mfs don't mess with those. using improperly recompiled versions can trash mfs and nuke your tivo.

the dbinfo is restored in the mfs_ftp tivosh script. it's easy to modify with joe and the info you need to restore the missing attributes is in tivoweb

[Will]

god, I am having a bitch of a time finding an ftp proggie that plays nice with this on os x....

Let me preface this by saying that 90% of the people I've helped, either on Wintels or Macs, are making basic mistakes and not reading carefully. Nothing to do with any defects in mfs_ftp or the ftp Clients.

But undeniably there are problems beyond that, that are complicating the situation, and Mac ftp Clients that work with mfs_ftp are rare indeed. Some people are doing everything else right and that is their only problem. But many are compounding a misunderstanding of the basics with ALSO using a non-friendly ftp Client on top of that; real trouble.

Fetch 4.03 works pretty well with mfs_ftp. Navigating the subdirectories is a minor nuisance, you have to do it manually. Also I occasionally have trouble with the first digit of a filename (fsid) getting clipped off and I haven't figured out how to enter a filename or a complete pathname manually for a download, in a way that gets accepted. But all-in-all Fetch 4.03 is a good solution.

There's a little gem called SimpleFTP (version 2.01) that works _very_ nicely with mfs_ftp. I like it a lot. Doesn't chop off that first digit. But it aborts file downloads at 2 GB! Hello!?

At least one poster here has said Transmit works. Not for me. It has the redundant subdirectory problem that many ftp Clients have with mfs_ftp.

I've tried more than a dozen other Mac ftp Clients, all NG. Sometimes the redundant subdirectories, to infinity. Some simply can't make any sense of the host's directory structure/syntax at all, some seem to see everything OK up until download time. Some clip off the first digit of all the subdirectories. BTW many Wintel ftp Clients have the same problems.

Riley seems to be one of those wonderful practical, pragmatic geniuses who have a good grasp of currently available tools and technology and a knack for putting them together to provide useful products. But brevity of coding and compliance with standards and protocols (ftp in this case) may be beyond his attention span. If this were a commercial product I was developing I'd have an anal analyst following him around and cleaning up after him. I'd pay them both well, but Riley quite a bit more.

As it is, for a free product he's hacking together for us, I guess as labor of love, I'm very grateful for it and perfectly willing to grope around for an ftp Client that can cope with what he's giving us.

Kudos and thanks to Riley!

[rc3105]

LOL - if this were a commercial project I'd fire MYSELF for such sloppy coding!

1.2.5.1 has been tested here with command line ftp from OS-X 10.2, fetch 3.0 from the jaguar 9.x classic environment and 7.6-8.6 installed on 6100/7500/8500/9500 & beige g3's.


--
Riley

[TivoDvlpr]

Well Rc, I see you did it. You realized the dream I had for tivo extraction so-so-long ago! And you even posted a reply to that response duly noting the idea!

(see post:http://www.dealdatabase.com/forum/sh...2182#post52182)

Cheers to you buddy!

Sorry I haven't been around, I'll be back in the summertime when I have more time available and my daughter is not in school.

later!

[rc3105]

folks, for those of you that don't know tivodvlpr (with help from mbm) is the author of mfs_import, which is what mfs_stdinsert is based on, which is the cornerstone of mfs_ftp's insert ability.


I suspect his original idea was more of a single coherent binary (as is my eventual goal) the current version is really just a proof of concept test that got loose!

--
Riley

[stumbleshoe]

Thanks Riley - this is great stuff. Do you take Paypal ?

I've been extracting like mad from my SA 1 w/ 3.0 - planning to setup my g/f SA soon so we can exchange stuff.

Any suggestions for setting up some form of password protection or should I look into setting up an FTP proxy ?

[rc3105]

Originally posted by stumbleshoe
Thanks Riley - this is great stuff. Do you take Paypal ?

I've been extracting like mad from my SA 1 w/ 3.0 - planning to setup my g/f SA soon so we can exchange stuff.

Any suggestions for setting up some form of password protection or should I look into setting up an FTP proxy ?

yep, can never have enough toys

the best way to move shows around it to go from tivo-pc, then pc-pc (ftp, p2p, e-mail attachment, sneakernet, carrier pigeon, whatever) then pc-tivo

password or access control depends on the method you use. you can use tivo-tivo directly across the net, but since there's no resume implemented yet it's just asking for a lot of partial transfers. (giving your tivo full net access is a bad idea anyway)