Mfs_Ftp: extract, archive, restore & transfer recordings

[wundernaut]

Ever since we lost use of the vstream plugin for VLC, I've been trying to get VLC to open files directly from mfs_ftp for viewing / streaming, but VLC didn't like mfs_ftp's command implementation set. I finally figured out that the problem was that VLC wants to know the size of the file that it's downloading, but SIZE was never implemented. I've implemented it, and so far I'm able to view any file. For a while, it would load the file every other try, but it seems pretty consistent now, so I'm not sure what the earlier problem was. This is a drop-in replacement for Jamie's patched mfs_ftp.tcl file, as it was based on that. Just unzip and restart mfs_ftp, and you can start entering ftp urls into VLC! Tested with VLC 0.9.4 and a hacked Series 1 TiVo.

I also took a stab at implementing a SIZE command a while back as well. Your code looks similar to what I did at first, and it works for client programs that just want a SIZE command to be supported and aren't too picky about the number that gets returned. However, I found this approach fails with client programs such as SmartFTP using passive mode, where the program expects the SIZE command to be accurate down to the byte.

There are two reasons why this approach fails:

1. You can get overflows in calculating the size. Normal precision tcl overflows at 4GB, and some files are larger than that. Some of the later versions of TiVo tcl support extended precision arithmetic, but not the version that's in a series 1 TiVo. If you search tcl code archives, you can find various packages that add extended arithmetic to early tcl, and they're not that big nor processor intensive. I did this for awhile, until I realized the next problem.

2. A tmf, ty, etc. file is in some sense synthetic. It doesn't really exist anywhere in the filesystem. These files consist of the actual TiVo binary, plus an XML header that's created on the fly and concatenated onto the beginning or end of the TiVo binary. As such, there's no TiVo system call to return their size. Worse, there's no easy way to even pre-calculate their size, say at startup. As I recall, the XML format contains the current date in ASCII, the size of which varies by month and day. So you might pre-calculate the size at startup, but it becomes inaccurate the next day by a few bytes. I did take a stab at pre-calculating file sizes, but you have to do it separately for all the various formats and it just became too complicated.

I'm not saying that your solution doesn't work for VLC or other programs that don't really care about how accurate SIZE is. But I think you'll find that you might suddenly see failures with other client programs. Client ftp programs have different strategies for dealing with ftp commands that are not supported. For example, if SIZE is not supported, SmartFTP just does an "ls -l" type command to get the rough size, and does not actually check this number against the number of bytes transferred. If the SIZE command is supported SmartFTP takes it as gospel and actually fails the transfer because the byte count is wrong, even though the file actually transferred OK. Thus in some cases you are actually better off not supporting the SIZE command if it's not accurate.

[lgkahn]

Any hope on getting this working again on 11.0 without crashing the box.. reily are you still around? thanks

[petestrash]

Any hope on getting this working again on 11.0 without crashing the box.. reily are you still around? thanks

Riley does not have an S3 box.

Peter.

[Jamie]

Any hope on getting this working again on 11.0 without crashing the box.. reily are you still around? thanks

Riley does not have an S3 box.

This was resolved in another thread.

[rc3105]

Riley does not have an S3 box.

Peter.

tis true, the shame!!!

tried gluing three series ones together but it just wasn't the same. almost snagged a cablecard S3 on craigslist before christmas, alas it wasn't meant to be

Aidan the toddlersaurus rex is 3 now, as mr mom there's not really free time to tinker anymore. maybe when he starts learning tcl (sigh)


edit: the little monkey will be a great grease monkey someday, when he's not too busy being a rocket surgeon

[philhu]

So, after reading the entire thread, is it possible to use mfs_ftp on a TivoHD?

I mean I read the correct old version is available, but most say, even with patches, that is it isn't very 'useful'

I used to use it on my S2 and it was an amazing program ("Riley, remember me? Folders.tcl?")

Why is it no longer useful?

[jt1134]

So, after reading the entire thread, is it possible to use mfs_ftp on a TivoHD?

sure is. I'm using it right now.

Why is it no longer useful?

there isn't a posted version that will work right out of the box. it's still quite useful though.

patch mfs_ftp-1.2.9p with this patch
then replace the mfs_* binaries with these 64bit versions
then fix mfs_ftp.tcl for sw versions in the double digits

insertion is a little quirky sometimes, but I've been using mfs_ftp to extract from my THD's every day for almost a year now.

[SevronD]

I've been using mfs_ftp for awhile now too - excellent program. Communicates with Filezilla on the PC quite nicely (~2Mbps wirelessly to my HR10-250 w/802.11g ethernet Game Adapter). I have mfs_ftp ver 1.2.9p-patchlevel-20070717b. The Tivo sw is 6.4a-01-2, whatever that means.

Anyway, I can download shows from HR10-250 to PC via mfs_ftp all day. Works great.

Insertion works too, however the show info (Episode Title, Genre, etc...), which I guess is in the XML, doesn't get into the Tivo, even though it's in the file I'm inserting into the Tivo. I'm inserting home-filmed hockey games which I've converted with FFMPEG (Rev 12665.7z), which creates the TY files.

The games play fairly well on the HR10-250, although the FFx1 does not speed up the playback (x2 and x4 go fast and super fast forward as usual). So, my issues are:

1) Why doesn't the Tivo accept my XML data?
2) Why doesn't the FF x1 actually speed up the playback?

Any advice on where to turn? I've searched the TCF and DDB boards for info the XML but haven't found much. Thanks! It's a great app, even with these 2 issues!

[Butch]

sure is. I'm using it right now.

there isn't a posted version that will work right out of the box. it's still quite useful though.

patch mfs_ftp-1.2.9p with this patch
then replace the mfs_* binaries with these 64bit versions
then fix mfs_ftp.tcl for sw versions in the double digits

insertion is a little quirky sometimes, but I've been using mfs_ftp to extract from my THD's every day for almost a year now.

Is this for the HR10-250 units ?

[GBill]

Ok, I've searched the numerous mfs_ftp threads, and it looks as if a few people have had this problem, but I can't determine what they did to fix it...

I am using Filezilla and smartftp. Filezilla seems to cause problems sometimes (like every other expansion) when I try to expand a directory when connected to the MFS side. I do not have this problem when I connect to the tivo on port 21 with Filezilla, so I have to assume it's mfs_ftp somehow. But strangely, I do not seem to have this problem when I use smartftp. I really would rather use Filezilla, and I've read that others do not have problems with Filezilla. I also can successfully use Movieloader too.


Here is my port.3150.log:

0:34:05:PM - 220 Mfs_Ftp ver 1.2.9p-patchlevel-20070717b - {sock18} from "192.168.0.6:3160"
10:34:05:PM - 331 User name okay, need password.
10:34:07:PM - 230 Running in TiVo Mode.
10:34:07:PM - 215 UNIX
10:34:07:PM - 502 Command not implemented "FEAT"
10:34:07:PM - 257 "/" is current directory.
10:34:07:PM - 200 Type set to I
10:34:07:PM - 227 Entering Passive Mode (192,168,0,90,12,32).
10:34:07:PM - 150 Opening ASCII mode data connection for file list.
10:34:07:PM - 226 Transfer complete.
10:34:12:PM - 250 Directory change successful.
10:34:12:PM - 257 "/ty+" is current directory.
10:34:12:PM - 227 Entering Passive Mode (192,168,0,90,12,32).
10:34:12:PM - 150 Opening ASCII mode data connection for file list.
10:34:14:PM - updating cached recording info
.......................................................................

.......................................................................

10:34:19:PM - 226 Transfer complete.
10:34:21:PM - 250 Directory change successful.
10:34:21:PM - 257 "/txt" is current directory.
10:34:21:PM - 227 Entering Passive Mode (192,168,0,90,12,32).
10:34:21:PM - 150 Opening ASCII mode data connection for file list.
10:34:22:PM - updating cached recording info
.......................................................................

.......................................................................

bgerror invoked with error

" can not find channel named "sock20" "

errorInfo:
can not find channel named "sock20"
while executing
"puts -nonewline $info(dc) $temp"
(procedure "NLST" line 32)
invoked from within
"NLST $line $fsock "
("LIST" arm line 1)
invoked from within
"switch $cmd {
USER { USER $fsock $line }
PASS { PASS $fsock $line }
CWD { CWD $args $fsock }
CDUP { CDUP $fsock }
DELE { DELE $fsock $line..."
(procedure "parseline" line 6)
invoked from within
"parseline $line $sock "
(procedure "readlinefromsocket" line 10)
invoked from within
"readlinefromsocket sock18"

re-initializing mfs_ftp

close the current ftp connection and simply open another

"core dump"

info(version): 1.2.9p-patchlevel-20070717b
info(tswv): 6.4a-01-2-151
info(dbl): 0
info(ithrottle): 2
info(insert_priority): 10
info(multithreaded): 0
info(saveuntil): suggestion
info(name_detail): 5
info(bjuggle): 0
info(active): 0
info(ac_interval): 1800
info(gatewayip): 127.0.0.1
info(gatewayport): 3105

Can someone help me figure out why I am getting this error? Thanks in advance.

Bill

[Jamie]

Ok, I've searched the numerous mfs_ftp threads, and it looks as if a few people have had this problem, but I can't determine what they did to fix it...

I am using Filezilla and smartftp. Filezilla seems to cause problems sometimes (like every other expansion) when I try to expand a directory when connected to the MFS side.
...

mfs_ftp can take a long time to produce directory listings. Try turning off any timeouts in Filezilla. I don't use Filezilla, but here's one thread that might offer a method to do that: link.

[GBill]

mfs_ftp can take a long time to produce directory listings. Try turning off any timeouts in Filezilla. I don't use Filezilla, but here's one thread that might offer a method to do that: link.

I disabled the timeout function, with no change...

It's wierd. Filezilla appears to wait long enough for the list on one folder, but when I move to another folder, it pauses as if it is going to wait, but returns an error anyway:

Connecting to 192.168.0.90:3105...
Status: Connection established, waiting for welcome message...
Response: 220 Mfs_Ftp ver 1.2.9p-patchlevel-20070717b - {sock18} from "192.168.0.6:3534"
Command: USER anonymous
Response: 331 User name okay, need password.
Command: PASS
Response: 230 Running in TiVo Mode.
Status: Connected
Status: Retrieving directory listing...
Command: CWD /bat
Response: 250 Directory change successful.
Command: PWD
Response: 257 "/bat" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (192,168,0,90,12,32).
Command: LIST
Response: 150 Opening ASCII mode data connection for file list.
Response: 226 Transfer complete.
Status: Directory listing successful
Status: Retrieving directory listing...
Command: CWD /asx
Response: 250 Directory change successful.
Command: PASV
Response: 227 Entering Passive Mode (192,168,0,90,12,32).
Command: LIST
Response: 150 Opening ASCII mode data connection for file list.
Error: Connection closed by server
Error: Failed to retrieve directory listing

I just noticed that if I switch to Active Mode I do not have this problem... I am not sure staying in active mode is really the solution or not...

[SpoonsJTD]

Sevron, I've seen that behavior while watching various OTA programs every once in a while. FFx1 won't fast forward at all.

I have a different issue. When I am transferring a show and watching it during the transfer, the progress doesn't update. Then, when it gets to the end of the bar, it stops. I can restart, but it starts from the beginning. If I stop playing the show before getting to the end of the progress bar and restart it, the bar will be updated to reflect what had been transferred while I was watching. Short story, it doesn't look the progress\'time left' bar updates to reflect what has currently been transferred over. I only consider this an issue because I swear back in the 6.2 days when I first used mfs_ftp, i could swear the bar updated while watching the show as the show got transferred.

Anybody else experience this?

[newcore]

Hello,

i can't play ty files from my tivo 240040 S2 version 9.3.2a-01-2-140 kernel 2.4.20. I have mfs_ftp version 1.2.9p-patchlevel-20080512 and TivoWeb version 1.3.1 and both used to work fine but not anymore. I thought it was a filter issue on my PC, but i downloaded ty sample from the internet and it plays fine. graphedit doesn't even open files from my tivo saying something about ClassFactory cannot supply requested class. And Media Player Classic swears at me:
Media Type 0:
--------------------------
Unknown

AM_MEDIA_TYPE:
majortype: MEDIATYPE_Stream {E436EB83-524F-11CE-9F53-0020AF0BA770}
subtype: Unknown GUID Name {B769ADA9-E324-4555-8BEC-DC42E980F0D8}
formattype: TIME_FORMAT_NONE {00000000-0000-0000-0000-000000000000}
bFixedSizeSamples: 1
bTemporalCompression: 0
lSampleSize: 1
cbFormat: 0

though the sample file can be played and opened in graphedit with no problems.

I appreciate any suggestions or advise.

[SevronD]

Spoons,

Are you uploading TMF or TY files? If TMF (i.e. TY file plus accompanying XML data), then are you sure the value in the XML denoting the duration of the video is correct? For example, if your video was 57 minutes and 11 seconds long, in the XML file that goes with the video should contain a line:

<Duration>3431</Duration>

I'm guessing here, but perhaps if you're uploading a TY file the Tivo doesn't know how long the green bar should be until it's fully uploaded.

-Sevron