PDA

View Full Version : tyftpd (mfs_ftp.tcl alternative)



jkozee
08-15-2011, 12:36 AM
This is an alternative to mfs_ftp.tcl that began as a replacement written in C and based on oftpd. After adding uploading ability it made more sense to optimize the mfs_* calls and keep the ftp code in tcl. Along the way I added another file type (tmf+) which is meant to solve the bookmark juggle problem in mfs_ftp by sending the master block of the ty file at the beginning of the upload. Anyhow, this alternative should be fairly stable and fixes some flaws I found when trying to use mfs_ftp.tcl.
New Version: 1.9


################################################################################
# tyFTPd - Copyright (C) 2012 jkozee #
# This program is distributed WITHOUT ANY WARRANTY; without even the implied #
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. #
# #
# TERMS OF USE: personal (noncommercial) fair-use backup & restore #
# This software may not be used for copyright infringment or any illegal #
# purpose, bundled with any product, sold, distributed or redistributed #
# under any circumstances on any media without express written consent. #
################################################################################
# History: 1.9 20130113 Fixed missing quote as noted by tas3086. #
# History: 1.8 20120729 Catch and log TransactionRetry attempts. #
# History: 1.7 20120717 Fixed a bug related to not clearing CleanupStart when #
# doing multiple STOR in a single client connection. #
# History: 1.6 20120715 Changed the routines that call AddPart to use 64-bit #
# values by using wide(). This may break compatibility #
# with older units, however it allows for parts larger #
# than 2GB to be created. Disable it by using #
# "set GlobalData(fUseWide) 0" #
# History: 1.5 20120710 Fixed a bug that would crash when a TmsID exists in #
# the GuideIndex, but the object no longer exist int the #
# mfs database. #
# History: 1.4 20120709 Added real station and npk support. #
# Updated tyftpd station to be reusable. #
# Fixed a bug that would not handle maxSTOR correctly. #
# History: 1.3 20120627 Fixed errors in UpdateParts. #
# Fixed handling of tmf+ file uploads. Using this file #
# type allows for initial allocation of all parts, which #
# means the playbar is accurate almost immediately. #
# Added -M flag to only list the tmf+ directory. #
# Added code to handle incomplete STOR operations #
# better. Now it removes unused parts and nulls out #
# portions of the final part to hide the junk data that #
# may exist due to using streamcreate vs streamfill. #
# History: 1.2 20120610 Added -p and -c items to usage text. #
# Added TYPE for better EOL handling (FlashFXP). #
# Now using unified mfs32/64 code for mfs_ftp_helper. #
# Added -m flag to only list the tmf directory. #
# Added daemonize code unless loging to the screen. #
# Added code to set Parts Begin/End attribute set. #
# WARNING: tmf+ type handling suspect - DO NOT USE #
# History: 1.1 20111118 Fixed changed list EOLs to CRLF to support FlashFXP #
# Added [-p #] flag to change the listening port #
# Added [-c] to cleanup incomplete recordings on startup #
# The default behavior is to not cleanup so startup is #
# much quicker. The -c flag is only needed if tyFTPd #
# was aborted/died during upload. #
# History: 1.0 20110815 Initial release. #
# #
# Purpose: This script is an alternative to mfs_ftp.tcl. It began as an #
# experiment inspired by mfs_ftp.tcl and written in C based on oftpd #
# and linked to the mfs_utils library to see if I could improve #
# download speeds. Eventually I added upload abilities which #
# required a tcl based server to handle the mfs objects. At that #
# point I decided to just rewrite it all in tcl. This code has a #
# advantages over mfs_ftp.tcl. #
# 1) All of the required mfs_* functions are contained in a singe #
# executable and is the only other required file (included). #
# 2) Faster directory listings without caching. #
# 3) The directory listings are sent as they are being gathered #
# instead of leaving the client waiting while they are gathered #
# 4) More complete mfs object creation including series creation to #
# preserve folders on upload. #
# 5) Better error handling, especially when uploads are aborted they #
# are marked as completed and the recording status is stopped. #
# 6) Forced the FSID to be part of the filename to assure that the #
# correct file is downloaded/deleted from the client. #
# #
# Usage: -c Cleanup incomplete recordings from prior instance #
# -d Include debug log information #
# -l Log to /var/log/tyFTPd.log #
# -m Only list tmf directory #
# -M Only list tmf+ directory #
# -p Port to listen on (Default is 5013) #
# -s Log to the Screen #
# -v Inlcude verbose log information #
# -x Limit the reported filesize to 2GB for poorly written #
# clients (LeechFTP) #
# #
# Important: Make sure that you update mfs_ftp_helper with each new release. #
# It is now using the unified mfs-utils 32/64 version. #
# #
# Notes: Portions of this code based on, borrowed from, or inspired by #
# other projects found on DDB, including TWP, mfs_ftp, etc. I only #
# tested this on S3/11.0k and S2.5/7.2.2, probably needs tweaked for #
# DTiVos. #
# #
# MovieLoader: If you are using this with MovieLoader make sure you are using #
# MovieLoader.v0.17c.zip or later. You should change the port #
# for tyftp by using the -p 3105. #
# #
################################################################################

djl
08-15-2011, 01:27 PM
I gave this a quick run, and it ran error-free on my Tivo HD. Really nice, and the file listing comes up much faster than mfs-ftp. Any thoughts of adding TS 'extraction', as in this thread (http://dealdatabase.com/forum/showthread.php?p=295133)?

AlphaWolf
08-21-2011, 07:31 PM
I am somewhat curious, how processor intensive is it to run s3tots? Would it be possible to s3tots the stream in realtime while maintaining maximum transfer speed?

I've been intending on looking into it myself but never put any serious effort into it.

blueman2
09-05-2011, 12:21 AM
Hey, great tool! VERY VERY fast! I am getting 2.5MB/s transfer rate. The the file list populates very quickly as well. Rock solid so far. Well done!!

Jamie
09-05-2011, 01:12 AM
I am somewhat curious, how processor intensive is it to run s3tots? Would it be possible to s3tots the stream in realtime while maintaining maximum transfer speed?

I've been intending on looking into it myself but never put any serious effort into it.
Have you looked at this (http://69.164.213.157/forum/showthread.php?t=58805) thread?

myasus
09-26-2011, 06:41 PM
Did not notice the port changed at the first try :( Now I can connect to the server on my THD. But there is no show listed while the original mfs_ftp correctly shows two. Any idea what is wrong? Thanks for your help.

jkozee
09-27-2011, 11:47 PM
Sorry, I probably should have pointed out that this listens on port 5013 not 3105, and can only be changed by editing the code as I did not add a port option to the arg list.

To be honest I haven't looked at the code in quite a while, so I don't have any ideas off the top of my head. The show list generation is based on code from the unified mfs_* tools. Do the shows appear in the output from NowShowing from that package?

The only thing I encountered in my testing was ftp clients that would not list shows that exceeded 2GiB (LeechFTP). Try running this command:

mfs_ftp_helper -F LIST -A /tmf

If they appear in this list then invoke the script with the -x flag to prevent it from reporting sizes over 2GiB.

myasus
09-28-2011, 07:02 PM
Do the shows appear in the output from NowShowing from that package?
Yes. And regular mfs_ftp can list correctly.


The only thing I encountered in my testing was ftp clients that would not list shows that exceeded 2GiB (LeechFTP).
Try running this command:
mfs_ftp_helper -F LIST -A /tmf
If they appear in this list then invoke the script with the -x flag to prevent it from reporting sizes over 2GiB.
Not likely since regular mfs_ftp can list my 2 shows (both ~4G) correctly. "mfs_ftp_helper -F LIST -A /tmf" worked correctly, even with "-x 2GiB" option.

Might because the patches I applied to mfs_ftp.tcl? Following the instruction on dvrpedia, I applied these:
mfs_ftp.20070121.patch.tar.gz
mfs_ftp-20070207-pasv-fix.patch.gz
mfs_ftp-20070717b-ya-pasv-fix.patch.gz

jkozee
09-29-2011, 12:27 AM
Wierd. The script uses mfs_ftp_helper to produce the directory listing. Did you rename mfs_ftp_helper.mips64 to mfs_ftp_helper and place it in the same directory as tyftpd.tcl. What do you see in /var/log/tyftpd.log when you run the script with logging turned on "tyftpd.tcl -l -v -d &"

myasus
09-29-2011, 07:26 PM
Did you rename mfs_ftp_helper.mips64 to mfs_ftp_helper and place it in the same directory as tyftpd.tcl.
I did. The log shows:


09/29/11 23:39:41 - (2559) >>> 220 tyFTPd 1.0 FTP server ready.
09/29/11 23:39:41 - Now serving 192.168.1.200 port 2559
09/29/11 23:39:41 - (2559) <<< USER anonymous
09/29/11 23:39:41 - (2559) >>> 331 Please specify the password.
09/29/11 23:39:41 - (2559) <<< PASS flashfxp-user@flashfxp.com
09/29/11 23:39:41 - (2559) >>> 230 Login successful.
09/29/11 23:39:41 - (2559) <<< SYST
09/29/11 23:39:41 - (2559) >>> 215 UNIX Type: L8
09/29/11 23:39:41 - (2559) <<< REST 100
09/29/11 23:39:41 - (2559) >>> 350 Restart okay, awaiting file retrieval request.
09/29/11 23:39:41 - (2559) <<< REST 0
09/29/11 23:39:41 - (2559) >>> 350 Restart okay, awaiting file retrieval request.
09/29/11 23:39:41 - (2559) <<< PWD
09/29/11 23:39:41 - (2559) >>> 257 "/"
09/29/11 23:39:43 - (2559) <<< TYPE A
09/29/11 23:39:43 - (2559) >>> 200 I only have TiVo mode.
09/29/11 23:39:43 - (2559) <<< PASV
09/29/11 23:39:43 - Updating the server ip address from 127,0,0,1...
09/29/11 23:39:43 - Entering Passive mode for ip=192,168,1,201 port=34286
09/29/11 23:39:43 - (2559) >>> 227 Entering Passive Mode (192,168,1,201,133,238).
09/29/11 23:39:43 - In Pasv Accept with {sock17} {sock21} {192.168.1.200} {2560}
09/29/11 23:39:43 - (2559) <<< LIST -al
09/29/11 23:39:43 - Data Socket Connected
09/29/11 23:39:43 - Suspending timeout for port 2559
09/29/11 23:39:43 - (2559) >>> 150 Opening data channel
09/29/11 23:39:43 - (2559) >>> 226 List transfer complete.
09/29/11 23:39:43 - Restarting timeout for port 2559

jkozee
09-29-2011, 08:27 PM
What client are you using? I think it might be the flag being sent with the LIST command. You can try removing the $flag from the call to mfs_ftp_helper in the proc LIST.

myasus
09-29-2011, 08:52 PM
What client are you using? I think it might be the flag being sent with the LIST command. You can try removing the $flag from the call to mfs_ftp_helper in the proc LIST.
I use flashfxp. It sends "LIST -al" command. Seems mfs_ftp.tcl has no problem with that.

myasus
09-29-2011, 09:45 PM
Just tried to use "LIST" only in flashfxp, still nothing listed.

jkozee
09-29-2011, 09:59 PM
Not sure. Maybe it doesn't like the format that mfs_ftp_helper is listing. If you would still like to test tyftpd you can try another client. I'm not sure what platform your using, but I tested with LeechFTP and filezilla, which are both free. If I get time I may try debugging with flasfxp. Is it a free client?

myasus
09-30-2011, 10:00 PM
If you would still like to test tyftpd you can try another client. I'm not sure what platform your using, but I tested with LeechFTP and filezilla, which are both free. If I get time I may try debugging with flasfxp. Is it a free client?
Just tested with LeechFTP, it can list dirs, but not the shows. The logs are here:

~ Connecting...
~ Connected to 192.168.1.201, waiting for response...
< 220 tyFTPd 1.0 FTP server ready.
> USER anonymous
< 331 Please specify the password.
> PASS *****
< 230 Login successful.
> REST 1
< 350 Restart okay, awaiting file retrieval request.
> REST 0
< 350 Restart okay, awaiting file retrieval request.
> SYST
< 215 UNIX Type: L8
> PWD
< 257 "/"
~ Login completed.
> PORT 192,168,1,200,9,136
< 200 PORT OK
> TYPE A
< 200 I only have TiVo mode.
> LIST
< 150 Opening data channel
< 226 List transfer complete.
> CWD ty+
< 250 Directory successfully changed.
> PWD
< 257 "/ty+"
> PORT 192,168,1,200,9,140
< 200 PORT OK
> TYPE A
< 200 I only have TiVo mode.
> LIST
< 150 Opening data channel
< LISTERROR: -rwxr-xr-x 1 0 0 004320133120 Sep 24 2011 {SportsCenter}{2011-09-24}{}{09.00 PM Sat Sep 24, 2011}{ESPNWHD}{237550}.ty+
< LISTERROR: -rwxr-xr-x 1 0 0 003785359360 Sep 21 2011 {CBS Evening News With Scott Pelley}{2011-09-21}{}{05.30 PM Wed Sep 21, 2011}{WCBSDT}{229266}.ty+
< 226 List transfer complete.

jkozee
09-30-2011, 10:07 PM
You will need the -x flag if you are using leech and have shows that are over 2 GiB. This flag will report the size as a 2G max. The shows will transfer, but the progress bar will be incorrect.

myasus
09-30-2011, 11:35 PM
You will need the -x flag if you are using leech and have shows that are over 2 GiB.
Thanks. It is working now with LeechFTP. One more question is LeechFTP only list tmf/tmf+/ty+/xml these 4 dirs. Is that possible to download ty format only?

jkozee
10-01-2011, 10:45 PM
Are you sure that you need a ty only? What are you trying to do with it? I think mfs_ftp lists a ty only file but it is still a ty+ as they are pretty compatible. The difference between a ty and ty+ is that there is show information in the form of an xml file in the last chunk of the ty+. Pretty sure that most tools will just ignore that data if you named it ty. A tmf is a tar file that has the xml at the beginning so that it can processed early during a transfer so that the show information looks correct. The tmf+ I added is a tar file with the xml and master chunk headers of all the ty parts up front, so that the entire show could be allocated before the transfer begins so that you could watch the show as it was transferring without having to do the bookmark juggle that mfs_ftp did. IIRC correctly it worked on the s2 units but not on the THD units. If you really need a ty, you can make it from any of the other formats.

myasus
10-03-2011, 08:22 PM
Thanks for clarifying these formats. I did not know s3tots works with ty+ :( I only need ts to xvid compressing so I do not care about the show information :P

Soapm
10-03-2011, 09:49 PM
Do I need to load mfs_ftp or do I just stick this in var and run it? If that is the case this is sure a lot easier to install...

jkozee
10-04-2011, 12:37 AM
This is an aternative to mfs_ftp that listens on port 5013, so it doesn't rely on any part of it. The only requirement is that you rename the appropriate mfs_ftp_helper.mipsxx to mfs_ftp_helper depending on your platform and place it in the same directory as tyftpd.tcl.

I'm hesitant to say it is an mfs_ftp replacement because it wasn't written to be, and I don't want to take anything away from riley. This is not a new version of, or patch for mfs_ftp, although it was certainly inspired by it. I orginally wrote a C version that was based on oftd and mfs_* tools that was download only. Later, I added upload, which required a tcl code for mfs db manipulation, so I added internal functions similar to the internal_DeleteShowing, which was a pain to debug and maintain, so I wrote a tclserver app to handle the mfs stuff over sockets. At that point I realized that riley had it right, so I took what I learned and wrote this version.

This code has a few advantages over mfs_ftp (better error handling on upload failures, fast file listings with no cache required, better program/series object creation, and so far less patching required). There are a few things that could be improved as well. Startup could be optimized because I was aggressive and walk all shows on startup to cleanup any interrupted transfers, it would probably be better to add a switch or to keep track of uploads some other way so that it knows when it has a possible interruption at startup. Also, I did very limited testing on different platforms, mostly just THD. And as pointed out above, it may not work with all clients.

All the source code is included, so feel free to poke around and if you have any patches I will apply them and update the package.

Soapm
10-04-2011, 01:03 AM
Where can get the helper file if I'm doing a fresh install so don't have mfs_ftp installed? I was hoping with this I wouldn't need to install it at all. I remember what a pain that was...

jkozee
10-04-2011, 08:46 AM
Everything you need is in post #1 of this thread. Both mfs32 and mfs64 mips binaries of mfs_ftp_helper are included, as well as the patch file for mfs_* tools t recreate them.

jkozee
10-19-2011, 12:26 AM
A new version is now available in post 1 that works with FlashFXP, allows for a different listening port, and speeds up startup.

Soapm
10-30-2011, 08:51 PM
I searched but don't see a support thread for tyftpd so I guess I will ask how to use it here. I moved it all over and made it executable and what not but this is what I got when I tried running the patch


FamRoom#/var/tyftpd/mfs_ftp_helper.patch
diff: mfs-utils.orig/generate_xml.c: No such file or directory
diff: mfs-utils/generate_xml.c: No such file or directory
/var/tyftpd/mfs_ftp_helper.patch: ---: command not found
/var/tyftpd/mfs_ftp_helper.patch: +++: command not found
/var/tyftpd/mfs_ftp_helper.patch: @@: command not found
/var/tyftpd/mfs_ftp_helper.patch: //: is a directory
/var/tyftpd/mfs_ftp_helper.patch: //: is a directory
/var/tyftpd/mfs_ftp_helper.patch: //: is a directory
/var/tyftpd/mfs_ftp_helper.patch: +//: No such file or directory
/var/tyftpd/mfs_ftp_helper.patch: +//: No such file or directory
/var/tyftpd/mfs_ftp_helper.patch: @@: command not found
/var/tyftpd/mfs_ftp_helper.patch: const: command not found
/var/tyftpd/mfs_ftp_helper.patch: line 15: syntax error near unexpected token `;'
/var/tyftpd/mfs_ftp_helper.patch: line 15: ` };'

I also tried running the tcl file with /var/tyftpd/tyftpd.tcl and that caused my Tivo to reboot so I assume that wasn't right either.

I guess what I need are some instructions since I'm not really a Linux person. How do we use this?

ScanMan
10-31-2011, 09:05 AM
You don't want to be running 'mfs_ftp_helper.patch' in fact you don't even need it on your machine. Upload 'tyftpd.tcl' and 'mfs_ftp_helper.mips64' (for TivoHD); rename 'mfs_ftp_helper.mips64' to 'mfs_ftp_helper' using the linux 'mv' command and then run 'tyftpd.tcl'.
mv mfs_ftp_helper.mips64 mfs_ftp_helper
./tyftpd.tcl&which runs it in the background; simple as that, works for me. Also, configure your ftp client to connect to port 5013, the default listening port for this app.

jkozee
10-31-2011, 11:54 AM
ScanMan's instructions are spot on and should get you up and running.

Because this is not the first time you have been tripped up by patch files, I'll offer a little more guidance. Patch files are not executables. They are a list of differences between a file or set of files and another file or set of files created by the diff command. They are applied to the original file(s) using the patch command. In this case I have included the executable and the patch file, which shows the changes I made to the original source code. If I had not included the executable (as is the case when only a patch file is available) you would download the source code for mfs_* tools, apply the patch, and compile the source code to get the executable.

Hope this helps you avoid future frustrations.

Soapm
10-31-2011, 05:44 PM
Sorta... Help me understand your response...

Is the "patch command" = patch /file_A /file_B. Is that the patch command or is there an actual command for patching?

you said download the source code for mfs_*. Does that stand for mfs_ftp or is there an actual application named mfs_*

You said after applying the patch I would compile the source, is that what dos2unix and chmod 755 does? Compiles it???

Sorry to be so green but this is like learning a foreign language so I am trying to understand word meaning...

Soapm
10-31-2011, 05:46 PM
You don't want to be running 'mfs_ftp_helper.patch' in fact you don't even need it on your machine. Upload 'tyftpd.tcl' and 'mfs_ftp_helper.mips64' (for TivoHD); rename 'mfs_ftp_helper.mips64' to 'mfs_ftp_helper' using the linux 'mv' command and then run 'tyftpd.tcl'.
mv mfs_ftp_helper.mips64 mfs_ftp_helper
./tyftpd.tcl&which runs it in the background; simple as that, works for me. Also, configure your ftp client to connect to port 5013, the default listening port for this app.

I thought I tried that with /var/tyftpd/tyftpd.tcl which causes me to reboot.

Does the & make a difference? Does it matter where I put the files?

ScanMan
10-31-2011, 08:15 PM
Regarding your previous post, don't worry about patch you shouldn't have to compile anything. You don't even need mfs_* tools to implement this. The helper file takes care of everything. Shouldn't cause a reboot, something else must be going on. It doesn't matter where you put them, I have both files in /var/hacks. Simplify your startup, my current setup only has telnet, tivowebplus and tivoftpd running.

Soapm
10-31-2011, 09:25 PM
Cool, not sure what was happening yesterday but it fired right up today. Thanks for all the help...

jkozee
10-31-2011, 11:22 PM
You are welcome.

Soapm
10-31-2011, 11:31 PM
You are welcome.

And thanks also for the great application. I'm digging this a lot better than mfs_ftp... Good work!!!

Soapm
11-01-2011, 12:40 AM
I thought it was working great but it's been 20 minutes now and it still hasn't pulled up my NPL. Is it supposed to take this long? I have a 2TB drive but it isn't halfway full yet.

Ps... I'm using flashXP. Are their better setting I should be using?

I see above someone had to switch from using FlashXP so I will try that tomorrow. The wife is in bed so I don't want to keep running up the stairs...


11/01/11 04:59:52 - tyFTPd version 1.1 Startup
11/01/11 04:59:53 - Server Ready for Action
11/01/11 05:03:14 - (1829) >>> 220 tyFTPd 1.1 FTP server ready.
11/01/11 05:03:14 - Now serving 192.168.254.103 port 1829
11/01/11 05:03:14 - (1829) <<< USER anonymous
11/01/11 05:03:14 - (1829) >>> 331 Please specify the password.
11/01/11 05:03:14 - (1829) <<< PASS flashfxp-user@flashfxp.com
11/01/11 05:03:14 - (1829) >>> 230 Login successful.
11/01/11 05:03:14 - (1829) <<< SYST
11/01/11 05:03:14 - (1829) >>> 215 UNIX Type: L8
11/01/11 05:03:14 - (1829) <<< FEAT
11/01/11 05:03:14 - (1829) >>> 502 Command not implemented "FEAT".
11/01/11 05:03:14 - (1829) <<< REST 100
11/01/11 05:03:14 - (1829) >>> 350 Restart okay, awaiting file retrieval request.
11/01/11 05:03:14 - (1829) <<< REST 0
11/01/11 05:03:14 - (1829) >>> 350 Restart okay, awaiting file retrieval request.
11/01/11 05:03:14 - (1829) <<< PWD
11/01/11 05:03:14 - (1829) >>> 257 "/"
11/01/11 05:03:14 - (1829) <<< TYPE A
11/01/11 05:03:14 - (1829) >>> 200 I only have TiVo mode.
11/01/11 05:03:14 - (1829) <<< PASV
11/01/11 05:03:14 - Updating the server ip address from 127,0,0,1...
11/01/11 05:03:14 - Entering Passive mode for ip=192,168,254,113 port=33175
11/01/11 05:03:14 - (1829) >>> 227 Entering Passive Mode (192,168,254,113,129,151).
11/01/11 05:03:14 - In Pasv Accept with {sock16} {sock20} {192.168.254.103} {1830}
11/01/11 05:03:14 - (1829) <<< LIST -al
11/01/11 05:03:14 - Data Socket Connected
11/01/11 05:03:14 - Suspending timeout for port 1829
11/01/11 05:03:14 - (1829) >>> 150 Opening data channel
11/01/11 05:05:00 - (1831) >>> 220 tyFTPd 1.1 FTP server ready.
11/01/11 05:05:00 - Now serving 192.168.254.103 port 1831
11/01/11 05:05:00 - (1831) <<< USER anonymous
11/01/11 05:05:00 - (1831) >>> 331 Please specify the password.
11/01/11 05:05:00 - (1831) <<< PASS flashfxp-user@flashfxp.com
11/01/11 05:05:00 - (1831) >>> 230 Login successful.
11/01/11 05:05:00 - (1831) <<< SYST
11/01/11 05:05:00 - (1831) >>> 215 UNIX Type: L8
11/01/11 05:05:00 - (1831) <<< FEAT
11/01/11 05:05:00 - (1831) >>> 502 Command not implemented "FEAT".
11/01/11 05:05:00 - (1831) <<< REST 100
11/01/11 05:05:00 - (1831) >>> 350 Restart okay, awaiting file retrieval request.
11/01/11 05:05:00 - (1831) <<< REST 0
11/01/11 05:05:00 - (1831) >>> 350 Restart okay, awaiting file retrieval request.
11/01/11 05:05:00 - (1831) <<< PWD
11/01/11 05:05:00 - (1831) >>> 257 "/"
11/01/11 05:05:00 - (1831) <<< TYPE A
11/01/11 05:05:00 - (1831) >>> 200 I only have TiVo mode.
11/01/11 05:05:00 - (1831) <<< PORT 192,168,254,103,7,40
11/01/11 05:05:00 - (1831) >>> 200 PORT OK
11/01/11 05:05:00 - (1831) <<< LIST -al
11/01/11 05:05:00 - Data Socket Connected
11/01/11 05:05:00 - Suspending timeout for port 1831
11/01/11 05:05:00 - (1831) >>> 150 Opening data channel

jkozee
11-01-2011, 08:56 AM
Try changing from Passive Mode (PASV) to Active Mode (PORT).

Soapm
11-02-2011, 01:25 AM
Try changing from Passive Mode (PASV) to Active Mode (PORT).

No go.......

Soapm
11-05-2011, 02:00 PM
I'm still not able to get this to list the directory but I had some time today to make log files. Maybe someone can spot something I'm doing wrong.

This is filezilla which I changed to verbose so it would provide the most logging.


2011-11-05 11:20:53 4000 3 Status: Connecting to 192.168.254.113:5013...
2011-11-05 11:20:53 4000 3 Status: Connection established, waiting for welcome message...
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 220 tyFTPd 1.1 FTP server ready.
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Command: USER anonymous
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 331 Please specify the password.
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Command: PASS **************
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 230 Login successful.
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Command: SYST
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 215 UNIX Type: L8
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Command: FEAT
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 502 Command not implemented "FEAT".
2011-11-05 11:20:53 4000 3 Status: Connected
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::ResetOperation(0)
2011-11-05 11:20:53 4000 3 Trace: CControlSocket::ResetOperation(0)
2011-11-05 11:20:53 4000 3 Trace: CFileZillaEnginePrivate::ResetOperation(0)
2011-11-05 11:20:53 4000 3 Status: Retrieving directory listing...
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::ChangeDirSend()
2011-11-05 11:20:53 4000 3 Command: PWD
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 257 "/"
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::ResetOperation(0)
2011-11-05 11:20:53 4000 3 Trace: CControlSocket::ResetOperation(0)
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::ParseSubcommandResult(0)
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::ListSubcommandResult()
2011-11-05 11:20:53 4000 3 Trace: state = 1
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferSend()
2011-11-05 11:20:53 4000 3 Trace: state = 1
2011-11-05 11:20:53 4000 3 Command: TYPE I
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 200 I only have TiVo mode.
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferParseResponse()
2011-11-05 11:20:53 4000 3 Trace: code = 2
2011-11-05 11:20:53 4000 3 Trace: state = 1
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferSend()
2011-11-05 11:20:53 4000 3 Trace: state = 2
2011-11-05 11:20:53 4000 3 Command: PASV
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 227 Entering Passive Mode (192,168,254,113,129,188).
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferParseResponse()
2011-11-05 11:20:53 4000 3 Trace: code = 2
2011-11-05 11:20:53 4000 3 Trace: state = 2
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferSend()
2011-11-05 11:20:53 4000 3 Trace: state = 4
2011-11-05 11:20:53 4000 3 Command: LIST
2011-11-05 11:20:53 4000 3 Trace: CTransferSocket::OnConnect
2011-11-05 11:20:53 4000 3 Trace: CTransferSocket::OnReceive(), m_transferMode=0
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::OnReceive()
2011-11-05 11:20:53 4000 3 Response: 150 Opening data channel
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferParseResponse()
2011-11-05 11:20:53 4000 3 Trace: code = 1
2011-11-05 11:20:53 4000 3 Trace: state = 4
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::SendNextCommand()
2011-11-05 11:20:53 4000 3 Trace: CFtpControlSocket::TransferSend()
2011-11-05 11:20:53 4000 3 Trace: state = 5
2011-11-05 11:21:14 4000 3 Error: Connection timed out
2011-11-05 11:21:14 4000 3 Trace: CControlSocket::DoClose(2050)
2011-11-05 11:21:14 4000 3 Trace: CFtpControlSocket::ResetOperation(2114)
2011-11-05 11:21:14 4000 3 Trace: CControlSocket::ResetOperation(2114)
2011-11-05 11:21:14 4000 3 Trace: CFtpControlSocket::ResetOperation(2114)
2011-11-05 11:21:14 4000 3 Trace: CControlSocket::ResetOperation(2114)
2011-11-05 11:21:14 4000 3 Error: Failed to retrieve directory listing
2011-11-05 11:21:14 4000 3 Trace: CFileZillaEnginePrivate::ResetOperation(2114)
2011-11-05 11:21:18 4000 3 Trace: CControlSocket::DoClose(64)
2011-11-05 11:21:18 4000 3 Trace: CControlSocket::DoClose(64)


This is why I got in the telnet terminal.


11/05/11 17:20:35 - tyFTPd version 1.1 Startup
11/05/11 17:20:35 - Server Ready for Action
11/05/11 17:20:52 - (1184) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 17:20:52 - Now serving 192.168.254.103 port 1184
11/05/11 17:20:52 - (1184) <<< USER anonymous
11/05/11 17:20:52 - (1184) >>> 331 Please specify the password.
11/05/11 17:20:52 - (1184) <<< PASS anon@localhost
11/05/11 17:20:52 - (1184) >>> 230 Login successful.
11/05/11 17:20:52 - (1184) <<< SYST
11/05/11 17:20:52 - (1184) >>> 215 UNIX Type: L8
11/05/11 17:20:52 - (1184) <<< FEAT
11/05/11 17:20:52 - (1184) >>> 502 Command not implemented "FEAT".
11/05/11 17:20:52 - (1184) <<< PWD
11/05/11 17:20:52 - (1184) >>> 257 "/"
11/05/11 17:20:52 - (1184) <<< TYPE I
11/05/11 17:20:52 - (1184) >>> 200 I only have TiVo mode.
11/05/11 17:20:52 - (1184) <<< PASV
11/05/11 17:20:52 - Updating the server ip address from 127,0,0,1...
11/05/11 17:20:52 - Entering Passive mode for ip=192,168,254,113 port=33212
11/05/11 17:20:52 - (1184) >>> 227 Entering Passive Mode (192,168,254,113,129,188).
11/05/11 17:20:52 - In Pasv Accept with {sock16} {sock20} {192.168.254.103} {1185}
11/05/11 17:20:52 - (1184) <<< LIST
11/05/11 17:20:52 - Data Socket Connected
11/05/11 17:20:52 - Suspending timeout for port 1184
11/05/11 17:20:52 - (1184) >>> 150 Opening data channel

This is the application log while I was trying different FTP clients so I don't know which goes with the above. I tried making a fresh log but I can no longer get the application to log even pasting in the command tyftpd.tcl -l -v -d &.


11/05/11 16:50:33 - tyFTPd version 1.1 Startup
11/05/11 16:50:33 - Server Ready for Action
11/05/11 16:52:30 - (1049) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 16:52:30 - Now serving 192.168.254.103 port 1049
11/05/11 16:52:30 - (1049) <<< USER anonymous
11/05/11 16:52:30 - (1049) >>> 331 Please specify the password.
11/05/11 16:52:30 - (1049) <<< PASS leech@nowhere.org
11/05/11 16:52:30 - (1049) >>> 230 Login successful.
11/05/11 16:52:30 - (1049) <<< REST 1
11/05/11 16:52:30 - (1049) >>> 350 Restart okay, awaiting file retrieval request.
11/05/11 16:52:30 - (1049) <<< REST 0
11/05/11 16:52:30 - (1049) >>> 350 Restart okay, awaiting file retrieval request.
11/05/11 16:52:30 - (1049) <<< SYST
11/05/11 16:52:30 - (1049) >>> 215 UNIX Type: L8
11/05/11 16:52:30 - (1049) <<< PWD
11/05/11 16:52:30 - (1049) >>> 257 "/"
11/05/11 16:52:30 - (1049) <<< PORT 192,168,254,103,4,26
11/05/11 16:52:30 - (1049) >>> 200 PORT OK
11/05/11 16:52:30 - (1049) <<< TYPE A
11/05/11 16:52:30 - (1049) >>> 200 I only have TiVo mode.
11/05/11 16:52:30 - (1049) <<< LIST
11/05/11 16:52:30 - Data Socket Connected
11/05/11 16:52:30 - Suspending timeout for port 1049
11/05/11 16:52:30 - (1049) >>> 150 Opening data channel
11/05/11 17:00:33 - (1145) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 17:00:33 - Now serving 192.168.254.103 port 1145
11/05/11 17:00:33 - (1145) <<< USER anonymous
11/05/11 17:00:33 - (1145) >>> 331 Please specify the password.
11/05/11 17:00:33 - (1145) <<< PASS anon@localhost
11/05/11 17:00:33 - (1145) >>> 230 Login successful.
11/05/11 17:00:33 - (1145) <<< SYST
11/05/11 17:00:33 - (1145) >>> 215 UNIX Type: L8
11/05/11 17:00:33 - (1145) <<< FEAT
11/05/11 17:00:33 - (1145) >>> 502 Command not implemented "FEAT".
11/05/11 17:00:33 - (1145) <<< PWD
11/05/11 17:00:33 - (1145) >>> 257 "/"
11/05/11 17:00:33 - (1145) <<< TYPE I
11/05/11 17:00:33 - (1145) >>> 200 I only have TiVo mode.
11/05/11 17:00:33 - (1145) <<< PASV
11/05/11 17:00:33 - Updating the server ip address from 127,0,0,1...
11/05/11 17:00:33 - Entering Passive mode for ip=192,168,254,113 port=33206
11/05/11 17:00:33 - (1145) >>> 227 Entering Passive Mode (192,168,254,113,129,182).
11/05/11 17:00:33 - (1145) <<< LIST
11/05/11 17:00:33 - In Pasv Accept with {sock20} {sock22} {192.168.254.103} {1146}
11/05/11 17:00:33 - checkDataSock waiting
11/05/11 17:00:33 - Data Socket Connected
11/05/11 17:00:33 - Suspending timeout for port 1145
11/05/11 17:00:33 - (1145) >>> 150 Opening data channel
11/05/11 17:01:48 - (1148) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 17:01:48 - Now serving 192.168.254.103 port 1148
11/05/11 17:01:48 - (1148) <<< USER anonymous
11/05/11 17:01:48 - (1148) >>> 331 Please specify the password.
11/05/11 17:01:48 - (1148) <<< PASS anon@localhost
11/05/11 17:01:48 - (1148) >>> 230 Login successful.
11/05/11 17:01:48 - (1148) <<< PWD
11/05/11 17:01:48 - (1148) >>> 257 "/"
11/05/11 17:01:48 - (1148) <<< TYPE I
11/05/11 17:01:48 - (1148) >>> 200 I only have TiVo mode.
11/05/11 17:01:48 - (1148) <<< PASV
11/05/11 17:01:48 - Entering Passive mode for ip=192,168,254,113 port=33207
11/05/11 17:01:48 - (1148) >>> 227 Entering Passive Mode (192,168,254,113,129,183).
11/05/11 17:01:48 - (1148) <<< LIST
11/05/11 17:01:48 - In Pasv Accept with {sock23} {sock25} {192.168.254.103} {1149}
11/05/11 17:01:48 - checkDataSock waiting
11/05/11 17:01:48 - Data Socket Connected
11/05/11 17:01:48 - Suspending timeout for port 1148
11/05/11 17:01:48 - (1148) >>> 150 Opening data channel
11/05/11 17:02:55 - (1150) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 17:02:55 - Now serving 192.168.254.103 port 1150
11/05/11 17:02:55 - (1150) <<< USER anonymous
11/05/11 17:02:55 - (1150) >>> 331 Please specify the password.
11/05/11 17:02:55 - (1150) <<< PASS anon@localhost
11/05/11 17:02:55 - (1150) >>> 230 Login successful.
11/05/11 17:02:55 - (1150) <<< PWD
11/05/11 17:02:55 - (1150) >>> 257 "/"
11/05/11 17:02:55 - (1150) <<< TYPE I
11/05/11 17:02:55 - (1150) >>> 200 I only have TiVo mode.
11/05/11 17:02:55 - (1150) <<< PASV
11/05/11 17:02:55 - Entering Passive mode for ip=192,168,254,113 port=33208
11/05/11 17:02:55 - (1150) >>> 227 Entering Passive Mode (192,168,254,113,129,184).
11/05/11 17:02:55 - (1150) <<< LIST
11/05/11 17:02:55 - In Pasv Accept with {sock26} {sock28} {192.168.254.103} {1151}
11/05/11 17:02:55 - checkDataSock waiting
11/05/11 17:02:55 - Data Socket Connected
11/05/11 17:02:55 - Suspending timeout for port 1150
11/05/11 17:02:55 - (1150) >>> 150 Opening data channel
11/05/11 17:03:40 - (1152) >>> 220 tyFTPd 1.1 FTP server ready.
11/05/11 17:03:40 - Now serving 192.168.254.103 port 1152
11/05/11 17:03:40 - (1152) <<< USER anonymous
11/05/11 17:03:40 - (1152) >>> 331 Please specify the password.
11/05/11 17:03:40 - (1152) <<< PASS anon@localhost
11/05/11 17:03:40 - (1152) >>> 230 Login successful.
11/05/11 17:03:40 - (1152) <<< PWD
11/05/11 17:03:40 - (1152) >>> 257 "/"
11/05/11 17:03:40 - (1152) <<< TYPE I
11/05/11 17:03:40 - (1152) >>> 200 I only have TiVo mode.
11/05/11 17:03:40 - (1152) <<< PASV
11/05/11 17:03:40 - Entering Passive mode for ip=192,168,254,113 port=33209
11/05/11 17:03:40 - (1152) >>> 227 Entering Passive Mode (192,168,254,113,129,185).
11/05/11 17:03:40 - In Pasv Accept with {sock29} {sock31} {192.168.254.103} {1153}
11/05/11 17:03:40 - (1152) <<< LIST
11/05/11 17:03:40 - Data Socket Connected
11/05/11 17:03:40 - Suspending timeout for port 1152
11/05/11 17:03:40 - (1152) >>> 150 Opening data channel
11/05/11 17:04:22 - Too many connections - 192.168.254.103 port 1154 refused
11/05/11 17:04:22 - (1154) >>> 421 Already serving maximum clients 5.
11/05/11 17:04:22 - Disconnected 192.168.254.103 port 1154
11/05/11 17:04:27 - Too many connections - 192.168.254.103 port 1155 refused
11/05/11 17:04:27 - (1155) >>> 421 Already serving maximum clients 5.
11/05/11 17:04:27 - Disconnected 192.168.254.103 port 1155

Jamie
11-05-2011, 06:52 PM
You probably should turn off any timeouts you have enabled in your ftp client: link for Filezilla. (http://forum.hostek.com/showthread.php?230-How-to-disable-FTP-timeout-in-Filezilla).

jkozee
11-07-2011, 11:37 PM
Not sure what you have going on. I just tested with 243 files on my thd running the latest FileZilla and had no major issues. Although it seems slow in PASV mode. Have you tried running the mfs_ftp_helper command directly to see if there are mfs issues reported and that you are seeing output. As for not logging, make sure you don't have any other instances running, as a new instance I believe dies silently.

Roger Dylan
02-14-2012, 03:46 AM
Anybody given any thought to interfacing this nice piece of work with Movieloader?

RandC
02-15-2012, 08:13 PM
Anybody given any thought to interfacing this nice piece of work with Movieloader?

stop mfs_ftp if running
stop MovieLoader

edit tyftpd.tcl
approx line # 1228 providing I have latest release
change to "set ServerData(Port) 3105"
save file

start tyftpd.tcl
start MovieLoader and wait for refresh, and get to transferring :)

Roger Dylan
02-18-2012, 09:10 AM
I thought about that and said to myself: "Self, it can't be that simple."

Thanks.

Roger Dylan
02-18-2012, 10:00 AM
I'm hesitant to say it is an mfs_ftp replacement because it wasn't written to be, and I don't want to take anything away from riley.Can't. He was there when the page was blank and nobody can ever match what he did, just as he couldn't match what tridge and the others did to make his work possible. Primitive environments and they were the pioneers.

But here we are.

How is tyftpd.tcl (with mfs_ftp_helper) NOT a complete replacement for mfs_ftp.tcl (and its required support elements)? I use this stuff sporadically over periods of time and forget everything in between so I ask this question honestly: since tyftpd.tcl also apparently works with MovieLoader, why isn't this the tool to go to, now?

Roger Dylan
02-19-2012, 03:58 AM
... and get to transferring :)Again, thanks very much. It was "as easy as that" in terms of from .tmf file on computer hard disk to DTivo. Can't quite get MovieLoader to work DTivo-to-DTivo yet, starts setting up the transfer promisingly but aborts with a "425 Can't open data connection" error.

I'm also getting "unparseable" errors on dates from MovieLoader, I think caused by the inclusion of the fsid in the filename. Are you seeing that?

The thing I like most about tyftpd.tcl is that it's robust, transfers tmf files that fail with mfs_ftp.tcl and even transfers some (at least) toxic tmf files that have caused reboots in the past.

RandC
02-20-2012, 11:24 PM
Can't quite get MovieLoader to work DTivo-to-DTivo yet, starts setting up the transfer promisingly but aborts with a "425 Can't open data connection" error.

I'm also getting "unparseable" errors on dates from MovieLoader, I think caused by the inclusion of the fsid in the filename. Are you seeing that?

Fired up MovieLoader and tyftpd. I am getting the "425 Can't open data connection" and "550 Error: No reply from mfs_ftp_helper."

No errors on MovieLoader cache log file

transfer file log below


2/20/12 6:01:32 PM: Copying {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
2/20/12 6:01:32 PM: from ftp://192.168.0.112:3105/tmf
2/20/12 6:01:32 PM: to ftp://192.168.0.111:3105/tmf
< (TestTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (TestTiVo:3105) USER anonymous
< (TestTiVo:3105) 331 Please specify the password.
> (TestTiVo:3105) PASS xxxxxxxx
< (TestTiVo:3105) 230 Login successful.
> (TestTiVo:3105) CWD /tmf
< (TestTiVo:3105) 250 Directory successfully changed.
< (PortasTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (PortasTiVo:3105) USER anonymous
< (PortasTiVo:3105) 331 Please specify the password.
> (PortasTiVo:3105) PASS xxxxxxxx
< (PortasTiVo:3105) 230 Login successful.
> (PortasTiVo:3105) CWD /tmf
< (PortasTiVo:3105) 250 Directory successfully changed.
> (PortasTiVo:3105) TYPE I
< (PortasTiVo:3105) 200 I only have TiVo mode.
> (PortasTiVo:3105) PASV
< (PortasTiVo:3105) 227 Entering Passive Mode (192,168,0,111,15,230).
> (TestTiVo:3105) TYPE I
< (TestTiVo:3105) 200 I only have TiVo mode.
> (TestTiVo:3105) PORT 192,168,0,111,15,230
< (TestTiVo:3105) 200 PORT OK
> (TestTiVo:3105) RETR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (TestTiVo:3105) 150 Opening BINARY mode data connection for /tmf/{Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf.
> (PortasTiVo:3105) STOR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (PortasTiVo:3105) 425 Can't open data connection
> (TestTiVo:3105) QUIT
< (TestTiVo:3105) 550 Error: No reply from mfs_ftp_helper.
> (PortasTiVo:3105) QUIT
< (PortasTiVo:3105) 221 Goodbye.
2/20/12 6:01:50 PM: java.io.IOException: Error from remote store: 425 Can't open data connection

crashHD
05-09-2012, 05:52 PM
Excellent tool! I tried many times, but was never able to successfully install mfs_ftp without causing errors or reboots. This works great!

What is the preferred format to use if you want to later on reinsert? tmf or tmf+? (for that matter, what is the difference between the two?)

crashHD
05-09-2012, 09:41 PM
Attempting to run on a DTivo with version 6.2a gives the following result:

bash# tyftpd.tcl -s
bad option "repeat": must be compare, first, index, last, length, match, range,
tolower, toupper, trim, trimleft, trimright, wordend, or wordstart
while executing
"string repeat "#" 512"
(file ".//tyftpd.tcl" line 1291)
bash#

On a 6.4a machine, it runs just fine, without error.

Smee
06-01-2012, 12:07 AM
Fired up MovieLoader and tyftpd. I am getting the "425 Can't open data connection" and "550 Error: No reply from mfs_ftp_helper."

No errors on MovieLoader cache log file

transfer file log below


2/20/12 6:01:32 PM: Copying {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
2/20/12 6:01:32 PM: from ftp://192.168.0.112:3105/tmf
2/20/12 6:01:32 PM: to ftp://192.168.0.111:3105/tmf
< (TestTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (TestTiVo:3105) USER anonymous
< (TestTiVo:3105) 331 Please specify the password.
> (TestTiVo:3105) PASS xxxxxxxx
< (TestTiVo:3105) 230 Login successful.
> (TestTiVo:3105) CWD /tmf
< (TestTiVo:3105) 250 Directory successfully changed.
< (PortasTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (PortasTiVo:3105) USER anonymous
< (PortasTiVo:3105) 331 Please specify the password.
> (PortasTiVo:3105) PASS xxxxxxxx
< (PortasTiVo:3105) 230 Login successful.
> (PortasTiVo:3105) CWD /tmf
< (PortasTiVo:3105) 250 Directory successfully changed.
> (PortasTiVo:3105) TYPE I
< (PortasTiVo:3105) 200 I only have TiVo mode.
> (PortasTiVo:3105) PASV
< (PortasTiVo:3105) 227 Entering Passive Mode (192,168,0,111,15,230).
> (TestTiVo:3105) TYPE I
< (TestTiVo:3105) 200 I only have TiVo mode.
> (TestTiVo:3105) PORT 192,168,0,111,15,230
< (TestTiVo:3105) 200 PORT OK
> (TestTiVo:3105) RETR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (TestTiVo:3105) 150 Opening BINARY mode data connection for /tmf/{Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf.
> (PortasTiVo:3105) STOR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (PortasTiVo:3105) 425 Can't open data connection
> (TestTiVo:3105) QUIT
< (TestTiVo:3105) 550 Error: No reply from mfs_ftp_helper.
> (PortasTiVo:3105) QUIT
< (PortasTiVo:3105) 221 Goodbye.
2/20/12 6:01:50 PM: java.io.IOException: Error from remote store: 425 Can't open data connection

Has anyone actually used it with MovieLoader before? It would be really cool if they did. mfs_ftp is really slow.

Smee

Roger Dylan
06-05-2012, 05:22 AM
Has anyone actually used it with MovieLoader before? It would be really cool if they did.Yeah. As I said, it works fine for me hard drive to Tivo, but I can't get it to work Tivo to Tivo.

Smee
06-05-2012, 10:21 PM
Could it be Passive mode? Can that be switched off? I'll have to mess with it when I have a chance and see if that's it.

Take care..

Smee

Smee
06-07-2012, 01:33 AM
OK, so I think I might know why this isn't working with MovieLoader. Well, sorta. I did get it to work, just not between Tivos. I tried Tivo-Tivo, and this is the logs between the Tivo initiating the request and the Tivo trying to serve the file.

Tivo requesting the program is 172.16.0.38


06/07/12 03:59:01 - (3341) >>> 220 tyFTPd 1.1 FTP server ready.
06/07/12 03:59:05 - (3341) <<< USER anonymous
06/07/12 03:59:05 - (3341) >>> 331 Please specify the password.
06/07/12 03:59:05 - (3341) <<< PASS xxxxxxxx
06/07/12 03:59:05 - (3341) >>> 230 Login successful.
06/07/12 03:59:05 - (3341) <<< CWD /tmf
06/07/12 03:59:05 - (3341) >>> 250 Directory successfully changed.
06/07/12 03:59:05 - (3341) <<< TYPE I
06/07/12 03:59:05 - (3341) >>> 200 I only have TiVo mode.
06/07/12 03:59:05 - (3341) <<< PASV
06/07/12 03:59:05 - (3341) >>> 227 Entering Passive Mode (172,16,0,38,15,21).
06/07/12 03:59:06 - (3341) <<< STOR {The Legend of Korra}{2012-06-02}{When Extremes Meet}{10.00 AM Sat Jun 02, 2012}{NIKe}.tmf
06/07/12 03:59:06 - checkDataSock waiting
06/07/12 03:59:07 - checkDataSock waiting
06/07/12 03:59:07 - checkDataSock waiting
06/07/12 03:59:08 - checkDataSock waiting
06/07/12 03:59:08 - checkDataSock waiting
06/07/12 03:59:09 - checkDataSock waiting
06/07/12 03:59:09 - checkDataSock waiting
06/07/12 03:59:10 - checkDataSock waiting
06/07/12 03:59:10 - checkDataSock waiting
06/07/12 03:59:11 - checkDataSock waiting
06/07/12 03:59:11 - checkDataSock waiting
06/07/12 03:59:12 - checkDataSock waiting
06/07/12 03:59:12 - checkDataSock waiting
06/07/12 03:59:13 - checkDataSock waiting
06/07/12 03:59:13 - checkDataSock waiting
06/07/12 03:59:14 - checkDataSock waiting
06/07/12 03:59:14 - checkDataSock waiting
06/07/12 03:59:15 - checkDataSock waiting
06/07/12 03:59:15 - checkDataSock waiting
06/07/12 03:59:16 - checkDataSock waiting
06/07/12 03:59:16 - checkDataSock waiting
06/07/12 03:59:17 - checkDataSock waiting
06/07/12 03:59:17 - checkDataSock waiting
06/07/12 03:59:18 - checkDataSock waiting
06/07/12 03:59:18 - checkDataSock waiting
06/07/12 03:59:19 - checkDataSock waiting
06/07/12 03:59:19 - checkDataSock waiting
06/07/12 03:59:20 - checkDataSock waiting
06/07/12 03:59:20 - checkDataSock waiting
06/07/12 03:59:21 - checkDataSock waiting
06/07/12 03:59:21 - checkDataSock waiting
06/07/12 03:59:21 - (3341) >>> 425 Can't open data connection
06/07/12 03:59:21 - (3341) <<< QUIT
06/07/12 03:59:21 - (3341) >>> 221 Goodbye.


On the Tivo that has the show I want to pull is 172.16.0.36:

06/07/12 03:58:57 - (3340) >>> 220 tyFTPd 1.1 FTP server ready.
06/07/12 03:59:01 - (3340) <<< USER anonymous
06/07/12 03:59:01 - (3340) >>> 331 Please specify the password.
06/07/12 03:59:01 - (3340) <<< PASS xxxxxxxx
06/07/12 03:59:01 - (3340) >>> 230 Login successful.
06/07/12 03:59:01 - (3340) <<< CWD /tmf
06/07/12 03:59:01 - (3340) >>> 250 Directory successfully changed.
06/07/12 03:59:06 - (3340) <<< TYPE I
06/07/12 03:59:06 - (3340) >>> 200 I only have TiVo mode.
06/07/12 03:59:06 - (3340) <<< PORT 172,16,0,38,15,21
06/07/12 03:59:06 - (3340) >>> 200 PORT OK
06/07/12 03:59:06 - (3340) <<< RETR {The Legend of Korra}{2012-06-02}{When Extremes Meet}{10.00 AM Sat Jun 02, 2012}{NIKe}.tmf
06/07/12 03:59:06 - Opening BINARY mode data connection for /tmf/{The Legend of Korra}{2012-06-02}{When Extremes Meet}{10.00 AM Sat Jun 02, 2012}{NIKe}.tmf.
06/07/12 03:59:06 - (3340) >>> 150 Opening BINARY mode data connection for /tmf/{The Legend of Korra}{2012-06-02}{When Extremes Meet}{10.00 AM Sat Jun 02, 2012}{NIKe}.tmf.
06/07/12 03:59:06 - (3340) >>> 550 Error - token count incorrect.
06/07/12 03:59:22 - (3340) <<< QUIT
06/07/12 03:59:22 - (3340) >>> 221 Goodbye.


When I tried pulling the shows off of my PC, MovieLoader had no issues and worked very well. So what I think is that tyFTPd does not have fxp implemented or at least not implemented right. That's where MovieLoader tries to set up an FTP session between the two Tivos so that doesn't have to serve as the proxy for the data transfer. Someone who knows more about how to trace could say for sure. Look here (http://dealdatabase.com/forum/printthread.php?t=51987&pp=15&page=48)for one thread talking about that. Real bummer. I also tried a few other things to see if tyftpd & MovieLoader would work together by setting multiple instances on each Tivo using ports 5013 & 3105, but no dice.

Take care..

Smee

jkozee
06-07-2012, 09:43 AM
I've never tried MovieLoader, but from the log that you posted above you are trying to pull a show that tyFTPd can't find on the tivo acting as the server. The "550 Error - token count incorrect." indicates that you are not passing a filename that tyFTPd expects. The filename is constructed as {Title}{Short Date}{EpisodeTitle}{Long Date}{ChannelCallSign}{FSID}.extension. The show you are requesting doesn't have the FSID included. This format was an option in mfs_ftp.tcl I believe, but not the default. The problem is that if you don't include the FSID you cannot be sure that you are working with the show you intend. For example if you have two partial recordings for the same showing, then without the FSID incluced, you will have two identical shows listed with the LIST command, but RETR and DELE would only operate on one of them. The STOR function ignores that FSID, so it doesn't care if it is included or not on the upload side.

So, the first step to getting MovieLoader working is to make sure that it using the same filenames as tyFTPd. If you want to change the format of the filenames in tyFTPd, you will need to make those changes in the helper app and re-compile it. I'm not sure where MovieLoader is getting it's filenames from, I would have thought it would get them from tyFTPd running on the server tivo. If you are tring to mix tyFTPd and and mfs_ftp for trouble shooting PASV on tyFTPd, then chage the format of the file listings for mfs_ftp. IIRC, there is a matching format in mfs_ftp, but I'm not sure if it a command line option or if you need to edit the file directly.

jkozee
06-07-2012, 01:41 PM
A couple of quick things to try:

Remove these lines from proc PasvConnect to prevent IP checks:

# Verify this is from who it should be
if {![string equal $ip $ClientData(ip)]} then {
# Nope, so close it and wait some more
logit 1 "Pasv Accept rejected on ip grounds"
close $sock
return
}


Also, you can increase the timeout that proc checkDataSock has by changing the line:

if {$cnt > 30} {

to a higer value. As it is written it will only wait for about 15 seconds, try

if {$cnt > 300} {

jkozee
06-07-2012, 07:29 PM
It's also worth noting that I also changeg the {Long Date} format to have the time seaprated by "." instead of ":"

Smee
06-07-2012, 09:25 PM
It's also worth noting that I also changeg the {Long Date} format to have the time seaprated by "." instead of ":"

Oh wow. I didn't even notice the name part. So that certainly is a problem. So from what I see, this is the name that tyFTPd is serving:


{American Digger}{2012-06-06}{Buried Bucks in the Bayou}{09.30 PM Wed Jun 06, 2012}{Spike}{79795}.tmf

And this is what mfs_ftp serves as a name:


{American Digger}{2012-06-06}{Buried Bucks in the Bayou}{09.30 PM Wed Jun 06, 2012}{Spike}.tmf

I can see the FSID at the end like you said, but it seems like MoveLoader is parsing through the name and rebuilding and since fsid is not what it expects, it dumps it. That may explain the error I am seeing in MovieLoader that I just noticed as well and I think someone here also brought it up before. For every file that it combs through as part of its discovery, it is dropping this error:


java.text.ParseException: Unparseable date: "PM Wed Jun 06, 2012"
at java.text.DateFormat.parse(Unknown Source)
at MediaFile.parseName(MediaFile.java:304)
at MediaFile.<init>(MediaFile.java:130)
at FtpMediaFile.<init>(FtpMediaFile.java:61)
at FtpMediaFolder.update(FtpMediaFolder.java:107)
at VirtualMediaFolder.update(VirtualMediaFolder.java:47)
at CacheUpdater.perform(CacheUpdater.java:79)
at FtpThread.run(FtpThread.java:131)

The error above is for the file I included at the beggining. I don't know about the : that you mention though as I don't see it in the output I posted. I am going to try the changes for the connection you posted above but I don't think that is going to make a difference. Are the sources for tyFTPd available somewhere in order to change the filename output?

Thanks...

Smee

RandC
06-07-2012, 09:44 PM
Fired up MovieLoader and tyftpd. I am getting the "425 Can't open data connection" and "550 Error: No reply from mfs_ftp_helper."

No errors on MovieLoader cache log file

transfer file log below


2/20/12 6:01:32 PM: Copying {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
2/20/12 6:01:32 PM: from ftp://192.168.0.112:3105/tmf
2/20/12 6:01:32 PM: to ftp://192.168.0.111:3105/tmf
< (TestTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (TestTiVo:3105) USER anonymous
< (TestTiVo:3105) 331 Please specify the password.
> (TestTiVo:3105) PASS xxxxxxxx
< (TestTiVo:3105) 230 Login successful.
> (TestTiVo:3105) CWD /tmf
< (TestTiVo:3105) 250 Directory successfully changed.
< (PortasTiVo:3105) 220 tyFTPd 1.0 FTP server ready.
> (PortasTiVo:3105) USER anonymous
< (PortasTiVo:3105) 331 Please specify the password.
> (PortasTiVo:3105) PASS xxxxxxxx
< (PortasTiVo:3105) 230 Login successful.
> (PortasTiVo:3105) CWD /tmf
< (PortasTiVo:3105) 250 Directory successfully changed.
> (PortasTiVo:3105) TYPE I
< (PortasTiVo:3105) 200 I only have TiVo mode.
> (PortasTiVo:3105) PASV
< (PortasTiVo:3105) 227 Entering Passive Mode (192,168,0,111,15,230).
> (TestTiVo:3105) TYPE I
< (TestTiVo:3105) 200 I only have TiVo mode.
> (TestTiVo:3105) PORT 192,168,0,111,15,230
< (TestTiVo:3105) 200 PORT OK
> (TestTiVo:3105) RETR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (TestTiVo:3105) 150 Opening BINARY mode data connection for /tmf/{Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf.
> (PortasTiVo:3105) STOR {Ultimate Weapons}{2012-02-16}{Sniper Rifles}{10.00 PM Thu Feb 16, 2012}{MILT}{3556164}.tmf
< (PortasTiVo:3105) 425 Can't open data connection
> (TestTiVo:3105) QUIT
< (TestTiVo:3105) 550 Error: No reply from mfs_ftp_helper.
> (PortasTiVo:3105) QUIT
< (PortasTiVo:3105) 221 Goodbye.
2/20/12 6:01:50 PM: java.io.IOException: Error from remote store: 425 Can't open data connection

{FSID} is definitely listed in STOR command for me.

jkozee
06-07-2012, 10:01 PM
I think the filename missing the FSID is only due to mixing both apps. I was able to get it working using tyFTPd as the client and server, with the changes I posted earlier, and allowing it to build a new cache. As for the source code, everyting is included in the download. tyFTPd.tcl is a tcl script and can be edited directly. However, the source for mfs_ftp_helper is in the form of a patch file that needs to be applied against the unified mfs tools and will need to be compiled using a cross compiler or native mips compiler. This is where the filenames come from.

You could edit the MovieLoader source to allow for the different {Long Date} format differences. Also, I would think you could have it only process the tmf directory, and get the xml files from the xml directory instead of having it download a partial tmf to extact it.

EDIT: Oh, wait I just noticed the post with both {Long Dates} being the same from both mfs_ftp and tyFTPd. Maybe someone can look at the JAVA code for MovieLoader and see about allowing for the FSID.

jkozee
06-07-2012, 10:20 PM
crashHD, as memtioned in the first thread, the tmf+ was my attempt to solve the bookmark issue. When a tmf is transfered, each part is allocated as it arrives from the master block info of that part. The tmf is a tar file with the xml file and then each partNN.ty file. Then tmf+ is a tar file with the xml then the master block info of each ty file, then each ty file. This would allow one to allocate the all of the parts first. I believe this worked on S2, but not on the THD, but I no longer have an S2 to verify this behavior.

Smee
06-07-2012, 10:46 PM
I think the filename missing the FSID is only due to mixing both apps. I was able to get it working using tyFTPd as the client and server, with the changes I posted earlier, and allowing it to build a new cache. As for the source code, everyting is included in the download. tyFTPd.tcl is a tcl script and can be edited directly. However, the source for mfs_ftp_helper is in the form of a patch file that needs to be applied against the unified mfs tools and will need to be compiled using a cross compiler or native mips compiler. This is where the filenames come from.

You could edit the MovieLoader source to allow for the different {Long Date} format differences. Also, I would think you could have it only process the tmf directory, and get the xml files from the xml directory instead of having it download a partial tmf to extact it.

EDIT: Oh, wait I just noticed the post with both {Long Dates} being the same from both mfs_ftp and tyFTPd. Maybe someone can look at the JAVA code for MovieLoader and see about allowing for the FSID.

Well, for me it will be a long while before I can figure out how to set this up in eclipse and compile. I have never coded in java before. :-/ I did see this in MediaFile.java which looks like it may be where it is parsing out the name of the file it is reading:


{
Matcher matcher = MFSFTP_FORMAT.matcher( name );
if ( matcher.matches() )
{
try
{
setTitle( matcher.group( 1 ));
setOriginalAirDate( originalAirDateFormat.parse( matcher.group( 2 )));
setEpisode( matcher.group( 3 ));
setRecordedDate( recordedDateFormat.parse( matcher.group( 4 )));
station = matcher.group( 5 );
if ( "{}".equals( station ))
station = "";
}
catch ( Exception e )
{
}
}

I'm going to post over in the MovieLoader thread, but that has been a dead thread since 2008.

Thanks...

Smee

jkozee
06-08-2012, 12:00 AM
You should still be able to get tyFTPd working with MovieLoader with the changes I suggested earlier. You might need to delete the chache directories and allow them to be rebuilt, but it did work for me.

Roger Dylan
06-08-2012, 04:38 AM
You should still be able to get tyFTPd working with MovieLoader with the changes I suggested earlier. You might need to delete the chache directories and allow them to be rebuilt, but it did work for me.THANKS!

That worked fine.

I made the changes in message #53, but not the "." for ":" substitution for Long Date in the following message, and for the first time, tyftpd worked with MovieLoader Tivo-to-Tivo, which was what I really needed (always HAS worked from hard drive to Tivo). This was HR10-250 to HR10-250.

This is great!!

DealDataBase.com LIVES!!!

RandC
06-08-2012, 06:15 PM
any way to stop tyFTPd without causing a reboot?

Smee
06-08-2012, 11:11 PM
You should still be able to get tyFTPd working with MovieLoader with the changes I suggested earlier. You might need to delete the chache directories and allow them to be rebuilt, but it did work for me.

Woohoo!! Changes worked. This is awesome. Best thing is, it doesn't slow down my TiVo like mfs_ftp. :-) One thing though, do you know why the recordings all show up with a duration of 0:00 (Partial)??

Thanks again!!

Smee


Many thanks!!!

Smee
06-09-2012, 04:45 PM
One thing though, do you know why the recordings all show up with a duration of 0:00 (Partial)??

I just noticed that when I initiate the transfer, the duration is listed correctly, but after it completes, it is set to 0:00. I was looking at tyftpd.tcl, but really don't see why this would be happening. I see where it sets it, but not why it is resetting it to back to 0:00 after the transfer completes. Any ideas?

Take care...

Smee

jkozee
06-09-2012, 09:17 PM
any way to stop tyFTPd without causing a reboot?

Why on earth would you want to stop such a kewl program:D No, seriously, I usually just disconnect the clients and issue a kill <pid> to get rid of the server and haven't had any reboot issues. How are you stopping it when you get a reboot and what platform. I only have THD's to test with.

crashHD
06-09-2012, 10:52 PM
Woohoo!! Changes worked. This is awesome. Best thing is, it doesn't slow down my TiVo like mfs_ftp. :-) One thing though, do you know why the recordings all show up with a duration of 0:00 (Partial)??
!

I've noticed that too. They also loose the station affiliation to which station they were recording from. It just says "tyFTPd" where it would normally have the station callsign.

Smee
06-10-2012, 10:03 AM
I've noticed that too. They also loose the station affiliation to which station they were recording from. It just says "tyFTPd" where it would normally have the station callsign.

Well, I think the Station stuff was never implemented. Looking at line 1198 I see this in tyftpd.tcl:


# TODO - We could add a function to locate the real first station based on the CallSign
# Create the dummy Station object
set station [db $ServerData(db) create Station]
dbobj $station set CallSign "tyftpd"
dbobj $station set Name "tyftpd"

So on mine, and I am sure yours, the station always remains tyftpd. The CallSign data is in the XML so I don't understand the comment. jkozee must have had something else in mind.

Take care..

Smee

RandC
06-10-2012, 10:44 AM
Why on earth would you want to stop such a kewl program:D No, seriously, I usually just disconnect the clients and issue a kill <pid> to get rid of the server and haven't had any reboot issues. How are you stopping it when you get a reboot and what platform. I only have THD's to test with.
I have been starting the program via telnet, when I close out the telnet window the TiVo reboots. Using directv hr10-250 6.4a SW, I will try to kill <pid> then close telnet window and see what happens.

jkozee
06-10-2012, 11:11 AM
How aer you calling the command when you run it from a window. IIRC, if you are running ANY tcl script in the foreground of a telnet window, and you close the windows without killing the process (ctrl-c), then the unit will reboot. This may have nothing to do with tyFTPd.

jkozee
06-10-2012, 11:16 AM
Smee is correct, the station isn't implemented. This is the same behavior as mfs_ftp IIRC. In order to do stations properly, one would have to search the unit and try to locate the station by callsign and then use that object exists, or else create a dummy one. I don't even think that tivo's own TivoDesktop bother with stations, but it would be trivial to add if one was so inclined.

RandC
06-10-2012, 05:40 PM
How aer you calling the command when you run it from a window. IIRC, if you are running ANY tcl script in the foreground of a telnet window, and you close the windows without killing the process (ctrl-c), then the unit will reboot. This may have nothing to do with tyFTPd.you are correct sir!! I was running ./tyftpd.tcl -l -d & command prompt and when I closed the window it rebooted. I confirmed killing <pid> before closing window keeps the TiVo running, also confirmed ./tyftpd.tcl -l command and then (ctrl-c) to kill process also keeps the TiVo alive. Thanks jkozee

Smee
06-10-2012, 07:27 PM
jkozee,

Do you know why the recordings all show up with a duration of 0:00 (Partial) after the transfer completes even though the transfer starts off with the correct duration after the transfer first starts?? I really can't figure out why it is happening. At least it is happening to me on my DTIVOs running 6.4a.

Thanks..

Smee

jkozee
06-10-2012, 08:04 PM
Smee,

No. I haven't had a chance to look at that. I don't think I touch anything but state in CleanupRecording. You can try setting the properties (startime stoptime duration etc) again there and see if that fixes it.

jkozee
06-10-2012, 10:24 PM
Smee (or anyone else for that matter),

Are you working on the MovieLoader patch for the FSID's in the filename? I haven't seen a request in that thread, so I thought you might be having a go at it yourself. I don't use MovieLoader and don't really do JAVA, but I did dowload it to see what issues it has with tyFTPd. Looks like you could add another regex pattern for the FSID case and get rid of those "Unparseable date" errors fairly easy.

I will upate tyFTPd to allow for daemonizing and will add a flag to only show the tmf directory which should improve the MovieLoader compatibility. I'll include the patch to ignore PASV ip address checks also and I just finished a porting mfs_ftp_helper to the mips.32/mips.64 unified code.

Roger Dylan
06-11-2012, 07:52 AM
While I'm having good performance with a couple of HR10-250 DirecTivos running 6.4a, tyftpd.tcl is failing to run on a Hughes DirecTivo HDVR2 running 6.2.

Message:

bad option "repeat": must be compare, first, index, last, length, match, range, tolower, toupper, trim, trimleft, trimright, wordend, or wordstart while executing "string repeat "#" 512" (file "./tyftpd.tcl" line 1291)

Remarking out the code line

set ServerData(Delim) [string repeat "#" 512]

results in tyftpd.tcl running, and my ftp client reads the contents and tries to begin a download but fails to do so.

I've tried to rewrite the code line without using the "repeat" option but my understanding of exactly what it's doing is faulty.

Any thoughts? Thanks.

jkozee
06-11-2012, 09:53 AM
Roger,

"string repeat" must not be available in the older version of tcl interpreter. The line of code is simply saying set the varible to a string of 512 "#" characters. The line could be written similar to mfs_ftp as:

set ServerData(Delim) "################################"
append ServerData(Delim) $ServerData(Delim) ; append ServerData(Delim) $ServerData(Delim)
append ServerData(Delim) $ServerData(Delim) ; append ServerData(Delim) $ServerData(Delim)


There may be other commands that aren't available, but this should get you past this one. Let me know if that works and I'll put it in with the changes I am working on.

jkozee
06-11-2012, 09:56 AM
Smee,

Does mfs_ftp.tcl also have the same "0:00 (partial)" recording status when the ftp upload completes. If not, what version/patch set are you running for mfs_ftp.tcl so I can compare them.

Roger Dylan
06-11-2012, 10:35 AM
The line of code is simply saying set the varible to a string of 512 "#" characters.I tried that, in a very gross way, and then got some kind of background error ... I'll try the exact code you suggest when I can get back to my Tivos later on.

EDITED TO ADD: With that code tyftpd.tcl runs and my ftp client sees the files, but the file transfers don't even start. Nothing immediately helpful in the logs but I'll keep working on it.

Smee
06-11-2012, 08:23 PM
Smee (or anyone else for that matter),

Are you working on the MovieLoader patch for the FSID's in the filename? I haven't seen a request in that thread, so I thought you might be having a go at it yourself. I don't use MovieLoader and don't really do JAVA, but I did dowload it to see what issues it has with tyFTPd. Looks like you could add another regex pattern for the FSID case and get rid of those "Unparseable date" errors fairly easy.


jkozee,

No, I haven't posted on the MovieLoader forums. I wanted to give it a crack, just haven't had time to set up an environment. I've not messed with Java before so I figured it would be perfect opportunity to start, but will be posting there soon anyway because of some other unrelated issue regarding menu weirdness.



I will upate tyFTPd to allow for daemonizing and will add a flag to only show the tmf directory which should improve the MovieLoader compatibility. I'll include the patch to ignore PASV ip address checks also and I just finished a porting mfs_ftp_helper to the mips.32/mips.64 unified code.

Daemonizing would be good. Thanks for that. :-)

Smee

Smee
06-11-2012, 08:39 PM
Smee,

Does mfs_ftp.tcl also have the same "0:00 (partial)" recording status when the ftp upload completes. If not, what version/patch set are you running for mfs_ftp.tcl so I can compare them.

jkozee,

No. It reports the correct duration after completion. As a matter of fact, it also updates the CallSign station number as well. Never really noticed that. I am using 1.2.9p-patchlevel-20070717b. At least that is what it reports inside mfs_ftp.tcl. I set up mfs_ftp using the instructions in dvrpedia.

Take care...

Smee

jkozee
06-11-2012, 09:57 PM
Roger - From your earlier post I would guess that "string equals" is also not supported on your platform, so you will probably need to change those lines as well.

Smee - I haven't looked at the mfs_ftp code in quite some time, but I don't think that it handles station items correctly, in that it doesn't create the required NKP items needed for later versions of the OS and I don't believe that TivoDesktop .tivo files carry the station info when they are transfered. I will try to have a look at the 0:00 partial issue and see if it is obvious to me. I am trying to wrap up the changes I have made and at least get it working for the most part with MovieLoader.

jkozee
06-13-2012, 12:26 AM
Looks like the "0:00 (partial)" issue is due to the Begin/End attributes of Parts not being set. I will add this to my update for tyFTPd. (Note: Missing Begin/End attributes for parts may also be why my attempt of pre-allocating the parts objects to avoid the bookmark juggle stuff did not work. I am not inclined to pursue it, but the tmf+ type may prove fruitful for those who may wish to do so)

Roger Dylan
06-16-2012, 03:01 AM
jkozee has posted a patched version of MovieLoader in the MovieLoader thread that seems to handle fsid's well, a big step forward.

Meanwhile, I have had little luck getting tyftpd.tcl to work on an older DTivo with 6.2, but have found that with with a patched-as-best-I-can version of mfs_ftp.tcl running on the older machine it serves MovieLoader fine with tyftpd.tcl running on the client HR10-250. Whether this gets me all the advantages of tyftpd.tcl (mainly handling toxic videos that mfs_ftp.tcl can't, and sometimes reboots or even GSODs) I have not yet tested out.

crashHD
06-16-2012, 07:14 AM
Just a point of curiosity...what's keeping your from updating that 6.2 unit?

Since 6.2a lost locals, I've been so glad to see work pick up on this, and movieloader. Thanks for your efforts guys!!

jkozee
06-24-2012, 11:16 PM
Roger, were you not able to run tyftpd after rewriting the "string equals" and "string repeat" lines? What is the "patched-as-best-I-can" version of mfs_ftp you are running. Did you have to custom patch it, I was under the impression it worked on 6.2 with the current patch set.

jkozee
06-25-2012, 11:30 PM
I have posted a new version that should provide better compatibility with MovieLoader (use the -m option). This version should also correct the "partial" status for completed recordings. I only tested this on THD, so some work may need to be done on other platforms. If you are using MovieLoader, be sure to use the patched version I posted in that thread to allow for the FSID in the filename.

Roger Dylan
06-26-2012, 09:25 AM
Roger, were you not able to run tyftpd after rewriting the "string equals" and "string repeat" lines?Once I fixed "repeat" correctly, typtp ran without error, but file transfers aborted immediately. I decided to upgrade the system on the Hughes HDVR2 instead; I had kinda hoped to just leave it alone but when I looked at it I saw the drive is too small anyway so I'll put a bigger drive in and take the occasion to upgrade the system. Hopefully that will eliminate the problem with tyftpd. I'll report the results.


What is the "patched-as-best-I-can" version of mfs_ftp you are running? Did you have to custom patch it, I was under the impression it worked on 6.2 with the current patch set.1.2.9p with the public patches. It works OK on 6.2 and even other machines with 6.4, but there are occasional problems on inserts and occasional toxic files that wreak havoc (tyftpd OTOH downloads the same video without creating toxic files, one of the reasons I would like to transition to it). I am using the patched mfs_ftp.tcl to get my content off the old HDVR2 and then I'll put in a larger drive with 6.4 and hope to live happily ever after with tyftpd.

jkozee
06-27-2012, 11:41 AM
I just noticed a bug in the UpdateParts proc, so I removed the download until I have a chance to fix and test it.

Roger Dylan
06-27-2012, 03:23 PM
[I am going to upgrade the HDVR2 to] 6.4 and hope to live happily ever after with tyftpd.Having just upgraded the HDVR2 to run 6.4a, it's now running tyftpd.tcl fine, as expected. File transfers are working both with an ftp client and via movieloader.

jkozee
06-27-2012, 07:48 PM
Is anyone trying to watch live as they are inserting files? I'm curious about how the playbar acts. Deciding if I should have another run at tmf+ file types to avoid the bjuggle that mfs_ftp did. On my THD when I am inserting a file and start watching it (tmf) it, they playbar only goes to the end of the current part that that was inserting when I started the playback. It never updates even if the xfer completes before I reach the end of the playbar.

Roger Dylan
06-28-2012, 12:03 AM
Is anyone trying to watch live as they are inserting files? I'm curious about how the playbar actsHere's the behavior I see (this is inserting into an HR10-250 from an HDVR2 via movieloader):

• The green portion of the progress bar is at a partial point (either how much had been transferred at that point or the end of the latest segment, I'm not sure).
* The green portion does not advance while I'm watching, even though the file continues to insert.
• When viewing reaches the end of the green bar the menu screen comes up asking whether to delete or not (as if the movie had come to an end).
• When you choose not delete, you then get the play screen; resume is not an option.
• When you choose play it starts from the beginning and you have to FF to viewing point; the green portion has considerably advanced.

• Repeats cycle, until movie has completely transferred; when you hit that cycle the green portion has filled the progress bar and after you FF you are allowed to watch to the end.

It's always been this way, I think, and I've gotten used to it.

crashHD
06-28-2012, 01:28 PM
Roger summed it up nicely. That was the behavior I observed when testing it out a few weeks ago. I was extracting from an HR10, and reinserting back onto itself. This was not via movieloader, though. Just tivo-->PC-->tivo

jkozee
06-28-2012, 09:57 PM
Thanks Roger for the deatiled response. This matches what I see on my THD.

I just found a bug that might cause a spontaneous reboot and possibly force a mfscheck. This code:

foreach val $meta(Series.$itm) {
dbobj $curseries add $itm $val
}

Should be:

foreach val $meta(Series.$itm) {
dbobj $curseries set $itm $val
}


Hopefully nobody experienced this but me. On the upside, tracking this bug down caused me to realize why the tmf+ files were not working as expected...

crashHD
06-30-2012, 11:52 AM
v1.2 coming back soon? ;)

jkozee
06-30-2012, 01:09 PM
Nope. 1.3 coming soon...

crashHD
06-30-2012, 01:27 PM
even better! [waits eagerly]

jkozee
07-01-2012, 01:04 AM
I have updated the first post with version 1.3. This new version fixes the bug I found and corrects the tmf+ file type processing. Using tmf+ files allows for the play bar to be populated early, fixing the problems associated with playing back while uploading. Keep in mind that the parts are allocated but not initiated to nulls when they are created. This means that forwarding past the upload point will allow previous recorded shows to display. This is expected behavior and avoids the time overhead when creating a part with streamfill. If an upload is stopped, I null out part of the stream to reduce the appearance of the "junk" in the stream. Also, for a tmf+ that aborts, I attempt to remove the remaining parts that were allocated but never uploaded, however this cannot be done if the show is being actively viewed when the upload aborts. In this case, you can expect those parts to have odd playback behaviors.

For those of you using MovieLoader, I have created a patched (http://www.dealdatabase.com/forum/showthread.php?51987-MovieLoader-HME-meets-MFS_FTP&p=315457#post315457)version that will allow for the new tmf+ file types. Your MovieLoader.properties file should have entires like this for tyftpd using tmf+ files:

FriendlyName=ftp://tivo_ip:5013/tmf+/

crashHD
07-01-2012, 08:37 AM
helper is no longer 32bit/64bit specific? I notice only one mfs_ftp_helper file in the distribution.

edit: Went ahead and tried it. Not a programmer, so I didn't know what "unified" meant when discussing mfs_ftp_helper. Turns out to mean exactly what it sounds like. Works fine so far. Haven't tried setting up Movieloader yet, but it's on the schedule for the day...

crashHD
07-01-2012, 11:34 AM
Is it necessary to use port 3105 to use movieloader?

I put port 5013 in movieloader.properties for each tivo, and movieloader loaded and cached each tivo, running tyftpd on port 5013.

Attempted transfers fail, using (correctly) port 5013 as the source, but attempting to push (incorrectly) to port 3105.

Does it make a difference if I change tyftpd to use port 3105, or movieloader to push to port 5013 (is the latter even possible)?

jkozee
07-01-2012, 11:54 AM
You can use the -p 3105 to use port 3105. I don't actually use MovieLoader, but from your description it sounds like it assumes port 3105 for push and I doubt you could correct it there without changing the source code. Just have tyftpd use port 3105 and update movieloader.properties.

crashHD
07-01-2012, 12:00 PM
That's what I was planning on trying, when I saw this snippet:


# MovieLoader: If you are using this with MovieLoader make sure you are using #
# MovieLoader.v0.17c.zip or later. Update your config files to: #
# NAME=ftp://ip.address:5013/tmf+/ #
# You can also use the -m or -M flag to limit the directories. #
# #
################################################################################

and didn't know if that meant movieloader should run on that port.

FWIW, movieloader set up that way does use 5013 for the source, and I was able to push to the ftp server on my PC. movieloader must have 3105 hard coded as the target port for sending to a tivo.

I'm off to set tyftpd to port 3105 and test that out.

crashHD
07-01-2012, 12:04 PM
what's the best way to shut it down? mfs_ftp always had the possibility to reboot the box if shutdown incorrectly, but I see no "shutdown.txt" to download here

Roger Dylan
07-01-2012, 12:46 PM
So far everything looks good. I set up all my Tivos with the new tyftpd.tcl, and set up movieloader on the server. First try: Tivo-Tivo; it filled the progress bar with green. No interruptions. When the movie completed it was listed with the appropriate duration. Great. Doesn't change anything from server to Tivo because I store as tmfs for use with other processes (eventual conversion to standard mpegs), but being able to watch shows from any room while they're still resident on the Tivos is what I was looking to do, seamlessly; this is terrific!

crashHD
07-01-2012, 01:30 PM
My testing is also working great.

I am having a minor issue. Playback of of a show that has been inserted from another tivo gives the message "This program's channel no longer exists in your lineup, and may have been locked. Please enter your 4 digit password <snip>". The short story is, playback of any transferred show requires the box to be unlocked with the passcode if parental controls have been turned on.

I have small children in the house, but can only supervise *most* of their tv viewing, so I use the parental controls. The tyftpd transferred show still retains its ratings in the NPL, but not the channel ID. My boxes are locked down according to ratings, not according to channels, but apparently it still wants to know the channel if parental controls are activated.

jkozee
07-05-2012, 11:05 PM
crashHD,

I turned on Kidzone on my THD and I do not see this behavior. It wouldn't be too difficult to add the real station if the station tmsid exists on the target unit, however there is also the NpkChannelDefinition. This contains additional information about the station used for the recording, such as channel major and minor number as well as the local serverid for the station, which means that it cannot be imported as is, it has to be reconstructed. This is a little more complicated and I chose to skip it. Does the NpkChannelDefinition exist in your recordings?

At this point I am not inclined to implement it, although I said the same thing about tmf+ files... If I get time maybe I will just add the correct station without the NpkChannelDefinition and see if that is enough to satisfy parental controls.

jkozee
07-08-2012, 08:14 PM
Never heard back if there is a NpkChannelDefinition on the DTiVo units, but here is a beta version that will create the showing with a real staion if the tmsid exisits for it and create a NpkChannelDefinition object as well. Let me know if/how it works.

crashHD
07-08-2012, 08:42 PM
Sorry. I read that post a day or two ago, and had been meaning to figure out how to answer the question. I'm afraid I don't know how to find the answer.

crashHD
07-08-2012, 10:33 PM
If you guide me a bit about where to look and what to look for, I can answer your question about the NpkChannelDefinition.

I tested out the new version. I moved 4 shows, all tivo-->tivo in a FXP transfer.

They have a channel listed, and it's the correct channel number, with a "-0" appended at the end.

I transferred two 30 minute shows. Both show up in NPL as partial recordings 0:07 in length. Playing back the episodes, they display a 30 minute play bar, and the full episode plays back like normal. It just has the duration incorrect and the show listed as partial in the NPL. One show retained it's ratings info, the other appeared as "no rating" after the transfer.

I transferred to 60 minute shows. They both playback correctly, appear to have correct channel information (plus "-0"), but have lost their rating info. One lists as 0:16 minutes length in the NPL, but still plays back the correct, full hour length.

jkozee
07-08-2012, 11:35 PM
I better do some more testing...

jkozee
07-08-2012, 11:56 PM
Were you transferring simultaneous or sequential?

RandC
07-09-2012, 12:06 AM
Never heard back if there is a NpkChannelDefinition on the DTiVo units, but here is a beta version that will create the showing with a real staion if the tmsid exisits for it and create a NpkChannelDefinition object as well. Let me know if/how it works.Yes there is Npk info on the DTiVo's. I don't remember seeing your post but if you need any specific information let me know.

crashHD
07-09-2012, 12:11 AM
Sequential. I queued them up in flashfxp all in one shot, and they transferred one after the other.

jkozee
07-09-2012, 10:09 PM
I added the latest beta version to the first post. This should fix the channel numbers with the "-0" being added. I know you said you were uploading sequential, but the partial recordings are most easily explained as a problem with the maxSTOR not being honored and allowing for simultaneous uploads. Please try the latest beta in the same manner as the previous beta and report the results.

Smee
07-10-2012, 01:03 AM
Jkozee,

Thanks. I'm gone for a few weeks only to come back to a butch of changes. :-)

Take care...

Smee

crashHD
07-10-2012, 09:18 PM
Thanks. I'm hoping to get a chance to test it tomorrow.

jkozee
07-10-2012, 11:53 PM
Posted a new version with a bug fix.

jkozee
07-15-2012, 11:12 PM
Posted a new version that will allow part sizes larger than 2GB that may be used on S3 units.

jkozee
07-18-2012, 12:17 AM
Posted a new version with a bug fix.

Captain 80s
07-22-2012, 09:02 PM
1st time installation. Could not get version 1.7 to run "No such file or directory". It's right there in the directory I created along with the helper file. I can see it via Telnet and Filezilla. Pulling my hair out. Permissions good. Going crazy at this point.
On a whim, I FTPd over 1.6 to the same directory... it ran fine (-p 3105). Movieloader properties configured accordingly for 1 Tivo running tyftpd and 3 running mfs_ftp. Transfers working great.
Anybody else have the same problem with 1.7 or have any ideas?

dwellar
07-23-2012, 10:26 PM
Seems that both the helper and tcl files have CTRL-M lines.
I know I didn't unpack the files on windows so they were
not added by something I did.

On the Tivo, run 'dos2unix filename' on each of the files to clean them up.
After I did that, I didn't get "No such file or directory" anymore.

jkozee
07-23-2012, 11:07 PM
Well that was sloppy. The tcl script has been fixed in the 1st post, the helper should have been ok.

dwellar
07-27-2012, 12:06 PM
For anyone still having trouble with tyftpd, do NOT run dos2unix on the helper file. That will corrupt the helper because the helper file is a binary file (not ASCII). Dos2unix is for ASCII type files only. Easiest way to fix this if the helper was inadvertently corrupted is to download and install the latest tyftpd.zip file.

Rakeesh
07-28-2012, 05:02 PM
I am trying - and failing miserably - to integrate s3tots into this. Any help? My c skills just aren't up to par.

BCC successfully integrated it into mfs_ftp, but I have issues with mfs_ftp.

BTW one thing I am trying to do is get the REST function working properly (an attempt at random access, basically I want to stream with xbmc) though I can't get anything to work at all.

jkozee
07-28-2012, 11:45 PM
Rakeesh,

Sorry, but I am not sure I understand your post. Have you encountered a bug in RETR or are you attempting to implement REST? The RETR command is uesed to retrieve a file from the server. The REST command is used to adjust the start position for both RETR and STOR. Implementing REST properly would take some thought considering the different file types and dynamic nature of those files depending on if they are STOR or RETR.

Rakeesh
07-29-2012, 02:01 PM
Rakeesh,

Sorry, but I am not sure I understand your post. Have you encountered a bug in RETR or are you attempting to implement REST? The RETR command is uesed to retrieve a file from the server. The REST command is used to adjust the start position for both RETR and STOR. Implementing REST properly would take some thought considering the different file types and dynamic nature of those files depending on if they are STOR or RETR.

Sorry yeah I meant REST.

djl
07-29-2012, 03:30 PM
This would be a great addition, if you can get it working. If I recall, BCC used a simple pipe into s3tots. It wasn't the fastest solution, but it did the job.


I am trying - and failing miserably - to integrate s3tots into this. Any help? My c skills just aren't up to par.
BCC successfully integrated it into mfs_ftp, but I have issues with mfs_ftp.
BTW one thing I am trying to do is get the REST function working properly (an attempt at random access, basically I want to stream with xbmc) though I can't get anything to work at all.

DanT
09-26-2012, 12:02 AM
Is there a version of mfs_ftp_helper available for the Series 1 DTiVos? My wife's mfs_ftp keeps crashing (was working fine for years) so I thought I'd give this a try. But IIRC, the Series 1 units are PPCs and not MIPS.

tas3086
09-29-2012, 08:43 PM
Check line 244: (in proc ProcessCommands {client} )

HELP {reply $client "502 Sorry, no help is available.}
should it be:
HELP {reply $client "502 Sorry, no help is available."}


Thanks for a great program.

Dave20042004
10-08-2012, 10:57 PM
For anyone still having trouble with tyftpd, do NOT run dos2unix on the helper file. That will corrupt the helper because the helper file is a binary file (not ASCII). Dos2unix is for ASCII type files only. Easiest way to fix this if the helper was inadvertently corrupted is to download and install the latest tyftpd.zip file.

I just downloaded the latest version (1.8). Put the two files in \var\hack When I run it I have the following problem. Tried again after rebooting with similar results. Don't have the earlier version to try. Thanks in advance, Dave


MBTivo /var/hack #ll
total 307
drwxr-xr-x 5 0 0 1024 Oct 9 02:27 .
drwxr-xr-x 16 0 0 1024 Oct 9 02:41 ..
drwxr-xr-x 2 0 0 1024 Oct 7 04:28 Saved_Apps
drwxr-xr-x 16 0 0 1024 Sep 20 02:29 TivoWebPlus
drwxr-xr-x 2 0 0 1024 Sep 20 02:29 etc
-rwxr-xr-x 1 0 0 329 Oct 3 00:57 hacks.fil
-rwxr-xr-x 1 0 0 199662 Oct 9 02:27 mfs_ftp_helper
-rwxrwxr-x 1 0 0 67271 Oct 9 02:27 tyftpd.tcl
MBTivo /var/hack #./tyftpd.tcl &
[1] 515
MBTivo /var/hack #bash: ./tyftpd.tcl: No such file or directory

[1]+ Exit 127 ./tyftpd.tcl
MBTivo /var/hack #./tyftpd.tcl -l -d &
[1] 522
MBTivo /var/hack #bash: ./tyftpd.tcl: No such file or directory

[1]+ Exit 127 ./tyftpd.tcl -l -d

Dave20042004
10-11-2012, 08:11 AM
I found something that may help others..... I know, it's not often.:o

When using Filezilla in the Auto mode, file sizes were a little larger than the original. Trying again as Binary, the file sizes stayed the same. Now it seems to run fine. I started a session in Filezilla using port 3105 and saw the TMF etc. directories. File listings came up OK and the kill pid worked well for me also. I'll be trying movieloader next. Very kewl jkozee thanks!


bash-2.02# ./tyftpd.tcl -p 3105 -s -v &
[1] 22143
bash-2.02# 10/11/12 23:00:54 - tyFTPd version 1.8 Startup
10/11/12 23:00:54 - Server Ready for Action

bash-2.02# ps aux | grep tyftpd
0 22143 5.5 6.3 43264 8072 ttyp0 S 23:00 0:01 /tvbin/tivosh ./tyftpd.tcl -p 3105 -s -v
0 22144 0.0 6.3 43264 8072 ttyp0 S 23:00 0:00 /tvbin/tivosh ./tyftpd.tcl -p 3105 -s -v
0 22146 0.0 6.3 43264 8072 ttyp0 S 23:00 0:00 /tvbin/tivosh ./tyftpd.tcl -p 3105 -s -v
0 22147 0.0 6.3 43264 8072 ttyp0 S 23:00 0:00 /tvbin/tivosh ./tyftpd.tcl -p 3105 -s -v
0 22152 0.0 0.4 2636 516 ttyp0 S+ 23:01 0:00 grep tyftpd
bash-2.02# kill 22143
bash-2.02# ps aux | grep tyftpd
0 22158 0.0 0.4 2636 516 ttyp0 S+ 23:01 0:00 grep tyftpd

clandress
10-22-2012, 09:21 AM
Hi. I installed tyftpd from post #1 onto my Hughes SD40 running 6.4a. I've hacked the Hughes with Superpatch and PTVUpgrade for 6.4a. Tyftpd seems to run OK but when I download a recorded program to .tmf or .ty+ and try to play it with vlc the video has pixelated areas in most of the scenes. It looks almost like some of the scenes are overlapping some of the others. It also appears that the audio drops out after the first minute or so (but I'm not entirely sure of that). My PC is running Linux-Mint version 12.0 and is up to date.

Is the latest version of tyftpd in post #1 or is there some other link to use to download the latest and greatest?
Is this a problem others are having with standard definition streams?

I'd like to convert the .ty+ or .tmf files to mpeg or mp4.
I've tried using tytompg gives the same pixelated scenes and no audio. I'm pretty sure I am using an old version of that.
Is that the correct tool to use? If not can you point me to a better program for the conversion?

I've also used TYTool 10R4 but it seems to get a lot of errors with the video downloads.

Thanks,
Chuck

Roger Dylan
11-09-2012, 01:16 PM
when I download a recorded program to .tmf or .ty+ and try to play it with vlc the video has pixelated areas in most of the scenes. It looks almost like some of the scenes are overlapping some of the others. It also appears that the audio drops out after the first minute or so (but I'm not entirely sure of that).VLC capability of playing .ty files correctly comes and goes with different versions. No versions, as far as I know, handle .tmf files correctly.


Is this a problem others are having with standard definition streams?Not I or anybody I know.


I'd like to convert the .ty+ or .tmf files to mpeg or mp4.
I've tried using tytompg gives the same pixelated scenes and no audio. I'm pretty sure I am using an old version of that.
Is that the correct tool to use?Yes. Early versions did not handle .tmf and I think I remember there used to be audio problems. Try a recent version. The consistency of your results through VLC and tytompg is difficult for me to understand and there may be some single underlying problem I'm not grasping. But on the surface it seems you may be getting different symptoms from different source problems (for example maybe trying to play .tmf files in VLC and remembering incorrectly that you ALSO got the same results with .ty files) and combining them in your mind into a single set of consistent symptoms.

So I may be wrong but I don't think tyftpd is your problem. Try different versions of VLC (and don't try to play .tmf's) and tytompg. I use a Mac for this so can't help you with version numbers.

jkozee
01-13-2013, 05:17 PM
Is there a version of mfs_ftp_helper available for the Series 1 DTiVos? My wife's mfs_ftp keeps crashing (was working fine for years) so I thought I'd give this a try. But IIRC, the Series 1 units are PPCs and not MIPS.

Sorry, I don't have a PPC toolchain setup, nor a unit to test with. If you are able to complile it, or find someone who can, I will certainly include it in the disribution.

jkozee
01-13-2013, 05:19 PM
Check line 244: (in proc ProcessCommands {client} )

HELP {reply $client "502 Sorry, no help is available.}
should it be:
HELP {reply $client "502 Sorry, no help is available."}


Thanks for a great program.

Yes, of course. Thanks for the feedback. I have updated the latest release with your patch.

spitfires
01-27-2013, 05:10 AM
Sorry, I don't have a PPC toolchain setup, nor a unit to test with. If you are able to complile it, or find someone who can, I will certainly include it in the disribution.
Where can I get mfs_ftp_helper.c please? Unless I'm going blind I can't see it in the mfs-utils repository.

jkozee
01-27-2013, 02:59 PM
spitfires,

mfs_ftp_helper is not in the mfs-utils, it is part of the tyftpd package. The mfs_ftp_helper*.c files are created when the patch the is included included patch is applied to the mfs-utils source. You should see them created after you run this from the mfs-utils directory (assuming the tyfpd package was extacted one level up):


patch -p1 < ../tyftpd-20130113/mfs_ftp_helper.patch

You can probably apply the patch against mfs-utils-20050604 with some slight modification. I used to compile it against the 32bit version and and 64 bit version separately, using the same patch.

A quick look at the patch and it looks like you may only need to chage this area of the patch (Keep in mind untested, use at your own risk, etc.):

1146:
+ int len=mfs_inode_run_length(&inode, run); //number of sectors left
+ u64 sec=mfs_inode_run_start(&inode, run); //sector to write to
becomes:
+ int len=inode.u.runs[run].len; //number of sectors left
+ u64 sec=inode.u.runs[run].start; //sector to write to

jkozee
01-27-2013, 03:19 PM
I just applied the patch to mfs-utils-20050604 and it fails to patch Makefile because of a changes. The easiest thing to do would manually edit your Makefile after applying the patch and make your BINS and COMMONS match this:



COMMON = mfs.c object.c util.c bitmap.c io.c partition.c \
crc.c pri.c export.c schema.c query.c tzoffset.c tar.c \
credits.c read_xml.c generate_xml.c generate_NowShowing.c attribute.c log.c \
mfs_ftp_helper_list.c mfs_ftp_helper_retr.c mfs_ftp_helper_stor.c

BINS = \
mfs_info mfs_ls mfs_streams mfs_dumpobj mfs_dumpschema mfs_tzoffset \
mfs_import mfs_uberexport mfs_burstcmds \
mfs_export mfs_stream mfs_tarstream mfs_tmfstream \
tserver vserver NowShowing mfs_ftp_helper ciphercheck \
vplay \
mfs_dump mfs_poke \
mfs_bitmap mfs_purge mfs_getslice mfs_findzero


That should get it compiling for you. Let us know the progress you make because others have expressed an interest.

Tweaks may also be required in tyftpd.tcl as well, I'm not sure though it's been a while since I looked at.

spitfires
01-27-2013, 05:43 PM
The mfs_ftp_helper*.c files are created when the patch the is included included patch is applied to the mfs-utils source.
So it does :). Sorry about that, I must've had brain fade this morning. Thanks for the tips - I'll have a play and report back.

darrin75
04-22-2013, 01:47 AM
Whats everybody using for converting avis,mpg to ty files???We needtivoserver intergraded with this to do on the fly converting.

Smee
04-24-2013, 01:09 AM
Jkozee,

Recently I have started getting an error while transferring some old video from one TiVo to another. It seems to happen at the time it is trying to insert the show metadata and consequently, tyftpd just hangs after the error. I have to go in and kill tyftpd and restart it to recover. This is the error I am getting from the terminal as the error happens:


error generating serial data from XML
while executing
"tvidl::xmltoserial {<TvBusEnvelope xs:type="TvNpkChannelDefinition"><channelNumber><major>265</major></channelNumber><strCallSign>A&E</strCallSign><id..."
("eval" body line 1)
invoked from within
"eval tvidl::$command $args"
(procedure "tvidl" line 3)
invoked from within
"tvidl xmltoserial $xml"
invoked from within
"if {$stationfsid!=""} {
set station [db $ServerData(db) openid $stationfsid]
dbobj $showing set Station $station

# NpkChannelDefin..."
("uplevel" body line 7)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set rec [db $ServerData(db) openid $ClientData(recfsid)]
set showing [dbobj $rec get Showing]
dbobj $rec set State 3 ;#..."
(procedure "UpdateMetadata" line 82)
invoked from within
"UpdateMetadata $client "" $data"
(procedure "STOR" line 65)
invoked from within
"STOR $client"
("STOR" arm line 1)
invoked from within
"switch $ClientData(cmd) {
USER {reply $client "331 Please specify the password."}
PASS {reply $client "230 Login successful."}
NOOP {reply..."
(procedure "ProcessCommands" line 33)
invoked from within
"ProcessCommands sock16"



Not really sure why it would error out. Any idea what could be causing it or do you need more debug information?

Take care...

Smee

lrhorer
05-01-2013, 10:50 PM
2. Your going to have to Open the tivo voiding you warranty.

Please do me a favor and fix your tagline. It should be either:

2. You're going to have to open the TiVo voiding your warranty.

or else:

2. You are going to have to open the TiVo voiding your warranty.

darrin75
05-01-2013, 11:12 PM
Well, what a reply:-) I will fix it or remove it. Seems my tagline is all there is to post about these days.

tas3086
05-10-2013, 09:21 AM
Jkozee,

Recently I have started getting an error while transferring some old video from one TiVo to another. It seems to happen at the time it is trying to insert the show metadata and consequently, tyftpd just hangs after the error. I have to go in and kill tyftpd and restart it to recover. This is the error I am getting from the terminal as the error happens:



Not really sure why it would error out. Any idea what could be causing it or do you need more debug information?

Take care...

Smee

The problem happens when you move files between systems that have different software versions. There is a fix for this in the mfs_ftp forum somewhere, but was not included in any patches. A similar fix can probably be made to tyftpd.

darrin75
05-23-2013, 03:51 PM
What's the latest update