MovieLoader: HME meets MFS_FTP

**cheer** · 12-13-2006, 08:41 PM

Originally Posted by mr_zorg

I'm not familiar with fxp... In order to go directly from tivo-to-tivo, it would need to telnet into the tivo to kick off the transfer, no doubt. As it stands right now (and it sounds like you're aware, but just in case you're not), the HME doesn't actually run on the TiVo at all, so issuing the command via telnet would be the only way to do it, I would think.

No, fxp allows you to set up a device-to-device transfer purely via ftp logins. No telnet required.

As for doubling the transfer time, even the fastest extract time I've heard of isn't so fast that doubling it would cause any problems for a decent 100Mbps network. Of course, I don't have two tivo's so I can't test that theory.

Maybe. I'll play around with it and see how they go.

**mr_zorg** · 12-14-2006, 12:14 AM

Originally Posted by cheer

No, fxp allows you to set up a device-to-device transfer purely via ftp logins. No telnet required.

Hmm, that sounds interesting. I'll have to read up on that.

Originally Posted by cheer

Maybe. I'll play around with it and see how they go.

Though I hadn't considered having more than two transferring simultaneously. That would, most likely saturate your PC if not the whole lan.

**cheer** · 12-14-2006, 09:01 AM

Originally Posted by mr_zorg

Though I hadn't considered having more than two transferring simultaneously. That would, most likely saturate your PC if not the whole lan.

Definitely! I'm probably overthinking this -- as you point out, a double-xfr involving the PC is still probably not going to be the bottleneck, given Tivo xfr speed.

Regarding drez's suggestion of using chrised's mfs_ftp patches...sanderton also patched mfs_ftp a while back in order to support tivo<->tivo transfers on S1 and S2-when-sendkey-still-worked units. My 2 minute review suggests that he patched RETR to send part size/information and STOR to send headers prior to transfer, and tmf2fsid to process the part info at the beginning.

This might be worth dinking with. His modifications allowed for more reliable viewing of a stream during insertion. On the other hand I haven't tried playing one with MovieLoader/stock mfs_ftp yet, so perhaps I'll shut up for now until I play.

**Jamie** · 12-14-2006, 09:23 AM

Originally Posted by cheer

Definitely! I'm probably overthinking this -- as you point out, a double-xfr involving the PC is still probably not going to be the bottleneck, given Tivo xfr speed.

Also, on a switched full duplex network, the traffic is on different links or in different directions, so as long as the PC and NIC can transfer at full speed in both directions on all links, there should be no conflicts in the TIVO->PC->TIVO transfer.

Still, a direct TIVO->TIVO connection seems prefered.

**cheer** · 12-14-2006, 01:22 PM

Originally Posted by Jamie

Still, a direct TIVO->TIVO connection seems prefered.

I think so too. Part of it is the network engineer in me, who just can't accept the double-xfr on principle.

On an unrelated note, I'm having some issues getting this working with my HR10-250. Haven't done much troubleshooting yet, so no questions from me until I have some detail to back it up. But if this works, it'll be great to yank my boatanchored SD-DVR80 out of the cabinet...

**drez** · 12-14-2006, 02:09 PM

Originally Posted by mr_zorg

Hmm, that sounds interesting. I'll have to read up on that.

It's pretty simple, here's how you could do it manually:

Login into both mfs_ftps, let's say .112 is the Tivo with the recording (sending) and .111 is the tivo you want to send the recording to (receiving)

So, you login, then you CWD (the command is case sensitive, means change working directory) to /tmf (tmf is the best/preferred format to FXP) on both FTPs:

Code:

CWD /tmf

server responds with: 250 Directory change successful.

So, you give .111 (receiving) a PASV (means passive mode) command and then note the IP/port seperated by commas the server responds with.

Code:

PASV

server responds with: 227 Entering Passive Mode (192,168,1,111,12,32).

(Kinda useless info: The first four numbers represent a regular IP (with commas instead of periods,) the last two represent the port number it's going to expect the next transfer to occur on.

(12 * 256) + 32 = 3104

yes, 3104, not 3105. 3104 is the data channel. 3105 is the control channel.
... you don't have to do any converting to a real port number though, read ahead. the other ftp will accept the port number in that same format.)

Next, you give .112 (sending) a PORT command (a.k.a. Active Mode, instead of Passive Mode) with the info that .111's PASV command gave you:

Code:

PORT 192,168,1,111,12,32

server responds with: 200 PORT command successful.

Let's say {Entourage}{2006-08-27}{Sorry, Ari}{08.00 PM Sun Aug 27, 2006}{HBO}.tmf is what you want to FXP:

Next, you give .111 (receiving) a STOR command that lets the server know to expect data in the port from the previous PASV and what to name the incoming data.
(Note: the transfer doesn't start until the next command on .112)

Code:

STOR {Entourage}{2006-08-27}{Sorry, Ari}{08.00 PM Sun Aug 27, 2006}{HBO}.tmf

server responds with: 150 Opening BINARY mode data connection for "{Entourage}{2006-08-27}{Sorry, Ari}{08.00 PM Sun Aug 27, 2006}{HBO}.tmf"

Next, you give .112 (sending) a RETR command to have send data to the IP/port given in the previous PORT command:

Code:

RETR {Entourage}{2006-08-27}{Sorry, Ari}{08.00 PM Sun Aug 27, 2006}{HBO}.tmf

server reponds with: 150 About to open data connection.

And that's it. The transfer will start after that RETR command.

If you wanted to hack in FXP support, you could simply have a telnet client login into port 3105 (the control channel) of the receiving mfs_ftp and do the commands for the receiving FTP.

For the sending FTP, you'd need a FTP client to get the filelist (for the recording's mfs_ftp filename.) If mfs_ftp supported STAT -l (it doesn't,) you could get a filelist in the control channel.

If you wanted to see it in action, just login to two mfs_ftps in SmartFTP. Then drag a .tmf recording between the tabs and drop it in the other mfs_ftp's tmf folder. Watch the FTP log.

**cheer** · 12-15-2006, 09:55 PM

OK, something is likely Not Right on my PC, but I'm not clear what.

MovieLoader fires up fine, and the Tivo sees it, and I can browse videos in multiple directories without a hitch. But if I try to transfer one, it immediately says Completed (status bar never moves), and on the PC I see this:

Code:

MovieLoader[#1,uri=null] warning: resource 2057 not found.
MovieLoader[#1,uri=null] warning: resource 2057 not found.
LOG: Received event for unknown resource id. Id = 2057, event = 2057.RESOURCE_IN
FO(complete, {})
Putting "G:\ty\{Star Trek}{1966-11-10}{The Corbomite Maneuver}{06.00 PM Sat Dec
09, 2006}{WCIUDT2}.ty" to "ftp://192.168.1.246:3105/{Star Trek}{1966-11-10}{The
Corbomite Maneuver}{06.00 PM Sat Dec 09, 2006}{WCIUDT2}.ty"
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(Unknown Source)
        at java.net.SocketOutputStream.write(Unknown Source)
        at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
        at java.io.BufferedOutputStream.write(Unknown Source)
        at sun.net.TelnetOutputStream.write(Unknown Source)
        at java.io.FilterOutputStream.write(Unknown Source)
        at java.io.FilterOutputStream.write(Unknown Source)
        at java.io.BufferedOutputStream.write(Unknown Source)
        at FtpThread.run(FtpThread.java:74)
LOG: connection to receiver closed
LOG: HME receiver disconnected

I'm guessing the resource not found stuff is at the crux of the matter, but is that some kind of JRE/Java resource missing, or a network resource, or...?

**cheer** · 12-16-2006, 12:32 AM

OK, I fixed that. It was an mfs_ftp problem. But now I have a new problem. When I select the video, it bounces back to a normal Tivo loopset and tells me there's been an error running the application. On the PC I now see this:

Code:

LOG: HME receiver connected
org.xml.sax.SAXParseException: The entity name must immediately follow the '&' i
n the entity reference.
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAX
ParseException(Unknown Source)
        at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalErro
r(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(
Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(
Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(U
nknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l.scanEntityReference(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l$FragmentContentDispatcher.dispatch(Unknown Source)
        at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImp
l.scanDocument(Unknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(U
nknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(U
nknown Source)
        at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown So
urce)
        at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Un
known Source)
        at DetailsScreen.<init>(DetailsScreen.java:142)
        at MainMenuScreen.handleAction(MainMenuScreen.java:166)
        at com.tivo.hme.bananas.BView.handleEvent(BView.java:210)
        at com.tivo.hme.bananas.BScreen.handleEvent(BScreen.java:146)
        at com.tivo.hme.sdk.View.postEvent(View.java:684)
        at com.tivo.hme.sdk.View.postEvent(View.java:686)
        at com.tivo.hme.sdk.View.postEvent(View.java:686)
        at MainMenuScreen$MovieList.handleKeyPress(MainMenuScreen.java:70)
        at com.tivo.hme.sdk.HmeObject.handleEvent(HmeObject.java:115)
        at com.tivo.hme.bananas.BView.handleEvent(BView.java:217)
        at com.tivo.hme.sdk.View.postEvent(View.java:684)
        at com.tivo.hme.sdk.View.postEvent(View.java:686)
        at com.tivo.hme.bananas.BApplication.dispatchKeyEvent(BApplication.java:
196)
        at com.tivo.hme.bananas.BApplication.dispatchEvent(BApplication.java:159
)
        at com.tivo.hme.sdk.Application.handleChunk(Application.java:786)
        at com.tivo.hme.host.sample.Listener.readEvents(Listener.java:306)
        at com.tivo.hme.host.sample.Listener.handleHME(Listener.java:288)
        at com.tivo.hme.host.sample.Listener.handle(Listener.java:381)
        at com.tivo.hme.host.http.server.HttpConnection.run(HttpConnection.java:
123)
        at java.lang.Thread.run(Unknown Source)
LOG: connection to receiver closed
LOG: HME receiver disconnected

Any ideas?

**mr_zorg** · 12-16-2006, 02:31 AM

Originally Posted by cheer

Code:

org.xml.sax.SAXParseException: The entity name must immediately follow the '&' in the entity reference.

Any ideas?

That's a clear sign that the XML descriptor file embedded in the stream is malformed. Most likely the program title or description contains an ampersand in it and mfs_ftp is not escaping it properly. In xml, just like html, & has special meaning, and if the ampersand character itself is meant & should be used instead. Unlike most web browsers, however, most standard java xml parsers won't tolerate such nonsense. It may be possible to put the parser into a more lax parsing mode, or, ideally, we patch mfs_ftp to spit out well formed xml.

In the mean time, you can manually edit the xml and change & to & or and. Alternatively, put the show back on the tivo manually (if it's not still there), use TWP to edit the program title/description and re-download it.

**jsauser11** · 12-16-2006, 02:42 PM

Has anybody tried getting MovieLoader to show up as a Galleon application?
The Galleon web site says you should be able to add HME apps into Galleon by copying the .jar file into the Galleon HME directory and adding a line to the launcher.txt file, then restarting the service. Here is what it says:

Copy the jar file of the app into the Galleon "hme" directory. For apps in the TiVo SDK, this will be the samples.jar file.
Add an entry for the new app in the launcher.txt file. When you open the launcher.txt file in a text editor, you should see something like the following:
#com.tivo.hme.samples.music.Music music #com.tivo.hme.samples.weather.Weather #com.tivo.hme.samples.skullbones.SkullBones #com.tivo.hme.samples.videobackground.VideoBackground #com.tivo.hme.samples.effects.Effects #com.tivo.hme.samples.tictactoe.TicTacToe

What would be the correct entry for the MovieLoader .jar file?

**gfb107** · 12-17-2006, 10:27 AM

Code:

MovieLoader

Note that if you do this you don't have control over the working directory. On XP it'll be %SystemRoot%\system32

So, until support is added for a parameter that names the properties file or initial directory, you'll just have to put MovieLoader.properties there.

**jsauser11** · 12-17-2006, 02:15 PM

Tried adding the code "MovieLoader" to the Galleon launcher.txt file:

###################################

#

# File: launcher.txt

#

# Copyright 2004 TiVo, Inc.

#

# This file contains a list of application classes and arguments in

# the following form:

#

# <class> <argument-list>

#

####################################

#com.tivo.hme.samples.animate.Animate

#com.tivo.hme.samples.clock.Clock

#com.tivo.hme.samples.fractal.Fractal

#com.tivo.hme.samples.hello.HelloWorld

#

# This application requires a parameter to specify the pictures directory

#

#com.tivo.hme.samples.pictures.Pictures pictures

#

# This application requires a parameter to specify the music directory

#

#com.tivo.hme.samples.music.Music music

#com.tivo.hme.samples.weather.Weather

#com.tivo.hme.samples.skullbones.SkullBones

#com.tivo.hme.samples.videobackground.VideoBackground

#com.tivo.hme.samples.effects.Effects

#com.tivo.hme.samples.tictactoe.TicTacToe

#com.tivo.hme.samples.fontinfo.FontInfo

#com.tivo.hme.samples.rss.RSS

#com.tivo.hme.samples.bananas.BananasSample
MovieLoader

and adding the MovieLoader.properties to the system32 directory, then restarting Galleon. No MovieLoader app showing in the Galleon app menu. MovieLoader.jar is present in the Galleon hme directory. Am I calling the MovieLoader application incorrectly from lanucher.txt?

**tall1** · 12-17-2006, 05:47 PM

Originally Posted by cheer

OK, something is likely Not Right on my PC, but I'm not clear what.

MovieLoader fires up fine, and the Tivo sees it, and I can browse videos in multiple directories without a hitch. But if I try to transfer one, it immediately says Completed (status bar never moves), and on the PC I see this:

Code:

MovieLoader[#1,uri=null] warning: resource 2057 not found.
MovieLoader[#1,uri=null] warning: resource 2057 not found.
LOG: Received event for unknown resource id. Id = 2057, event = 2057.RESOURCE_IN
FO(complete, {})
Putting "G:\ty\{Star Trek}{1966-11-10}{The Corbomite Maneuver}{06.00 PM Sat Dec
09, 2006}{WCIUDT2}.ty" to "ftp://192.168.1.246:3105/{Star Trek}{1966-11-10}{The
Corbomite Maneuver}{06.00 PM Sat Dec 09, 2006}{WCIUDT2}.ty"
java.net.SocketException: Connection reset by peer: socket write error
        at java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.net.SocketOutputStream.socketWrite(Unknown Source)
        at java.net.SocketOutputStream.write(Unknown Source)
        at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
        at java.io.BufferedOutputStream.write(Unknown Source)
        at sun.net.TelnetOutputStream.write(Unknown Source)
        at java.io.FilterOutputStream.write(Unknown Source)
        at java.io.FilterOutputStream.write(Unknown Source)
        at java.io.BufferedOutputStream.write(Unknown Source)
        at FtpThread.run(FtpThread.java:74)
LOG: connection to receiver closed
LOG: HME receiver disconnected

I'm guessing the resource not found stuff is at the crux of the matter, but is that some kind of JRE/Java resource missing, or a network resource, or...?

I have the same problem. You said it was a problem with mfs_ftp? What did you change to fix it? thanks.

EDIT: hmmm, got it to work on the other HD Tivo. I guess I'll reinstall mfs_ftp.

**gfb107** · 12-17-2006, 07:06 PM

Originally Posted by jsauser11

Tried adding the code "MovieLoader" to the Galleon launcher.txt file:

Code:

###################################
#
# File: launcher.txt
# 
# Copyright 2004 TiVo, Inc.
#
# This file contains a list of application classes and arguments in
# the following form:
#
# <class> <argument-list> 
#
####################################
#com.tivo.hme.samples.animate.Animate
#com.tivo.hme.samples.clock.Clock
#com.tivo.hme.samples.fractal.Fractal
#com.tivo.hme.samples.hello.HelloWorld
#
# This application requires a parameter to specify the pictures directory
#
#com.tivo.hme.samples.pictures.Pictures pictures
#
# This application requires a parameter to specify the music directory
#
#com.tivo.hme.samples.music.Music music
#com.tivo.hme.samples.weather.Weather
#com.tivo.hme.samples.skullbones.SkullBones
#com.tivo.hme.samples.videobackground.VideoBackground
#com.tivo.hme.samples.effects.Effects
#com.tivo.hme.samples.tictactoe.TicTacToe
#com.tivo.hme.samples.fontinfo.FontInfo
#com.tivo.hme.samples.rss.RSS
#com.tivo.hme.samples.bananas.BananasSample

MovieLoader

and adding the MovieLoader.properties to the system32 directory, then restarting Galleon. No MovieLoader app showing in the Galleon app menu. MovieLoader.jar is present in the Galleon hme directory. Am I calling the MovieLoader application incorrectly from lanucher.txt?

Did you stop and start Galleon after making the change?

**jsauser11** · 12-17-2006, 07:39 PM

Yes. I did stop/start the Galleon service after any change in the launcher.txt file.

Thread: MovieLoader: HME meets MFS_FTP

Thread Tools

Rate This Thread

Display

MovidLoader as a Galleon App?

MovieLoader as GalleonApp

Posting Permissions