PDA

View Full Version : mfs_ftp: no utf-8 accents



Blaq
06-21-2006, 10:35 AM
(Using mfs_ftp 1.2.9p, SA TiVo S1 3.0, Filezilla 2.2.9.)

TiVo encodes program information in UTF-8, a smart move since the world uses more than ASCII to communicate. Specifically, French-Canadian shows use lots of accents in their titles and other info.

Unfortunately, when I use mfs_ftp, my client receives a file listing that seems to be missing some UTF-8 flag, resulting in entries that read like this:


{Y paraît que}{2005-11-06}{Échos du passé}{11.00 PM Fri Jun 09, 2006}{TFO}.ty

These appears on TiVo as:


Y paraît que
"Échos du passé" (2005) etc.

(Since HTML is off in the forums, I can't use entities; let's hope the accented characters are readable to you.) I use Filezilla, which supports UTF-8, so it seems there's nothing I can do on my end, and the problem lies with mfs_ftp.

TiVo uses UTF-8 as its native character set. Shouldn't mfs_ftp reflect this? Thanks for any info.

cheer
06-21-2006, 12:11 PM
This may not be mfs_ftp itself so much as the underlying mfs_* tools. Are you running the latest unified mfs_* tools by the way?

But that's just a guess -- I don't have any shows recorded at the moment with accents or other characters that would require UTF-8.

Jamie
06-21-2006, 12:28 PM
This may not be mfs_ftp itself so much as the underlying mfs_* tools. Are you running the latest unified mfs_* tools by the way?

But that's just a guess -- I don't have any shows recorded at the moment with accents or other characters that would require UTF-8.The mfs_* tools (e.g. mfs_uberexport & mfs_import) don't have anything to do with the show metadata (title, description, etc). That's all handled in the tcl code.

To the OP: mfs_ftp hasn't been updated in a long time, so I don't think there is a lot of hope that someone else is going to fix this for you. If it is important to you, you may want to take matters in your own hands, and work out how to handle utf-8 in tcl yourself.

Blaq
06-21-2006, 01:31 PM
Thanks, cheer and Jamie, for your replies. Since I don't know TCL, it'll be easier for me to manually rename offending files than try to patch the code. Jamie, if mfs_ftp hasn't been updated in so long, is there another utility that has become more popular, for people running 3.0?

cheer
06-21-2006, 01:48 PM
You could try TyTool...not sure how it handles UTF-8.

Blaq
06-21-2006, 01:52 PM
No joy - tyTool also takes Now Showing data as is, not bothering to convert from UTF-8.

Jamie
06-21-2006, 01:54 PM
Jamie, if mfs_ftp hasn't been updated in so long, is there another utility that has become more popular, for people running 3.0?Not much development for S1's anymore, AFAIK. You could always checkout the oztivo and other non-us sites where S1's are the only game in town.

AlphaWolf
06-21-2006, 03:59 PM
This may not be possible to do without adding significant bits of extra code. Other than maybe massaging the text output somehow (e.g. detecting U+ chars and then converting them during the FTP listing,) I am not certain if tivosh even supports unicode.

Check this out. I found a show in the guide called "El Chapulín Colorado"

doing a dumpobj of it gets this:



% dumpobj 44549
Series 44549/10 {
Genre = 371 124
IndexPath = /Server/ATSH068173:4:0:0
ServerId = ATSH068173
ServerVersion = 0
ThumbData = Predicted 1 33
Title = {El ChapulÃ*n Colorado}
TmsId = SH068173
Version = 233
}


mfs_ftp is just putting out whatever it is given. You'd have to basically make up for what tivosh lacks.

Jamie
06-21-2006, 04:07 PM
This may not be possible to do without adding significant bits of extra code. Other than maybe massaging the text output somehow (e.g. detecting U+ chars and then converting them during the FTP listing,) I am not certain if tivosh even supports unicode.A quick google indicates that tcl has had utf-8 support in it at least since 8.1. Under tivo software 7.x anyway, tivosh reports that the tcl_version is 8.4. It's possible tivo stripped out utf-8 support to make the executable lighter weight, but I doubt it.

I've looked a little at the ftp protocol spec, and it's possible that this would "just work" if mfs_ftp.tcl responded to a "feat" command with the appropriate "UTF8" response. This would tell the clients that the server knew utf-8 and would send path names encoded in utf-8. Ascii is a subset of utf-8, so it should continue to work the way it does now with pure ascii file names. Should be simple to test.

Jamie
06-21-2006, 11:36 PM
Blaq: Filezilla has an option to force UTF8 under Site Manager/advanced. My guess is that mfs_ftp is sending the utf-8 encoded characters just fine and Filezilla just needs to be told to use that encoding. One way is by patching mfs_ftp as I suggested before; another way is the "force UTF8" option on the client side.

{Edit: I tried it with some manually edited utf-8 titles, and it seems to work fine.}

Blaq
06-27-2006, 01:03 PM
Jamie, what a great solution! I was so focused on the TiVo side that I didn't think to check whether Filezilla had been upgraded to add forced UTF-8 support. It works fine -- all my French shows now bear correct accented characters and, once downloaded using mfs_ftp, bear the correct filenames on my PC, so tyTool's lack of UTF-8 support is no longer a problem, either.

Thanks!

tsanga
08-06-2007, 03:13 AM
I'm looking for some suggestions.

Forcing UTF8 encoding in FileZilla ensures the files download with the correct character set. However, upon re-insertion, the TiVo fails to display the text properly. What's interesting is that in TWP, the show title has all the correct accents, but just not in Now Playing in the TiVo UI.

Any ideas what else I can try?