Using samba to access MFS.

[whisla13]

Hey all. I compiled a MIPS version of Samba and am using it on my Tivo. Because Windows networking uses the Samba protocol, my Tivo now comes up as a workgroup computer. Now accessing the files on my Tivo is much more easy and quick compared to FTP.

Now to figure out how to get the Samba server to access content on the MFS partitions. This would allow *.ty files to come up in a shared folder on your network. Streaming would be much simpler because show titles would come up as *.ty files in a shared folder. All you do is simply double-click the file you want to watch and your default media player will launch (provided you have tyShow), and the show will stream through Samba's protocol. vserver and ccxstream (if you're streaming to your XBox) would no longer be needed. On the other hand, this wouldn't be too efficient for extracting shows (its better using FTP because you can see transfer speeds and resume if something goes wrong).

I have no idea how to go upon doing something like this. I know the MFS filesystem is undocumented and you have to go through tivosh to access it. I think Samba supports scripting of some sort which might be able to make this possible, but I'm not sure.

Any ideas would be great.

Thanks,
- Andrew

[Jamie]

Now to figure out how to get the Samba server to access content on the MFS partitions.
...
I have no idea how to go upon doing something like this.
...
Any ideas would be great.

Thanks,
- Andrew
You'd need to write a samba VFS module (see the samba developer guide (http://at.samba.org/samba/devel/docs/Samba-Developers-Guide.pdf)). The code to access the MFS data structures is in this (http://www.dealdatabase.com/forum/showthread.php?t=39487) package, incidently originally written by the initial author of samba. For example, the mfs_streams program lists the streams available. The mfs_uberexport program exports a file in ty, ty+ or tmf format. The code for these programs could be adapted into a samba VFS module.

[feldy]

Would you mind posting the MIPS samba module you compiled?
Thanks,
-David

[whisla13]

Jamie, thanks for the response. I'll look into it and I'll post anything I come up with.

Feldy, I posted the MIPS binaries. Samba needs the shared library libnsl.so.1 which I included in the archive, just place it in your /lib directory. You will also need to place smb.conf (configuration file for Samba) in your /etc/rc.d folder and customize it to work for you. Execute the binaries like this:

nmbd -D
smbd -D

The -D tells it to run in daemon mode. After that just refresh your workgroup computers in My Network Places and tivo should come up.

- Andrew

[tivomaster]

Jamie, thanks for the response. I'll look into it and I'll post anything I come up with.

Feldy, I posted the MIPS binaries. Samba needs the shared library libnsl.so.1 which I included in the archive, just place it in your /lib directory. You will also need to place smb.conf (configuration file for Samba) in your /etc/rc.d folder and customize it to work for you. Execute the binaries like this:

nmbd -D
smbd -D

The -D tells it to run in daemon mode. After that just refresh your workgroup computers in My Network Places and tivo should come up.

- Andrew

This is GREAT.... Thanks Andrew....
Now is there a way to get the TiVo to mount a share on my Doz box via SMB?

[whisla13]

This is GREAT.... Thanks Andrew....
No prob :-)

Now is there a way to get the TiVo to mount a share on my Doz box via SMB?
The Samba suite comes with a program called "smbsh" that is supposed to make a directory (/smb) appear to have the contents of SMB shares on your network. I tried running this on the Tivo, it asks me for a Username and Password, and after I do it outputs "smbw: shared variables corrupt (Success)". I have no idea what this means, but it doesn't appear to work. The smbsh reference says "You must use a shell that is dynamically linked in order for smbsh to work correctly."

I went ahead and posted "smbsh" and "smbwrapper.so" for you to try on your Tivo. See if you can get it to work.

- Andrew

[sheriff]

Hey I just wanted to drop a note and let you know how awesome this feature will be if you can get it to work in the MFS partitions....I have my whole house wired with tivo's, xbox's and my server.
I have a tivo in the main room and then xbox's in all my bedrooms running xbox media center on em. My server holds all my movies, music and photo's which then streams to my xbox's via samba. if you get this working I will beable to go into video's on my xbox and see all media on the server and the tivo. This would be awesome!!! I hate ccxstream because it has TERRIBLE sink issues (fast forward or anything and boom the audio is off) I am almost possitive samba would cure this.

Good luck! I hope you get it :)

Sheriff

Hey fast question...I put all the files where you said but when I run the commands you stated to run I get this error:
smbd: error while loading shared libraries: cannot open shared object file: cannot load shared object file: No such file or directory

Any idea's?

[zen_monk]

The -D tells it to run in daemon mode. After that just refresh your workgroup computers in My Network Places and tivo should come up.


Works like a champ - thanks for posting !

[whisla13]

Hey sheriff. I was looking into creating VFS modules for Samba, but my lack of C++ knowledge has restricted me from coming up with anything. I was hoping a developer could pick this idea up and code a module or something that samba could use to access the MFS partitions. I have much programming expierence in VB.NET and have been trying to pick up on C++. I'm hoping to learn enough to create a Samba module with the help of mfs_ftp's code if somebody else doesn't come out with one first.

I also have XBox Media Center and HATE ccxstream too. It degrades the video quality and the sync issues are pretty bad, plus it crashes all the time. I noticed pausing and then playing the video again seems to sync it back up.

About the error you're getting, you don't have a required shared object in your /lib folder. The only one I needed was the one I included in the archive. Check to see if your /lib folder is missing any of these:

ld.so.1
libdl.so.2
libnsl.so.1
libcrypt.so.1
libc.so.6

If you need any, you can PM me and I'll send them to you. If you're not missing any of these, then I'm not sure what the problem is.

- Andrew

[whisla13]

Works like a champ - thanks for posting !

No prob, glad its working for you :-)

[tivomaster]

No prob :-)


The Samba suite comes with a program called "smbsh" that is supposed to make a directory (/smb) appear to have the contents of SMB shares on your network. I tried running this on the Tivo, it asks me for a Username and Password, and after I do it outputs "smbw: shared variables corrupt (Success)". I have no idea what this means, but it doesn't appear to work. The smbsh reference says "You must use a shell that is dynamically linked in order for smbsh to work correctly."

I went ahead and posted "smbsh" and "smbwrapper.so" for you to try on your Tivo. See if you can get it to work.

- Andrew

I had the same problem. No joy....

[sheriff]

Hey whisla13,

I sent you a pm with my email address and everything. I am missing one file you listed which is:
libdl.so.2
Thanks for any help you can provide :)

Sheriff

[fearlessrogue]

Ok, I am going to propose something stupid. As you have samba working on your tivo, why not write a script that does the following after the tivo is fully booted up:
using ftp mount the mfsftp at /somePlace.
Edit your smb.conf so that /somePlace is shared on the network.

Perhaps?

Oh, and why did this get moved to the Newbie forum?

[alldeadhomiez]

Ok, I am going to propose something stupid. As you have samba working on your tivo, why not write a script that does the following after the tivo is fully booted up:
using ftp mount the mfsftp at /somePlace.
Edit your smb.conf so that /somePlace is shared on the network.

http://linux-universe.com/HOWTO/Kernel-HOWTO/ftpfs.html

Be forewarned that triple-buffering over two extraneous sockets will probably make things work very slowly. Also, mfs_ftp is a very simple server implementation, so this might not work at all (especially for insertions).

Oh, and why did this get moved to the Newbie forum?

It didn't belong in "Series2 Development" as nothing was being developed.

[whisla13]

Ok, I am going to propose something stupid. As you have samba working on your tivo, why not write a script that does the following after the tivo is fully booted up:
using ftp mount the mfsftp at /somePlace.
Edit your smb.conf so that /somePlace is shared on the network.

I've thought of this already :-) I had troubles getting it to run on the Tivo, and didn't bother working on it that long. I thought it wouldn't be worth it, as ADH said, it would run very slowly or probably not at all.

- Andrew

[sheriff]

Okay, well I got samba working on Dtivo and man this is handy.
All I can say is someone has to know how to make this work so we can access video's. This would change everything. streaming to windows / Linux. Streaming to xbox. getting video's off of it. Man it would be nice. Can anyone help figure this out??

Sheriff

[sheriff]

Okay, I just for fun to see how well video would stream from the tivo to an xbox using samba threw a 10 meg video clip in my var directory (an AVI file) and it streamed to my xbox awesome! I fast forwarded and the audio stayed right on sink! One VERY cool thing that in my opinion would make this so great is:
A: you could stream your recorded shows right to an xbox without sink issues (no more ccxstream)
B: If you have alot of hard drive space in your tivo (like 250 gigs :rolleyes: ) you could make a folder on the tivo to hold your other media that wasnt in TY format and it would stream fine to your xbox. In other words it would be a tivo and a server for all formats of video.

This is all pending someone can make samba work accessing where the video's are stored...This should be a priority

I am getting very excited about this. Just wish I knew how to do it!

Sheriff

[fearlessrogue]

Perhaps we should start a bounty?

I wonder how hard this would be to code, I have c experience but have never written anything like this... Winter break sounds soooo tempting.

[sheriff]

Well I dont know a thing about it. I am at the mercy of the smart people.

[whisla13]

Here is some information on how I compiled a MIPS binary.

I used the information in this thread http://www.dealdatabase.com/forum/showthread.php?t=36422 (look at post #9) that informed how to compile samba for a series 1 Tivo. The same instructions apply for compiling MIPS (series 2) binaries. There is an archive in that thread that gives you everything you need to get it to compile and run properly. You must run the patch and you must compile with the libtivohack library. All of which is included in that archive. I used Samba version 2.2.10 (as described in that thread, seems there was problems with version 3).

I used a MIPS cross compiler for cygwin (I'm using Windows XP). To create a MIPS cross compiler for cygwin I used the information from this thread http://www.dealdatabase.com/forum/showthread.php?t=27280. You have to build the MIPS cross compiler yourself from within cygwin. A much simpler way for people with access to a Linux box would be to grab a MIPS cross compiler.

After I compiled I placed smbd and nmbd on my Tivo, along with the config file in /etc/rc.d and a shared library that was missing.

- Andrew

[fearlessrogue]

I had a conversation with the origianl author of samba and most of the mfs crap:
-------------------------------------------------------------
To: tridge@samba.org
Howdy. I am wondering if you have ever written a vfs module for the
mfs filesystem, as you appear to have a significant amount of
experience with both projects. It is my intention to try to write the
module, but I am hopping that you might have already created it.

Cheers,
-Rob
-------------------------------------------------------------
To: Rob Wultsch <wultsch@gmail.com>
Rob.

You mean the TiVo MFS filesystem?

That begs three questions:

- have you got a working implementation of a mfs library that
includes write/create etc?

- what do you propose the MFS object interface would look like in
CIFS?

- why on earth would you want to do this? I think vserver is a much
better approach.
------------------------------------------------------------------
To: tridge@samba.org
On Mon, 6 Dec 2004 11:40:58 +1100, tridge@samba.org <tridge@samba.org> wrote:
> Rob.
>
> You mean the TiVo MFS filesystem?
Yep.

> That begs three questions:
>
>
> - have you got a working implementation of a mfs library that
> includes write/create etc?
I would think that the current state
(http://www.dealdatabase.com/forum/showthread.php?t=39487) of being
able to import and export a ty would be sufficient. I am not an
expert on either mfs or samba, but with winter break approaching I
will have time to see if I can duct tape something together. It would
not be pretty, and it would take a long time, but I think I could do
it.

> - what do you propose the MFS object interface would look like in
> CIFS?
In general I would love if what would show up in a share would look
like mfs_ftp. Basically show all the streams in the all the possible
formats.

> - why on earth would you want to do this? I think vserver is a much
> better approach.
Simple answer: Xbox Media Center (XBMC) likes smb, and I do not like mfs_ftp.
Long answer: It is now possible to play a ty on an Xbox using
ccxstream, but it is fugly. Fast forward is not great and it crashes
occasionally. I imagine that it would be possible to implement vpalyer
in XBMC, but smb is already there and it works very well.
Mfs_ftp is under a very restrictive licence, and I have had issue with
it. I want something that is girlfriend proof, which means a gui.

-------------------------------------------------------------------
To: Rob Wultsch <wultsch@gmail.com>
Rob,

> I would think that the current state
> (http://www.dealdatabase.com/forum/showthread.php?t=39487) of being
> able to import and export a ty would be sufficient. I am not an
> expert on either mfs or samba, but with winter break approaching I
> will have time to see if I can duct tape something together. It would
> not be pretty, and it would take a long time, but I think I could do
> it.

no, i think you'll find that is a long way from being sufficient.

I think you'll also find that a filesystem interface isn't very good
for video streaming, especially given the low CPU resources of a
TiVo. Doing something based on vserver makes much more sense to me.

> Mfs_ftp is under a very restrictive licence, and I have had issue with
> it. I want something that is girlfriend proof, which means a gui.

the core code in mfs_ftp that does the real work is based on my mfs
library, and that is under the GPL. I have asked the mfs_ftp author to
fix the mfs_ftp distribution to make that clear.

Cheers, Tridge
---------------------------------------------------------------
I do not agree with his assesment (not to say that I am smart, I bet I am just wasting my time). i have watched a ty off of a samba shared and it worked fairly well so a bit of a speed hit might not be fatal.

At this point I think very basic read only filesystem usage would make everyone happy. (meaning ls, cd, and being able to do litle more than stream in the way that already exists)

[sheriff]

Well that cleared up a few things. I would have to agree Vserver does work alot better buuut I have no idea how you would get xbox media center to use vserver? :confused:

Sheriff

[Jamie]

I'm sure tridge is right that CIFS access to MFS is not the most efficient way to stream video data. I would guess that you should be able to stream SD data in real time on a series 2 that way, but I could be wrong, I haven't tried it. The performance may depend on the request sizes the client is making on the other end.

If you have some time on your hands and some C programming skills, why not just try it? It will likely be a good learning experience even if it doesn't turn out to be useful. Here's a samba VFS tutorial: link (http://www.geocities.com/orville_torpid/papers/vfs_tutorial.html). All you really need to implement is open/close/lseek/read/opendir/readdir/closedir. All of the write/modify functions can just return an error.

[fearlessrogue]

Would someone post:
libdl.so.2
libnsl.so.1
Thanks.

[sheriff]

here ya go.

[fearlessrogue]

I did some testing to see how well samba would work on a tivo and I have not been impressed with the transfer rates on my 2.4.4 box (what is that 3.1.1c or some such?).

I am maxing out at .8 MB/sec. What is everyone else getting? (bottom line for real time playback is >.4 MB/sec (1.5GB/hr /60min/60sec) assuming nothing fancy like fast forward or rewind). I think.

I wonder if I have something screwed up in my configuration or if this is the norm.

[Jamie]

I did some testing to see how well samba would work on a tivo and I have not been impressed with the transfer rates on my 2.4.4 box (what is that 3.1.1c or some such?).

I am maxing out at .8 MB/sec. What is everyone else getting? (bottom line for real time playback is >.4 MB/sec (1.5GB/hr /60min/60sec) assuming nothing fancy like fast forward or rewind). I think.

I wonder if I have something screwed up in my configuration or if this is the norm.
What usb ethernet device are you using? Is it usb2? What drivers? What's your raw network performance (e.g. from netperf)? It's hard to assess whether the samba performance is good or bad without knowing these things. 800KB/sec is actually pretty good for usb 1.1.

[whisla13]

There are optimizations you can include in your config file, take a look at http://home.arcor.de/36bit/samba.html#4.3.

- Andrew

[sheriff]

Even if Samba is not the best choice for some it may be for others. I personally would LOVE samba access to my video's!

[whisla13]

I would love to use samba as well. In my opinion it'd be a much faster and easier way to access videos when using Windows, because Windows is made to use Samba. If you're using a different OS then you would probably benfit nothing from using Samba. Also just accessing the Tivo filesystem is easier, for instance, if you need to change settings in a config file, you simply double click the config file in the tivo samba share, edit it, and just hit save in the editor and you're done. You can treat files on the Tivo drive as if they were stored locally on your own harddrive.

Just naming out reasons why I love to use samba over FTP.

- Andrew

[tivomaster]

Thanks Andrew for this utility.
I did notice that you did not run the binary though the strip utility so I did and am attaching. Doesn't gain much but every byte counts.

When I run smbd and nmbd they both put out the below warnings.

[2004/12/11 15:13:32, 0] lib/charset.c:load_client_codepage(213)
load_client_codepage: filename /usr/samba/lib/codepages/codepage.850 does not exist.
[2004/12/11 15:13:32, 0] lib/util_unistr.c:load_unicode_map(617)
load_unicode_map: filename /usr/samba/lib/codepages/unicode_map.850 does not exist.
[2004/12/11 15:13:32, 0] lib/util_unistr.c:load_unicode_map(617)
load_unicode_map: filename /usr/samba/lib/codepages/unicode_map.ISO8859-1 does not exist

The util still works great but I am trying to figure out what they mean. Any clues?

Again, thanks.
TM

[whisla13]

I'm not sure what its for either. It still works without these files obviously, so I didn't bother trying to fix these warnings.

- Andrew

[boboli]

When I try to run smbd -D I get

"cannot execute binary file"

I see this error has cropped up on other issues, but they seem to be issue related and not general problems.

Am I correct in this assumption? and What is the fix for this problem?


B

[whisla13]

Anybody still interested in this idea? fearlessrogue, did you come up with anything? Unfortunantly I haven't...

Thanks.

- Andrew

[fearlessrogue]

I have not had much time over the break to work on this. I got a crosscompiler up on my hppa box and then the SCSI controller died. I am going to play with this more.

[mchipser]

hello,

ok i got the smb working great i can see it on my pc through "my network places"

now i am looking for the files and i have tried to find the biggest folder but i still cnat seem to find the .ty files

can someone please tell me what file they are in???


Myk3

[cheer]

ok i got the smb working great i can see it on my pc through "my network places"

now i am looking for the files and i have tried to find the biggest folder but i still cnat seem to find the .ty files

can someone please tell me what file they are in???
This won't work. The videos are stored on the proprietary MFS partitions, which you cannot mount and serve via samba.

[mchipser]

ok thanks for clearing that up for me. i am a newb and didnt know that they are a diff. partition. thanks again.

Myk3

[zafdor]

Has anyone gotten this to work? I think a TIVO would be great for a NAS as long as you did not need a lot of bandwidth.

I've clocked similar speeds on my wired/USB S2 (800Kbps), don't some of the later S2s have a native ent (or even USB 2.x)? I'm guessing they are faster?