PDA

View Full Version : mfs_ftp crash & fsid breaks



The Jake
06-06-2004, 12:41 PM
Hi there. I wouldn't consider myself a newbie, but I haven't done any TiVo work since maybe 2001 when I worked on jpegwriter and the logos module in tivoweb. My 100gig upgrade drive failed this weekend, getting me kind of excited about playing around with the Tivo. I guess I've been out of circulation long enough to be considered a newbie again.

I'm still on a HDR112, but currently only back to the originally 14 gigs until my new 120gig replacement drive arrives this week.

So, old newbie Problem#1:

I do have to say that mfs_ftp is a brilliant idea and well done. However, I seem to only be able to get about 150-200 megs of any given ty stream before the tivo crashes hard. Current video freezes, all open telnet sessions freeze. Only course of action is to pull the plug. Any suggestions on what's going wrong? Using command line FTP on MacOS X with PASV off.

I have been successful in using mfs_stream and netcat to get tystreams off the Tivo. However, I seem to have problems in the breaks between fsids.

So old newbie problem #2:

what am I doing wrong with my extracted tystreams. Both tystudio and the old, familiar 'tyconvert' seem to have problems. I get errors like "skipping chunk bad timebase" from tyc, while tyStudio simply leaves out huge chunks of video. For example, one program I recently downloaded came in two fsids. The first was large, and tyConvert was fine, but when it got to the last few minutes of the program, it was from the second fsid. tyStudio went from 98% to finished almost instantly. The output file was missing the last 30 seconds of the show and the credits.

Any help is appreciated. I'm off to get a cross compiler working under OSX. ;)

Thanks in advance,
Jake

eastwind
06-06-2004, 01:52 PM
I would suggest trying TyTool. jdiner has been working on it and updating it, while development on TyStudio has been stagnant for a long time (1-1 1/2 years IIRC). Don't know about the problem with mfs_ftp unless maybe you are using a really old version. I think the current one is 1.9.2p. Only other thing I can think of (this just came to me) is that you have an old OS on you TiVo and you need to upgrade. I don't think any of the tools handle 1.3 tystreams very well. Current software for the SA S1 is 3.0, and it gives some extra function to your TiVo as well.

ew

The Jake
06-06-2004, 06:35 PM
Thank you for the suggestion. I gave TyStudio the first go because it had a Mac version. However, I did give TyTool a try just now. Also a nice tool...

tserver seems to suffer from the same issue that mfs_ftp does. After a certain amount of download, I get a crash. With tServer, I got to about 4%, which is about 20 megs according to the tyTool progress indicator.


Doing the Lowest PriorityFix...
Priority set...
Waiting for an incomming connection!
SERVER: We got a message! buf = 'SHOWING'
Waiting for an incomming connection!
SERVER: We got a message! buf = 'TYSTRM2 192.168.0.3 4602 383328/385628'
-> '383328'
exporting fsid 383328 of size 536870912 to stdout
4%

20 Megs is less than the 150-200megs I get out of mfs_ftp, but still the symptom is the same. Hard crash, video freezes, open telnet sessions freeze, no new network connections (ftp, telnet, etc) are accepted.

On the video front, the data coming from an tyTool has the same problems when I feed it a ty file from mfs_stream and netcat. Even though the ty stream is made up of two tystreams, the second one doesn't seem to read. In the case of this show, the credits and about 30 seconds of the end of the show are cut off.

I am running the TivoOS 3.0.something. I'm pretty sure its the latest for the SA series one. I am using Mfs_Ftp ver 1.2.9p and TyTool 9r13 (and the associated tserver which came with).

update I thought I had it figured out-- I had another version of mfs_stream in the search path. However, after removing it, mfs_ftp still stalled at about 350 megs, this time.

Thanks again for any guidance,
Jake

eastwind
06-07-2004, 07:22 AM
Are you only trying to get this one show? Or is this happening on every show you try? If only one it could actually be a bad stream stored on the TiVo. TiVos are very resilient (sp?) about playing their own files. Even ones that won't ever extract properly.

Next thing I would try is TyTool's Get Parts option to get the pieces one at a time. Or mfs_ftp's different way's to get the same show. If you're trying .TY format, try .TMF or vice-versa.

ew

cojonesdetoro
06-07-2004, 10:09 AM
Make sure the place you are writing to allows your logged in user to write large files. (no ulimit on Linux or perms on Win XP/NT).

The Jake
06-07-2004, 05:17 PM
TMF download seemed to be a success, but I think it fooled me. It didn't freeze. Inside, the episode in question comes out as two parts. The second part still has decode issues ("Skipping chunk bad timebase" out of tyc). I haven't tried in TyTool yet, as I have to use the other machine. Even so the resulting mpeg is complete.

However, trying again on a different program via TMF has resulted in the same crash. It may be due to bad streams. I will clear off the Tivo and try again with new recordings to see if the recordings themselves.

However, its discouraging to say the least.

update OK, so things are improving. I had been just running mfs_ftp without an "&" and relying on it to background itself. I ran it now with an &, and when it crashed, it did not take down the whole tivo. However, it has still hosed my network. No telnet sessions accepted. I will have to reboot it I guess now.

For reference, its a TivoNet adapter and card from 9thTee. No drivers, as 3.0 has them native.

Thanks, Jake

The Jake
06-08-2004, 06:14 PM
Ok, so I now suspect a problem with either the support tools and/or the network.

First, I did a mfs_stream (the one included with mfs_ftp) out to /dev/null. Three FSID's, of a hour long program. It went along with no problem. (Much faster, by the way, then when netcat-ing to another machine)

Next I did a manual mfs_tarstream of these same FSIDs out to netcat. I got to about 800 megs (this was an hour long program) before it crashed. About 60% into the second FSID.

Next, (after pulling the plug) I did another identical mfs_tarstream out to /dev/null. No problems.

So, I am lead to believe that massive amounts of network traffic eventually hose something and cause a freeze. I am running a TivoNet card with the included NE2000, as I mentioned earlier. Its old, but seems to work.

Also, I did see a previous post complaining of error packets when doing an ifconfig. During my last netcat operation, I did a few 'ifconfig's. Lots of packets, but no errors.

Does anyone know any reason why dumping data to the network would cause such a problem? Is there a way around it? I've done this twice without a freeze.

I guess I could go out and buy a cache card (with the associated faster network card) but I'd rather not spend the money just now... And there is no reason I can see that my old legacy TivoNet card should not work.

Thanks again, as always.
Jake

eastwind
06-08-2004, 07:44 PM
Did you try to mfs_stream the 3 FSIDs out to netcat? That's the way I learned to do it way back when.
Have you tried adjusting your MTU? Try a lower value or get a tool to help optimize it for your configuration. (Can't remember right now where I saw the link to the tool, but it was on this sight.)

ew

The Jake
06-08-2004, 09:39 PM
MTU is an excellent idea, that I hadn't considered. I lowered the MTU to 1200 as a test. I got to 440megs with mfs_ftp before it crashed. Again running it with an '&' seems to mitigate the hard crash problem. However, mfs_ftp did infact stall, and apparently die... it brought down my telnet session as well...

What is odd, is that telnet did come back... and after logging in, MTU was back to 1500 again. That makes it sound like eth0 was brought down and back up.

Puzzling... I'm going to experiment with different MTU sizes. See if I can find one that works, but it doesn't seem to be it. I'll also try a few netcats. Up till now, mfs_stream/netcat has always worked, but I'm more and more convinced its a network issue that can intermittently hit any stream dumping method.

Jake

eastwind
06-08-2004, 09:47 PM
MTU is an excellent idea, that I hadn't considered. I lowered the MTU to 1200 as a test. I got to 440megs with mfs_ftp before it crashed. Again running it with an '&' seems to mitigate the hard crash problem. However, mfs_ftp did infact stall, and apparently die... it brought down my telnet session as well...

What is odd, is that telnet did come back... and after logging in, MTU was back to 1500 again. That makes it sound like eth0 was brought down and back up.

Puzzling... I'm going to experiment with different MTU sizes. See if I can find one that works, but it doesn't seem to be it. I'll also try a few netcats. Up till now, mfs_stream/netcat has always worked, but I'm more and more convinced its a network issue that can intermittently hit any stream dumping method.

JakeI don't guess there's any chance that you have more than one networked TiVo, is there?

ew

The Jake
06-08-2004, 09:50 PM
Heh, sadly no. But a new Series 2 is looking real nice about now. ;)

Jake

eastwind
06-08-2004, 11:42 PM
Longshot. Thought that you might have two TiVos with the same MAC address causing network difficulties.

ew

The Jake
06-09-2004, 09:29 AM
Yeah, I saw posts regarding MAC conflicts causing problems. My DHCP server assigns IP based on MAC address, so I am familiar with all the cards on my network and their MAC. No conflicts.

Its not like I have all that much video to dump right now anyway, but I would like to get it to work. When I get my new 120gig drive to replace the drive that died, I plan on making it the new A-drive increasing the swap space. I hold out hope that more swap will magically fix things, but I'm not hopeful.

Jake

eastwind
06-09-2004, 02:18 PM
Yeah, I saw posts regarding MAC conflicts causing problems. My DHCP server assigns IP based on MAC address, so I am familiar with all the cards on my network and their MAC. No conflicts.

Its not like I have all that much video to dump right now anyway, but I would like to get it to work. When I get my new 120gig drive to replace the drive that died, I plan on making it the new A-drive increasing the swap space. I hold out hope that more swap will magically fix things, but I'm not hopeful.

Jake
Well, if you want to save the video before upgrading, you can put the drive in a Linux box and mfs_stream them to another mount point. That way if it is a network problem you will have bypasses it.

ew

The Jake
06-09-2004, 09:50 PM
Well, i think MTU is out as the problem I've tried as low as MTU-512.

FYI, when it crashes, pings do not get through either, just checked that.

I'm gonna setup shell on dss and see if that works during the crash. I'm not suspecting that it will. Running out of ideas.

A nice $130 series 2 is looking nice about now. Hacked up with a nice big drive, a USB wifi card, and (a hopefully working) mfs_ftp would be one nice unit to play with. ;)

Jake

eastwind
06-10-2004, 01:33 AM
Did you ever try the Get Parts in Tytool? What about Single Socket mode?

ew

Bantis
06-10-2004, 06:26 AM
I am having this same problem using TyTool on my Series 2 SA. Sometimes I can get the whole thing, other times it freezes and eth0 dies. I'm at whits end trying to figure it out. Could it have anything to do with running MFS_FTP and TyTool at the same time (the servers)? I installed originally using Sleepers ISO btw. Wish there was an easy answer to all this :(

Bantis
06-10-2004, 06:28 AM
One other thing I don't quite understand (sort of a *nix Newbie, but not totally). I'm using a usb ethernet adapter on my SA S2, and its address I assigned to 192.168.2.10 in the eth config through sleepers iso, and I can communicate to it via that address, but Tivo also gets assigned a DHCP address when I go into the networking menu in Setting on the Tivo unit. Is it actually getting 2 IP's? Could this be my problem at all?

Bantis
06-10-2004, 06:31 AM
One more thing of note is the same thing is plaguing my father's Tivo, a Philips SA S1. And this was using TyStudio at the time (over a year ago, we finally just gave up at that point). So this is obviously not directly related to the pulldown software. Or its eery coincidences

The Jake
06-10-2004, 12:15 PM
I did try get parts in tytool to the same result.

I suspect that the video freeze is not the cause of the crash, but rather the sympom. I looked at the source to mfs_ftp and it seems to open the database right at the beginning and leave it open.

I'm an old-timer when it comes to programming, but if I recall, if a tivosh program crashes while the database is open, it will bring down the Tivo.

My suspicion is that a network problem of some sort is causing mfs_ftp (or one of its support programs) to crash. Since the database is open, it crashes the TiVo.

I did write a quick "CharGen" program to try and create network traffic and bring down the TiVo outside of a video extraction. However, my program didn't dump the data to the port nearly as quickly as video extraction does. In fact, after 2 hours it had only dumped 300 or so megs! I guess I need to make my traffic generator more efficient by sending large chunks of data rather than individual characters.

I'll keep expermienting to see if I can figure this one out. My new hard drive comes today, so I'll tackle that job first. Then get back to debugging this network issue.

Jake

sanderton
06-10-2004, 12:35 PM
I did try get parts in tytool to the same result.

I suspect that the video freeze is not the cause of the crash, but rather the sympom. I looked at the source to mfs_ftp and it seems to open the database right at the beginning and leave it open.

I'm an old-timer when it comes to programming, but if I recall, if a tivosh program crashes while the database is open, it will bring down the Tivo.

My suspicion is that a network problem of some sort is causing mfs_ftp (or one of its support programs) to crash. Since the database is open, it crashes the TiVo.

I did write a quick "CharGen" program to try and create network traffic and bring down the TiVo outside of a video extraction. However, my program didn't dump the data to the port nearly as quickly as video extraction does. In fact, after 2 hours it had only dumped 300 or so megs! I guess I need to make my traffic generator more efficient by sending large chunks of data rather than individual characters.

I'll keep expermienting to see if I can figure this one out. My new hard drive comes today, so I'll tackle that job first. Then get back to debugging this network issue.

Jake

One thing to try is to run mfs_ftp in the foreground in a console with the debug level turned right up; when it does crash you will at least see what was happeneing at that moment!

eastwind
06-10-2004, 03:07 PM
One thing to try is to run mfs_ftp in the foreground in a console with the debug level turned right up; when it does crash you will at least see what was happeneing at that moment!
Yeah, set it up to 5 and watch on the telnet session or if you're running in the background just check the log. You might see something in the log right now, but maybe not unless you've already set the debug level. Be aware that the log gets erased/renewed every time you start mfs_ftp.

ew

The Jake
06-10-2004, 09:44 PM
New 120 gig drive with an expanded swap file doesn't fix the problem... not that I expected it to.

I tried one download, which failed at about 300megs.

Then I set the debug level to 5. Got to 1.475 gigs of a 1 hour program! Thought I had it... but it froze at 1.475 gigs.

Same problem. Frozen video, network, etc.

Level 5 shows nothing interesting. "About to open data connection." then 1.475 gigs of data... then a crash.

Back to the drawing board, sadly.

At least I have a good recording size again, and the new Diamondmax drive is much quieter. ;)

Jake

Bantis
06-17-2004, 05:21 AM
I really wish this problem even made remote amounts of sense these days :( Mine has still been pretty random; sometimes it works, sometimes it doesn't. Ah well.

RangerOne
06-22-2004, 04:18 AM
Jake--I have the *exact same* configuration as you and am running into the *exact* same issue as you, it seems. See my post (http://www.dealdatabase.com/forum/showthread.php?p=170834#post170834). I also have an HDR112 with a 9th Tee supplied TiVoNET board, though with an NE2000 clone I had already from elsewhere.

What is the manufacturer of your NE2000 clone, if identifiable at all? I have a second one, different circuit board, but also NE2000-compatible and will give that one a try to see if it helps at all. I am wondering if this is a TiVoNET issue or the NIC used with TiVoNET...sadly, I don't have a TurboNET handy to try my theory.

RangerOne

RangerOne
06-22-2004, 05:43 AM
OK, some more testing--results posted in my thread (http://www.dealdatabase.com/forum/showthread.php?t=35588) there...I'd like to follow the issue there since it's a new post with lots of details and now atop the forum list so hopefully others will see it fresh. Check out the results there.

RangerOne
06-22-2004, 08:32 AM
The Jake-- in case you're reading up on this thread, please check out my thread (linked above). I have some questions about your setup:

1) Is your TiVo subscribed?
2) What exactly is the version of the TiVo software you're running (say, as shown in the Messages & Setup | System Information page)?
3) How is the network setup in your case? That is, what kind of topology do you have between the TiVo and your PC/Mac, etc.? Are they connected to a common hub? How are the IP addresses assigned? What kind of hub/switch/networking hardware are you using?

I really don't want to have to resort to removing the drive every time to extract my recordings until I get a turbonet card (and I'd rather not have to get one for this less frequently used TiVo). I'd like to figure out what the issue is.

Thanks!
RangerOne