PDA

View Full Version : corrupt files from mfs_ftp/vserver, but fine on the tivo



samhobbit
09-20-2005, 06:45 PM
First, sorry for the double post. I have a lot more information to post, but have abandoned the first post because I can't seem to change the title.

I am using a Series2 DTV with 6.2 and the latest bins for all programs mentioned (and in many cases different versions were tried).

I don't have much experience with mfs_ftp, but have recently started trying to use it. The problem I am having is that files fetched with mfs_ftp, vserver, and the mfs utils piped over nc all turn out the same every time. A rare few are without error, but most have small sections (usually one second or less) of other shows mixed in. It causes weird audio and video effects that usually clear up, but I have to stress that every single file plays flawlessly on the tivo, and I have checked the filesystem (mfsassert -please ?) and no red flags were raised and everything happens the same.

I have downloaded about 50 files to test out, and here are the results.

I made a little shell script that puts mplayer 1.0pre6a (1.0pre7 support for .ty files is broken) into ffmpeg decoding mode to use its error reporting. Because of a bug in mplayer, it can't decode .ty files in this mode, so it has to read the file from a fifo while mencoder converts it into an avi. The -benchmark -nosound -vo null options allow it to decode faster than realtime, ec=1 is for the error reporting, and fast/gray just decrease the decoding quality to speed it up. The same results can bve achieved (slower) by only using the -vc ffmpeg12,mpeg12, -lavdopts ec=1 options.

Then I ran "find . -name \*.ty -exec ./check.sh {} \;" and waited.

This is check.sh - the sed part just makes it more readable
#!/bin/sh
renice +20 $$ >/dev/null 2>&1
rm test.fifo >/dev/null 2>&1
mkfifo test.fifo >/dev/null 2>&1
mencoder -oac copy -ovc copy -o test.fifo "$1" >/dev/null 2>&1 &
mplayer -benchmark -nosound -vo null -vc ffmpeg12,mpeg12, -lavdopts ec=1:fast:gray - < test.fifo 2>&1 | sed 's/\r/\r\n/g' > "$1.check"
rm test.fifo >/dev/null 2>&1

It is still running, but here are some of the results. Most of the errors look more or less like this first one.

V:1092.7 32750/32750 15% 0% 0.0% 0 0 48%
V:1092.8 32751/32751 15% 0% 0.0% 0 0 48%
V:1092.8 32752/32752 15% 0% 0.0% 0 0 48%
V:1092.8 32753/32753 15% 0% 0.0% 0 0 47%
[mpegvideo @ 0x8568e00]mb incr damaged
[mpegvideo @ 0x8568e00]slice mismatch
[mpegvideo @ 0x8568e00]invalid cbp at 6 29
[mpegvideo @ 0x8568e00]concealing 2147483647 errors
[mpegvideo @ 0x8568e00]Warning MVs not available
V:1092.9 32754/32754 15% 0% 0.0% 0 0 49%
V:1092.9 32755/32755 15% 0% 0.0% 0 0 49%
V:1092.9 32756/32756 15% 0% 0.0% 0 0 49%
V:1093.0 32757/32757 15% 0% 0.0% 0 0 49%
V:1093.0 32758/32758 15% 0% 0.0% 0 0 49%
V:1093.0 32759/32759 15% 0% 0.0% 0 0 49%
V:1093.1 32760/32760 15% 0% 0.0% 0 0 49%
V:1093.1 32761/32761 15% 0% 0.0% 0 0 49%
V:1093.1 32762/32762 15% 0% 0.0% 0 0 49%
V:1093.2 32763/32763 15% 0% 0.0% 0 0 49%
V:1093.2 32764/32764 15% 0% 0.0% 0 0 49%
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]00 motion_type at 9 4
[mpegvideo @ 0x8568e00]00 motion_type at 0 21
[mpegvideo @ 0x8568e00]00 motion_type at 11 22
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]00 motion_type at 15 23
[mpegvideo @ 0x8568e00]00 motion_type at 0 24
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]00 motion_type at 14 25
[mpegvideo @ 0x8568e00]00 motion_type at 0 26
[mpegvideo @ 0x8568e00]00 motion_type at 0 27
[mpegvideo @ 0x8568e00]00 motion_type at 0 28
[mpegvideo @ 0x8568e00]00 motion_type at 0 29
[mpegvideo @ 0x8568e00]concealing 2147483647 errors
[mpegvideo @ 0x8568e00]Warning MVs not available
V:1093.2 32765/32765 15% 0% 0.0% 0 0 49%
V:1093.3 32766/32766 15% 0% 0.0% 0 0 49%
V:1093.3 32767/32767 15% 0% 0.0% 0 0 49%
V:1093.3 32768/32768 15% 0% 0.0% 0 0 49%
V:1093.4 32769/32769 15% 0% 0.0% 0 0 49%


That error cleared up after some audio/video blips, but in the next one it caused mplayer to crash.


V:1517.0 45464/45464 17% 0% 0.0% 0 0 49%
V:1517.0 45465/45465 17% 0% 0.0% 0 0 49%
V:1517.0 45466/45466 17% 0% 0.0% 0 0 49%
V:1517.1 45467/45467 17% 0% 0.0% 0 0 49%
V:1517.1 45468/45468 17% 0% 0.0% 0 0 49%
V:1517.1 45469/45469 17% 0% 0.0% 0 0 49%
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
[mpegvideo @ 0x8568e00]ac-tex damaged at 3 10
[mpegvideo @ 0x8568e00]ac-tex damaged at 11 22
[mpegvideo @ 0x8568e00]ac-tex damaged at 10 23
[mpegvideo @ 0x8568e00]slice mismatch
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary


MPlayer interrupted by signal 11 in module: decode_video
- MPlayer crashed by bad usage of CPU/FPU/RAM.
Recompile MPlayer with --enable-debug and make a 'gdb' backtrace and
disassembly. Details in DOCS/HTML/en/bugreports_what.html#bugreports_crash.
- MPlayer crashed. This shouldn't happen.
It can be a bug in the MPlayer code _or_ in your drivers _or_ in your
gcc version. If you think it's MPlayer's fault, please read
DOCS/HTML/en/bugreports.html and follow the instructions there. We can't and
won't help unless you provide this information when reporting a possible bug.

Here is an example of a file ending on its own after recovering earlier on.

V:1809.4 54229/54229 17% 0% 0.0% 0 0 11%
V:1809.4 54230/54230 17% 0% 0.0% 0 0 11%
V:1809.5 54231/54231 17% 0% 0.0% 0 0 11%
V:1809.5 54231/54231 17% 0% 0.0% 0 0 0%


BENCHMARKs: VC: 322.477s VO: 0.386s A: 0.000s Sys: 12.783s = 335.647s
BENCHMARK%: VC: 96.0763% VO: 0.1151% A: 0.0000% Sys: 3.8085% = 100.0000%

Exiting... (End of file)

Then there are some files with errors so small that I wonder if they are even related to the problem. I can watch and listen very closely and nothing appears to be corrupted, but mplayer spits out an error message anyway. It looks like this.

V: 96.1 2881/2881 14% 0% 0.0% 0 0 49%
V: 96.1 2882/2882 14% 0% 0.0% 0 0 49%
V: 96.2 2883/2883 14% 0% 0.0% 0 0 49%
V: 96.2 2884/2884 14% 0% 0.0% 0 0 49%
V: 96.2 2885/2885 14% 0% 0.0% 0 0 49%
[mpegvideo @ 0x8568e00]MPEG motion vector out of boundary
V: 96.3 2886/2886 14% 0% 0.0% 0 0 49%
V: 96.3 2887/2887 14% 0% 0.0% 0 0 49%
V: 96.3 2888/2888 14% 0% 0.0% 0 0 49%
V: 96.4 2889/2889 14% 0% 0.0% 0 0 49%
V: 96.4 2890/2890 14% 0% 0.0% 0 0 49%

I have no idea what is causing this, and (again) the tivo plays the files fine so I suspect that there is nothing wrong with the file system. In case it matters, I did upload some shows when I upgraded, but nothing bad seemed to happen because of it (other than some odd settings on the files).

I would like to just try to re-install and see if that clears the problem up, but of course I would need to back up my current shows first...

Jamie
09-20-2005, 07:00 PM
The problem I am having is that files fetched with mfs_ftp, vserver, and the mfs utils piped over nc all turn out the same every time. A rare few are without error, but most have small sections (usually one second or less) of other shows mixed in. It causes weird audio and video effects that usually clear up, but I have to stress that every single file plays flawlessly on the tivo, and I have checked the filesystem (mfsassert -please ?) and no red flags were raised and everything happens the same.I'm just guessing here, but it is normal for tyfiles to have "junk" between file parts. The ty file readers are suppose to read the master chunks for the stream size info and skip over the junk between parts. Is it possible the ty reader you are using isn't doing that correctly?

samhobbit
09-20-2005, 10:41 PM
I think you nailed it. I guess that means that the MPlayer support for .ty files is only meant for files that have already been cleaned up. So the things I was seeing are probably old deleted files that are being written over for the new file, and the tivo is leaving gaps that it doesn't pad out, leaving those "junk" sections.

What I have read so far shows that my only option is the closed source vsplit, since tystudio doesn't support series 2. Correct?

Jamie
09-20-2005, 11:09 PM
I think you nailed it. I guess that means that the MPlayer support for .ty files is only meant for files that have already been cleaned up. So the things I was seeing are probably old deleted files that are being written over for the new file, and the tivo is leaving gaps that it doesn't pad out, leaving those "junk" sections.

What I have read so far shows that my only option is the closed source vsplit, since tystudio doesn't support series 2. Correct?I'm not sure, but I believe the mplayer ty filter available on this site used to work for me. I haven't used it lately, and I can't be certain it didn't have this problem. The standard mplayer that ships with distributions (e.g. I run fedora core 4) claims to support ty, but it is badly broken.

You might check out tivotool (http://www.tivotool.com/down/down.html). While it mainly is marketed to Mac users, there is an older version that is command line only and runs on linux. I tried it a while back, and it seemed to work, although I didn't beat on it heavily. The author frequents these boards and tivotool has a support thread (http://www.dealdatabase.com/forum/showthread.php?t=43907&highlight=tivotool), though the linux version is old and may no longer be supported.

I think I remember seeing jdiner say that tytools may support other platforms soon.

samhobbit
09-21-2005, 12:13 AM
Thanks Jamie you have been a big help to me again. Apparently the mplayer .ty support _is_ broken. I downloaded tivotool for the vsplit binary (and ran it in a chroot jail) and it fixed the problems in the few files I have tried it on so far. Hopefully there will be an open source option soon. I don't mind running binaries I can't trust on my firewalled usb host-to-host tivo, but running them on my linux boxes creeps me out.

Although MPlayer does support .ty streams to a certain extent, it doesn't know how to filter out the junk from previously deleted files. If it were up to me (if I knew c good enough) mfs_ftp would have a new directory along with ty, ty+, tmf, etc for a nice clean stream.