Page 1 of 9 123 ... LastLast
Results 1 to 15 of 130

Thread: ffmpeg with MPEG to TY+ encoding support

  1. #1
    Join Date
    Nov 2002
    Posts
    1,077

    ffmpeg with MPEG to TY+ encoding support

    Enclosed are changes to ffmpeg which allows it to encode to .ty+ format. ffmpeg is superior to mplex for encoding to .ty+ format in that:
    • ffmpeg's multiplexor, with the enclosed changes, can handle directv streams with video frames that sometimes contain 3 fields/frame. This is a common mpeg encoding scheme employed by directv but not seen in dvd compliant mpeg. mplex malfunctions in such cases because it fails to synthesize the correct timestamps to match the variable frame duration.
    • With ffmpeg, one can now encode directly from .mpg or .vob to .ty+ without any separate demuxing step.

    CREDITS:
    • bcc for: The new libavformat encoding library for ffmpeg, TY master chunk processing library, XML record generation, TY chunk record streaming library.
    • Nova1 for:
      Original code demonstrating the relatively straightforward mapping from an elementary mpeg video stream to TY records
      Original changes to more efficiently pack TY chunks

    USAGE:
    Notes: When specifying an a/v sync offset with -itsoffset, the order of the arguments matter.
    Unlike mplex, ffmpeg won't work hard to glean the audio stream format, so if you're specifying that separately, you'll need to indicate whether it's ac3 or mpeg. In the following examples I do so by file name extension.
    Converting from a .vob to .ty directly requires ffmpeg to identify the a/v streams. If it fails to do so (common it seems) you may still have to demux the streams first.
    ffmpeg -i src.mpg -acodec copy -vcodec copy dest.ty
    The above converts .mpg to .ty directly, without transcoding the video or audio
    ffmpeg -i src.ac3 -itsoffset 0.798 -i src.m2v -acodec copy -vcodec copy dest.ty
    In the above, the video stream start delayed 798ms after AC3 audio stream.
    ffmpeg -i src.m2v -itsoffset 0.637 -i src.m2a -acodec copy -vcodec copy dest.ty
    MPEG audio stream start delayed by 637ms after video stream.
    COMPILING:
    ffmpeg compiles under linux, and for windows, using cygwin.

    1. Unfortunately the ffmpeg project does not seem to be maintaining snapshots of their code, so first checkout the code from their SVN repository. The diffs were made against revision 5510, so if you checkout the same revision, the diffs should patch cleanly even if the mainline has diverged since then:
      svn checkout -r 6543 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
      Or you could try the tivoserver snapshot of ffmpeg source.
    2. cd ffmpeg
    3. patch -p1 < ffmpeg-diffs-xx.txt
    4. ./configure --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
    5. make
    6. make install


    There is a support thread over here: http://www.dealdatabase.com/forum/sh...ad.php?t=50010
    Attached Files Attached Files
    Last edited by bcc; 10-06-2006 at 05:00 AM.

  2. #2
    Join Date
    Nov 2002
    Posts
    1,077
    Windows & linux binaries are here.
    For windows, you'll need the DLL files from cygwin.rar as well. For windows, unpack the two .rar files, and put the .dll and .exe files in the same dir.
    Attached Files Attached Files
    Last edited by bcc; 10-17-2006 at 09:52 PM.

  3. #3
    Join Date
    Aug 2004
    Posts
    4,086
    Very cool. I'm embarrassed that I volunteered to look at this, then dropped the ball. Glad someone got to it. I'll give it a try.

  4. #4
    Join Date
    Nov 2002
    Posts
    1,077
    Quote Originally Posted by Jamie
    Very cool. I'm embarrassed that I volunteered to look at this, then dropped the ball. Glad someone got to it. I'll give it a try.
    Thanks. You weren't the only other person who said you were going to try tackling it But you should probably be glad you didn't try - I had to do a bunch of wading through the ffmpeg code before I could get things to work and there were a couple nasty issues. In the end the diffs to ffmpeg came out pretty clean (cleaner than with mplex). So the ffmpeg API is fairly modular but their internals are poorly documented.

  5. #5
    Join Date
    Aug 2004
    Posts
    4,086
    Quote Originally Posted by bcc
    So the ffmpeg API is fairly modular but their internals are poorly documented.
    Yeah, searching for good libav* documentation is about as far as I got.

    To make another possibly empty promise, maybe I can look at integrating this into tivoserver. Tivoserver requires that ffmpeg can write to an unseekable stream, but, it doesn't need the at-end master chunk trickplay update. It seems to me they named a file extension type for ty without masterchunks, but I can remember what it is. Maybe another AVOutputFormat for ty without masterchunks is the way to go.

  6. #6
    Join Date
    Nov 2002
    Posts
    1,077
    For tivoserver, I was going to make the code skip the master chunk processing if the file descriptor was a fifo, but punted because I think that would make the source less portable. A little less clean would be to make the code assume it's dealing with a fifo if the lseek fails.

  7. #7
    Join Date
    Aug 2004
    Posts
    4,086
    Quote Originally Posted by bcc
    For tivoserver, I was going to make the code skip the master chunk processing if the file descriptor was a fifo, but punted because I think that would make the source less portable. A little less clean would be to make the code assume it's dealing with a fifo if the lseek fails.
    tmf would also be a nice addition, since, IME, mfs_ftp works better with tmf than it does with ty+. Naturally, that would require a seekable output stream. I can probably look at this. It doesn't sound too hard and should be able to reuse all the ty+ code with just a little extra processing at the end and between file parts.

  8. #8
    Join Date
    Nov 2002
    Posts
    1,077
    Quote Originally Posted by Jamie
    tmf would also be a nice addition, since, IME, mfs_ftp works better with tmf than it does with ty+. Naturally, that would require a seekable output stream. I can probably look at this. It doesn't sound too hard and should be able to reuse all the ty+ code with just a little extra processing at the end and between file parts.
    I agree, tmf would be a bonus. It just always struck me as a bit wrong to spread tar file format knowledge to all the utilities to placate the insertion software. But fixing that backend would be harder, so...

  9. #9
    Join Date
    Aug 2004
    Posts
    4,086
    Quote Originally Posted by bcc
    I agree, tmf would be a bonus. It just always struck me as a bit wrong to spread tar file format knowledge to all the utilities to placate the insertion software. But fixing that backend would be harder, so...
    The important thing (I think) is that the metadata (show duration, etc) are known at the beginning of the stream rather than at the end. The insertion code seems to be more reliable if all that stuff can be set up in MFS before the stream is inserted. I'm not sure there is a backend fix for that problem. Tar format is kind of irrelevant. We could make up yet-another-ty format with the xml preceeding the ty parts, but the tar file headers are simple to generate, so sticking with tmf seems better than inventing yet another file format.

  10. #10
    Join Date
    Nov 2002
    Posts
    1,077
    I'm not disagreeing about tmf being a useful addition, and I certainly wouldn't suggest inventing a 3rd format. It's too bad that there are two already, with 1 that is problematic for utilities that wish to process recording streams, and 1 that is problematic for file transport and editing.

  11. #11
    Join Date
    Mar 2006
    Posts
    63
    What I've been able to watch has been flawless! The only issue I'm seeing is the master chunks after the first segment (chunk 4097 and up) don't seem to be valid so tivoserver is unable to parse the streams correctly and insertion (through tivoserver) causes the tivo to reboot when the transfer reaches the first corrupt master chunk. They're also crashing my ty chunk analyzer at the moment but what I saw before the app died was all FF bytes. I'll post additional details when I get a chance to dig into this a little further. I think everything is kosher with streams that aren't big enough to fill > 4096 chunks, though I haven't tested one just yet.
    Last edited by phat_bastard; 06-22-2006 at 03:50 PM.
    try out my automated tserver client here

    Hey, guess what you're all accessories to? ~ Bender B. Rodriguez

  12. #12
    Join Date
    Mar 2006
    Posts
    63
    After further investigation, the problem seems to be that the master chunk for the second fsid is empty - chunk size and chunk count are both zero. The crashing of my packet analyzer was happening on a padding chunk at the end of the sequence (no problems there).
    try out my automated tserver client here

    Hey, guess what you're all accessories to? ~ Bender B. Rodriguez

  13. #13
    Join Date
    Mar 2006
    Posts
    63
    I think it might be something here:

    Code:
    tymaster.c::rewrite_master()
     	if (!parse_chunk(buf, paychunk, &lastseq, segsize)) {
     	    if (chunkcnt && (chunkcnt-1) == pos) { // < ???
     		/* expected end of data chunks */
     	    } else {
     		dbgprint("Warning: Found invalid chunk at %u\n", pos);
     	    }
     	    break;
            }
    Because I'm getting this:
    Code:
    Warning: Found invalid chunk at 4095
    Generating master chunk for chunks 0:4094
    Successfully wrote new XML
    from muxing a stream that has 5972 chunks

    Maybe just need to move the break up 3 lines?
    try out my automated tserver client here

    Hey, guess what you're all accessories to? ~ Bender B. Rodriguez

  14. #14
    Join Date
    Nov 2002
    Posts
    1,077
    Quote Originally Posted by phat_bastard
    The only issue I'm seeing is the master chunks after the first segment (chunk 4097 and up) don't seem to be valid
    Oops, I fudged the sanity checking trying to hush a cosmetic warning. Here's the fix:
    Code:
    *** libavformat/tymaster.c	2006-06-21 09:33:27.000000000 -0700
    --- libavformat/tymaster.c	2006-06-22 13:22:58.000000000 -0700
    ***************
    *** 70,82 ****
          boolean expected, have_seq;
      
          recs = getshort(buf);
          if (recs == 0xffff) {
    ! 	return FALSE;
          }
          p = &buf[4];
          time = get64(&p[8]);
          if (!time_validate(time, lasttime, ticks_per_chunk)) {
    - 	expected = (chunkcnt == (segsize+1));
      	if (dbg(2)) {
      	    dbgprint("Hit %stime discontinuity at chunk %d\n",
      		     expected ? "expected " : "",
    --- 70,82 ----
          boolean expected, have_seq;
      
          recs = getshort(buf);
    +     expected = (chunkcnt == (segsize+1));
          if (recs == 0xffff) {
    ! 	return expected;
          }
          p = &buf[4];
          time = get64(&p[8]);
          if (!time_validate(time, lasttime, ticks_per_chunk)) {
      	if (dbg(2)) {
      	    dbgprint("Hit %stime discontinuity at chunk %d\n",
      		     expected ? "expected " : "",

  15. #15
    Join Date
    Mar 2006
    Posts
    63
    That makes more sense, thanks! Testing...

    Incidentally the ffmpeg-tivoserver.tgz snapshot has a few changes in libavformat that prevent merging your patch, but it's not a big deal. The svn snapshot seemed to compile okay under cygwin after I got off my duff and installed it (svn).
    try out my automated tserver client here

    Hey, guess what you're all accessories to? ~ Bender B. Rodriguez

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •