Page 1 of 22 12311 ... LastLast
Results 1 to 15 of 326

Thread: hdemux: New tystream demuxer

  1. #1
    Join Date
    Nov 2002
    Posts
    1,076

    hdemux: New tystream demuxer

    hdemux is a ty stream demux program. It will convert a .ty/.ty+ file into its elementary audio and video streams. The results of this program can be fed to an open source multiplexer, to remux the streams into (hopefully) standard mpeg2. There are two multiplexers that have been known to work with hdemux output: mjpegtools' (http://mjpeg.sourceforge.net) mplex, and ffmpeg (http://ffmpeg.mplayerhq.hu/)

    For both multiplexers, a patch is needed to allow their mpeg2 multiplexers to employ enough virtual buffering to avoid under-runs at standard HD data rates. Ie you *will* need to run a specially patched version of these programs if you intend to multiplex HD recordings. I have pre-built versions of these available in other DDB threads.

    For ffmpeg, another special patch is needed to deal with some encoder's habit of encoding a variable number of fields per frame. (directv is the most notable encoder, but some OTA encoders such as FOX's employ this encoding as well). Standard DVD-compliant muxers/demuxers do not handle this.

    For mplex, there is no such patch, and the resultant mpeg video will not play at the correct rate.

    Latest version of hdemux is released as part of the tytompg package, here.
    Mplex binary compiled for windows with HD support is here.
    ffmpeg binary here.

    GOALS:
    A tool that can convert an HD tystream to standard mpeg format as simply as possible, with solid A/V sync. This conversion should not require transcoding. Once converted, your favorite mpeg editing/transcoding/playback tools will hopefully be able to do the rest (conversion to vob, re-transcoding, non-linear editing). I say hopefully in part because it is not safe to assume that other tools work with 1920x1080i resolution. Also because I don't really know what will/won't work

    Non-goals:
    Editing support (editing typically produces discontinuities in the stream, and I believe this is best handled after the content has been converted from tystream format to a standard, more portable format).

    Typical Usage:
    hdemux -i <show>.ty -v <show>.m2v -a <show>.m2a
    then:
    mplex -r <data-rate> -O <delay> -f 8 <show>.m2a <show>.m2v -o <show>.mpg
    OR
    ffmpeg -acodec copy -vcodec copy [-f ac3] -i <show>.m2a -itsoffset <delay> -i <show>.m2v <show>.vob

    where,

    <data-rate> is the target muxing data rate, as reported from the demux stage
    <delay> is the a/v timestamp delta, as reported from the demux stage

    See the ffmpeg/mplex threads at DDB for more details on the multiplex step.

    example:
    Code:
    % hdemux -i cbs.ty -v c.m2v -a c.m2a
    Version 0.17
    Source is cbs.ty
    Video frame size is 1920x1080  - high definition. Frame rate 29.970030
    Video bit rate is 65000000 bits/sec., 65000 Kbit/sec.
    AC3 audio at 48kHz, 1280 bytes/sync frame
    6246400 audio stream bytes 299222905 video stream bytes
    Reported datarate 65320 Kbit/sec. (65000+320)
    Proceed with remuxer:
            mplex -f 8 -O 358ms -r 153701 c.m2a c.m2v -o <outfile>.mpg
    OR
            ffmpeg -acodec copy -vcodec copy -f ac3 -i c.m2a -itsoffset 0.3587 -i c.m2v <outfile>.vob
    % file c.m2*
    c.m2a: ATSC A/52 aka AC-3 aka Dolby Digital stream, 48 kHz,, complete main (CM) 2 front/0 rear, LFE on,, 320 kbit/s reserved Dolby Surround mode
    c.m2v: MPEG sequence, v2, MP@HL interlaced Y'CbCr 4:2:0 video, HD-TV 1920P, 29.97 fps
    % ffmpeg -acodec copy -vcodec copy -f ac3 -i c.m2a -itsoffset 0.3587 -i c.m2v c.vob
    FFmpeg version SVN-r6543, Copyright (c) 2000-2006 Fabrice Bellard, et al.
    [text deleted]
    Input #0, ac3, from 'c.m2a':
      Duration: 00:02:36.1, start: 0.000000, bitrate: 320 kb/s
      Stream #0.0: Audio: ac3, 48000 Hz, mono, 320 kb/s
    Input #1, mpegvideo, from 'c.m2v':
      Duration: 00:00:36.8, start: 0.000000, bitrate: 64977 kb/s
      Stream #1.0: Video: mpeg2video, yuv420p, 1920x1080, 65000 kb/s, 29.97 fps(r)
    Output #0, vob, to 'c.vob':
      Stream #0.0: Video: mpeg2video, yuv420p, 1920x1080, q=2-31, 65000 kb/s, 29.97 fps(c)
      Stream #0.1: Audio: ac3, 48000 Hz, mono, 320 kb/s
    ...
    To-do:
    Robustness so that the tool can handle streams with data errors.


    Disclaimer:
    This program is release-candidate quality, and is being provided in the interest of information sharing. Any errors in the ty stream and it'll probably blow up instead of correcting the a/v discontinuities.
    Last edited by bcc; 12-09-2006 at 04:41 PM.

  2. #2
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235

    Excellent

    This is exactly what the HDTIVO folks need to make playable mpeg2 files.

    That is the most useful format. One can make VOB files if one wants for DVD authoring,Play the mpeg2 directly in WinDVD, dump it to a D-VHS for direct playback by the deck or use on one of the PC HDTV cards to play it back.

    Your effort is much appreciated!!

    Hate to ask this but:

    I do Unix at work and Windows at home so I would like to recompile this in Windows.

    Looking at your makefile, I don't see any dependencies on any Linux libraries.

    Am I correct in that assumption?

    Thanks a lot

    Oops - almost missed something. Is there a windows version of mplex?


    JQ
    Last edited by redstone; 07-22-2004 at 02:10 PM. Reason: forgot about mplex

  3. #3
    Join Date
    Nov 2002
    Posts
    1,076
    Thanks for the appreciation. No, there's no linux binary dependency except for the mplex program. Hmm, I hadn't noticed but it looks like mplex wasn't designed to support windows. One might have to look at what tystudio did to get mplex compiled for windows.

    I take it back: I see there is win32 support in mplex code & config scripts. I got mislead by a readme comment about lack of support for anything but linux.
    Last edited by bcc; 07-22-2004 at 06:13 PM.

  4. #4
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    Quote Originally Posted by bcc
    I take it back: I see there is win32 support in mplex code & config scripts. I got mislead by a readme comment about lack of support for anything but linux.

    Where?

    I have been hunting through sourceforgre for 2 hours but the only windows mplex (source or binary) I found is in this forum in a zip file.

    It is the binary but an older version.

  5. #5
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    bcc,
    Any chance you could try to build hdmux in windows?

    I have tried but there are too many differences between the data types in linux vs windows so I am getting a zillion compile errors.


    I am using Visual Studio 6

    eg:
    uint64_t is undefined
    long long syntax is not supported so I tried long double but that trips a bunch of errors.

    I did find many of the data types (like u_char) in different .h files like <winsock.h> and <basetsd.h> under windows



    Kinda stuck here.

  6. #6
    Join Date
    Jan 2002
    Posts
    1,777
    Quote Originally Posted by redstone
    bcc,
    Any chance you could try to build hdmux in windows?

    I have tried but there are too many differences between the data types in linux vs windows so I am getting a zillion compile errors.


    I am using Visual Studio 6

    eg:
    uint64_t is undefined
    long long syntax is not supported so I tried long double but that trips a bunch of errors.

    I did find many of the data types (like u_char) in different .h files like <winsock.h> and <basetsd.h> under windows



    Kinda stuck here.
    Try cygwin.

  7. #7
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    bcc,
    Any chance you could try to build hdmux in windows?
    Here you go. No data type errors using cygwin (just had to fix a portability problem with open()). Have I mentioned that I think the windows development environment blows?

    The latest windows release is bundled into the hdemux zip. Just extract the .exe from that release, and also cygwin1.dll from the cygwin.zip. See:
    http://www.dealdatabase.com/forum/sh...8&postcount=52
    Last edited by bcc; 11-22-2004 at 08:02 PM.

  8. #8
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    Where?
    The source code, per my install instructions (file INSTALL) is at http://prdownloads.sourceforge.net/m...s-1.6.2.tar.gz I have yet to get it to compile with cygwin (some undefined references at link time).

  9. #9
    Join Date
    Jan 2003
    Posts
    388
    Quote Originally Posted by bcc
    (just had to fix a portability problem with open())
    Yeah, I saw that too when I tried to compile. How did you fix, just remove the LARGEFILE option?

    Thanks.

  10. #10
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by rung
    Yeah, I saw that too when I tried to compile. How did you fix, just remove the LARGEFILE option?
    I made its use #ifdef linux. I wish LBA48 support was clean&portable... See updated source zip if you want to see exactly.

  11. #11
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    Quote Originally Posted by bcc
    The source code, per my install instructions (file INSTALL) is at http://prdownloads.sourceforge.net/m...s-1.6.2.tar.gz I have yet to get it to compile with cygwin (some undefined references at link time).

    I gave up on Windows so I set up another PC with Linux.

    I can not get past the first step in building mjpegtools.

    Could you please tell me what options you put for ./configure ?

    or better yet, please build the patched mplex and post it?

    Thanks,
    JQ

  12. #12
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    I gave up on Windows so I set up another PC with Linux.
    Ah, good.
    Quote Originally Posted by redstone
    I can not get past the first step in building mjpegtools.
    Probably you're missing a required program or library. The compilation was straightforward:
    Code:
    tar xzf mjpegtools-1.6.2.tar.gz
    cd mjpegtools-1.6.2
    ./configure
    cd mplex
    patch < $path_to_hdemux/mplex.patch
    cd ..
    make
    Quote Originally Posted by redstone
    or better yet, please build the patched mplex and post it?
    Here you go. Make sure you put the shared libraries in some place already in your shared library path (/usr/local/lib?)
    Last edited by bcc; 10-29-2006 at 03:19 AM.

  13. #13
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235

    Finally,some results to report

    Thanks for the help.

    I finally got mplex to build. It was not a Linux issue. Turns out that one should not use WINZIP in Windows to untar the mjpegtools distribution cause it somehow messes up the configure script.

    So, I have built the patched mplex and your hdemux under Linux.

    FYI for folks that don't want to fool with building a complete Linux system.
    Get the Kloppix Linux CD IMage. You boot your PC from that CD image and you have a complete Linux system.
    I built the 2 tools under that.


    Results are very good but as bcc noted, work still needs to be done.

    I converted the following .ty files into .mpg files with bcc's tools:
    1)Six Feet Under
    2)HDNET test pattern
    3)Biker
    4) In and Out
    5) Law and Order
    6) Rides
    7) Dion Live

    Results:
    I am playing these all with my MYHD card cause that is already connected to my HDTV and my Audio Receiver.

    If using MYHD, please make you use at least Version 1.63 of the SW.

    All seven of these short clips are nearly perfect or have at most a few <1 second audio discontinuties in them.

    First time I have ever seen the biker clip play in sync


    Great job bcc.

    JQ
    Last edited by redstone; 07-25-2004 at 06:51 PM. Reason: Updates

  14. #14
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    1) Six Feet Under is basically unplayable - it only plays for 3 seconds and no AV.
    5) Law and Order has serious lip sync problems
    That's pretty strange, I still maintain that I have neither of these problems with these clips using MYHD. I can even play them back using windows media player/powerdvd/windvd, tho 2.4ghz is not quite enough to keep the audio from dropping out sometimes (when using software playback).
    Last edited by bcc; 07-25-2004 at 03:48 PM.

  15. #15
    Join Date
    Nov 2002
    Posts
    1,076
    I'm running the 1.63 release of MYD (driver&app). Maybe you're running one of the million betas?
    A couple more ideas:
    You applied my patch to mplex before compiling right? I'm actually thinking that's less likely to be the problem as you said you had problems with the law.mpg I posted as well (yet I do not).
    Have you tried software playback? wm9/powerdvd/windvd/elecard?
    Last edited by bcc; 07-25-2004 at 08:14 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •