Page 1 of 9 123 ... LastLast
Results 1 to 15 of 326

Thread: hdemux: New tystream demuxer

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Join Date
    Nov 2002
    Posts
    1,076

    hdemux: New tystream demuxer

    hdemux is a ty stream demux program. It will convert a .ty/.ty+ file into its elementary audio and video streams. The results of this program can be fed to an open source multiplexer, to remux the streams into (hopefully) standard mpeg2. There are two multiplexers that have been known to work with hdemux output: mjpegtools' (http://mjpeg.sourceforge.net) mplex, and ffmpeg (http://ffmpeg.mplayerhq.hu/)

    For both multiplexers, a patch is needed to allow their mpeg2 multiplexers to employ enough virtual buffering to avoid under-runs at standard HD data rates. Ie you *will* need to run a specially patched version of these programs if you intend to multiplex HD recordings. I have pre-built versions of these available in other DDB threads.

    For ffmpeg, another special patch is needed to deal with some encoder's habit of encoding a variable number of fields per frame. (directv is the most notable encoder, but some OTA encoders such as FOX's employ this encoding as well). Standard DVD-compliant muxers/demuxers do not handle this.

    For mplex, there is no such patch, and the resultant mpeg video will not play at the correct rate.

    Latest version of hdemux is released as part of the tytompg package, here.
    Mplex binary compiled for windows with HD support is here.
    ffmpeg binary here.

    GOALS:
    A tool that can convert an HD tystream to standard mpeg format as simply as possible, with solid A/V sync. This conversion should not require transcoding. Once converted, your favorite mpeg editing/transcoding/playback tools will hopefully be able to do the rest (conversion to vob, re-transcoding, non-linear editing). I say hopefully in part because it is not safe to assume that other tools work with 1920x1080i resolution. Also because I don't really know what will/won't work

    Non-goals:
    Editing support (editing typically produces discontinuities in the stream, and I believe this is best handled after the content has been converted from tystream format to a standard, more portable format).

    Typical Usage:
    hdemux -i <show>.ty -v <show>.m2v -a <show>.m2a
    then:
    mplex -r <data-rate> -O <delay> -f 8 <show>.m2a <show>.m2v -o <show>.mpg
    OR
    ffmpeg -acodec copy -vcodec copy [-f ac3] -i <show>.m2a -itsoffset <delay> -i <show>.m2v <show>.vob

    where,

    <data-rate> is the target muxing data rate, as reported from the demux stage
    <delay> is the a/v timestamp delta, as reported from the demux stage

    See the ffmpeg/mplex threads at DDB for more details on the multiplex step.

    example:
    Code:
    % hdemux -i cbs.ty -v c.m2v -a c.m2a
    Version 0.17
    Source is cbs.ty
    Video frame size is 1920x1080  - high definition. Frame rate 29.970030
    Video bit rate is 65000000 bits/sec., 65000 Kbit/sec.
    AC3 audio at 48kHz, 1280 bytes/sync frame
    6246400 audio stream bytes 299222905 video stream bytes
    Reported datarate 65320 Kbit/sec. (65000+320)
    Proceed with remuxer:
            mplex -f 8 -O 358ms -r 153701 c.m2a c.m2v -o <outfile>.mpg
    OR
            ffmpeg -acodec copy -vcodec copy -f ac3 -i c.m2a -itsoffset 0.3587 -i c.m2v <outfile>.vob
    % file c.m2*
    c.m2a: ATSC A/52 aka AC-3 aka Dolby Digital stream, 48 kHz,, complete main (CM) 2 front/0 rear, LFE on,, 320 kbit/s reserved Dolby Surround mode
    c.m2v: MPEG sequence, v2, MP@HL interlaced Y'CbCr 4:2:0 video, HD-TV 1920P, 29.97 fps
    % ffmpeg -acodec copy -vcodec copy -f ac3 -i c.m2a -itsoffset 0.3587 -i c.m2v c.vob
    FFmpeg version SVN-r6543, Copyright (c) 2000-2006 Fabrice Bellard, et al.
    [text deleted]
    Input #0, ac3, from 'c.m2a':
      Duration: 00:02:36.1, start: 0.000000, bitrate: 320 kb/s
      Stream #0.0: Audio: ac3, 48000 Hz, mono, 320 kb/s
    Input #1, mpegvideo, from 'c.m2v':
      Duration: 00:00:36.8, start: 0.000000, bitrate: 64977 kb/s
      Stream #1.0: Video: mpeg2video, yuv420p, 1920x1080, 65000 kb/s, 29.97 fps(r)
    Output #0, vob, to 'c.vob':
      Stream #0.0: Video: mpeg2video, yuv420p, 1920x1080, q=2-31, 65000 kb/s, 29.97 fps(c)
      Stream #0.1: Audio: ac3, 48000 Hz, mono, 320 kb/s
    ...
    To-do:
    Robustness so that the tool can handle streams with data errors.


    Disclaimer:
    This program is release-candidate quality, and is being provided in the interest of information sharing. Any errors in the ty stream and it'll probably blow up instead of correcting the a/v discontinuities.
    Last edited by bcc; 12-09-2006 at 04:41 PM.

  2. #2
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235

    Excellent

    This is exactly what the HDTIVO folks need to make playable mpeg2 files.

    That is the most useful format. One can make VOB files if one wants for DVD authoring,Play the mpeg2 directly in WinDVD, dump it to a D-VHS for direct playback by the deck or use on one of the PC HDTV cards to play it back.

    Your effort is much appreciated!!

    Hate to ask this but:

    I do Unix at work and Windows at home so I would like to recompile this in Windows.

    Looking at your makefile, I don't see any dependencies on any Linux libraries.

    Am I correct in that assumption?

    Thanks a lot

    Oops - almost missed something. Is there a windows version of mplex?


    JQ
    Last edited by redstone; 07-22-2004 at 02:10 PM. Reason: forgot about mplex

  3. #3
    Join Date
    Nov 2002
    Posts
    1,076
    Thanks for the appreciation. No, there's no linux binary dependency except for the mplex program. Hmm, I hadn't noticed but it looks like mplex wasn't designed to support windows. One might have to look at what tystudio did to get mplex compiled for windows.

    I take it back: I see there is win32 support in mplex code & config scripts. I got mislead by a readme comment about lack of support for anything but linux.
    Last edited by bcc; 07-22-2004 at 06:13 PM.

  4. #4
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    Quote Originally Posted by bcc
    I take it back: I see there is win32 support in mplex code & config scripts. I got mislead by a readme comment about lack of support for anything but linux.

    Where?

    I have been hunting through sourceforgre for 2 hours but the only windows mplex (source or binary) I found is in this forum in a zip file.

    It is the binary but an older version.

  5. #5
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    bcc,
    Any chance you could try to build hdmux in windows?

    I have tried but there are too many differences between the data types in linux vs windows so I am getting a zillion compile errors.


    I am using Visual Studio 6

    eg:
    uint64_t is undefined
    long long syntax is not supported so I tried long double but that trips a bunch of errors.

    I did find many of the data types (like u_char) in different .h files like <winsock.h> and <basetsd.h> under windows



    Kinda stuck here.

  6. #6
    Join Date
    Jan 2002
    Posts
    1,777
    Quote Originally Posted by redstone
    bcc,
    Any chance you could try to build hdmux in windows?

    I have tried but there are too many differences between the data types in linux vs windows so I am getting a zillion compile errors.


    I am using Visual Studio 6

    eg:
    uint64_t is undefined
    long long syntax is not supported so I tried long double but that trips a bunch of errors.

    I did find many of the data types (like u_char) in different .h files like <winsock.h> and <basetsd.h> under windows



    Kinda stuck here.
    Try cygwin.

  7. #7
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    bcc,
    Any chance you could try to build hdmux in windows?
    Here you go. No data type errors using cygwin (just had to fix a portability problem with open()). Have I mentioned that I think the windows development environment blows?

    The latest windows release is bundled into the hdemux zip. Just extract the .exe from that release, and also cygwin1.dll from the cygwin.zip. See:
    http://www.dealdatabase.com/forum/sh...8&postcount=52
    Last edited by bcc; 11-22-2004 at 08:02 PM.

  8. #8
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by redstone
    Where?
    The source code, per my install instructions (file INSTALL) is at http://prdownloads.sourceforge.net/m...s-1.6.2.tar.gz I have yet to get it to compile with cygwin (some undefined references at link time).

  9. #9
    Join Date
    Jun 2004
    Location
    Planet Earth
    Posts
    235
    Quote Originally Posted by bcc
    The source code, per my install instructions (file INSTALL) is at http://prdownloads.sourceforge.net/m...s-1.6.2.tar.gz I have yet to get it to compile with cygwin (some undefined references at link time).

    I gave up on Windows so I set up another PC with Linux.

    I can not get past the first step in building mjpegtools.

    Could you please tell me what options you put for ./configure ?

    or better yet, please build the patched mplex and post it?

    Thanks,
    JQ

  10. #10
    Join Date
    Jul 2004
    Location
    S. California
    Posts
    1

    Tech savi new HD Tivo owner lurkng and learning

    Tech savi new HD TIVO (my first tivo )owner that wants to archive, I'm reading and reading and learning, do I understand correctly that hdemux runs only on a lenux os? and archiving my tivo (.ts?) HD files with all the programs neccesary need to and only can be done on Lenux OS?

    Thanks for the answer.
    Dean
    Now I have to figure how to get the .ts files to my computer!

  11. #11
    Join Date
    Dec 2004
    Posts
    9
    Actually directshow filters are chosen by their merit, and if the application that is doing the directshow graph building is worth its salt, it will manually build the filter graph and force the use of a codec chain that it knows to work.

    Here's a handy tool:
    http://www.divx-digest.com/software/...r_manager.html
    NOTE microsoft's core codec dlls will sometimes take priority no matter what you set their merit values to. So be forewarned...
    Last edited by konfoo; 09-13-2005 at 03:42 AM.

  12. #12
    Join Date
    Nov 2002
    Posts
    1,076
    Quote Originally Posted by konfoo
    Actually directshow filters are chosen by their merit, and if the application that is doing the directshow graph building is worth its salt, it will manually build the filter graph and force the use of a codec chain that it knows to work.
    Right, and we all know how well applications agree upon their relative merits. gspot is the best app. I've seen for troubleshooting merits&filters.

  13. #13
    Join Date
    Nov 2002
    Posts
    1,076
    Here's a new version of hdemux that computes the recommended a/v offset differently. Roughly, this version computes the offset based upon the pts values written to the start of the audio/video streams instead of trying to compute the offset based upon packet arrival times in the tivo records. Let me know if this works better. Source&linux binary below:

    [See later version below]
    Last edited by bcc; 09-21-2005 at 09:48 PM.

  14. #14
    Join Date
    May 2004
    Posts
    233
    Something can't be right, I get vastly different offsets just by changing how many initial chunks to skip...

    hdemux -d -s 0 -n 2000 -i tivotool.ty -a ks.ac3 -v ks.m2v
    Proceed with remuxer:
    mplex -f 8 -O 1696ms -r 153701 ks.ac3 ks.m2v -o <outfile>.mpg

    hdemux -d -s 1 -n 2000 -i tivotool.ty -a ks.ac3 -v ks.m2v
    Proceed with remuxer:
    mplex -f 8 -O 1696ms -r 153701 ks.ac3 ks.m2v -o <outfile>.mpg

    hdemux -d -s 10 -n 2000 -i tivotool.ty -a ks.ac3 -v ks.m2v
    Proceed with remuxer:
    mplex -f 8 -O 3835ms -r 153701 ks.ac3 ks.m2v -o <outfile>.mpg

    hdemux -d -s 20 -n 2000 -i tivotool.ty -a ks.ac3 -v ks.m2v
    Proceed with remuxer:
    mplex -f 8 -O 2715ms -r 153701 ks.ac3 ks.m2v -o <outfile>.mpg

  15. #15
    Join Date
    Nov 2002
    Posts
    1,076
    That's expected behavior because the code is now just looking at the beginning presentation timestamps from the streams, not at the delivery times to the buffer. Since chunks differ on where the first video sequence is delivered, the results vary from chunk to chunk.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •