TY enabled mplex for mpeg to TY stream conversion

**bcc** · 02-27-2005, 05:06 PM

Here are some changes to mplex that allow it to generate TY streams as output. This is being presented as two sets of diffs to 1.6.2 version of the mplex program. Input to this program are elementary mpeg video and an audio stream. The audio stream may be either mpeg2 or ac3. For video, I've tested with resolutions of 1920x1088, 1280x720, 480x480, and 720x480, and all work with my hr10-250. So you can use this mpeg->ty conversion starting with DVD, previously converted tivo mpeg, SVCD, or synthetic video streams (avisynth for example).

We haven't yet looked into whether audio sync drifts for longer recordings. Good chance there is an issue there.
I've included linux and windows binaries for mplex.

CREDITS:
The ifdef TY part is nearly all written (and GPL licensed) by nova1. The second part is taken from my ty1to2 program, with the copy_rec() chunk spanning changes that nova1 previously posted. I contributed some c++ vs c glue and some of the AC3 audio support as well. Thanks also to rung for some early discussions.
USAGE:
Use '-f 10' argument to mplex to select TY stream format output. For example, below I take some HD ty, convert to mpeg, and back:

Code:

% hdemux -i Fox-OC.ty -v f.m2v -a f.m2a
Version 0.9
AC3 audio at 48kHz, 1792 bytes/sync frame
Video frame size is 1280x720 - high definition
Video bit rate is 65000000 bits/sec., 65000 Kbit/sec.
Audio timestamp offset: 513.533333 ms
Reported datarate 65448 Kbit/sec. (65000+448)
Proceed with remuxer:
        mplex -f 8 -O 513ms -r 153701 f.m2a f.m2v -o <outfile>.mpg
426688 audio stream bytes 10857140 video stream bytes
% mplex -f 10  -O 513ms f.m2a f.m2v -o f.ty
   INFO: [lt-mplex] mplex version 1.6.2 (2.2.3 $Date: 2004/01/13 20:45:26 $)
   INFO: [lt-mplex] File f.m2a looks like an AC3 Audio stream.
   INFO: [lt-mplex] File f.m2v looks like an MPEG Video stream.
   INFO: [lt-mplex] Video stream 0: profile 10 selected - ignoring non-standard options!
   INFO: [lt-mplex] Found 1 audio streams and 1 video streams
   INFO: [lt-mplex] Selecting TY and generic MPEG2 output profile
   INFO: [lt-mplex] Multiplexing video program stream!
   INFO: [lt-mplex] Scanning for header info: AC3 Audio stream 00 (f.m2a)
   INFO: [lt-mplex] AC3 frame size = 1792

   INFO: [lt-mplex] AC3 AUDIO STREAM:
   INFO: [lt-mplex] Bit rate       :    57344 bytes/sec (448 kbit/sec)
   INFO: [lt-mplex] Frequency      :     48000 Hz
   INFO: [lt-mplex] Scanning for header info: Video stream e0 (f.m2v) 
   INFO: [lt-mplex] VIDEO STREAM: e0
   INFO: [lt-mplex] Frame width     : 1280
   INFO: [lt-mplex] Frame height    : 720
   INFO: [lt-mplex] Aspect ratio    : 16:9 display
   INFO: [lt-mplex] Picture rate    : 59.940 frames/sec
   INFO: [lt-mplex] Bit rate        : 65000000 bits/sec
   INFO: [lt-mplex] Vbv buffer size : 999424 bytes
   INFO: [lt-mplex] CSPF            : 0
   INFO: [lt-mplex] SYSTEMS/PROGRAM stream:
   INFO: [lt-mplex] rough-guess multiplexed stream data rate    : 66801896
   INFO: [lt-mplex] Setting best-guess data rate.
   INFO: [lt-mplex] Run-in Sectors = 998 Video delay = 68199 Audio delay = 26533
   INFO: [lt-mplex] New sequence commences...
   INFO: [lt-mplex] Audio bd: buf=  16384 frame=000000 sector=00000000
   INFO: [lt-mplex] Video e0: buf=2719744 frame=000000 sector=00000000
   INFO: [lt-mplex] STREAM e0 completed @ frame 150.
   INFO: [lt-mplex] STREAM bd completed @ frame 235.
   INFO: [lt-mplex] Multiplex completion at SCR=677450.
   INFO: [lt-mplex] Audio bd: buf=    256 frame=000235 sector=00000236
   INFO: [lt-mplex] Video e0: buf=2719744 frame=000150 sector=00000171
   INFO: [lt-mplex] AUDIO_STATISTICS: bd
   INFO: [lt-mplex] Audio stream length 422912 bytes.
   INFO: [lt-mplex] Frames         :      236
   INFO: [lt-mplex] BUFFERING min 256 Buf max 256
   INFO: [lt-mplex] VIDEO_STATISTICS: e0
   INFO: [lt-mplex] Video Stream length:    10855544 bytes
   INFO: [lt-mplex] Sequence headers:       11
   INFO: [lt-mplex] Sequence ends   :        0
   INFO: [lt-mplex] No. Pictures    :      151
   INFO: [lt-mplex] No. Groups      :       11
   INFO: [lt-mplex] No. I Frames    :       11 avg. size 82887 bytes
   INFO: [lt-mplex] No. P Frames    :       40 avg. size114992 bytes
   INFO: [lt-mplex] No. B Frames    :      101 avg. size 52686 bytes
   INFO: [lt-mplex] Average bit-rate : 22830800 bits/sec
   INFO: [lt-mplex] Peak bit-rate    : 28366400  bits/sec
   INFO: [lt-mplex] BUFFERING min 192 Buf max -11760
   INFO: [lt-mplex] MUX STATUS: no under-runs detected.
   INFO: [lt-mplex] Generating master chunk for chunks 0:86

%

Latest version is here:
http://www.dealdatabase.com/forum/sh...4&postcount=63

**bcc** · 02-27-2005, 05:17 PM

Download mplex 1.6.2 source from:http://prdownloads.sourceforge.net/m...ar.gz?download
Project page:http://mjpeg.sf.net
patch with:

Code:

patch -p1 < patch1.txt
patch -p1 < patch2.txt

and for gcc 3.4 users, you probably also need this enclosed patch:
patch -p1 < patch-gcc.txt

**bcc** · 02-28-2005, 05:08 AM

Ok, I've added a windows binary.
Now, does anybody know of a good mpeg2 encoder for HD? I've been unable to get tmpgenc or cce to encode 1920x1080. If I could get that to work I could insert some top quality test patterns into the tivo. The HDNet test pattern is only 1280x1080 so it's not a good resolution test.

**bcc** · 02-28-2005, 01:18 PM

Looks like someone's support questions got deleted. Lets try a support thread over here

**nova1** · 02-28-2005, 08:37 PM

Here are some diffs on bcc's mplex with patch1 and 2 applied. These patches handle the case where an MPEG SEQUENCE_END header is seen and where a GOP header is not preceded by a SEQUENCE_HEADER(seen on a couple of VCDs).

Edit: Forgot to remove some code when handling the seq_pos == -1 case.

Added patch4 which adds a '-x xml-file' option to read in a TY+ XML file and adjust the StreamFileSize setting and then append it to the TY file.

Edit: forgot to include xml.c from ty1to2-1.1 in patch4.txt

Edit: added patch5.txt to generate better TY record timestamps. This should fix the sync issues seen when generating TY files from AVI and DVD sources.

Edit: uploaded new patch5.txt that changes definition of TIVO_SD_PER_CHUNK and modified time_validate().

Edit: uploaded patch6.txt that fixes bug in copy_rec() function that manifests itself when the current chunk has a VID_SEQ record already and the last record is a VID_SEQ record which continues on to the next chunk.

All the patches have been rolled into a larger patch file. Start at this thread http://www.dealdatabase.com/forum/sh...&postcount=101

**bcc** · 03-01-2005, 04:25 AM

Ok, I've built 0.2 versions that include the patch3 changes.

**nova1** · 03-01-2005, 10:55 AM

I was working with a 3.2GB file under Fedora Core 2 and I needed to replace the fopen() calls in main.cpp with fopen64() to get rid of the file too large error.

**Jamie** · 03-01-2005, 11:35 AM

Originally Posted by nova1

I was working with a 3.2GB file under Fedora Core 2 and I needed to replace the fopen() calls in main.cpp with fopen64() to get rid of the file too large error.

It's usually better to enable LFS through the gcc command line, typically in your Makefile:

Code:

CFLAGS += -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE

Large File Support in Linux.

**nova1** · 03-01-2005, 12:52 PM

Originally Posted by Jamie

It's usually better to enable LFS through the gcc command line, typically in your Makefile:

Code:

CFLAGS += -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE

Large File Support in Linux.

Thanks. I'll look to see what i did wrong.
I added the following to bbtool19 bits.cpp to dmux the vob and the fopen() still would not work unless I changed it to fopen64().

Code:

#define __USE_FILE_OFFSET64
#define _LARGEFILE64_SOURCE
#define __USE_LARGEFILE64
#define __REDIRECT

**Jamie** · 03-01-2005, 01:18 PM

Originally Posted by nova1

I added the following to bbtool19 bits.cpp to dmux the vob and the fopen() still would not work unless I changed it to fopen64().

Code:

#define __USE_FILE_OFFSET64
...

__USE_FILE_OFFSET64 doesn't look right to me.

On a POSIX system, getconf LFS_CFLAGS should tell you the right flags to use on your platform. If it's like mine (FC3), you'll get:

Code:

fedora[101] ~ % getconf LFS_CFLAGS
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

A typical thing to do for portability is to wrap this in the Makefile:

Code:

CFLAGS += `getconf LFS_CFLAGS`

**bcc** · 03-01-2005, 01:26 PM

Originally Posted by nova1

Here are some diffs on bcc's mplex with patch1 and 2 applied. These patches handle the case where an MPEG SEQUENCE_END header is seen and where a GOP header is not preceded by a SEQUENCE_HEADER(seen on a couple of VCDs).

Apply using patch -p1 <patch3.txt.

In the seq_pos = -1 case, the patch is bad: the code still falls through to the original memcpy.

**bcc** · 03-01-2005, 01:31 PM

Originally Posted by Jamie

__USE_FILE_OFFSET64 doesn't look right to me.

On a POSIX system, getconf LFS_CFLAGS should tell you the right flags to use on your platform. If it's like mine (FC3), you'll get:

Code:

fedora[101] ~ % getconf LFS_CFLAGS
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64

A typical thing to do for portability is to wrap this in the Makefile:

Code:

CFLAGS += `getconf LFS_CFLAGS`

We need something that is portable to windoze as well.

**bcc** · 03-01-2005, 01:32 PM

Also I'm already setting _LARGEFILE64_SOURCE in the TY I/O code, and that's been working fine with open() under linux, cygwin, and microsoft visual studio.

**Jamie** · 03-01-2005, 01:42 PM

Originally Posted by bcc

We need something that is portable to windoze as well.

That's messier. Here's a good reference.

**bcc** · 03-01-2005, 01:45 PM

Originally Posted by nova1

Thanks. I'll look to see what i did wrong.
I added the following to bbtool19 bits.cpp to dmux the vob and the fopen() still would not work unless I changed it to fopen64().

Code:

#define __USE_FILE_OFFSET64
#define _LARGEFILE64_SOURCE
#define __USE_LARGEFILE64
#define __REDIRECT

Try just #define _LARGEFILE64_SOURCE /* Needs to precede inclusion on features.h */
then #include <sys/types.h> before the other includes.

Thread: TY enabled mplex for mpeg to TY stream conversion

Thread Tools

Rate This Thread

Display

TY enabled mplex for mpeg to TY stream conversion

Compiling

Posting Permissions