ffmpeg with MPEG to TY+ encoding support

[bcc]

Enclosed are changes to ffmpeg which allows it to encode to .ty+ format. ffmpeg is superior to mplex for encoding to .ty+ format in that:
ffmpeg's multiplexor, with the enclosed changes, can handle directv streams with video frames that sometimes contain 3 fields/frame. This is a common mpeg encoding scheme employed by directv but not seen in dvd compliant mpeg. mplex malfunctions in such cases because it fails to synthesize the correct timestamps to match the variable frame duration.
With ffmpeg, one can now encode directly from .mpg or .vob to .ty+ without any separate demuxing step.
CREDITS:
bcc for: The new libavformat encoding library for ffmpeg, TY master chunk processing library, XML record generation, TY chunk record streaming library.
Nova1 for:
Original code demonstrating the relatively straightforward mapping from an elementary mpeg video stream to TY records
Original changes to more efficiently pack TY chunks

USAGE:
Notes: When specifying an a/v sync offset with -itsoffset, the order of the arguments matter.
Unlike mplex, ffmpeg won't work hard to glean the audio stream format, so if you're specifying that separately, you'll need to indicate whether it's ac3 or mpeg. In the following examples I do so by file name extension.
Converting from a .vob to .ty directly requires ffmpeg to identify the a/v streams. If it fails to do so (common it seems) you may still have to demux the streams first.
ffmpeg -i src.mpg -acodec copy -vcodec copy dest.ty
The above converts .mpg to .ty directly, without transcoding the video or audio
ffmpeg -i src.ac3 -itsoffset 0.798 -i src.m2v -acodec copy -vcodec copy dest.ty
In the above, the video stream start delayed 798ms after AC3 audio stream.
ffmpeg -i src.m2v -itsoffset 0.637 -i src.m2a -acodec copy -vcodec copy dest.ty
MPEG audio stream start delayed by 637ms after video stream.
COMPILING:
ffmpeg compiles under linux, and for windows, using cygwin.


Unfortunately the ffmpeg project does not seem to be maintaining snapshots of their code, so first checkout the code from their SVN repository. The diffs were made against revision 5510, so if you checkout the same revision, the diffs should patch cleanly even if the mainline has diverged since then:
svn checkout -r 6543 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
Or you could try the tivoserver snapshot of ffmpeg source.
cd ffmpeg
patch -p1 < ffmpeg-diffs-xx.txt
./configure --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
make
make install


There is a support thread over here: http://www.dealdatabase.com/forum/showthread.php?t=50010

[bcc]

Windows & linux binaries are here.
For windows, you'll need the DLL files from cygwin.rar as well. For windows, unpack the two .rar files, and put the .dll and .exe files in the same dir.

[Jamie]

Very cool. I'm embarrassed that I volunteered to look at this, then dropped the ball. Glad someone got to it. I'll give it a try.

[bcc]

Very cool. I'm embarrassed that I volunteered to look at this, then dropped the ball. Glad someone got to it. I'll give it a try.Thanks. You weren't the only other person who said you were going to try tackling it :) But you should probably be glad you didn't try - I had to do a bunch of wading through the ffmpeg code before I could get things to work and there were a couple nasty issues. In the end the diffs to ffmpeg came out pretty clean (cleaner than with mplex). So the ffmpeg API is fairly modular but their internals are poorly documented.

[Jamie]

So the ffmpeg API is fairly modular but their internals are poorly documented.Yeah, searching for good libav* documentation is about as far as I got.

To make another possibly empty promise, maybe I can look at integrating this into tivoserver. Tivoserver requires that ffmpeg can write to an unseekable stream, but, it doesn't need the at-end master chunk trickplay update. It seems to me they named a file extension type for ty without masterchunks, but I can remember what it is. Maybe another AVOutputFormat for ty without masterchunks is the way to go.

[bcc]

For tivoserver, I was going to make the code skip the master chunk processing if the file descriptor was a fifo, but punted because I think that would make the source less portable. A little less clean would be to make the code assume it's dealing with a fifo if the lseek fails.

[Jamie]

For tivoserver, I was going to make the code skip the master chunk processing if the file descriptor was a fifo, but punted because I think that would make the source less portable. A little less clean would be to make the code assume it's dealing with a fifo if the lseek fails.tmf would also be a nice addition, since, IME, mfs_ftp works better with tmf than it does with ty+. Naturally, that would require a seekable output stream. I can probably look at this. It doesn't sound too hard and should be able to reuse all the ty+ code with just a little extra processing at the end and between file parts.

[bcc]

tmf would also be a nice addition, since, IME, mfs_ftp works better with tmf than it does with ty+. Naturally, that would require a seekable output stream. I can probably look at this. It doesn't sound too hard and should be able to reuse all the ty+ code with just a little extra processing at the end and between file parts.I agree, tmf would be a bonus. It just always struck me as a bit wrong to spread tar file format knowledge to all the utilities to placate the insertion software. But fixing that backend would be harder, so...

[Jamie]

I agree, tmf would be a bonus. It just always struck me as a bit wrong to spread tar file format knowledge to all the utilities to placate the insertion software. But fixing that backend would be harder, so...The important thing (I think) is that the metadata (show duration, etc) are known at the beginning of the stream rather than at the end. The insertion code seems to be more reliable if all that stuff can be set up in MFS before the stream is inserted. I'm not sure there is a backend fix for that problem. Tar format is kind of irrelevant. We could make up yet-another-ty format with the xml preceeding the ty parts, but the tar file headers are simple to generate, so sticking with tmf seems better than inventing yet another file format.

[bcc]

I'm not disagreeing about tmf being a useful addition, and I certainly wouldn't suggest inventing a 3rd format. It's too bad that there are two already, with 1 that is problematic for utilities that wish to process recording streams, and 1 that is problematic for file transport and editing.

[phat_bastard]

What I've been able to watch has been flawless! The only issue I'm seeing is the master chunks after the first segment (chunk 4097 and up) don't seem to be valid so tivoserver is unable to parse the streams correctly and insertion (through tivoserver) causes the tivo to reboot when the transfer reaches the first corrupt master chunk. They're also crashing my ty chunk analyzer at the moment but what I saw before the app died was all FF bytes. I'll post additional details when I get a chance to dig into this a little further. I think everything is kosher with streams that aren't big enough to fill > 4096 chunks, though I haven't tested one just yet.

[phat_bastard]

After further investigation, the problem seems to be that the master chunk for the second fsid is empty - chunk size and chunk count are both zero. The crashing of my packet analyzer was happening on a padding chunk at the end of the sequence (no problems there).

[phat_bastard]

I think it might be something here:

tymaster.c::rewrite_master()
if (!parse_chunk(buf, paychunk, &lastseq, segsize)) {
if (chunkcnt && (chunkcnt-1) == pos) { // < ???
/* expected end of data chunks */
} else {
dbgprint("Warning: Found invalid chunk at %u\n", pos);
}
break;
}

Because I'm getting this:
Warning: Found invalid chunk at 4095
Generating master chunk for chunks 0:4094
Successfully wrote new XML

from muxing a stream that has 5972 chunks

Maybe just need to move the break up 3 lines?

[bcc]

The only issue I'm seeing is the master chunks after the first segment (chunk 4097 and up) don't seem to be validOops, I fudged the sanity checking trying to hush a cosmetic warning. Here's the fix:*** libavformat/tymaster.c 2006-06-21 09:33:27.000000000 -0700
--- libavformat/tymaster.c 2006-06-22 13:22:58.000000000 -0700
***************
*** 70,82 ****
boolean expected, have_seq;

recs = getshort(buf);
if (recs == 0xffff) {
! return FALSE;
}
p = &buf[4];
time = get64(&p[8]);
if (!time_validate(time, lasttime, ticks_per_chunk)) {
- expected = (chunkcnt == (segsize+1));
if (dbg(2)) {
dbgprint("Hit %stime discontinuity at chunk %d\n",
expected ? "expected " : "",
--- 70,82 ----
boolean expected, have_seq;

recs = getshort(buf);
+ expected = (chunkcnt == (segsize+1));
if (recs == 0xffff) {
! return expected;
}
p = &buf[4];
time = get64(&p[8]);
if (!time_validate(time, lasttime, ticks_per_chunk)) {
if (dbg(2)) {
dbgprint("Hit %stime discontinuity at chunk %d\n",
expected ? "expected " : "",

[phat_bastard]

That makes more sense, thanks! Testing...

Incidentally the ffmpeg-tivoserver.tgz snapshot has a few changes in libavformat that prevent merging your patch, but it's not a big deal. The svn snapshot seemed to compile okay under cygwin after I got off my duff and installed it (svn).

[darrin75]

Does this seem to be more reliable and stable for tivoserver??

[Jamie]

Does this seem to be more reliable and stable for tivoserver??In principle, eliminating tymplex should make the whole tivoserver transcode/remux process more reliable. The integration hasn't been done yet.

[bcc]

Incidentally the ffmpeg-tivoserver.tgz snapshot has a few changes in libavformat that prevent merging your patch, but it's not a big deal. The svn snapshot seemed to compile okay under cygwin after I got off my duff and installed it (svn).Hmm, I can tell that maintaining diffs against an ever changing baseline is going to be a problem.

[bcc]

I've made version 1.1 of the diffs, which fix the >4095chunk bug that phat_bastard saw, and also allows ffmpeg to write .ty to a fifo/pipe for tivoserver. (Diffs attached to first post of this topic).

[phat_bastard]

The only drawback I've noticed is that skip ahead / back trick play is very hit and miss. Most of the time I get taken back to the beginning of the stream. I'm guessing this has something to do with timestamps?

[bcc]

The only drawback I've noticed is that skip ahead / back trick play is very hit and miss. Most of the time I get taken back to the beginning of the stream. I'm guessing this has something to do with timestamps?
Trick play seems to break if the green trick play bar is sized wrong. Which in turn is likely if the XML data for your recording indicates the wrong length. Does that length look OK? Green bar size right?
Additionally trick play functionality requires the GOP start points to be set right in the master chunk records. I assume you're testing with the fix from yesterday. It's possible that ffmpeg is not "seeing" the start of all the GOPs. One should be able to compare mplex-ty vs ffmpeg-ty output to see if there's a difference in GOP counts.

[Jamie]

One thing I noticed is that ffmpeg with your patch is about 2-3 times slower when muxing mpeg. For example, I transcoded an avi to mpeg (not ty) with these results:
tivoserver_ffmeg snapshot
-----------------------------
real 10m22.209s
user 8m57.106s
sys 0m4.792s

latest svn without bcc patch
-----------------------------------------
real 10m25.782s
user 8m46.545s
sys 0m4.684s

latest svn +bcc patch
--------------------------------
real 26m22.175s
user 23m13.751s
sys 0m4.644sI profiled the ffmpeg+bcc code, and >80% of the time is in av_interleave_packet_per_dts. I inserted some debug there, and found that with the bcc patch, I'm getting lists with thousands of packets to interleave (typically 1 video and thousands of audio packets). The code isn't written in a way that makes that effecient. Without the bcc patch, the lists are typically 10-20 packets and less than 1% of the time is spent in the interleave code.

I was suspicious of the "scale video frame rate by 10" modification in the patch, and indeed, if I take that out, the performance goes back where it should be. I'm not quite sure what the right solution is. Perhaps both the video and audio frame rate need to be scaled by 10 to avoid the two streams being out of wack when being feed to the interleaver? Or perhaps the rounding issue that motivated the "scale by 10" hack can be attacked more directly?

[phat_bastard]

Trick play seems to break if the green trick play bar is sized wrong. ... Green bar size right?

Since the patch to fix the invalid master chunk the duration for all the streams I've watched has been correct. I even took a look at the duration of the original PS and they concur.

It's possible that ffmpeg is not "seeing" the start of all the GOPs. One should be able to compare mplex-ty vs ffmpeg-ty output to see if there's a difference in GOP counts.

I'll give this a look sometime in the next couple days.

Another thing I've noticed that may or may not be related is that on the streams that this is most evident in, the audio doesn't start up right away (though it is in sync when it does) when you begin playing it. Also, when I skip ahead and as luck would have it I'm not taken back to the beginning of the stream, most of the time the audio doesn't come back for a few seconds, or sometimes for up to a minute.

I also thought I'd mention that after muxing some dvd rips with this over the weekend, I've noticed that the trick play does seem to be somewhat more reliable on these, but the skip ahead 30 seconds function skips more like 5-10 seconds.

[bcc]

One thing I noticed is that ffmpeg with your patch is about 2-3 times slower when muxing mpeg. For example, I transcoded an avi to mpeg (not ty) with these results:Nice find.
Looks to me that the problem isn't with my changing the scale of the PTS timestamps per se. The DTS timestamps examined in the routine you mention are examined *after* they've already been scaled to actual DTS units (not units of .1 seconds).

Looks like the issue is with the frame rate as set in opt_input_file(). In my changes I'm setting this to the output frame rate, instead of leaving it uninitialized at the PAL rate of 1/25. Seems this change breaks things if your input file is .mpg. In my main test cases the input files were elementary streams and this was working fine.

So if you test muxing mpeg using separate a/v input files, I think you'll see it work OK.

Now, I was updating the frame in opt_input_file() because there was a bunch of output processing in ffmpeg.c that breaks when using '-vcodec copy'. The code breaks because it was using the uninitialized frame rate on the output side. I was trying to provide a centralized fix for this issue.

[bcc]

Jamie - Try version 1.2 of my diffs. I made my time base initializing and scaling changes more specific, so that they shouldn't be overwriting a good time base picked up from a .mpg file.

For .ty encoding, I got rid of an unnecessary level of buffer copying. Figured I better get to that before you profiled that part of the code :)

phat_bastard - if you've been feeding .mpg instead of .m2v to ffmpeg, you may have been having the same problems that jamie saw. This could also lead to incorrect a/v interleaving, not just bad performance. Might explain your trick play issues.

[Jamie]

Jamie - Try version 1.2 of my diffs. I made my time base initializing and scaling changes more specific, so that they shouldn't be overwriting a good time base picked up from a .mpg file.Much better. Thanks. I'm getting pretty good results converting .avi directly to .ty now.

[shstevens]

is there a support thread or can i ask questions here? :^)

thanks,
shawn

[bcc]

Ok, I've just added a support thread over here:
http://www.dealdatabase.com/forum/showthread.php?t=50010
since that's the convention around here.

[phat_bastard]

Additionally trick play functionality requires the GOP start points to be set right in the master chunk records.

Can you elaborate on this? I wasn't aware of the master chunks containing anything of importance other than the chunk count.

It's possible that ffmpeg is not "seeing" the start of all the GOPs. One should be able to compare mplex-ty vs ffmpeg-ty output to see if there's a difference in GOP counts.

It doesn't seem to be losing any GOPs or I-Frames. I also tested with the build I made from last week, and the latest binary you posted and the output matches exactly. I'm using elementary streams for this exclusively, BTW.

[BTUxNine]

The master chunk contains a bitmap indicating which chunks contain an I frame, used almost exclusively for trick play. When a recording is xferred using MRV, as opposed to mfs_ftp, that information is NOT sent... the tivo regenerates it. If you try to xfer with both tivoserver and mfs_ftp, and the trick play is different, it could be problems with that bitmap.

[bcc]

Ok, I've posted a new version of the source patches, version 1.3. Changes: Completely skip generation of master chunks if writing to a pipe/fifo (for tivoserver)
New switch -x to include template XML.Someone let me know if this new XML support does what is wanted. I coded this differently than before in that I'm simply merging the template XML with the dynamically generated XML. The dynamically generated XML is appended to the old, with the assumption that the later values will supersede the template values during insertion. This keeps ffmpeg from having to use libxml.

[shstevens]

i will give that a try tonight (currently encoding some stuff with the old version of ffmpeg and i don't want to run two vesions at once and freak windows out, heh). any chance you could put a windows binary together? unfortunately i have never gotten into compiling (i can build a mean network though :^)

if anyone is an ffmpeg guru and can help me out, i would love to figure out the right command line arguments to downconvert the bitrate. currently i have had to express all of the video attributes (aspect, resolution, bitrate, and frame rate) to get it to properly pull the video out. i love the idea of not doing anything to a stream unless i have to though (in this case down converting the bitrate is something i consider i have to do)...

thanks,
shawn

[phat_bastard]

When a recording is xferred using MRV, as opposed to mfs_ftp, that information is NOT sent... the tivo regenerates it. If you try to xfer with both tivoserver and mfs_ftp, and the trick play is different, it could be problems with that bitmap.

Strange, of the handful of short sample streams I've tested before and after transferring with MRV, the first N bytes of the (first and only) master chunk match exactly. After byte N the synthetic ty master chunk contains all nulls, so I'd assume the regenerated master chunk (after tivoserver > mrv > tivo) is just uninitialized random bits. There are more than [nchunks / 8] bytes after the header, so there must be something else here, but I don't think it's relevant.

This is such a minor issue that it seems foolish to even look at it. I'm suprised no-one else is noticing it, but then my OCD may be worse than my shrink has led me to believe. :)

BCC: Thanks for adding the XML stuff btw, very nice. I'll try to test it out and report back if no-one beats me to it.

[shstevens]

thanks for the 1.3 binary post update. trying the xml now :^)

shawn

[bcc]

thanks for the 1.3 binary post update. trying the xml now :^)
Welcome. I wasn't awake enough last night to post a notice.

[shstevens]

Not sure if your chunk issues are related to this, but the output seems a little strange when it is generating the master chunks.

Multiplexing Audio and Video into TY format...
FFmpeg version SVN-r, Copyright (c) 2000-2004 Fabrice Bellard
configuration: --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
libavutil version: 49.0.0
libavcodec version: 51.9.0
libavformat version: 50.4.0
built on Jun 29 2006 01:28:36, gcc: 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)
could not open '-y'
Input #0, mpeg, from 'd:\vobs\Slapshot.VOB':
Duration: 02:03:00.1, start: 0.280633, bitrate: 7802 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480, 9800 kb/s, 29.97 fps(r)
Stream #0.1[0x80]: Audio: ac3, 48000 Hz, stereo, 192 kb/s
Output #0, ty, to 'd:\tys\Slapshot.ty':
Stream #0.0: Video: mpeg2video, yuv420p, 720x480, q=2-31, 2500 kb/s, 29.97 fps(c)
Stream #0.1: Audio: ac3, 48000 Hz, stereo, 192 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
Generating master chunk for chunks 0:4095379.5 bitrate= 0.0kbits/s
Generating master chunk for chunks 4096:8191
Generating master chunk for chunks 8192:12287
Generating master chunk for chunks 12288:16383
Generating master chunk for chunks 16384:19081
Successfully wrote new XML
frame=221204 q=2.0 Lsize= 0kB time=7380.8 bitrate= 0.0kbits/s
video:1777972kB audio:172988kB global headers:0kB muxing overhead -100.000000%
'el' is not recognized as an internal or external command,
operable program or batch file.

D:\vobs>

not sure what the:
el' is not recognized as an internal or external command,

is, but it could be a typo in my batchfile....

Just thought the part on creating master chunks looked odd, and it mentioned chunks so I thought i would post it :^)

Shawn

[bcc]

The batch file error would be from your batch script.
I also note you got a "could not open '-y'" error, so your command line must be messed up.
The "chunks 0:4095379.5" part is because ffmpeg and my libraries are cross talking with their diagnostic output. Looks like I should make my libraries use stderr to appease ffmpeg.

[shstevens]

yes, i do have some commandline issues. i forgot the XML file name so -y was the next arguement and it thought that was my xml file.

wasn't sure on the chunk stuff. looked odd, so i posted it. you know much better what is really going on though, heh.

working it out now... have to go get a sammich though....

shawn

[phat_bastard]

No luck with xml yet - but I may be doing something stupid. The dynamic xml seems to get inserted correctly, then...

6 [main] ffmpeg 4004 _cygtls::handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
216806 [main] ffmpeg 4004 open_stackdumpfile: Dumping stack trace to ffmpeg.exe.stackdump

Here's the xml I was feeding it:
<?xml version="1.1" tivoversion="6.2-01-2-151"?>
<Object type="Recording" id="_top">
<SubObject type="Showing" id="Showing">
<Object type="Program" id="Program">
<DescLanguage>English</DescLanguage>
<Description>Description</Description>
<Title>Title</Title>
</Object>
</SubObject>
<StartTime>0</StartTime>
<StopTime>0</StopTime>
<StreamFileSize>0</StreamFileSize>
</Object>

Is it my xml?

[bcc]

Is it my xml?Yes, your XML needs to be bounded by 512 # marker bytes. If you could see the ffmpeg output before the crash you should have seen a hint, such as:Generating master chunk for chunks 0:27
Found XML but rewrite failed, appending instead...
Successfully wrote new XMLNot sure why this makes ffmpeg crash, but it's crashing after your file is complete (but without the XML template you want) :)

[BTUxNine]

re: appended xml... I'm afraid just appending the fields may not work well with tivoserver... certain fields (actor, genre, etc.) have multiple entries, so it's set up NOT to supercede any of the data... it aggregates it

re: logging... if those messages are being sent to stdout, then it'd certainly help if they were changed over to stderr (or the ffmpeg logging) because stdout messages would break piping to stdout

[phat_bastard]

bcc: I attempted to apply the 1.3 diffs to the current SVN and I'm running into some errors. Would you mind verifying that they merge against the current snapshot? I'm not really familiar with tweaking diff files, but it looks like there might be a few lines miffed in this latest one.

I'm wanting to look at the xml append routines and possibly mod them to:
1) use one or the other but not both (internal or external xml)
2) hopefully insert only the showing duration and file size tags if using external xml
3) generate the delimiter chars

[bcc]

re: appended xml... I'm afraid just appending the fields may not work well with tivoserver... certain fields (actor, genre, etc.) have multiple entries, so it's set up NOT to supercede any of the data... it aggregates itThe objects dynamically updated with my code are simple objects that are not comprised of lists. The user supplied template comes first, then these dynamically changed objects. So it wouldn't be hard to have the objects get superseded in the order they are listed, as with mfs_ftp.
Yes it would be cleaner to just #define HAVE_XML_LIB and include my xml.c, and libxml. Then the objects will be overwritten instead. ffmpeg could be compiled for tivoserver this way.

Or maybe ffmpeg shouldn't need to deal with XML at all if it's being used with tivoserver.re: logging... if those messages are being sent to stdout, then it'd certainly help if they were changed over to stderr (or the ffmpeg logging) because stdout messages would break piping to stdoutHmm, non-error informative message to stderr doesn't make much sense to me. And ffmpeg is actually already inconsistent about this - in ffmpeg's usage output, half the output goes to stdout, and half to stderr.
Personally I don't want to make my log messages directly reference ffmpeg's log facility, as the libraries are useful without ffmpeg, but it'd be OK for dbgprint() to hook in to ffmpeg I suppose. For now I've simply made dbgprint use stderr.

[bcc]

bcc: I attempted to apply the 1.3 diffs to the current SVN and I'm running into some errors. Would you mind verifying that they merge against the current snapshot? I'm not really familiar with tweaking diff files, but it looks like there might be a few lines miffed in this latest one.I checked and it looks like they made some cosmetic changes that make 1 of the hunks fail to apply. I would suggest using an older SVN revision. I'm currently on 5510. So you could just use '-r 5510' during checkout.

I'm wanting to look at the xml append routines and possibly mod them to:
1) use one or the other but not both (internal or external xml)
2) hopefully insert only the showing duration and file size tags if using external xml
3) generate the delimiter chars
1. You get just auto-generated XML if you leave off the new -x argument
2. ? Show duration & file size are already handled in the auto-generated XML
3. Already handled in the auto-generated XML

I've also got some new code that handles a user supplied template with or without the '#' marker bytes.

[phat_bastard]

I checked and it looks like they made some cosmetic changes that make 1 of the hunks fail to apply. I would suggest using an older SVN revision. I'm currently on 5510. So you could just use '-r 5510' during checkout.

Thanks, will try it out.

1. You get just auto-generated XML if you leave off the new -x argument
2. ? Show duration & file size are already handled in the auto-generated XML
3. Already handled in the auto-generated XML

I've also got some new code that handles a user supplied template with or without the '#' marker bytes.

How about this idea: Since, for all intents and purposes the only thing most of us are trying to accomplish with external xml is to populate Description, MpaaRating, and Title, why not just pull those three fields from a file instead of messing around with the entire tempate and the additional overhead of dealing with xml? I've never been a big fan of it to start with, so I'm sure a good old fashioned config file with it's [variable]=[value][\n] layout would be just fine for everyone, and should be relatively simple to implement.

[phat_bastard]

Hell, for that matter what would be wrong with specifying these things as command line arguments? The mechanisms for this are already in place.

[BTUxNine]

I don't think adding more xml handling to ffmpeg is the best idea... among other things, the farther you stray from ffmpeg, the more issues in keeping the patch up to date.

I'd propose an alternative... leave the xml stuff out of ffmpeg completely, and write a program to add/modify the xml at the end... should be trivial enough to do in most scripting languages

[phat_bastard]

I checked and it looks like they made some cosmetic changes that make 1 of the hunks fail to apply. I would suggest using an older SVN revision. I'm currently on 5510. So you could just use '-r 5510' during checkout.

Same result. No matter though.

I added the bounding characters to my xml templates and the tags that I was concerned with work just fine with tivoserver using the appended xml method. Guess btu's comments incited an unwarranted panic in me. :eek:

The lack of bounding characters is probably why I was never able to get xml working with mplex. :)

[T u r b o]

The ffmpeg patch works much better for me than the modified mplex did. Nice work!

I know this isn't 100% on topic here but you guys are familiar with the mpeg code in ffmpeg. I have some VOB files that incorrectly identify their aspect ratio in the MPEG video stream. This causes my TiVo to display them letterboxed.

Can anyone provide some insight into the best place to put a routine to patch the aspect ratio before it gets written out to the .ty file?

Also, I'm running on Mac OSX and could contribute a patched ffmpeg binary (PPC) if anybody is interested.

EDIT: Oh yeah I meant to add that I'm using -vcodec copy to avoid re-encoding the video. If I use -vcodec mpeg2video and -aspect 4:3 it works as expected.

[shstevens]

Same result. No matter though.

I added the bounding characters to my xml templates and the tags that I was concerned with work just fine with tivoserver using the appended xml method. Guess btu's comments incited an unwarranted panic in me. :eek:

The lack of bounding characters is probably why I was never able to get xml working with mplex. :)

I am still having trouble with XML appending (see my post here http://www.dealdatabase.com/forum/showpost.php?p=259111&postcount=42 for more details please).

I believe I have added the boundry characters correctly, but the XML is still failing to read. This is the last hurdle I have in using this awesome new option. It has made a really big difference to me and I am grateful.

Thanks,
Shawn

[bcc]

Same result. No matter though.Not sure what you mean. If you check out rev. 5510 of the svn source, the code should patch cleanly.

On supporting an XML related config file, or command line features: I think this is too much XML featurism for a program that is supposed to just be a multiplexor/demultiplexor/format converter. It made sense to me to provide minimal XML support, as some XML objects cannot be left unupdated or unset after a remultiplex, without introducing problems. Those are included in the ones already dynamically updated by the XML support.
Basically I agree with BTUxNine. However if one was to remove XML support from ffmpeg, the post processing to update the dynamic objects wouldn't be as trivial as suggested. To compute the Duration object's value, you have to go back to the recording and look at the elapsed time in seconds.
I added the bounding characters to my xml templates and the tags that I was concerned with work just fine with tivoserver using the appended xml method.Glad you got it working

[bcc]

I have some VOB files that incorrectly identify their aspect ratio in the MPEG video stream. This causes my TiVo to display them letterboxed. Sounds like you want dvdpatcher. I'm sure you could hack ffmpeg to do the same - have at it if you want.

[phat_bastard]

Not sure what you mean. If you check out rev. 5510 of the svn source, the code should patch cleanly.

Still the same problem. Now it works, not sure what I did differently. Whatever it was, the world is better off without my meddlings in C.

On supporting an XML related config file...

Nevermind my rantings about that. I took btu's comments to mean that tivoserver would balk at the way the dynamic xml is appended to the template xml. It doesn't, so no problem to solve.

The only thing I thought was off and needed to be fixed was the need for bounding characters in the template xml. I've read and searched here quite a bit trying to get the xml stuff working in mplex and never saw anything that suggested the need for bounding characters. I'm guessing that's why I never got mplex to insert my xml. :) I modded my converter script in about two minutes to conform to this requirement.

At any rate, thanks for all your work on this. You've saved me lots of CPU cycles, gigs of disk space and the frustration of watching the shimmer / jitter caused by de-interlacing. I can now mux stuff directly in about 2 minutes that used to take an hour or more re-encoding. I'm still seeing a lot of instances where the skip ahead button takes you back to the beginning of the stream, but since I can cut the commercials and still have flawless playback of all my interlaced DVB-S rips, I'm more than happy to adapt the wife to using the ffwd button. :)

I'm guessing the trick play issue is related to something you mentioned about ffmpeg rounding timestamps. I'm not complaining, but if you're insterested - if I try to convert directly from the interleaved source instead of demuxing the elementary streams I see rafts of this:

error, pts < dts

and this:

error, non monotone timestamps 9714705 >= 9714705

in my logfiles, and if ffmpeg manages to finish the mux process then playback is horrendous. If I demux using external tools all is well, so I'm happy either way.

[bcc]

Those look like legit gripes about how the timestamps in your recordings are layed out. I wonder what format you're feeding to ffmpeg (mpeg, .ts), and what you used to multiplex the streams before ffmpeg. Looks like whatever multiplexor it was, it might have been having the same sort of problem as mplex was with variable fields per frame. Passing elementary streams to ffmpeg avoids this problem as it leaves it up to ffmpeg to generate the timestamps instead.

As for your trick play issues, I've still not seen enough info to do more than continue to speculate. I wonder if you're only seeing an issue on these special case dvb transfers.

[T u r b o]

Sounds like you want dvdpatcher. I'm sure you could hack ffmpeg to do the same - have at it if you want.

Yeah I've written C code to patch the mpeg headers that's no problem, I was really just inquiring if you had any input on where the patching should occur since you're (significantly) more familiar with ffmpeg than I am.

[bcc]

Well I would think the clean thing would be to add a video codec to libavcodec that patches the video headers without causing a reencode. Thankfully I didn't have to dig into libavcodec much so I don't really have much experience with it.

[phat_bastard]

...I wonder what format you're feeding to ffmpeg (mpeg, .ts), and what you used to multiplex the streams before ffmpeg. Looks like whatever multiplexor it was, it might have been having the same sort of problem as mplex was with variable fields per frame.

It's all mpeg program stream; the newer tivo rips came out of TyTool10r4 and the dish stuff was muxed with either womble's mpeg-vcr or PVRExplorer Pro (http://sourceforge.net/projects/pvrexplorer-pro). The only reason I tried this out was to save a few minutes in conversion, but it's probably better to demux prior to ffmpeg since it gives me the opportunity to drop unwanted audio tracks. The demuxer I use does a pretty good job of finding the correct audio delay (or video delay) as well.

Passing elementary streams to ffmpeg avoids this problem as it leaves it up to ffmpeg to generate the timestamps instead.

As for your trick play issues, I've still not seen enough info to do more than continue to speculate. I wonder if you're only seeing an issue on these special case dvb transfers.

Yes, this is only happening with hybrid (interlaced / progressive) source. Since the playback is flawless, my vote is not to worry about it since there's almost no need for the trick play buttons. If you're just really bored or have some other reason to look into it, you can test with the source ty file of the sample I sent you back when you located the issues in mplex.

[bcc]

I've uploaded a new version, 1.4. Changes: xml markers now optional for -x, better diagnostics for xml rewrite
Diagnostic logging to stderr to match ffmpeg
Fix frame rate processing when frame rate specified via command line (-r)

[bcc]

I've updated the checkout instructions so that it's obvious how to checkout the SVN revision from which the diffs were made (no conflicts from changes made to the mainline after my diffs were generated).

[rbautch]

I'm not complaining, but if you're insterested - if I try to convert directly from the interleaved source instead of demuxing the elementary streams I see rafts of this:

error, pts < dts

and this:

error, non monotone timestamps 9714705 >= 9714705

in my logfiles, and if ffmpeg manages to finish the mux process then playback is horrendous. If I demux using external tools all is well, so I'm happy either way.I get similar errors, but only on high-def program streams extracted, edited, and vob-muxed with tytool. The problem goes away when I use the "Multiplex (new format) 2" option, but this (http://www.dealdatabase.com/forum/showpost.php?p=201526&postcount=662) vintage post leads me to believe that's not the best solution. Splitting the files with tytool resulted in an m2a audio stream that ty-ffmpeg didn't recognize. I'll try a few more options/variables and report back.

edit: I demuxed a tytool-created program stream with tmpgenc, and the resulting elemental streams worked fine with ty-ffmpeg. I can't notice any quality differences between this, the stream that was muxed with "Multiplex (new format) 2" in tytool, and the original recording.

edit2: Washing the tytool-created program stream through Videoredo also fixed the timestamp issue. Think I'll start using Videoredo to make my cuts.

[stealthdave]

Just wondering if anyone had looked at adding ty input capabilities to ffmpeg? The ty+ output works great! I'm using it to upload dailies of our short film (TiVo rocks for skipping through raw footage!), but it would be really useful for some of my other projects to have ffmpeg to be able to read the ty format as well. For starters, one could use it as a ty -> dvd converter in a single command-line function. It would also be useful for streaming TiVo programs in other formats, or even just for simple conversion. Would hdemux be a good place to borrow code from?

btw, this is a fantastic and extremely useful tool! Thanks for all the hard work! Are you considering submitting this patch to Fabrice (the ffmpeg maintainer) for inclusion into the ffmpeg source tree?

- Stealth Dave

[lgkahn]

already done.. use mplex with the -f 10 (.ty format flag) to split the file into standard m2v m2a files. and then use ffmpeg to put back together.

this is a special version of mplex that supports it...

[BTUxNine]

umm... mplex is a muxer, not a demuxer. He referenced the right tool (hdemux) for demuxing already, but is interested in a single tool to do both jobs (i.e. adding ty as a supported ffmpeg decode codec).

Not sure there's enough interest to support that, though you're welcome to give it a shot, stealthdave.

[rc3105]

Just wondering if anyone had looked at adding ty input capabilities to ffmpeg? The ty+ output works great! I'm using it to upload dailies of our short film (TiVo rocks for skipping through raw footage!), but it would be really useful for some of my other projects to have ffmpeg to be able to read the ty format as well. For starters, one could use it as a ty -> dvd converter in a single command-line function. It would also be useful for streaming TiVo programs in other formats, or even just for simple conversion. Would hdemux be a good place to borrow code from?
...
- Stealth Dave
ty parsing in ffmpeg would definatly be handy. those who aren't technical enough to implement it can use mplayer

mplayer/mencoder supports direct playback/transcoding of ty files. usually it'll auto-detect the format, if not use the option "-demuxer 33"

mplayer -demuxer 33 whatever.ty

mencoder whatever.ty -o whatever.mpg -demuxer 33 -of mpeg -oac lavc -ovc lavc -lavcopts vcodec=mpeg2video:vhq:autoaspect:trell:acodec=mp2:sameq

[bcc]

Just wondering if anyone had looked at adding ty input capabilities to ffmpeg? The ty+ output works great! I'm using it to upload dailies of our short film (TiVo rocks for skipping through raw footage!), but it would be really useful for some of my other projects to have ffmpeg to be able to read the ty format as well. For starters, one could use it as a ty -> dvd converter in a single command-line function. It would also be useful for streaming TiVo programs in other formats, or even just for simple conversion. Would hdemux be a good place to borrow code from?Yes it would probably be feasible to merge hdemux support into ffmpeg, and that would be the best way to get ty support into ffmpeg if one wanted to. It would improve ease of use. On the other hand, I already have code that merges mplex into hdemux, and have written my own multiplexer, and also other tools such as tivotool automate the process as well. I would hesitate to add yet another way to do the same thing without first better evaluating mplex's multiplexing results vs ffmpeg's. I thought I would have seen more insight on that from here, but most of the feedback has instead largely been about bell&whistle items such as XML. Also there's some value in keeping things separate so that it's clear where problems have occurred, as they invariably do.

I may get to it anyways, but not this month. That's not to say someone else can't step up...btw, this is a fantastic and extremely useful tool! Thanks for all the hard work! Are you considering submitting this patch to Fabrice (the ffmpeg maintainer) for inclusion into the ffmpeg source tree?

- Stealth DaveThanks. As my encoding&decoding libraries were written for standalone applications, and for integration with other tools as well, they do not strictly conform to ffmpeg's coding style, and I'm not sure I'd want to try to make them conform. I think it'd get to be too much like a day job to try to juggle all that 3rd party integration (with no payback, and for code that some would claim violates DMCA). Maybe the ffmpeg project wouldn't even want the code because of those DMCA concerns. On the other hand the ffmpeg API has so far proven much cleaner than mplex. If it wasn't for ffmpeg's incomplete API documentation I may have never started working with the seemingly easier mplex first...

[bcc]

ty parsing in ffmpeg would definatly be handy. those who aren't technical enough to implement it can use mplayer

mplayer/mencoder supports direct playback/transcoding of ty files. usually it'll auto-detect the format, if not use the option "-demuxer 33"

mplayer -demuxer 33 whatever.ty

mencoder whatever.ty -o whatever.mpg -demuxer 33 -of mpeg -oac lavc -ovc lavc -lavcopts vcodec=mpeg2video:vhq:autoaspect:trell:acodec=mp2:sameq
I know mplayer can be used as a poor man's demultiplexer, but I don't think it's a very good choice. Mplayer code takes a very heuristic approach to parsing ty, which may be good enuf for media players, but it doesn't understand master chunks, chunk segment lengths, the garbage chunks that can appear at the end of streams, and the various formats that audio PES headers may be written in. hdemux understands all this.

The fact that you often have to pass "-demuxer 33" to mplayer is a bug that is a symptom of its heuristic parsing.

In case it's not clear, I recommend hdemux instead :)

[dishdude]

Can an XML file be used to assign a folder and program descriptions once uploaded to the tivo? This works great i would just rather have a folder called "movies" instead of all of them showing up under the root of now showing.

[lgkahn]

couple of questions trying first dvd /vob conversions all other methods resulted in audio video sync issues.. will let you know the results..

I am getting the following error on one rip

error, non monotone timestamps 89280

any ideas

also since your example is a straight copy the bit rate seems too high at 7000 -9000 kb/s per dvd.. should I try setting it lower but then I assume I cannot use the straight stream copy as you specified in the example..

thanks

[lgkahn]

also I have a movie with subtitles anyone know how to use -scodec to get the subtitles converted

I tried -scodec copy
but it didn't work
thanks

[ronkuba]

I search and can't find how to install ffmpeg in Windows XP. If someone could be so kind and point me in the right direction or give me the instructions on how to install it and how to convert to .ty. Thanks in advance for your help.

[darrin75]

don't double post, it can get your in trouble on this board.

[bcc]

I search and can't find how to install ffmpeg in Windows XP. If someone could be so kind and point me in the right direction or give me the instructions on how to install it and how to convert to .ty. Thanks in advance for your help.The second post in this thread provides pre-compiled windows binaries. The first post shows you how to use the program. Further details about conversion help belong in the support thread.

[bcc]

I've uploaded a new version, 1.5. Changes:
Synced to newer ffmpeg snapshot (allowing some vc1/wmv3 decoding)
Unfortunately the source code repository for ffmpeg seems like a minefield of problems (patches that break compiles, and mess up frame rates). So this will have to be considered an alpha release, as I'm uncertain what might have been broken in the mainline that I haven't noticed yet.
Fixes for mpeg2 encoding: Allow for HD video frame rates, fix bug in multiplexer's notion of the "predecode" queue. ffmpeg is now the preferred multiplexer for hdemux, replacing mplex. (This 1.5 release supports both ty->mpg conversion with hdemux+ffmpeg, as well as encoding to .ty)
For windows users who compile from source, you'll need to reverse apply the following patch to libavcodec/mpegaudiodec.c.Index: mpegaudiodec.c
===================================================================
--- mpegaudiodec.c (revision 6256)
+++ mpegaudiodec.c (revision 6290)
@@ -430,9 +430,9 @@
for(i=0; i<512*16; i++){
int exponent= (i>>4);
double f= pow(i&15, 4.0 / 3.0) * pow(2, (exponent-400)*0.25 + FRAC_BITS + 5);
- expval_table[exponent][i&15]= lrintf(f);
+ expval_table[exponent][i&15]= llrint(f);
if((i&15)==1)
- exp_table[exponent]= lrintf(f);
+ exp_table[exponent]= llrint(f);
}

for(i=0;i<7;i++) {

[pahunt]

Please ignore - posted in the wrong thread

[Lowlight]

Here's how I used this version of ffmpeg to rip a DVD straight to a ty+ file on my FC5 linux box, wrap it into a tmf file and then insert it into my HR10-250. Worked like a champ. The one thing that screwed me up for a bit was not realizing that ffmpeg was making ty+ not plain ty files. I also had problems getting mfs_ftp to insert the ty+ file, thus the use of the ty+2tmf.pl script (found here on ddb).

vobcopy -l -o -|~/ffmpeg -y -i - -vcodec copy -acodec copy HarryPotter-GobletOfFire.ty
ty+2tmf.pl HarryPotter-GobletOfFire.ty
ncftpput -E -P 3105 192.168.0.x /tmf HarryPotter-GobletOfFire.tmf


I've also managed to use similar steps to those above to convert a wmv3 HD trailer to a 1280x720, 7000k tmf that I was able to insert and play on my HR10-250. The command looked something like this.

~/ffmpeg -i trailerHD.wmv -vcodec mpegvideo -b 7000k -r 29.97 -aspect 16:9 -acodec mp2 -ab 192 foo.ty

Here's the output:
Seems that stream 1 comes from film source: 1000.00 (1000/1) -> 29.97 (30000/1001)
Input #0, asf, from 'trailerHD.wmv':
Duration: 00:01:01.2, start: 3.000000, bitrate: 2043 kb/s
Stream #0.0: Audio: wmav2, 44100 Hz, stereo, 96 kb/s
Stream #0.1: Video: wmv3, yuv420p, 1280x720, 29.97 fps(r)
File 'foo.ty' already exists. Overwrite ? [y/N] y
Output #0, ty, to 'foo.ty':
Stream #0.0: Video: mpeg2video, yuv420p, 1280x720, q=2-31, 7000 kb/s, 29.97 fps(c)
Stream #0.1: Audio: mp2, 44100 Hz, stereo, 192 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Stream #0.0 -> #0.1


It would be cool if this ffmpeg could directly generate the tmf file but I'm happy with all the hard work that's been put into this already. I'm still figuring out the xml "stuff" that it can already do now. Personally I think I'd like to see mfs_ftp get a little more xml-fu but this isn't the thread for that. Edit: I found what I wanted here http://dealdatabase.com/forum/showpost.php?p=271451&postcount=27

[joeybubbabob]

Is there any chance the patch can be updated to apply to a more recent svn revision of ffmpeg?

There seems to be a bug in ffmpeg rev. 6543 that has since been fixed. If I try encoding a file with blank video (just plain black frames), but with audio then the output is severely truncated (from 30 mins down to 7 secs). It's not a ty issue as it happens with mpg being the destination format as well. The problem also appears on regular videos: a few seconds from the start and end get cut off, presumably because of the black frames at the start and end of the video. I tested with a rev 7165 build and the problem was no longer there.

[bcc]

It would be cool if this ffmpeg could directly generate the tmf file but I'm happy with all the hard work that's been put into this already. I saved that part for Jamie since he volunteered. Or maybe someone else wants to do it, it shouldn't be hard.

[bcc]

Is there any chance the patch can be updated to apply to a more recent svn revision of ffmpeg?Go ahead and do the merge and post the results (preferably with linux&windows binaries); I have my hands full with tytompg.

[joeybubbabob]

Go ahead and do the merge and post the results (preferably with linux&windows binaries); I have my hands full with tytompg.

I've made a patch that applies to the current ffmpeg svn, it compiles and runs fine but it spits out
Compiler did not align stack variables. Libavcodec has been miscompiled and may be very slow or crash. This is not a bug in libavcodec, but in the compiler. Do not report crashes to FFmpeg developers.
whenever you run it. I'm told that using gcc 4.1 and up will fix this message, which I do not have.

Apply the patch with "patch -p2 < ffmpeg-diffs-1.6.txt".

The patch also includes a different fix for the llrint problem: I added this small implementation of llrint (http://www.cygwin.com/ml/cygwin/2006-11/msg00665.html).

[drez]

Hey, I tried your patch and got:

[dre@localhost ffmpeg]$ patch -p2 < ffmpeg-diffs-1.6.txt
patching file config.h
patching file ffmpeg.c
patching file libavcodec/mpegaudiodec.c
Reversed (or previously applied) patch detected! Assume -R? [n] n
Apply anyway? [n] y
Hunk #1 FAILED at 295.
1 out of 1 hunk FAILED -- saving rejects to file libavcodec/mpegaudiodec.c.rej
patching file libavformat/Makefile
patching file libavformat/allformats.c
patching file libavformat/allformats.h
patching file libavformat/file.c
patching file libavformat/mpeg.c
patching file libavformat/mpeg.h
patching file libavformat/ty-enc.c
patching file libavformat/ty.c
patching file libavformat/tydefs.h
patching file libavformat/tyexterns.h
patching file libavformat/tymaster.c
patching file libavformat/tyos.h
patching file libavformat/typrotocol.h
patching file libavformat/tyutils.c
patching file libavformat/utils.c
patching file libavformat/xmladd.c


did I do something wrong? also, I had to run the configure script first with this patch or I would get another error about config.h missing. (the instructions in the first post say to run the patch first.)


deleted the folder, checked out the latest svn (7323) again, ran configure, and said yes this time. i started compiling it.. got a lot of warnings (way too many to list... I'll have to increase my scrollback,) it stopped here:

gcc -L"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavformat -L"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavcodec -L"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavutil -Wl,--warn-common -rdynamic -export-dynamic -Wl,--as-needed -Wl,-rpath-link,"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavcodec -Wl,-rpath-link,"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavformat -Wl,-rpath-link,"/home/dre/Desktop/tyffmpeg/ffmpeg"/libavutil -g -o ffmpeg_g ffmpeg.o cmdutils.o -lavformat -lavcodec -lavutil -lm -ldl
ffmpeg.o: In function `opt_input_xml':
/home/dre/Desktop/tyffmpeg/ffmpeg/ffmpeg.c:2563: undefined reference to `xml_buf'
/home/dre/Desktop/tyffmpeg/ffmpeg/ffmpeg.c:2568: undefined reference to `xml_size'
/home/dre/Desktop/tyffmpeg/ffmpeg/libavformat/libavformat.a(allformats.o): In function `av_register_all':
/home/dre/Desktop/tyffmpeg/ffmpeg/libavformat/allformats.c:113: undefined reference to `mpegty_muxer'
collect2: ld returned 1 exit status
make: *** [ffmpeg_g] Error 1


This is on FC6 / gcc version 4.1.1 20061011 (Red Hat 4.1.1-30)

[joeybubbabob]

Sorry about that, I'm new to this patching process.

Here is a new patch and instructions to patch and build.

1. svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
If the patch does not apply cleanly to the current svn, then checkout revision 7323 with the command line
svn checkout -r 7323 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg

2. cd ffmpeg

3. patch -p2 < ffmpeg-diffs-1.6a.txt

4. ./configure --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver --enable-avisynth

5. make

6. make install

[drez]

The patch worked fine with r7328 (http://svn.mplayerhq.hu/ffmpeg/trunk/?view=log). I compiled it in FC6.

I had to drop --enable-avisynth because you can only use that option when building in Cygwin/MinGW (http://ffmpeg.mplayerhq.hu/faq.html#SEC20).

I'm going to try MinGW once I reboot into XP. If that fails, I'll try Cygwin..

[Lowlight]

Drez - New binary doesn't work on FC4 box that the 1.5 version did work on.

$ ./ffmpeg-ty -h
Floating point exception

[drez]

Drez - New binary doesn't work on FC4 box that the 1.5 version did work on.

$ ./ffmpeg-ty -h
Floating point exception

It seems to be an issue with FC6 having a newer glibc. Somebody will have to compile it in an older environment.

http://www.x-tend.be/~raskas/blog/2006/11/30/perl-binary/
http://forums.fedoraforum.org/showthread.php?t=141402



I'm going to try MinGW once I reboot into XP. If that fails, I'll try Cygwin..

Regular ffmpeg compiled fine in MinGW but ty-ffmpeg would fail on ty-enc.o.... So I tried Cygwin.

Like joeybubbabob mentioned, if you didn't compile ffmpeg in GCC 4.1.1+, you'd get a "Compiler did not align stack variables" warning when you ran ffmpeg (ffmpeg would work correctly with that error though.)

So I looked for GCC 4.1.1 for Cygwin... After trying to compile gcc 4.1.1 myself, I eventually found this: http://www.generic-programming.org/software/ConceptGCC/download.php. It has a regular GCC 4.1.1 Cygwin binary and its own variant of GCC, ConceptGCC.



I've attached a Cygwin binary of ty-ffmpeg 1.6 (r7329) and the latest Cygwin DLLs (cygwin1.dll and cygz.dll).

[Lowlight]

It seems to be an issue with FC6 having a newer glibc. Somebody will have to compile it in an older environment.


$ ./ffmpeg-ty-1.5 --version
FFmpeg version SVN-r6543, Copyright (c) 2000-2006 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
libavutil version: 49.0.1
libavcodec version: 51.16.0
libavformat version: 50.5.0
built on Oct 5 2006 21:38:15, gcc: 4.1.0 20060304 (Red Hat 4.1.0-3)
./ffmpeg-ty-1.5: missing argument for option '--version'

$ ./ffmpeg-ty-1.6 --version
Floating point exception

$ ./ffmpeg-ty-1.6b --version
FFmpeg version SVN-r7332, Copyright (c) 2000-2006 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
libavutil version: 49.1.0
libavcodec version: 51.27.0
libavformat version: 51.6.0
built on Dec 18 2006 18:11:36, gcc: 4.0.2 20051125 (Red Hat 4.0.2-8)
./ffmpeg-ty-1.6b: missing argument for option '--version'

Ok I built an new version on FC4 from SVN-r7332. Hope this helps.

[Lowlight]

I recompiled the linux version with --enable-pthreads but haven't had a chance to test it so I will wait to upload that until it's tested. I figured those of us with SMP/dualcore systems would like to have pthreads. I would also like to try to get AAC support compiled in to make transcoding quicktime movie trailers possible without having to first run it through my non-ty ffmpeg.

[slseed]

What would be the command line usage to convert ty to dvd using ffmpeg-ty? I tried:

ffmpeg-ty-1.6b -i myshow.ty -target dvd myshow.mpg

but I get:

FFmpeg version SVN-r7332, Copyright (c) 2000-2006 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
libavutil version: 49.1.0
libavcodec version: 51.27.0
libavformat version: 51.6.0
built on Dec 18 2006 18:11:36, gcc: 4.0.2 20051125 (Red Hat 4.0.2-8)
myshow.ty: could not find codec parameters

[Jamie]

What would be the command line usage to convert ty to dvd using ffmpeg-ty? I tried: ...ty enabled ffmpeg can encode ty files, not decode them. I think you want tytompg (http://www.dealdatabase.com/forum/showthread.php?t=51740).

[helpdeskdan]

Dang.

gcc -L./libavformat -L./libavcodec -L./libavutil -Wl,--warn-common -rdynamic -export-dynamic -Wl,--as-needed -Wl,-rpath-link,/home/dan/ffmpeg/libavcodec -Wl,-rpath-link,/home/dan/ffmpeg/libavformat -Wl,-rpath-link,/home/dan/ffmpeg/libavutil -g -o ffmpeg_g ffmpeg.o cmdutils.o -lavformat -lavcodec -lavutil -lm -lz -ldl
ffmpeg.o: In function `opt_input_xml':
/home/dan/ffmpeg/ffmpeg.c:2580: undefined reference to `xml_buf'
/home/dan/ffmpeg/ffmpeg.c:2585: undefined reference to `xml_size'
collect2: ld returned 1 exit status
make: *** [ffmpeg_g] Error 1

[joeybubbabob]

Dang.

gcc -L./libavformat -L./libavcodec -L./libavutil -Wl,--warn-common -rdynamic -export-dynamic -Wl,--as-needed -Wl,-rpath-link,/home/dan/ffmpeg/libavcodec -Wl,-rpath-link,/home/dan/ffmpeg/libavformat -Wl,-rpath-link,/home/dan/ffmpeg/libavutil -g -o ffmpeg_g ffmpeg.o cmdutils.o -lavformat -lavcodec -lavutil -lm -lz -ldl
ffmpeg.o: In function `opt_input_xml':
/home/dan/ffmpeg/ffmpeg.c:2580: undefined reference to `xml_buf'
/home/dan/ffmpeg/ffmpeg.c:2585: undefined reference to `xml_size'
collect2: ld returned 1 exit status
make: *** [ffmpeg_g] Error 1

I remember coming across this before, but don't recall what the problem was. If you could post the steps that you followed to produce this error I could probably figure it out (the patch file you used, the patch command you used, the svn revision you used, etc).

[helpdeskdan]

A REPLY! At last! Thanks!

Honestly, I can't remember. I used the following steps in Ubuntu and it worked. Maybe I used a newer source last time. Also, I got mv4 to work. (Install libfaad, --enable-faad)

svn checkout -r 7323 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg

patch -p2 < ffmpeg-diffs-1.6a.txt

./configure --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver --enable-faad

make

[phunk311]

Is there any way to do this in Windows? I need to configure ffmpeg with support for mp4s.

EDIT: Nevermind, I found a website with win32 ffmpeg with everything I needed.

[mttjcksn]

I'm at my wits end...

I've tried:

patch 1.5 on SVN 6543 (both manually building, and the supplied binary version)
patch 1.6a & 1.6b on SVN 7332 (both manually building, and the supplied binary version)


Patching & compiling the src doens't pop up any obvious errors.

Running below the command results in the following output:


# ffmpeg-6543-1.5/ffmpeg -i ika-smr.vob -vcodec mpegvideo -b 3000k -acodec ac3 -ab 384 "show.ty+"
FFmpeg version SVN-r6543, Copyright (c) 2000-2006 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 --disable-ffplay --disable-ffserver
libavutil version: 49.0.1
libavcodec version: 51.16.0
libavformat version: 50.5.0
built on Mar 24 2007 11:19:52, gcc: 4.1.1 20070105 (Red Hat 4.1.1-51)

Seems that stream 0 comes from film source: 25.00 (25025/1001) -> 25.00 (25/1)
Input #0, mpeg, from 'ika-smr.vob':
Duration: 00:01:09.5, start: 2875.981611, bitrate: 3703 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x576, 6804 kb/s, 25.00 fps(r)
Stream #0.1[0x80]: Audio: ac3, 48000 Hz, mono, 384 kb/s
Output #0, ty, to 'show.ty+':
Stream #0.0: Video: mpeg2video, yuv420p, 720x576, q=2-31, 3000 kb/s, 25.00 fps(c)
Stream #0.1: Audio: ac3, 48000 Hz, mono, 384 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Press [q] to stop encoding
[mpeg2video @ 0x84ae558]ac-tex damaged at 43 7trate= 0.0kbits/s
[mpeg2video @ 0x84ae558]Warning MVs not available
[mpeg2video @ 0x84ae558]concealing 1305 DC, 1305 AC, 1305 MV errors
Generating master chunk for chunks 0:151
Successfully wrote new XML
frame= 1738 q=2.0 Lsize= 0kB time=69.5 bitrate= 0.0kbits/s
video:15822kB audio:3256kB global headers:0kB muxing overhead -100.000000%


I'd LOOOVE to come up with a way to verify the output file is valid right on the system, since everytime i transfer to the HD Directivo over MFS_FTP, it shows as a 'partial recording' with a time of 0:00, and of course, can't play it.

So far, I've tried the following to validate the file in some way...


# file show.ty+
show.ty+: data


I would have anticipated something about an MPEG stream, v2, or SOMETHING... "data" is pretty meaningless.. oh well, lets verify the XML is right...


# strings show.ty+ | tail -20
*det
5Y:$x
e|'O
,EiSi
SD 5Bt
aq:{
@k\2:
`v7H
qPwA
>:l@
#################################################################################################### #################################################################################################### #################################################################################################### #################################################################################################### #################################################################################################### ############
<?xml version="1.1" tivoversion="3.1.5e-01-2-357"?>
<Object type="Recording" id="_top">
<StreamFileSize>19584</StreamFileSize>
<Duration>69</Duration>
<StartTime>1</StartTime>
<StopTime>70</StopTime>
<CallSign>ffmpeg</CallSign>
</Object>
#################################################################################################### #################################################################################################### #################################################################################################### #################################################################################################### #################################################################################################### ############


Now, I'm running 6.3 on the HD Directivo, so that MAY have a hand in it, but I'm still stuck on the idea that i'm running the patched ffmpeg with the wrong flags. Can SOMEBODY please help me!? This seems like an obscure problem that is turning something that should be very simple into a real pain in the ass.

Thank you everyone!

[Lowlight]

Here's the command I use with 1.6b


ty-ffmpeg -i trailerHD.wmv -vcodec mpegvideo -b 7000k -r 29.97 -aspect 16:9 -acodec mp2 -ab 192 foo.ty


when I'm done I pass that through ty+2tmf.pl and upload that because mfs_ftp has had problems with ty/ty+ uploads. The best way to test the file before you upload it is to try to play it with mplayer. I have uploaded some tmf files that I've made like this and seen the 0:00 partial in the NSL but they do play and the timeline while playing is right. I've only been doing this with short clips from the internet. Also I use 7000k because I'm uploading 720p HD content to my HR10-250.

To rip a dvd at full quality to ram into my HD Tivo I use this on my Linux box.

vobcopy -l -o -|~/ty-ffmpeg -y -i - -vcodec copy -acodec copy HarryPotter-GobletOfFire.ty
ty+2tmf.pl HarryPotter-GobletOfFire.ty


Hope this helps

[iguy]

Is there any updates to the patch?

[joeybubbabob]

Is there any updates to the patch?

You mean to update it to apply to a newer ffmpeg revision? I can attempt to update the patch to the latest svn revision of ffmpeg if there is a demand for that.

[drez]

I'd appreciate that if you can joeybubbabob.

[nova1]

Here is a patch that skips over unknown mpeg start codes and does an xml_replace() instead of an xml_union(). I also had to add a chunk_seek() to flush_chunk() as the output fd filepos was resetting to 0. Synced with ffmpeg svn 9509.

1. svn checkout -r 9509 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
2. cd ffmpeg
3. patch -p1 < ffmpeg-diffs-1.7.txt
4. ./configure --enable-gpl --enable-a52 --disable-v4l --disable-dv1394 -disable-ffplay --disable-ffserver --enable-avisynth
5. make
6. make install

I've attached a build from FC6.

[rbautch]

I'm getting the following error when I try to convert several mpeg files to ty, using a straight -acodec copy and -vcodec copy, and the v1.5 Windows executable: [ty @ 0x6fd594]seq_pos=0 gop_pos=98 pic_pos=-1assertion "pic_pos != -1" failed:
file "ty-enc.c", line 275

0 [sig] ffmpeg 4916 open_stackdumpfile: Dumping stack trace to ffmpeg.exe.
stackdumpIt works just fine if I re-encode the mpeg stream, but I'd rather not do that. I created the original mpeg files using TMPGEnc and frameserved video from Virtualdub. (TMPGEnc settings attached) I tried using a quickstream fix in VideoRedo, and also demuxing the mpeg first, but to no avail. Any ideas?

[whitepelican]

Here is a Windows executable that includes the 1.7 patch Nova1 posted above.

[studog86]

Is there a tyffmpeg windows executable?
Is the post before me tyffmpeg 1.7 or ffmpeg 1.7.
I am a little confused.

[drez]

That is tyffmpeg 1.7, there is no ffmpeg 1.7.

[studog86]

Thank you for the quick reply.
Here is my problem.
I am getting this error with tyffmpeg 1.7.
The procedure entry point_isnand could not be located in the dynamic link library cygwin 1.dll

[drez]

Try putting these dlls in the same folder as tyffmpeg:

http://www.dealdatabase.com/forum/attachment.php?attachmentid=5957&d=1166472970

[studog86]

Try putting these dlls in the same folder as tyffmpeg:

http://www.dealdatabase.com/forum/attachment.php?attachmentid=5957&d=1166472970

That did it.
Thank you drez

[dsully]

For those of us with HR10-250's and no MRV, this is still a valid option.

Attached is a patch to HEAD (currently r11213) of ffmpeg.

[mikey]

For those of us with HR10-250's and no MRV, this is still a valid option.

Attached is a patch to HEAD (currently r11213) of ffmpeg.

any chance of a windows compile?
Thanks,
Mikey

[Soopaman]

For those of us with HR10-250's and no MRV, this is still a valid option.

Attached is a patch to HEAD (currently r11213) of ffmpeg.

Hey dsully, I was able to successfully apply your latest 1.8 patch, however it seg faults any time i deal with tivo files. Could we work out a test case to see if the seg fault is a result of the patch, or my environment. thanks

[jt1134]

I had the same problem when I compiled it under cygwin, using various versions of gcc. It would transcode to mpg/vob/whatever just fine, but would seg fault if trying to go to ty/ty+. Compiling it under Ubuntu worked just fine though.

-edit- nevermind, I must have screwed up my PATH or something. It seg faults under Ubuntu also when dealing with ty/ty+.

[phord]

This works under 1.7, but segfaults under 1.8:

./ffmpeg -i test.flv test.ty

If I can get 1.7 to work for me, I'll probably stick with it. If not, I'll see if I can fix 1.8 and post back.

[dsully]

Yeah, it appears that something has changed in ffmpeg to cause it to segfault.

I've not had time to track this down.

[jmw86069]

dsully or someone, I've not had luck building tyffmpeg with the latest ffmpeg source, but would be willing to go through it if you could describe the general process to a newbie? I program a lot (in Perl and R mostly lately) but am not familiar with patch files.

Should I basically go through the patch files and figure out what they modify, emulate the same thing with my own patch? Do I modify the file, then do a diff which creates a patch? Sorry for the n00b questions, I want to help, and I want to use tyffmpeg on Ubuntu Hardy alongside MythTV and my HR10-250, but no luck yet.

Thank you all for all this stuff -- too damn cool!

[nova1]

I've attached a patch file that builds against SVN 13747. However, I ran it against some mpeg files I had and the TY output is different(and doesn't work-stuttering and sync issues) when compared against the TY output from SVN 9509 and patch 1.7. I suspect a lot of things have changed within ffmpeg and will require someone more versed in audio/video than I to figure it out.

1. svn checkout -r 13747 svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
2. cd ffmpeg
3. patch -p1 < ../ffmpeg-diffs-1.9.txt
4. on FC7:
./configure --enable-gpl --enable-liba52bin --disable-ffplay --disable-ffserver
5. make
6. make install

[jmw86069]

I ended up building ty-ffmpeg 1.7 versus ffmpeg 9509, using whatever other newer libraries didn't cause errors with the older ffmpeg 9509... it works and seems to upload fine to the TiVo HR10-250 -- which fits my requirements.

I do follow a process suggested previously in this thread, transcoding using the latest-greatest ffmpeg, then converting to TY+ using ty-ffmpeg-1.7. It seems to work well, though I haven't tested it with files > 2 GB yet. (I'm assuming it works, just sayin' I haven't tested it yet.)

The other thing which may be important, I built it STATIC. So that build does two things:
1. It means it cannot be built with x264 and xvid (both of which require dynamic builds)
2. It can be run on basically any linux since it doesn't look for codec libraries all over the filesystem (which is the reason post 98's linux build didn't work for me.)

I did, however, pretty much follow the steps in post 98 to build ty-ffmpeg-1.7.