Page 3 of 30 FirstFirst 1234513 ... LastLast
Results 31 to 45 of 445

Thread: tytompg: TY to mpg conversion in one step, alpha release

  1. #31
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,831
    I've used OSX fairly extensively, and while I think it is a major POS (dunno why that 6% of all computer users seems to believe that they have the best in the world over the remaining 94% who do have a choice to use it if it is that much better, but whatever...) I'll point out that you can always download and run it on any intel/amd computer that supports SSE3. The development tools for compiling something on it are freely available after that point. I think their compilers make building a universal binary a one step process as well.

    Just google osx86 torrent or something like that.
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  2. #32
    Join Date
    Nov 2004
    Location
    St Louis
    Posts
    206
    OK, 0.3 has processed two of my FOX OTA files.

    Still have the same question, what is it about these new mpg's from tytompg that is different from an Mplex mpg from an hdemux output? (WOW, the military would be proud with the Acronym usage)

    My TMPGenc chokes on the 0.3 output (Lockup, no error, must kill the thread)

    I have fired up Windows Media Encoder ver 9 and it appears to be chugging away just fine.

    I will report back with the results.

    Interestingly WME will allow you to

    Preserve Timestamps
    or
    Make new ones.

    Hmm, any reasons why you would want to make new ones? Is it possible, that the way tytompg works, its better to make new timestamps (Referencing the audio holes) Or will that bring my problem back?

    If you cant answer that question, thats OK. Just curious if you have incite. I plan to do it again tomorrow with Make Timestamps to just see what happens.

    Thanks BCC! heh, BCC for President, err Senator for Missouri! (Too many political ads of late for my noggin, it overflowed)
    Last edited by RavenStL; 11-07-2006 at 10:16 PM.
    2 HR20-100's with OTA (Thank GOD I didnt get a HR21)
    3 HDVR2's with 6.2, Sub'd, Hacked and 160 gig Seagates.
    1 HR10-250 with 250 WD and 300 Seagate, fully Hacked
    3 HDVR2's with 6.2, Hacked and 160 gig Seagates.
    Who doesnt have 7 Tivos? and with 5 computers, contain 2 Terabytes of storage medium in their house??
    Thanks to all who makes up DDB!!!!!!!!!!!!!!!!!!!!!!!!!!!

  3. #33
    Join Date
    Nov 2002
    Posts
    1,079
    Quote Originally Posted by AlphaWolf View Post
    I've used OSX fairly extensively, and while I think it is a major POS (dunno why that 6% of all computer users seems to believe that they have the best in the world over the remaining 94% who do have a choice to use it if it is that much better, but whatever...) I'll point out that you can always download and run it on any intel/amd computer that supports SSE3. The development tools for compiling something on it are freely available after that point. I think their compilers make building a universal binary a one step process as well.

    Just google osx86 torrent or something like that.
    Something that doesn't require pirating a hacked distribution of osx with funky hardware support would be nice. (Although osx under vmware/xen sounds pretty interesting...) If apple wants 3rd party applications to be built for their OS they would be best served by providing the build environment.
    Yes the size of the osx user base vs the rest of the world adds some perspective doesn't it?

  4. #34
    Join Date
    Sep 2006
    Posts
    11
    Quote Originally Posted by bcc View Post
    If apple wants 3rd party applications to be built for their OS they would be best served by providing the build environment.
    Apple provides XCode, a completely free, high-quality build environment. It's the same one the Apple engineers themselves use. It's also gcc-based. However, you do need OS X to run it. Apple has never tried to provide even informal support for cross-compilation, and I really can't blame them. The situation with tytompg is just not common enough. Nearly all Mac developers are also Mac users, building GUI apps, and of the ones who are not, most either have open source code, or access to a Mac for building and testing. Also, you can get a Mac Mini for just $600, a machine which can easily be set up to triple boot OS X, Windows, and linux. From that point of view it's a great development machine.

    Maybe a bunch of us should chip in to get bcc a Mac Mini
    Last edited by pdicamillo; 11-07-2006 at 11:21 PM.

  5. #35
    Join Date
    Feb 2004
    Location
    New York City
    Posts
    579
    Quote Originally Posted by bcc View Post
    Something that doesn't require pirating a hacked distribution of osx with funky hardware support would be nice. (Although osx under vmware/xen sounds pretty interesting...) If apple wants 3rd party applications to be built for their OS they would be best served by providing the build environment.
    Yes the size of the osx user base vs the rest of the world adds some perspective doesn't it?
    Hrmm. That raises a good point. You CAN run the full Darwin OS for free (just as any linux) on a PC. It's freely available from Apple.com. While it doesnt have the pretty GUI, it'd be great for a build environment. I can send you the related instructions and such if you want (better solution than the darwin-cross packages I was talking about in my PM).

  6. #36
    Join Date
    Aug 2006
    Posts
    28
    Quote Originally Posted by bcc View Post
    I'm just doing straightforward file i/o syscalls, and I made sure to statically link, so I'm surprised there's an issue.
    Static linking to glibc is bad for a couple of reasons:
    • not all of glibc is available for static linking (although that shouldn't be a problem here)
    • it is probably a license violation: glibc is LGPL. Static linking makes the resulting binary a derived work, which means you must distribute source. I'm not sure why the GPL COPYING file is included in the rar file; if it is under the GPL (either because it is derived from other GPL code or because you created it all and want to use the GPL) then you must distribute the source.


    The problem is probably because newer Fedora glibcs (IIRC FC5+) support only the new threading library, which in turn requires newer kernels (basically the old LinuxThreads system was broke and NPTL replaces it but required some kernel changes). I think even if you don't use the new threads some of that kicks in. I'm not sure how to build for older glibcs under newer systems.

  7. #37
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,831
    Quote Originally Posted by pdicamillo View Post
    Apple provides XCode, a completely free, high-quality build environment. It's the same one the Apple engineers themselves use. It's also gcc-based. However, you do need OS X to run it. Apple has never tried to provide even informal support for cross-compilation, and I really can't blame them. The situation with tytompg is just not common enough.
    Actually it is very common. Just about any linux program could be readily cross compiled for OSX with little to no modification. This is true for even programs that use x windows, as the OSX install CD includes an optional X11 server.

    If you'll notice, many non-apple programs that come out these days and have a linux port usually run under x11 when they are ported to osx. Take matlab for example.

    I think the biggest problem is that OSX really isn't as great as mac users make it out to be. It really is incredibly easy for commercial linux developers to make their software available for macintoshes, but they don't most of the time simply because there is little to no demand for it.
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  8. #38
    Join Date
    Nov 2002
    Posts
    1,079
    Quote Originally Posted by RavenStL View Post
    OK, 0.3 has processed two of my FOX OTA files.

    Still have the same question, what is it about these new mpg's from tytompg that is different from an Mplex mpg from an hdemux output? (WOW, the military would be proud with the Acronym usage)
    Right, all this arguing about macs has been getting in the way of the real problem at hand (achieving correctness with demuxing&muxing troublesome OTA recordings). If 2 of your fox recordings with holes in them processed OK then it sounds like we're a lot closer than ever before.

    Anyways, hdemux generates mpeg2 video and ac3 or mpeg2 audio streams. These are called "elementary" streams in the spec, and they lack timing data. (No PTS, DTS or SCR). mplex adds back that timing data, by computing it up on its own, and interleaves the two streams at it's notion of the correct rate. It does some extra work to try and make video I frames available at the beginning of the mpeg2 packets. There are several problems with what mplex is doing in those steps on the elemental streams. 1) mplex, being rather dvd-centric, doesn't compute those timestamps correctly when it comes to some broadcasts that have varying field per frame rates. FOX is one of the broadcasters that employs this encoding technique. 2) mplex code newer than 1.6.2 actually doesn't even make the video I frames available at the right place, 3) There must be no holes in the a/v stream or else you'll lose a/v sync

    tytompg preserves the timing data during the demux step, passing it on to the multiplexer. This saves the multipler from lots of issues: The a/v sync should start out perfect and stay that way; basically issues 1) and 3) go away. As far as I have seen, I pack mpeg2 packets just like mplex used to back in 1.6.2, with respect to positioning of the records. There may be some issues with timing. Because the hr10-250 tivo is wrapping the video DTS timestamp every few minutes, it is making straightforward processing based upon it a problem. Right now I'm throwing it out and using the PTS timestamps instead. This is not quite right but I would have expected the timing to be close enough for decoders (assuming they are doing virtual buffering per the specs). tmpgenc may not be nearly as forgiving about that as xine/windvd, it may be assuming the program stream was freshly encoded with perfect timestamps. I'll give it a try. It may also be that tmpgenc thinks I'm not conforming to the spec's virtual buffer model (allowing the streams to underflow). I hadn't finished my mpeg compliance tester before trying this project so I haven't been able to easily verify this I do try to keep the virtual buffers part full.

    Short answer is that both tytompg and hdemux+mplex generate mpeg2 program streams as their end result.
    Quote Originally Posted by RavenStL View Post
    Interestingly WME will allow you to

    Preserve Timestamps
    or
    Make new ones.

    Hmm, any reasons why you would want to make new ones?
    Depends upon how they make the new ones. Assuming they totally recompute the timestamps this could fix the above virtual buffer issue. It also stands a good chance of re-introducing problem #1 with mplex shown above.
    Quote Originally Posted by RavenStL View Post
    Is it possible, that the way tytompg works, its better to make new timestamps (Referencing the audio holes) Or will that bring my problem back?
    I have code to regenerate the timestamps as well, but you'd have to take into account the original timestamps to measure the holes correctly. Not sure what algorithm one would want where you half use the original times and half make them up. Ok unless you're editing the streams. Otherwise I would think you'd either want the originals (because they are right or because the streams have holes), or regenerated ones (in the case where you don't have holes but the broadcaster screwed up a/v sync or somesuch).

    Thanks BCC! heh, BCC for President, err Senator for Missouri! (Too many political ads of late for my noggin, it overflowed)
    Thanks, haven't even been there. Too many adds here too; I got political telemarketing spam at 6:30am even.

  9. #39
    Join Date
    Jan 2002
    Posts
    183
    Quote Originally Posted by AlphaWolf View Post
    Actually it is very common. Just about any linux program could be readily cross compiled for OSX with little to no modification.
    Crosscompiled? I don't think so. It's quite true that most linux software will compile and run on the mac with little if any modification, but that's not cross compilation. (compiling for one platform on another) I'm not aware of any cross compilation tools that target the mac from other platforms. (Xcode cross-compiles for the other processor architecture, but that doesn't help us here.)

    Compiling under Darwin, perhaps running on on a VM would be worth a try, but really is the hard way to do it. I'd love to have bcc's new engine running on a mac, because hdemux 0.13 (the last version we have on the mac) doesn't seem to work with any of my HD streams.

    My fastest machine is a mac, so I'd prefer to run tytompg there, and there's already a nice GUI in TivoTool.

    Refurb minis are $520... time to pass the hat? If bcc's willing to support the platform, I'll put up the first $100.

    bcc: PM me if you are interested.

  10. #40
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,831
    Well, right, not cross compiled, but thats what I meant.

    Quote Originally Posted by bcc View Post
    Anyways, hdemux generates mpeg2 video and ac3 or mpeg2 audio streams. These are called "elementary" streams in the spec, and they lack timing data. (No PTS, DTS or SCR). mplex adds back that timing data, by computing it up on its own, and interleaves the two streams at it's notion of the correct rate. It does some extra work to try and make video I frames available at the beginning of the mpeg2 packets. There are several problems with what mplex is doing in those steps on the elemental streams. 1) mplex, being rather dvd-centric, doesn't compute those timestamps correctly when it comes to some broadcasts that have varying field per frame rates. FOX is one of the broadcasters that employs this encoding technique. 2) mplex code newer than 1.6.2 actually doesn't even make the video I frames available at the right place, 3) There must be no holes in the a/v stream or else you'll lose a/v sync
    Would it be possible to insert blank video or audio frames in order to force the video streams remain sync in third party muxers?
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  11. #41
    Join Date
    Sep 2006
    Posts
    11
    Quote Originally Posted by bcc View Post
    I made sure to statically link, so I'm surprised there's an issue.
    I'm wondering if static linking is what's causing the problem. Wouldn't dynamic linking get a version of glibc that works with my kernel? Would you be willing to do a build to test that?

    While my 2.6.7 kernel should be updated, it was current just two years ago, and our production Red Hat machines at work still use kernel 2.4, so I suspect this is going to affect quite a few other people.

  12. #42
    Join Date
    Feb 2002
    Posts
    285

    cross-compiling for Darwin (Mac OS X)

    Being a Mac convert (used to be a Linux guy), I decided to do some research into cross-compiling for Mac OS X on Linux and came across these links:

    http://www.racoonfink.com/fink/darwin-cross/
    http://myownlittleworld.com/miscella...ss-distcc.html
    http://www.sable.mcgill.ca/~dbelan2/...erpc-i686.html

    These are more about using the cross-compile environment with distcc to speed up compile times, but I imagine that it would be a big step forward in a stand-alone cross-compile environment as well. I would also look into using PearPC and/or QEMU with OpenDarwin (or Mac OSX) if the cross-compile environment turns out to be too complicated. QEMU is nice because it has built-in file sharing options (it will launch a custom smb daemon) so you can keep the source on your host system. I haven't used PearPC as much, so I don't know what facilities they have.
    - Stealth Dave

  13. #43
    Join Date
    Nov 2002
    Posts
    1,079
    Quote Originally Posted by AlphaWolf View Post
    Would it be possible to insert blank video or audio frames in order to force the video streams remain sync in third party muxers?
    Certainly.
    But it's beyond the scope of what a demuxer should do IMO. Because you have to do all the work of evaluating the timestamps (like a multiplexer would) and encode new ac3 and mpeg2 frames (like an encoder would). tytompg is a lot closer to being up for the task than hdemux.

  14. #44
    Join Date
    May 2004
    Posts
    233
    I tried a clip from The Tonight Show, OTA.

    hdemux -> mplex: no problems.
    tytompg: choppy audio, poor audio sync.

    I can provide a clip if you would like.

  15. #45
    Join Date
    Nov 2002
    Posts
    1,079
    Quote Originally Posted by burdellgp View Post
    Static linking to glibc is bad for a couple of reasons:
    It didn't used to be. Certainly back in the old days of shared C libraries it did improve binary portability to statically link. For example SunOS. I see there is some new religion about this. Smells like binary compatibility bugs in glibc to me.

    Quote Originally Posted by burdellgp View Post
    not all of glibc is available for static linking (although that shouldn't be a problem here)
    Right, non issue
    Quote Originally Posted by burdellgp View Post
    it is probably a license violation: glibc is LGPL. Static linking makes the resulting binary a derived work,
    Being deemed a derived work violates my intent so I guess I won't be distributing this way any more. I've removed my image for now.
    Quote Originally Posted by burdellgp View Post
    which means you must distribute source.
    Does NOT per paragraph 1 section 6 of the LGPL.

    Wow, looks like glibc is not really an appropriate choice for the standard C library and needs to be replaced.
    Quote Originally Posted by burdellgp View Post
    I'm not sure why the GPL COPYING file is included in the rar file;
    Because I intended to distribute the binaries with licensing terms and I was still using GPL terms. I'll replace this with some stricter terms that better apply.

    Quote Originally Posted by burdellgp View Post
    The problem is probably because newer Fedora glibcs (IIRC FC5+) support only the new threading library, which in turn requires newer kernels (basically the old LinuxThreads system was broke and NPTL replaces it but required some kernel changes). I think even if you don't use the new threads some of that kicks in. I'm not sure how to build for older glibcs under newer systems.
    Yes, looks like the thread change must have been the motivating factor. I still don't understand how breaking backwards binary compatibility like this was considered acceptable, and I see no workarounds.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •