Page 9 of 44 FirstFirst ... 789101119 ... LastLast
Results 121 to 135 of 654

Thread: tyremux a GPLed TyStream demuxer and to be remuxer

  1. #121
    Join Date
    May 2002
    Location
    West Hartford CT
    Posts
    620

    Can't wait ...

    Hoping to see 4.0 soon... not much of my holiday vacation left! need some time to BURN!

    also picking up my Sony 715 set-top player.. tomorrow morning, TyDVD compliant!

    In/Sync DVD ready!
    ~Hi8

    (4) Hughes SD DVR40
    (1) Hughes HR10-250
    (2) xbmc XBOX X2 & xbit

  2. #122
    Join Date
    Aug 2002
    Posts
    319
    Hello Folks,

    Okay I'm late but not due to the fact that the code isn't "working" and ported to both MacOS X and Solaris.

    I got everything to "working" last week, but then I ran into trouble with a/v sync when I multiplex it with mplex (mjpeg-tools) both v 1.6.0 and the latest cvs version.

    Hmm, the fact is that I can view how the frame rate is shifting back and forth from 29.97 to 23.976 in my DTivo TyStreams and correct it.

    Here is how I do it:

    First I executing a temporal reference check to see that every frame is in the right order (if not I fix it).

    After that I fetch the PTS time in temporal reference order

    MPEG temp refs for a small SEQ can look like this:
    2I 0B 1B 5P 3B 4B 8P 6B 7B
    Where I == I Frame, P == P Frame and B == B Frame, The I and P frames comes before the B frames but you will show them in temp ref order hence you should get the PTS time for 0 before 2 and so on.

    (PTS presentation time stamp i.e. the time the frame should be displayed).

    Now for a frame rate of 29.976 the diff in PTS time between two frames e.g. 0B and 2I should be 7506 (plus minus a few ticks). If it's 6006 we have a frame rate of 29.97.

    I have determine the frame rate of my Tystream during the initial probe of the stream. If it's set to 23.976 and I bounce into a frame rate of 29.97 during the check I will return a error.

    When I get a error I preform a frame rate fix in this case (29.97 to 23.976) I remove frames and adjust the PTS times accordingly.

    If I have made a frame rate correction I will preform a final check of the frame rate. Just so I can check if I corrected it, I'm also doing a secod temp reference check just for the sake of it.

    Anyways I can see the frame rate both before check/fix and after and I correct it. The final check shows that the frame rate I write to file is near perfect but still I get problem with the A/V sync when I multiplex it .

    I can mention that I on top of this make a check for drift in A/V sync - you see it's not always the diff is e.g. 7506 but instead 7509 or so. It's not much but over couple of hundred frames it can build up to a a/v sync drift of 30ms or so. Hence I make a correction of this too. In fact I can mend sync drifts of as little as 14ms depending on the frame rate.

    At the moment I simply don't get what is wrong but I guess I will figure it out during the week that comes (been sick the last couple of days on top of this #$%$#% ).

    Nope it's not the a/v drift check that is the touble - it's only has a very small effect and I have tested both with and without it.

    It wasn't a small task to fix all this temp ref, frame rate, drift and so on - it's more or less doubled the amount of code lines from 8500 to over 15000 lines of code.

    Anyways I hope you can wait a little while longer.

    Cheers Olaf

  3. #123
    Join Date
    Nov 2001
    Posts
    142
    First off, great work Olaf!

    When you are done, we will finally be able to burn instead of wasting time chopping/rmplexing/etc on these $%^& files. I think you finally explained why some streams mplex ok and others loose sync. Get well, you can't think right when the Crud is doing it's thing.

    Meantime I will continue to archive TY files.

    Thankx again,
    Jmayes

  4. #124
    Join Date
    Aug 2002
    Posts
    319
    Hello Folks,

    Hmm, after some tinkering/hacking of the mjpeg-tools mplex source code I think I'm on to it - I mean why I have a/v sync problem with my present code.

    It has to do with repeated frames, no not the one I add or subtract but if the mpeg stream contains frames that should be repeated or not (which also depends on the chroma format).

    Anyways things are geting slightly more complex the more I dig into the core of the problem. Back with an update soon.

    Cheers Olaf

  5. #125
    Join Date
    Dec 2002
    Location
    Cleveland, Ohio
    Posts
    17
    THanks for the update Olaf--

    Why is this so Non-Standard, I mean, i know that MPEG is a very forgiving format, but why do you think the tivo ppl decided to go this route? Do you think it was to help compensate for various signal strengths from the Input??

    Originally posted by olaf_sc
    Hello Folks,

    Hmm, after some tinkering/hacking of the mjpeg-tools mplex source code I think I'm on to it - I mean why I have a/v sync problem with my present code.

    It has to do with repeated frames, no not the one I add or subtract but if the mpeg stream contains frames that should be repeated or not (which also depends on the chroma format).

    Anyways things are geting slightly more complex the more I dig into the core of the problem. Back with an update soon.

    Cheers Olaf

  6. #126
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,829
    Its anybodies guess. I would suppose they strayed so far from spec because when directv started using mpeg2, there was no mpeg2 standard yet. That combined with they never intended for their transport streams to be recorded/streamed from any set top boxes, and, tivos own tystream format is not valid mpeg2 on its own anyways.
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  7. #127
    Join Date
    May 2002
    Location
    West Hartford CT
    Posts
    620

    strange that 32khz worked for me ...

    I've been experimenting with the tydemux (3.1) and IFOEdit .096B1 --

    I did an extract of a SA stream, limited to one part, 512megs worth of a movie that I recorded to my SA - modified fomat/quality of 720x480 32khz 3.5mbps - original source was a DVD via the S-Video input.

    ran tydemux against the .ty file, and just followed my normal procedure with IFOEdit, creating a few chapters vis CELLTIMES.TXT -- and to my amazment - it was perfectly insync. I've tried previously the same procedure with VSPLIT and upsampling the audio to 48khz - with A/V sync problems.. using the same .ty file as the source. I did notice that IFOEdit reported that it was offsetting , or starting the audio 1300 or so forgot the exact log message. I'll pay attention more closely next time.

    I'm going to do some more tests, and see if I can create a complete extract via parts, and demux & burn..... I can do this perfectly with tytool5/vsplit13c and IFOEdit -- with my DTiVo streams but my SA extracts always seem to be problematic with A/V sync. I figured I'd try the burn using the 32khz audio and see if my /DVD set-top player cared... it didn't and boy am I happpy.

    I'll use this method to convert some of my VHS and more so Hi8 tapes to DVD if it proves repeatable.

    ~Hi8

    (4) Hughes SD DVR40
    (1) Hughes HR10-250
    (2) xbmc XBOX X2 & xbit

  8. #128
    Join Date
    Aug 2002
    Posts
    319
    Okay I more or less know* what is causing the problem.

    I will do some test coding this evening to see if my theory is correct if so then I will most probably make a tmp hack and release 0.4.0 shortly even if it will not be perfect.

    If it's like I think it is well then there still is a lot todo , and some that I have done that wasn't necessary.

    Cheers Olaf

    *Better than saying that I know just in case ).

  9. #129
    Join Date
    Nov 2002
    Posts
    17
    Cheers olaf - everyone appreciates your hard work!!

  10. #130
    Join Date
    Jan 2003
    Location
    Hampshire, UK, Timezone: GMT
    Posts
    158

    Why split?

    Sorry if this is a stupid question but as I understand it, the problem outstanding is syncing audio and video over a long period of time?

    Surely this is a by-product of spliting and then remuxing the ty stream.

    Is the ty stream understood well enough now to produce a stream based codec to turn a ty stream directly into mpeg2? i.e avoid the split/remux but instead go from ty straight to MPEG2

    This would avoid the long term sync problems that are seen at present.

    It seems so obvious that I'm sure it's been thought of and found not possible but I would like to understand the reasons.

  11. #131
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,829

    Re: Why split?

    Originally posted by MrBassMan
    Sorry if this is a stupid question but as I understand it, the problem outstanding is syncing audio and video over a long period of time?

    Surely this is a by-product of spliting and then remuxing the ty stream.

    Is the ty stream understood well enough now to produce a stream based codec to turn a ty stream directly into mpeg2? i.e avoid the split/remux but instead go from ty straight to MPEG2

    This would avoid the long term sync problems that are seen at present.

    It seems so obvious that I'm sure it's been thought of and found not possible but I would like to understand the reasons.
    This is not what causes the sync problems, afaik the reason you split the tystream into elementary streams is because from there its easier to make any changes to it. Even if you don't do this, the sync issues will still remain if you do not make the stream conform to mpeg2 program stream standards.
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  12. #132
    Join Date
    Jan 2003
    Location
    Hampshire, UK, Timezone: GMT
    Posts
    158

    Re: Re: Why split?

    Originally posted by AlphaWolf
    This is not what causes the sync problems, afaik the reason you split the tystream into elementary streams is because from there its easier to make any changes to it. Even if you don't do this, the sync issues will still remain if you do not make the stream conform to mpeg2 program stream standards.
    1. There are loads of video editors out there that can handle MPEG2 input and loads of converters if you want a different format or want it splt. Most people would probably want MPEG2 for DVD generation.
    2. Making it conform to the standards is what I suggest. The standards are well defined, I was wondering if the ty stream, was understood well enough now to do the necessary reformating. I do not understand why the long term sync issues remain when converted to an MPEG2 stream, the method of synchronising MPEG2 audio and video seems to be well defined in the standards. Is it getting the input ty stream audio/video sync sorted the problem?

    If the input ty stream sync is the problem, would modeling a player help? i.e. Process the TY stream into a 'virtual player' that recodes to a MPEG2 output stream. This virtual player would have a fixed play buffer (0.5 second max) so the furthest out of sync the stream could ever get is 1/2 second. The 'virtual player' need not work in real time, the player's clock could be sped up to process streams as fast as your hardware could handle it, i.e. data is clocked in and out at the same speed, whatever clock speed you care to choose.

    As you can probably guess I know very little about video streams, however I do know a lot about writing real time software to control and/or model hardware so this is why I would select that approach.

  13. #133
    Join Date
    Aug 2002
    Posts
    319
    Hi RC

    If this was just a small problem i.e. the tydemux basically worked I would release it. The sad fact that when I say problems with a/v sync then I really mean it (i.e. the present non released codebase is not producing streams that are useful).

    Now as I said earlier this morning I kind of know what is wrong, I will do the test coding this evening. It's a relativly simple hack (tops out to 20 lines of code or so) - if this work I will do some final adjustments and release the code.

    However note that it means that we do some things the wrong way which is a thing I generally don't like at all. Hence I need to continue coding and release a version that make the right thing as soon as possible but a interim pre version will at least give people something that some what works as it should. Anyways I will see when I make my test this evening - will as I said be back with an update.

    Cheers Olaf






    Originally posted by rc3105
    Olaf, don't get too hung up trying to make things perfect. we're still a loooong ways from 1.0, and the more eyeballs involved the more likely sombody'll have a trick up their sleeve that solves the problem.

    thanks for all the hard work and boy are we all glad you took the high road.

    --
    Riley

  14. #134
    Join Date
    May 2002
    Posts
    234
    we only have sync problems with the dtivo, the stand alone version produces a nice standard mpeg that is easy to split/mux without sync problems

    if you decode/encode the data you solve all the sync problems.

    however it takes a HUGE amount of processor time and losses a significant amount of quality in the process (remember that mpeg is a lossy compression)

    we start with sort-of mpeg streams that are encoded with some of the best equipment you can get, from the best quality source available, for free (the studios do it)

    this project and jdiner's project both are trying to take the tivo-mangled mpeg and convert it to a normal mpeg.

    initially it was thought that just extracting the video and audio data out wold be good enough, but it has since been discovered that frame rate differences break this. the two projects have differing opinions on how extreme this frame rate change is and how best to manipulate the file to produce a standard mpeg file. that's where the interesting problems are happening.

    looking just at frame rate, the mpeg standard says that the datastream reports the framerate and all frames will be at that rate. tivo changes the rate without reporting any change. tivo just includes a timestamp on each frame and the playback hardware just plays each frame at the appropriate time. unfortunantly DVD hardware doesn't do this and honors the framerate it's told. As a result it's not a trivial thing to convert this. Olaf thinks that there are only two frame rates in use and the datastream toggles back and forth between them and that he can duplicate frames to change the slow framerate to the fast one, jdiner thinks that frame rates are far more variable and he is working on his own approach to the problem. both are getting there, but both are running into different problems.

  15. #135
    Join Date
    May 2002
    Location
    West Hartford CT
    Posts
    620

    no sync problems?

    Originally posted by dlang
    we only have sync problems with the dtivo, the stand alone version produces a nice standard mpeg that is easy to split/mux without sync problems
    I've been really confused by these statements, those even by jdiner. Always stating that the SA splitting and muxing is a piece of cake and is already there. The DTivo streams are the issue.

    Well it must be me ... all my Dtivo stream extractions and DVD burns are perfectly in-sync best I can detect. Using IFOEdit and Nero produces a GREAT DVD. I have been burning off my SAT60 for weeks now with a routine that is building quite a large number of disks! However getting the same consistancy from my SA is still a challange. mplex is the only muxer that seems to keep things in-sync. However it's limited to 2gig output files, which tends to be a pain for full length movies at 5.8mbps - have to extract in pieces. The other problem being the 32khz upsample stage.

    jdiner's muxer seemed to be the answer, but the last release isstill crippled to limit 2000chunk output. Just when I thought jdiner was getting somewhere his releases seemed to dry-up. I hope he's ok; last I recall he had an ear infection and still had a ringing in his ear.

    All this is quite alot of fun; no matter what the out come - I just wish it was abit more predictible.
    ~Hi8

    (4) Hughes SD DVR40
    (1) Hughes HR10-250
    (2) xbmc XBOX X2 & xbit

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •