Page 6 of 36 FirstFirst ... 4567816 ... LastLast
Results 76 to 90 of 531

Thread: MUX'ing, VSplit, and MPG2 files... #2!

  1. #76
    Join Date
    Sep 2002
    Posts
    1,735
    Originally posted by jdiner
    Ok. I can dump any number of frames to an external file. Perhaps down the road that is what should be done. Rather than doing any encoding myself. Dump the 5 frames that are required and then spawn off a call to an external compressor something with a nice API and then reload and mux using those new replacement frames and the original timestamp values.

    Then we could get perfect seemless cuts. The audio is not a problem as it does not have any prediction going on.

    Anyone know of there is an external API to tmpgenc or any of the other compressor tools?

    --jdiner
    Why don't you just take the reference code that's available and use it? Speed won't really be an issue because you're only talking about a few frames.

    It's linked in the MPEG section of my sig. I'm INCREDIBLY interested in perfect edits (TLC and Discovery don't do long fades that often) so here 'tis:

    ftp://mm-ftp.cs.berkeley.edu/pub/mul...mpeg2/software

    I've stumbled on some other freely-usable source so let me know if you need something else. I'll happily go digging.
    Last edited by FredThompson; 11-23-2002 at 04:16 AM.
    Collecting 9/11, Afghan/Iraq, Mail Call, Trains, Cooking, Woodworking, Fighting Illini - Let's chat
    A/V links: neuron2 doom9 VideoHelp DigitalMediaNet CreativeCow DVDShrink PgcEdit Streambox WMRecorder
    other links: SnapFiles NoNags HackADay Engadget Fontleech OfflineExplorerPro TechBargains PriceWatch

  2. #77
    Join Date
    Sep 2002
    Posts
    1,735
    That link is a dog. Here's the source code.
    Collecting 9/11, Afghan/Iraq, Mail Call, Trains, Cooking, Woodworking, Fighting Illini - Let's chat
    A/V links: neuron2 doom9 VideoHelp DigitalMediaNet CreativeCow DVDShrink PgcEdit Streambox WMRecorder
    other links: SnapFiles NoNags HackADay Engadget Fontleech OfflineExplorerPro TechBargains PriceWatch

  3. #78
    Join Date
    Jan 2002
    Posts
    4,809
    Originally posted by FredThompson
    Why don't you just take the reference code that's available and use it? Speed won't really be an issue because you're only talking about a few frames.
    I will have to have a look see at the source code. The reason I didn't want to use the reference source I have is that it is liscenced such that we could never, even with free tools use it. And it was crap. Not just big frames as the problem but bad frames. Very bad.

    If this is better on both counts I have no issues with using it. I would also like perfect cuts. I really would. I am as much a fan of that as anyone here. But I also know how much work is involved in all of this. And comparing the work needed to do my own versus what we have already... I kind of have the feeling of "why bother" on doing better cuts. But that is just me.

    --jdiner

  4. #79
    Join Date
    Jan 2002
    Posts
    83

    Temporal Reference Errors

    Jdiner:

    As I understand from your recent post, Temporal Reference errors are a result of bad data in the dtivo stream.

    However, I have had two ty streams which generate Temoral Reference Errors in DVDMaestro when split with vsplit13c, but do not have the errors when split with the earlier vsplit13.

    There are a few OOB's in the subject streams, but no holes reported (no vidbody off messages).

    The subject streams play fine with no sync problems on the final burned DVD-R's.

  5. #80
    Join Date
    Jan 2002
    Posts
    4,809

    Re: Temporal Reference Errors

    Originally posted by fredisdead
    Jdiner:

    As I understand from your recent post, Temporal Reference errors are a result of bad data in the dtivo stream.

    However, I have had two ty streams which generate Temoral Reference Errors in DVDMaestro when split with vsplit13c, but do not have the errors when split with the earlier vsplit13.

    There are a few OOB's in the subject streams, but no holes reported (no vidbody off messages).

    The subject streams play fine with no sync problems on the final burned DVD-R's.
    If you can find them in the stream cut them for me. I am not sure what is going on. But I have gone to far one way or the other in adding or removing what looked like bad data.

    It should not be happening.

    And again it is not bad data it is out of order data... However if I am cutting something I should not be then we would infact potentially be getting out of order data.

    i.e. 1-1 1-2 1-3 [cut goes here] 2-5 2-6

    and this looks likes out of order data...

    --jdiner

  6. #81
    Join Date
    Jan 2002
    Posts
    4,809
    I mentioned again recently in response to questions and comments from others the VFR nature of the DTivo/DTV video stream.

    To try and help clarify once again what is going on attached is a text file that shows the video frame count and the data sizes segmented by seconds. 1 line per second.

    As an example here are the first few lines from the file:
    Code:
     29 nodes && (488848 bytes)
     31 nodes && (519388 bytes)
     28 nodes && (289136 bytes)
     25 nodes && (277036 bytes)
    First line means: 29 frames in that seconds worth of video with a total of 488k for just the video data.

    Second line means: 31 frames for the 2nd second of video with a total of 519k of data for the 31 frames.

    Third line means: 28 frames of video at 289k of data.

    And so on...

    The Variable Bit Rate (VBR)is quite obvious ranging from 519k to 277k in just the first 4 seconds of output.

    The VFR is also obvious. 29 then 31 then 28 then 25. Look at the entire text file to see that the pattern just keeps going in that same vein. Now at every point along the way the output is reporting that it is 29.97 frames per second. If only...

    This is again why custom mux'ing is needed. And why movies work and regular episodic TV rarely does. The movie clips I have report themselves as 24fps and the average, again cutting out the lead-in and lead-out segments, is 24. Almost perfectly so.

    So there will be slight drift using another mux'er but not bad if all is well.

    If anyone would like to check this out for themselves on their own clips grab the latest (1N) version of vsplit and run it with the -v (NOT -V) flag. For level 1 verbosity. Then grep for "MUXV:" in the output. It will generate for you the same file as I have attached here.

    For movies check the lead-in. Probably fluctuating, then settles down the show itself.

    For TV shows it will most likely fluctuate wildly from start to finish.

    --jdiner

  7. #82
    Join Date
    Jan 2002
    Posts
    4,809
    To continue my last post for the SA Tivos...

    Attached is an SA Tivo output from an episode of Lois and Clark.

    Code:
    MUXV: This much video in the Q: 21 nodes && (720015 bytes)
    MUXV: This much video in the Q: 23 nodes && (788060 bytes)
    MUXV: This much video in the Q: 23 nodes && (786240 bytes)
    MUXV: This much video in the Q: 23 nodes && (787080 bytes)
    Note the smaller first frame this is the cut of the 2 lead-in B-Frames from the previous GOP.

    Then almost everything is 23's. The reason this is 23 and not 29-30 is that the code I am using just counts nodes not the frames in them. On the DTivo 1 frame per node. On the SA Tivo the B frame followed by the P frame is put into the same record on the Tivo.

    But the output stays almost perfect all the way through. The imperfections are my display code rather than the stream.

    Also noet the sizes. While not perfect this is CBR... 788, 786, 787 and so on. Nice and steady bit rate...

    --jdiner

  8. #83
    Join Date
    Jan 2002
    Posts
    4,809

    RELEASE: Muxer version 1N...

    Alright folks here it. Version 1N of the MUX'er.

    This release has the following key things in it:

    1- The mux'er has been further cleaned up and it is marginally faster now than before.

    2- The system requires more memory. In dealing with holes things now take a bit more ram. Not substantial but the need is there.

    3- The full stream is mux'ed out. Before the ending pieces were chopped off even with the length limits.

    4- The temporary mux'ing buffers are now flushed to the file at the very end of the run.

    5- These building buffers are now padded to be a full mpeg-2 sector in size. Amazing the kinds of fireworks this one caused.

    6- The MUXA: and MUVX: lines were moved to show during the -v level of debugging output. These show how much data was collected for each second for both video and audio. Just a way to check the VFR and VBR...

    7- Other sundry small fixes were added. Every time I go through the mux'ing I find something. Sadly this time it was if there was a full second of video and no audio (i.e. a hole) it caused vsplit to error out. This has been fixed.

    8- The "cut" from 5-10 seconds has been removed.

    9- It is still limited to 2000 chunks. But this phase is coming to and end. If there are not any major problems then it means that mux'ing is going as needed and we can deal with the last stages of the VOB and editing...

    --jdiner

  9. #84
    Join Date
    Jan 2002
    Posts
    4,809
    Ok. I have the code built to create the editing I-Frame key file.

    Any takers on building the GUI before I get started on it? There is other stuff for me to do so I am serious. If anyone wants to start the editing GUI that would be great with me.

    --jdiner

  10. #85
    Join Date
    Jan 2002
    Posts
    4,809
    Don't you just hate it when you think what you are doing is the best way and they you get the thought bubbling up from the back brain that will make it all better?

    I expect that I can make the mux'er 5-10% faster again. Was just kind of being dumb about buffer management.

    --jdiner

  11. #86
    Join Date
    Jun 2002
    Posts
    41
    Jdiner,

    I'll have a go at a gui..

    I'm on a clients site rebuilding a mirrored server at the mo(taking forever, giving me time to checkup on this board), but I'll be around later.

    I have trouble sleeping lately, so I have lots of time to code. (10am fri till 7am on saturday.. 21 Hours straight).

    I'm only good for windows ATM though, that might be a problem.

    PM me if your interested.

    If not I do have one idea, which I have stated before, but I'm not sure if it helps, as I don't know in detail how your code works.

    If we know that add breaks are at least 60 seconds long for example, we only need to take stills every 59 seconds on so...

    Which means 1 still per minute rather than per iframe.

    Then zero in on the iframes for that minute, and get the cut closer.

    Then we have a cutlist.

    Anyway, Servers at 90% so I'm packing up. Hope this helps!

    J
    Last edited by JasonJLee; 11-25-2002 at 11:42 AM.

  12. #87
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,829
    IMO you should write a front end spec and just distribute a command line only binary, this way it would be easy to have GUIs written for either windows or linux, and us command line jockeys don't have to run X servers where they aren't necessary
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  13. #88
    Join Date
    Jan 2002
    Posts
    4,809
    Originally posted by AlphaWolf
    IMO you should write a front end spec and just distribute a command line only binary, this way it would be easy to have GUIs written for either windows or linux, and us command line jockeys don't have to run X servers where they aren't necessary
    Exactly my plan. I wasn't clear enough but I was just getting started...

    What I have right now:

    1- A version of vsplit that can be run through a TyStream file. It outputs the actual time stamp and each I-Frame from within the file. Thus correct timestamps for a cutlist are easy to generate.

    2- At some point this will be run on the Tivo but for now you have to have a downloaded TyStream file. But having gotten this far the rest of the work to put it on the Tivo is a piece of cake...

    3- For right now I am assuming a process of "starting at the first timestamp then up to but not including the second timestamp" for the cutlist. This way even the first I-frame in the cut is gone but the second will be there. Looks to be about right with how things are arranged inside the files.

    4- The program also prints the timestamps to stdout like the rest of the debugging. This way you can get a timestamp list as a text file for manual cutting.

    5- Then you can create a simple text file cutlist as was mentioned before. 2 timestamps per line...

    6- The -k is the command for this. It stands for "Key Frames file" rather than something like "Kut list"...

    7- Then you re-run vsplit with the -c option for listing the cutlist file as an argument. It then cuts the unwanted data from the file.


    As for a front-end attachment to the system I am not sure of file structure yet. I am trying to decide on the best way to do it. Fundamentally what is needed is the timestamp and then the frames data. But we need to package it for scanning back and forth. File positions would make that faster, flags that could be locked onto would make it faster, sizes of the data chunk that follows would make it faster. I just haven't decided yet. If anyone has any suggestions on that then please feel free to chime in.

    But I do like the idea of others being able to do the front ends. I have enough work. And there are some here that are better at GUIs than I am and some that can actually do a UNIX gui. I haven't in years and if I had to again I would want to do it in Java as it would be the least painful. Not a problem if you run X and have java installed. But anyway I digress...

    --jdiner

  14. #89
    Join Date
    Jan 2002
    Posts
    4,809
    Ha! Ok the edit list file stuff is cleaned up and finished. So it is now possible to provide a cut list with as many entries as desired and have it cut them all. When cutting on a specific GOP boundry all seems to work well with no visual issues at all. (Being on a GOP boundry SHOULD do that...

    Also as I was playing with things I got 1 step further. I have a way to do visual editing in a very manual way.

    Here is the process I am doing:

    1- vsplit it normally with no mux'ing or other options and save the output files.

    2- vsplit with -k to get the textual only edit list. The lines look like:

    Code:
    Type = 1 (I) Size = 21212 - PTS  = (280532029) 00:51:57.022
    Type = 1 (I) Size = 18544 - PTS  = (280623621) 00:51:58.040
    Type = 1 (I) Size = 18800 - PTS  = (280713711) 00:51:59.041
    Type = 1 (I) Size = 20832 - PTS  = (280803801) 00:52:00.042
    Type = 1 (I) Size = 37344 - PTS  = (280895392) 00:52:01.059
    Type = 1 (I) Size = 40768 - PTS  = (281003500) 00:52:02.261
    But as they come out in the standard output with other information (the ... 100... and so on) I used grep, for win32, to get just the lines above.

    3- Load up the .m2v output from step1 with DVD2AVI.

    4- Using the right and left arrow keys jumps by I frames. (See where this is going?

    5- Count the steps for now and each right or left step directly relates to one of the above.

    6- Build a cut list with the times from above, from this one to this one, and save it out.

    7- Mux the fix with the -m and new -c options thus:

    vsplit1o -c cut.txt -m file.ty output.m2v junk.m2a > junk.txt

    and you get what appears to be a perfectly edited, on GOP boundries, MPEG-2 file.


    Now this is not that easy to do! Sad but true. Counting the stupid arrow key clicks from the DVD2AVI main window is a royal pain! I got lost so many times trying this out and I wanted to use key repeat etc...

    So the good news for this part too. I am almost done with an altered version of DVD2Avi that will show in the display window (at the side when previewing etc...) the current count of the I-Frame you are on. Then key-repeats allow for fast fast editing through the file.

    This is still not direct or perfectly easy as you have to make a cut list manually and then go through the output list to get the timstamps. But a simple program to pull out timestamps from the output list is easy enough to write. And work could be done to do other things with the DVD2Avi app itself. Have it save the "counts" directly to a file to generate the edit list.

    This really only helps the Windows people but it was a nice thing to note.

    --jdiner

  15. #90
    Join Date
    Jul 2001
    Posts
    657
    vsplitmux-1n working fine here! no problems...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •