MUX'ing, VSplit, and MPG2 files... #2!

[jdiner]

Ok. This text is a copy of the starting fo rhte the second new Thread I am created in the DTivo forum. Just like there this thread is most definately a discussion of how to produce to mux'ing/working/finished/finalized/ready-to-use MPEG2 files.

Further it is to be a place where we can all trade ideas, discuss code and applications, be it for the Tivo or just for MPEG2 in general. However there are some rules for this thread.

Just like before these are rules for this thread, not the forum. If you want to discuss things deemed off limits here make your own threads and set your own rules. But if you decide to participate here then please follow the guidelines.

So the rules for this thread are:

1- Leave the flames and arguments elsewhere... Again I am interested in getting things working better and moving forward, not a re-hash of the problems covered in other threads. So here be nice or stay out.

2- We discuss ANY tools dealing with mux'ing or viewing or editing MPEG2 files or TyStreams... But let's focus on things that are built for TiVo output as this is a Tivo Forum. I am not the only one working on this, and while I intend to have my discussions of things here I have no arguments again others doing the same. If you find a tool that is better that any you have seen before, then share.

3- NO DISCUSSION OF EXTRACTION. This thread is about MUX'ing and making working MPEG2 files. To do that you have to have elementary streams. So no matter how you get them you have to have them before coming here with ideas or problems.

4- Feel free to discuss various OS wishes/desires. At some point we should support all of them or at least as many as is possible.

5- Let's limit the discussion of VSplit to how it interracts with MUX'ing rather than usability or general feature set. We have to discuss VSplit as it is the core of my current MUX'ing engine. Also in the works is a MUX'er for exisitng VSplit output that will stitch it all back together as appropriate for a tivo file that can be done long after the a/v split has been done.

6- Feel free to ask for features but be aware that I am making no promises except to work on things in general... I mean it. Ask. But don't hound me or any other author, that accomplishes nothing. If you think of something that would be cool let me know. I will add it to the list.

I would like this thread to be about the previously mentioned "technical amazement" rather than usability in my tools, ExtractStream, etc... That is what the other thread and this forum is for. Here if you find something sweet dealing with MUX'ing/editing then let's all talk about it and move forward in getting to that "perfect" point where everyone is happy.

The technical discussion of the mux'ing and what is going on behind the scenes will be held in here. Some don't care about it. If all you want to discuss is tools for extraction/splitting then go to the other threads.

So if anyone has things on their wish list for MUX'ing now is the time and this is the place.

--jdiner

[chjones]

I suppose one of the main wishes for most SA owners (certainly for me) is 32kHz->48kHz inline audio conversion. I know you have someone working on this, Josh, but since we're now in the SA forum I thought I'd mention it one last time. Just removing one step from the overall process, but doing it *right*---exactly what everything you've done so far has seemed to do.

Edit (about 30 seconds later): Actually, I suppose I should have also mentioned 44.1kHz audio. Selfish of me. Once we are thinking of those two, is there any reason not to go to arbitrary frequencies?
CDJ

[Saturn]

[newbie question] Could someone post the status of the various MUXing methods - pros, cons, etc. I know that jdiner is about to release a new muxing program with a new algorithm, and I know that 3rd party apps can also do the job.

I saw the 74 page thread in the DTiVo forum, but if someone who's been keeping up with everything could post the Reader's Digest version, I'd greatly appreciate it. [/newbie question]

It appears to me as if the Extraction and Splitting programs are fairly mature, but MUXing still presents some obstacles.

My ultimate goal is to retain 100% of the quality of the TiVo's recordings (i.e. avoid re-encoding), and be able to play them back on the widest variety of players (SVCD probably). If it weren't, I'd just use my video capture equipment to recapture the shows and re-encode.

[jdiner]

Originally posted by chjones
I suppose one of the main wishes for most SA owners (certainly for me) is 32kHz->48kHz inline audio conversion. I know you have someone working on this, Josh, but since we're now in the SA forum I thought I'd mention it one last time. Just removing one step from the overall process, but doing it *right*---exactly what everything you've done so far has seemed to do.

Edit (about 30 seconds later): Actually, I suppose I should have also mentioned 44.1kHz audio. Selfish of me. Once we are thinking of those two, is there any reason not to go to arbitrary frequencies?
CDJ

Technically this is a "both" forum. But the answer to that is that the code will allow the for the arbitrary setting of both end-factors. So you can go from 32k to 44.1 or 48 and from 48 to 44.1. I have not looked at it well enough to know if it can go to 32 but doing so is... rather pointless in my mind.

I have not spent a great deal of time looking at the code. I have been trying to finish things so we can get some real testing done.

--jdiner

[jdiner]

Originally posted by Saturn
I saw the 74 page thread in the DTiVo forum, but if someone who's been keeping up with everything could post the Reader's Digest version, I'd greatly appreciate it. [/newbie question]


Ok. There was a synopsis from me right near the end. Go to the end of the thread and then carefully page back no more than 5 pages and you will find exactly the answer you are looking for. It was 2 long-ish messages from me right in a row. If I could figure out how to get the direct URL for you I would, but it is not something I have ever bothered to figure out.

--jdiner

[smeghead]

You can find the recent summary posts here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=70381#post70381) (part one) and here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=70382#post70382) (part two).

[Saturn]

Yes, I found the summaries on Page 65, smeghead provided links above, thanks.

But there was no mention of the current state of using 3rd party tools for muxing. For reference, I only have an SA TiVo, so the DTiVo complications don't come into play for me.

jdiner, I also want to thank you for the work you've done this far. I checked out video extraction a while ago, and it seemed like it had serious problems and was very complicated to get running. Your TyTool is has made things very easy.

For now, I'm extracting over my serial / ppp connection. Very slow, but it'll be worth it if I can get my Good Eats episodes off the TiVo and onto CD-R without commercials and without re-encoding.

I guess my question really is, is this possible right now with current tools?

[jdiner]

For an SA Tivo stream yes. Just grab it and split it.

Then mux in the right settings with tmpgenc and you are there.

If you want to cut commercials and what not it might be more problematic. I have no idea how well that is dealt with. To date most of what is going wrong is solely based on the VFR and VBR nature of the DTivo.

However, historically there have been problems which is why I have been working so hard on it all.

--jdiner

[Hi8]

Originally posted by Saturn

For now, I'm extracting over my serial / ppp connection. Very slow, but it'll be worth it if I can get my Good Eats episodes off the TiVo and onto CD-R without commercials and without re-encoding.

I guess my question really is, is this possible right now with current tools?

Wow, can't imagine doing via serial, but if you are determined...

YES, it can be done, and I do it quite often. I'm putting each episode of Seinfeld (2 per day) on SVCD. (by the way, GOOD EATS - is one of my favorite shows!) This is done with a few 3rd party tools.

TiVoWeb - (Resource Editor) modify recording to 480x480 2600k
TyTool5 - extract tystream
vsplit13c - split to .mpa & mpv
besweet - upsample 32khz-48khz - audio
mplex - mux to mpg
mpeg2vcr - (tools) fix GOP - level2
mpeg2vcr - edit out commercials
Nero - > burn to SVCD (turn off standard compliance)

The last issue (on compliance) to be used at your own risk. They play fine on my stand-alone hardware, but you my want to use Nero to fix things up at that point however.

Enjoy!

[Saturn]

Originally posted by Hi8
Wow, can't imagine doing via serial, but if you are determined...

YES, it can be done, and I do it quite often. I'm putting each episode of Seinfeld (2 per day) on SVCD. (by the way, GOOD EATS - is one of my favorite shows!) This is done with a few 3rd party tools.

TiVoWeb - (Resource Editor) modify recording to 480x480 2600k
TyTool5 - extract tystream
vsplit13c - split to .mpa & mpv


Hmmm. I have all my Good Eats episodes recorded with Best, which is way too high a bitrate for SVCD. Crap. I may end up re-encoding just to reduce the bitrate so an episode fits on a CD. I could change the bitrate with the resource editor, but that wouldn't do me any good for the many episodes I've let accumulate this far. I imagine that even if I cut the commercials, the combined audio+video would be way too big for a single CD, and I have no plans of getting a DVD burner anytime soon.

I'm using TyTool5 to do both the extracting and splitting. Is that bad?


besweet - upsample 32khz-48khz - audio
mplex - mux to mpg
mpeg2vcr - (tools) fix GOP - level2
mpeg2vcr - edit out commercials
Nero - > burn to SVCD (turn off standard compliance)

The last issue (on compliance) to be used at your own risk. They play fine on my stand-alone hardware, but you my want to use Nero to fix things up at that point however.

Enjoy!

Can't mpeg2vcr also do the mux'ing? Does it do a poor job compared to mplex? I can't seem to find mplex available for download, and I've also seen many posts on this board of people using it and creating unviewable and uneditable mpegs. Can you post your command line, and perhaps a place to download a Windows binary of mplex?

[BubbleLamp]

I give up, why is everyone posting to a Sticky Note? That's not their purpose.

Oh no, I just posted to it too! Circular logic will trap me here forever. ;)

[jdiner]

I made it a sticky on purpose. I wanted it to stay at the top of the display in this forum for now. So that it would be obvious where we had all moved too during the transition.

So it is my fault. Blame me. Everyone else does. :)

--jdiner

[Hi8]

Originally posted by Saturn

I'm using TyTool5 to do both the extracting and splitting. Is that bad?

Can't mpeg2vcr also do the mux'ing? Does it do a poor job compared to mplex? I can't seem to find mplex available for download, and I've also seen many posts on this board of people using it and creating unviewable and uneditable mpegs. Can you post your command line, and perhaps a place to download a Windows binary of mplex? [/B]

no, TyTool5 and vsplit13c - GOOD!


I never used mpeg2vcr to mux, not sure why fitting a 21min episode on SVCD would be a problem. My episodes are approx. 450m after editting. - plenty of room. You can try using Nero to "fix" the bitrate - to SVCD standard.

(edit) I mento to say I use Besweet to upsample to 44100khz


here (http://www.mpython.com/download/mplex.zip) is a copy mplex for win32

[Saturn]

Originally posted by Hi8
no, TyTool5 and vsplit13c - GOOD!


I never used mpeg2vcr to mux, not sure why fitting a 21min episode on SVCD would be a problem. My episodes are approx. 450m after editting. - plenty of room.

here (http://www.mpython.com/download/mplex.zip) is a copy mplex for win32

Well, fitting 21 minutes onto an SVCD isn't a problem, but I was trying to do it without re-encoding and suffering a loss of quality. If I have to re-encode, I think I can just use Vegas Video Factory to do the muxing, editing, resampling of audio, and re-encoding, and I'm already familiar with the interface. Then all I have to do is burn it to CD with Nero.

21 minutes won't fit onto an SVCD at TiVo's default Best quality - 5800KB/s, IIRC. If I have to re-encode, there's a thousand different ways to do it, but I was after the holy grail, much like jdiner, to avoid re-encoding altogether.

[Hi8]

Originally posted by Saturn

21 minutes won't fit onto an SVCD at TiVo's default Best quality - 5800KB/s, IIRC. If I have to re-encode, there's a thousand different ways to do it, but I was after the holy grail, much like jdiner, to avoid re-encoding altogether.

my BEST quality is still set for the default that my SVR-2000 came configured, that being 544X480 5.8m 32khz.

no mater how you slice it, that is not SVCD-able as is. what ever you do to the 544x480 will result in loss of quality to get to 480x480 SVCD standard. Also the other major issue is the bitrate, by getting to 2.6m another loss of quality.

I'm not sure what you intended could/would happen during those transistions - but to get to the SVCD standards quality will suffer.

I have several DVD players (Sony s7700,APEX 600a,APEX 1100w, NEXTBASE DVM4,KOSS 2163) each has it's ability to cope with "standard" formats. I've never tried, but perhaps one of them may actually play the TiVo SA BEST - format. However The problem with fitting anything worth watching time-wize on one CD-R cannot be forgotten as you stated.

It is clear you have a decision to make or should I say compromise? Because none of my players support miniDVD - my only path is SVCD which I find quite acceptable. Anything I want in the BEST quality possible I go to DVD.

[jdiner]

Alright. Things got moved. Many thanks to whoever did it...

So we are in our new home and things should begin to progress just a bit more smoothly... At least that is the plan.

--jdiner

[snoopy]

Jdiner, Is this not correct?

This was my understanding of one of your very recent posts in the original muxing thread... that the holes issues had been resolved and a final release was to follow. I have not seen it yet. Please correct me if I have missed something. Thanks.

[mattdb]

Jdiner,

I was starting to panic. I thought you had abandoned ship. Wheww.... Found your thread.

Matt

Are you still working on the DTiVo here too? I couldn't find the other one in the other forum. The old one had been closed.

Could you add a note to the bottom of that one to point here?

[durian]

Here's the printout from muxing a SA clip(-s -m) with vsplit1k or vsplit1l:

Processing 'test1.ty': (10 chunks per tick)
Jumping 1 chunks into input (131072 total bytes)
Detected Tivo Type: Standalone
Detected Audio Stream Type: MPEG Layer II
Final standardAudioSize = 880
Final standardFrameLength = 864
Final standardAudioDiff = 3240 or 00:00:00.036
First Video PTS: 00:00:01.182
MUX-add: Audio PTS: (107228) 00:00:01.191
MUX1 Found the B3-B8 header block and the 1st PES header block
MUX1 Video Embedded PES Header: (106396) 00:00:01.182
MUX2 Now we have the almost full first packet (lead-in missing)
for the B2-Frame...
MUX2 lead-in skip == 3089120
MUX2 PTS = 106396 == 00:00:01.182
ERROR: Out of memory getting a new MuxNode buffer!

[jdiner]

Originally posted by snoopy
Jdiner, Is this not correct?

This was my understanding of one of your very recent posts in the original muxing thread... that the holes issues had been resolved and a final release was to follow. I have not seen it yet. Please correct me if I have missed something. Thanks.

No. Not even close. This is still a work in progress. Yes I closed the next major issue but there are more still to come. I have never said I was done. Don't know where you got it from. Read the messages I have posted. They detail what is to be done... And no I have not made a new release yet. I was out of town for the last 2 days. Makes it hard to work on anything.

--jdiner

[jdiner]

Originally posted by mattdb
Jdiner,

I was starting to panic. I thought you had abandoned ship. Wheww.... Found your thread.

Matt

Are you still working on the DTiVo here too? I couldn't find the other one in the other forum. The old one had been closed.

Could you add a note to the bottom of that one to point here?

I am not sure why everyone thinks this is a Stand-Alone tivo forum. This is an extraction and processing forum. Applicable to both types of Tivos.

--jdiner

[Rowan]

It's good to see that there is now a new home dedicated to all things extracting and muxing.

[jdiner]

Originally posted by durian
ERROR: Out of memory getting a new MuxNode buffer!
Ok. That is bad. Basically it means a malloc failed. But to do that you have to have exhausted all physicall and virtual memory or have run out of the stack space defined for a program. Neither should happen. So... you might have a bad clip, bad in way I have never seen, or it might have been other issues.

Are you running under Windows? if so do a full shutdown and then re-try it when it all boots back up before running everything else. You just never know... :)

If you are running under linux, some other unix with a bin emulator going on, then please try it under windows to see what happens there.

If you can't tell, I really have no idea what is going wrong for you at this point. I have never seen that occur.

--jdiner

[jdiner]

Ok. Well. Good news folks. The code I put in place for dealing with holes led me directly to the code for editing...

Now I do not have an editor in place. Nothing GUI. Nothing visual. But I do have a textual "from timestamp1 to timestamp2" cutter in place. And miracle of miracles everything still works. :)

Once I get a bit more polish on it, we have the first cut of a way to remove lead-in, lead-out, commercials, and unwanted segments. :)

Not sure what to do next. To start the GUI. Or to build a better text cutter interface...

But I am excited. It works. Perfectly... :) You just gotta love it when that happens.

NOTE: I still have not gotten to the reseting PTS issue yet. I got a bit sidetracked...

I intend to make a release before getting to that next big issue. I was just excited and had to tell people.

--jdiner

[jdiner]

Oh, and I suppose I should mention. That once I got the hole code done. The changes to the mux'ing engine to handle the cutting were 25 lines of C source code.

Now that kind of thing is what happens when you design and build and rebuild and rebuild this stuff until it is right... :)

--jdiner

[dlang]

Great!!!

by the way we don't need no stinkin GUI for the editing :)

give us something that works with a text editor and I'll bet you will have a dozen or so GUI's that work with it within a week

seriously I think it would be a good idea to let people try useing it as is for a little bit and then they can suggest improvements to the text interface, let's not cast it in stone anytime soon.

[digitalAir]

just curious...

do timestamp1 and timestamp2 have to be GOP boundaries?

if not, does it round to the nearest, or what?

[jdiner]

Originally posted by dlang
Great!!!

by the way we don't need no stinkin GUI for the editing :)

give us something that works with a text editor and I'll bet you will have a dozen or so GUI's that work with it within a week

seriously I think it would be a good idea to let people try useing it as is for a little bit and then they can suggest improvements to the text interface, let's not cast it in stone anytime soon.
My thoughts exactly.

Anyone can write that part of it. Well ok anyone with a bit of talent for the game. But yeah. So I figure for now the text interface will stick.

I actually posted what I did hoping that some of the people that have offered to help write code recently would well... offer to write that part... :)

Right now it is 2 timestamps per line indicating a cut. I.e.

start end

or

00:52:18.932 00:55:18.932

to cut the very end of a stream starting at 52 minutes 18 seconds up to and including 55 minutes 18 seconds... Hummm. You know I never thought of it. But the cut times are in stream time. Which changes for every stream and has a different starting point... I can print out where a stream start very quickly. Ack... There is a problem there. Whoever winds up doing the GUI is going to have to be aware of a few things from the stream. Oh well. Guess it will be a bit more troublesome than I thought.

Any GUI volunteers want to jump on the bandwagon?

--jdiner

[jdiner]

Originally posted by digitalAir
just curious...

do timestamp1 and timestamp2 have to be GOP boundaries?

if not, does it round to the nearest, or what?
It turns out that many of us where right in what we guessed for cutting way way way back when.

It can be used to cut on a millisecond boundry not just on a GOP boundry. Do keep in mind that that is only a cut point. Meaning a frame is not shown for less time nor is half of it displayed. It is either all there or all not there etc... I have used it to cut at a GOP boundry and between 2 b-frames.

As was guessed at, by edpuffmonster IIRC, so long ago there is a small almost unnoticable glitch in the video to me it was not worth worrying about getting any better.

As for the rest of it I am just cleaning things up now and should have a test version out real soon.

I know I know. I have said that before. But hey in the end I always deliver so please, be patient.

--jdiner

[dlang]

unfortunantly I don't have the time right now to write a full GUI, but a series of scripts that grabbed 1 sec of video every min, displayed it then asked you if it was show/commercial/unknown, repeated a larger clip around the unknown, then narrowed down the transition from commercial to program, then produced a final mpeg with all the commercials cut out of it is within my time budget (for speed I will probably just set it up to run from a linux SVGA/VESA console not useing X at all)

not the fanciest interface around, but it should get the job done

[FreydNot]

Once you have released the new text based cutting I'll probably try using M2-edit to generate an edit decision list. It will be pretty easy to make an excel spreadsheet to account for the first frame being non-zero.

I see the server takes a single start and end point. Is the intention for the client to stitch together several pieces? Or will the server eventually take a multi-part list?

PS, when the threads moved my email notifications stoped working. I thought things were just quiet until I came back and poked around.

[jdiner]

Rather than wait. Here it is... Not fully cleaned up. And not configurable yet. But something for others to see what I am talking about with the editing.

At the moment:

- It does not take an external file for cut points.
- It has 1 cut built in internally from 5-10 seconds from the start of the clip wherever it is.
- It still only processes a small chunk of the file (2000 chunks).
- It contains the code needed to work around the following types of holes:

1- Audio only.
2- Video only.
3- Where the audio and video are gone from the same timeframe range.
4- Where the audio and video are gone from overlapping segments.

I couldn't think of another case to test so for now I am considering the hole problem closed.


This is still just a quick peek/test version. What you need to do is the following:

1- Locate your small test clip.
2- Run it through 1L and save the output.
3- Run it through 1M and save the output.
4- Play output-1L and see how and where it works etc...
5- Play output-1m and notice the cut from the 5 to 10 second marks.
6- Be suitably amazed.
7- Go find spouse and show them, pretend to accept their amazement at face value.
8- Ignore muttering as they walk away shaking head... :)

--jdiner

[jdiner]

Originally posted by FreydNot
I see the server takes a single start and end point. Is the intention for the client to stitch together several pieces? Or will the server eventually take a multi-part list?

PS, when the threads moved my email notifications stoped working. I thought things were just quiet until I came back and poked around.

I was afraid that the quiteness might mean people were not following it. Perhaps a post to the original forum saying we are not gone, just moved to here. I used to just right into the DTivo fourm rather than going from the home page in. I suspect others do the same thing.

As for the server. What do you mean? My server? It takes all of the FSIDs or conversely just the chosen FSIDs based on what button is hit. But that is TyTool. For VSplit, this tool, there is no server. It just takes the file and goes.

As for how the client would work I figure at some point I will still do what I was thinking before.

- Add a feature to the server to get just the I_Frames and related timestamp information.
- Add have a new client/window in TyTool that will take this output and create a cut list from it.
- Use this cut-list in whatever fashion while downloading the full thing and mux'ing.

Overall this seems like it would work. If the cut-lists could be saved to disk then you could mark the cut points in several shows very quickly and then do the longer more drawn out download/split/cut/mux process on several shows without user intervention. Yeah. I like that idea. :)

--jdiner

[FreydNot]

Ya, I was thinking about the tytool situation. Ignore my references to client and server.

Since you aren't limited to cutting on I frames, I could see a problem trying to add two (or more) cut out pieces together. Unless the second piece starts with an I frame there could be trouble. Maybe the easiest way to deal with this would be to keep the segements as individual pieces and don't try to make one big file.

I'm down with queuing up a bunch of cut lists and letting it run over night. In fact, I like your whole approach to the problem. Seems practical and useful.

[durian]

Rather than wait. Here it is... Not fully cleaned up. And not configurable yet. But something for others to see what I am talking about with the editing.

Is it possible to post a linux version as well?

[mpauley]

jdiner -

Looks great! Both files keep sync, FF and RR with no problems.

--mp

[Fugg]

I tried vsplit1m on a basic sa file I had laying around at work...

C:\Temp\555>vsplit1m -m mr.ty mr.mpg mr.mpa
Processing 'mr.ty': (10 chunks per tick)
Detected Tivo Type: Standalone
Detected Audio Stream Type: MPEG Layer II
Final standardAudioSize = 880
Final standardFrameLength = 864
Final standardAudioDiff = 3240 or 00:00:00.036
First Video PTS: 00:00:01.227
MUX1 Found the B3-B8 header block and the 1st PES header block I1-frame
MUX1 Video Embedded PES Header: (110476) 00:00:01.227
MUX2 Now we have the almost full first packet (lead-in missing), and the header
for the B2-Frame...
MUX2 lead-in skip == 2
MUX2 PTS = 110476 == 00:00:01.227
MUX2 Video Embedded PES Header: (104469) 00:00:01.160
MUX Found the 3rd PES header block (B3-frame)
MUX3 there are 1 lead-in bytes for the P frame.
MUX2 Video Embedded PES Header: (107472) 00:00:01.194
MUX Found the 4th PES header block (P4-frame) - We DO NOTHING with the data!
MUX2 Video Embedded PES Header: (113478) 00:00:01.260
MUX Have the body for P4-frame and Header for B5-Frame.
MUX5 Video Embedded PES Header: (116481) 00:00:01.294
MUX-add: Audio PTS: (110438) 00:00:01.227
(looks like it played for 5 sec?)
Cutting a video packet at: 00:00:06.232
Cutting a video packet at: 00:00:06.265
Cutting a video packet at: 00:00:06.399
(246 more a/v "cuts")
Cutting an audio packet at: 00:00:11.199
........ 100......... 200.........and on to the end.

5 sec in, 5 sec cut... it was perfect!

Really looking foward to "2 timestamps per line indicating a cut"!!!!
:D

[durian]

Originally posted by jdiner
Ok. That is bad. Basically it means a malloc failed. But to do that you have to have exhausted all physicall and virtual memory or have run out of the stack space defined for a program. Neither should happen. So... you might have a bad clip, bad in way I have never seen, or it might have been other issues.

Are you running under Windows? if so do a full shutdown and then re-try it when it all boots back up before running everything else. You just never know... :)


If you are running under linux, some other unix with a bin emulator going on, then please try it under windows to see what happens there.

If you can't tell, I really have no idea what is going wrong for you at this point. I have never seen that occur.

--jdiner

Done all that. It segfaults on linux. Furthermore, the clip that I am about to upload to you is at the very beginning of a show, so allocation errors are odd. Consequently, I thought that this might be an artifact of tytool -- thus the subject of my post.

I tried 1m on it and it seems to process it fine, except that PowerDVD cannot play it (quits with errors), although WinDVD can.

[jdiner]

Originally posted by durian
Done all that. It segfaults on linux. Furthermore, the clip that I am about to upload to you is at the very beginning of a show, so allocation errors are odd. Consequently, I thought that this might be an artifact of tytool -- thus the subject of my post.

I tried 1m on it and it seems to process it fine, except that PowerDVD cannot play it (quits with errors), although WinDVD can.

Ok. That is wierd. I would like to take a look at the TyStream file. This would be the first thing that PowerDVD can't play that is being outputed by the latest mux'ing code that I have seen. I might have missed a corner case or it might be something else altogether. At this point I can't say.

--jdiner

[mpauley]

jdiner -

Just wanted to let you know if you need any help with a GUI, let me know.

-mp

[jdiner]

Ok. One further note. I had intended to mention this and then I forgot. There is an issue dealing with playback if there is a hole. It is a playback issue and not a Vsplit/muxing issue.

If you get a single stream hole, either audio or video, that is < 3 seconds in length then both PowerDVD and WinDVD will skip the duration of the hole. They beging playback again where the hole ended. This is most noticable by a quick just across the gap and then what appears to be a 1/8 to a 1/4 of a second pause before playback resumes.

I have no idea why they would treat a gap this way. For a 4 second+ gap playback continues normally you just don't get the missing stream. So if anyone should see this it is not a bug in my code. Just a decision by authors of these other tools.

--jdiner

[snoopy]

Most older DVD players will play "regular" VCD's comprised of mpeg1 level encoding as opposed to mpeg2. The information is correct according to all the information I have read recently. I would like to create VCD's instead of SVCD's obtained by using the -d switch in vsplit. Is there any plans for such an option so that the disk will play in almost any machine until I can afford an expensive DVD burner?

[jdiner]

Originally posted by Fugg
I tried vsplit1m on a basic sa file I had laying around at work...

5 sec in, 5 sec cut... it was perfect!

Really looking foward to "2 timestamps per line indicating a cut"!!!!
:D
It was in there. Or perhaps I should say, after a fashion it was in there. I was in the middle of rebuilding it to be more production ready, I had a typical RAD prototype of the code which would not hold up under production use. Anyway, I figured rather than seem a tease and mention in and then put nothing out for awhile I would release a simple test version that would showcase the new feature. And so 1M was born. :)

Anyway, I am working on a few things this morning and should have more news later in the day.

--jdiner

[Kythorn]

Originally posted by jdiner

If you get a single stream hole, either audio or video, that is < 3 seconds in length then both PowerDVD and WinDVD will skip the duration of the hole. They beging playback again where the hole ended. This is most noticable by a quick just across the gap and then what appears to be a 1/8 to a 1/4 of a second pause before playback resumes.
--jdiner


Shouldn't be a problem with the kind of holes I get from off the locals spot beam...

[Rowan]

Originally posted by jdiner
Ok. One further note. I had intended to mention this and then I forgot. There is an issue dealing with playback if there is a hole. It is a playback issue and not a Vsplit/muxing issue.

If you get a single stream hole, either audio or video, that is < 3 seconds in length then both PowerDVD and WinDVD will skip the duration of the hole. They beging playback again where the hole ended. This is most noticable by a quick just across the gap and then what appears to be a 1/8 to a 1/4 of a second pause before playback resumes.

I have no idea why they would treat a gap this way. For a 4 second+ gap playback continues normally you just don't get the missing stream. So if anyone should see this it is not a bug in my code. Just a decision by authors of these other tools.

--jdiner

In the case were the audio has a hole would it not be possible to just fill it in with silence, this should allow the video to keep playing but with no audio?

Rowan

[Pr.Sinister]

Originally posted by snoopy
Most older DVD players will play "regular" VCD's comprised of mpeg1 level encoding as opposed to mpeg2. The information is correct according to all the information I have read recently. I would like to create VCD's instead of SVCD's obtained by using the -d switch in vsplit. Is there any plans for such an option so that the disk will play in almost any machine until I can afford an expensive DVD burner?

To get an MPEG-1 VCD File, you'll have to transcode the output
of the Mux or take the elementary streams and transcode those.

The DirecTiVo creates MPEG-2 files. The -d just says to MUX with
a specific sector size and add a SVCD header. There is no way
of muxing to VCD.

VCD Standard is MPEG-1, 1150kbps
SVCD Standard is MPEG-2, 2520kbps (althought i can pretty much
be anything).

Bottom line : You have to transcode to get MPEG-1 or you can
use a header trick but that doesn't work on all players.

-Pr.

[snoopy]

Originally posted by Pr.Sinister
To get an MPEG-1 VCD File, you'll have to transcode the output
of the Mux or take the elementary streams and transcode those.

The DirecTiVo creates MPEG-2 files. The -d just says to MUX with
a specific sector size and add a SVCD header. There is no way
of muxing to VCD.

VCD Standard is MPEG-1, 1150kbps
SVCD Standard is MPEG-2, 2520kbps (althought i can pretty much
be anything).

Bottom line : You have to transcode to get MPEG-1 or you can
use a header trick but that doesn't work on all players.

-Pr.

What would you use to accomplish this "transcoding" process to MPEG-1 VCD Standard? I probably won't attempt the header trick until I get a real one to work properly. Thanks.

[tlphipps]

Bottom line : You have to transcode to get MPEG-1 or you can

Wouldn't the point of doing VCD be to include more on the cd. If you just trick the player into thinking it's playing a VCD when it's actually playing an SVCD, you don't accomplish any space savings?

Am I just missing something here?

[snoopy]

Originally posted by tlphipps
Wouldn't the point of doing VCD be to include more on the cd. If you just trick the player into thinking it's playing a VCD when it's actually playing an SVCD, you don't accomplish any space savings?

Am I just missing something here?

The point would be to allow an older DVD to play the file. By "older", I mean one that does not play SVCD's.


Does anyone know what program I can use to "transcode" the mpeg2 file to be mpeg1 ? I try TMPEnc but I don't understand it well enough to set it to do that. What gives

[jdiner]

Originally posted by snoopy
Most older DVD players will play "regular" VCD's comprised of mpeg1 level encoding as opposed to mpeg2. The information is correct according to all the information I have read recently. I would like to create VCD's instead of SVCD's obtained by using the -d switch in vsplit. Is there any plans for such an option so that the disk will play in almost any machine until I can afford an expensive DVD burner?
They are 2 completely different thing. I would have to build and encoder and a decoder (encoder for mpeg-1 and decoder for mpeg-2) into the system to make it happen.

For me this is never going to happen. I don't want it. No one else that knew what was going on would either. It is a waste of time.

A waste of time because there are already better tools out there that do this. Going back to mpeg-1 is a re-encode step. You lose quality. You have drop the bit rate massively from what it is, and so you lose quality massively.

Go to wal-mart. Get out your wallet and buy a $49 apex. My friend just did and it will play stuff from the DTivo/SATivo wonderfully.

--jdiner

[jdiner]

Originally posted by Pr.Sinister
Bottom line : You have to transcode to get MPEG-1 or you can
use a header trick but that doesn't work on all players.

There is honestly no header trick that will work for 99% of the players. The encoding of mpeg-2 is different at the encoding layer than mpeg-1. I could mux to mpeg-1 headers but nothing would play. Because nothing is as it should be...

It is like putting sand in shoe boxes and claiming the sand is now shoes. Not hardly...

--jdiner

[jdiner]

Originally posted by snoopy
The point would be to allow an older DVD to play the file. By "older", I mean one that does not play SVCD's.

Does anyone know what program I can use to "transcode" the mpeg2 file to be mpeg1 ? I try TMPEnc but I don't understand it well enough to set it to do that. What gives

We should have just started here. A player that will not play SVCDs does not have the mpeg-2 system built in such a way to accept generic mpeg-2 files. They must match the DVD specs perfectly. These players also tend to have problems with slightly non-standard DVDs.

You have to convert, i.e. transcode, from mpeg-2 to mpeg-1. Download a demo copy of tmpgenc. It does this and has a nice newbie wizard interface. The demo used to be good for 30 days no idea now. Once you try it if you like it support them and buy it.

But for now let's consider the topic closed. It has nothing to do with mux'ing and dealing with mux'ed files. It is about VCD and that has nothing to do with what is going on in here. Make another thread...

--jdiner

[alunj]

Tested 1m on UK SA 2.5.5 . Looks good. Sync Good
Some noise on cut ! But good sync

A

[jdiner]

Originally posted by alunj
Tested 1m on UK SA 2.5.5 . Looks good. Sync Good
Some noise on cut ! But good sync

Yeah. There will be. Always unless it is a perfect GOP cut but that means re-ordering frames and there is no garantee that will work right either. How much noise all depends on nature of the stream.

The only other way to get rid of the noise is to re-encode a few frames. And while that is possible not something I plan to do as what is there is good enough for me.

--jdiner

[Rowan]

Originally posted by jdiner
Yeah. There will be. Always unless it is a perfect GOP cut but that means re-ordering frames and there is no garantee that will work right either. How much noise all depends on nature of the stream.

The only other way to get rid of the noise is to re-encode a few frames. And while that is possible not something I plan to do as what is there is good enough for me.

--jdiner

One other way to fix that is to re-encode that last frame before the cut and then the first frame after the cut to ramp the audio down to silence and then back up to the normal sound.

But now re-reading you post I think that is what you said.

Rowan

[skorous]

Originally posted by jdiner

The only other way to get rid of the noise is to re-encode a few frames. And while that is possible not something I plan to do as what is there is good enough for me.

--jdiner

Out of curiousity, are there any tools that can do the re-encoding on a range where the cut is going to be committed? I doubt the noise will bother me but my curiousity ie piqued. <grin>

Skorous

[Pr.Sinister]

Originally posted by snoopy
What would you use to accomplish this "transcoding" process to MPEG-1 VCD Standard? I probably won't attempt the header trick until I get a real one to work properly. Thanks.

In TMPGEnc, just go into the Project Wizard and choose
NTSC under Video-CD

Click Next, Drag & Drop or use the file browser to add
your m2v and m2a files and the other steps should all be
Next, next, next, etc... very easy... Depending on your CPU
power, i could take anywhere from 30mins to 7 hours..

hehehe

-Pr.

[jdiner]

Originally posted by skorous
Out of curiousity, are there any tools that can do the re-encoding on a range where the cut is going to be committed? I doubt the noise will bother me but my curiousity ie piqued. <grin>

The answer is "yes and no"... :) Fundamentally it is just an MPEG-2 encoder that is needed. But you need to get the uncompressed frames and then encode only a few frames.

What you would do is turn them into I-frames so that there is no prediction. This will make them substantially bigger. But it will give you a perfect cut.

--jdiner

[jdiner]

Ok. I supose I should have elaborated on why I consider what I have "enough".

Most of what I am interested in is TV shows. The ones I have all fade to black over half dozen frames and then fade back in from black over a half dozen.

I cut from the end of the black fade out, to the start of the black fade in, and bingo, the noise video wise is almost none existant. This will not always be possible. But by and large it will work. :)

However to do it at that level is something of a challenge for the GUI needed to pull it all off.

--jdiner

[Kythorn]

re: powerdvd/windvd playback oddness with 'holes'. I got bored and decided to play with the output of 1m on my dvd players, and both of them seem to not care whether the hole was 4 frames or 4 seconds, they just play the part of the stream they do have.

Afreey 2060 ld and the el cheapo apex from walmart.

They exhibit the tearing I'd expect from the cutting on non gop boundaries, but as long as it doesn't completely puke, I don't especially care.

I'm trying to see what happens when I fast forward or rewind over the gap, but it's pretty hard to tell when it's 5 seconds from the beginning. I'll test this more when we can define our own cuts.

[skorous]

Originally posted by jdiner
The answer is "yes and no"... :) Fundamentally it is just an MPEG-2 encoder that is needed. But you need to get the uncompressed frames and then encode only a few frames.

What you would do is turn them into I-frames so that there is no prediction. This will make them substantially bigger. But it will give you a perfect cut.

--jdiner

Can you ("you" being a generic term for any end-user) convert frames over a limited timeframe? If so, you could convert a dozen frames or so at each cut to get the perfect cut. That shouldn't make a large difference in size, should it?

Oh, and I guess I should ask if you might have seen a mythical beast capable of doing this? :)

Skorous

[jdiner]

Originally posted by skorous
Can you ("you" being a generic term for any end-user) convert frames over a limited timeframe? If so, you could convert a dozen frames or so at each cut to get the perfect cut. That shouldn't make a large difference in size, should it?

Oh, and I guess I should ask if you might have seen a mythical beast capable of doing this? :)

Ok. I can dump any number of frames to an external file. Perhaps down the road that is what should be done. Rather than doing any encoding myself. Dump the 5 frames that are required and then spawn off a call to an external compressor something with a nice API and then reload and mux using those new replacement frames and the original timestamp values.

Then we could get perfect seemless cuts. The audio is not a problem as it does not have any prediction going on.

Anyone know of there is an external API to tmpgenc or any of the other compressor tools?

--jdiner

[skorous]

JD,

Just so it's clear I'm not looking for any features to be added to vsplit though I appreciate the willingness. I'm perfectly happy with an additional step or two to create perfect cuts if indeed it bothers me as I'm the one being neurotic. :) Since I haven't even seen the cut noise (been busy) I'm only seeking information as to whether it can be fixed if it _does_ bother me. I'm just ecstatic to not have to deal with lipsync issues and I had a bugger of a time with that. Most of my machines were really erratic so I'd wind up encoding things a half dozen times before I got it right. A few steps for perfect cuts is kid stuff. <big grin>

That being said, am I correct in assuming that I could use TMpgEnc's Merge and Cut feature to cut a show into, for simplicities sake, 3 parts : before commercials, commercials + the black fade frames before and after, after commercials. Then re-encode the commercials section with TmpgEnc into solely I-frames, make my cut, and merge 'em back together again? How would that be affected by the VFR nature of DTivo? < joke > Not that I care since I'm a lowly SA owner < /joke > but it'll probably come up as part of someone elses curiousity.

Skorous

[sirfergy]

Question. Is one of the goals of the splitting.muxing to eliminate temporal references? If I am seeing them, do you want to know about them?

Thanks

[jdiner]

Originally posted by sirfergy
Question. Is one of the goals of the splitting.muxing to eliminate temporal references? If I am seeing them, do you want to know about them?

I am so sick of this stupid question. Mux'ing has nothing to do with temporal reference issues. There are some tools most notably Spruce Up that for some insane reason checks that field while doing it's conversion into a VOB. While this includes a mux'ing step this check has nothing to do with that.

The error in the simplest terms is the field order within a frame. A single image is made up of lots of tiny little segments of a picture. Each segment has it's own start code and contains data formatted in the mpeg-2 mode.

Every now and again for some unknown reason DTV send these out in "the wrong order". Instead of being 1, 2, 3, 4, 5, 6, etc... It is 1, 2, 4, 3, 5, 6...

This order-by-number is not defined in the specs. As near as I can tell Spruce Up just liked it this way. Although it might be a DVD spec thing... The data in the file can be "re-organized" to work with what was coming. A simple sort is enough. while complex to add due to the nature of the data alignment. At times I have also seen missing segments that could cause this same error.

Spruce Up's stopping on it is just plain stupid. At the very worst case "with bad hardware" it would cause 1 incorrectly rendered frame. That is all....

There is not much that I can do however to make things work for the Spruce Up program itself. Read all of the messages on why we even need to mux things in the first place to understand why. All of the work I am doing right is for the sole purpose of NOT NEEDING TO USE IT OR OTHER TOOLS LIKE IT.

I can't believe anyone even tries. It is a steaming pile of crap. I have tried to do a half a dozen different things with it. Not once has it worked start to finish, it crashes continuously, and well... most of the rest is just a rant.

So no. I am not going to be elminating them. I am going to be trying to get rid of the need for programs that care about them in the first place.

--jdiner

[jdiner]

Originally posted by skorous
JD,

Just so it's clear I'm not looking for any features to be added to vsplit though I appreciate the willingness.
It would be a down the road type of thing. I was just giving a quick thought to what could be done to have a more seemless cut. But 5-6 frames would be more than enough.

As for overall space from the previous message an I frame from DTV is about 60k on average, a P frame about 30 and a B frame about a 20k. So overall if there were 6 cuts:

- Lead-in
- 4 commercial breaks
- Lead-out

You would increase about what 500k? Or so?

As for what would happen. It would fail. If the 6 frames at the end were part of the VFR nature.

1.5 1.5 1.5 1.5 1.5 etc...

Then creating them elsewhere and trying just through them in would generate:

1 1 1 1 1 etc...

They would have to be made, and then handed back to the mux'er which would use the first set of timestamps and get:

1.5 1.5 1.5 1.5 1.5 etc...

It would just be I I I I for the frames instead of P B B P B B and so on.

--jdiner

[skorous]

It would be a down the road type of thing. I was just giving a quick thought to what could be done to have a more seemless cut. But 5-6 frames would be more than enough.

Cool. I just like to maintain my non-obnoxious status. ;)

As for overall space from the previous message an I frame from DTV is about 60k on average, a P frame about 30 and a B frame about a 20k. So overall if there were 6 cuts:
- Lead-in
- 4 commercial breaks
- Lead-out

You would increase about what 500k? Or so?



Ok, 500k isn't that bad. It's noticeable but when you think about a half a meg versus a 650M/700M it's not that bad.

As for what would happen. It would fail.

Yeah, that's kinda what I figured. Cool, thanks for the info.

Skorous

[stealthdave]

I have a simple feature request. :) Can a you add a flag to pipe the muxed output to stdout rather than a file? You would probably have to redirect the currently copious logging messages to a file or stderr (I believe that it is currently going to stdout), but there would be many advantages to this. i.e, piping a muxed stream directly into a transcoding utility. :)

I'm a web developer, so among other uses, I can see using this as a way to watch a program on the TiVo remotely by transcoding the stream in real-time to a low-bitrate MPEG-4 stream and stream it over the web. You would need a dedicated server to do this, but the processing power to do it in real-time is fairly cheap these days. The limiting factor would actually be the TiVo's network connection speed.

Anyway, that was a very off-topic explanation of why I would like the ability to pipe the muxed output to stdout. :)

- Stealth Dave

[snoopy]

Originally posted by stealthdave
I have a simple feature request. :) Can a you add a flag to pipe the muxed output to stdout rather than a file? You would probably have to redirect the currently copious logging messages to a file or stderr (I believe that it is currently going to stdout), but there would be many advantages to this. i.e, piping a muxed stream directly into a transcoding utility. :)

I'm a web developer, so among other uses, I can see using this as a way to watch a program on the TiVo remotely by transcoding the stream in real-time to a low-bitrate MPEG-4 stream and stream it over the web. You would need a dedicated server to do this, but the processing power to do it in real-time is fairly cheap these days. The limiting factor would actually be the TiVo's network connection speed.

Anyway, that was a very off-topic explanation of why I would like the ability to pipe the muxed output to stdout. :)

- Stealth Dave

WOW! Heck of an idea :)

[jdiner]

Originally posted by stealthdave
I have a simple feature request. :) Can a you add a flag to pipe the muxed output to stdout rather than a file? You would probably have to redirect the currently copious logging messages to a file or stderr (I believe that it is currently going to stdout), but there would be many advantages to this. i.e, piping a muxed stream directly into a transcoding utility. :)

You can't transcode the output of a DTivo stream. The SATivo should work but the DTivo won't. For the same reason as has been mentioned before. The system reports a frame rate that isn't real.

Or perhaps I should say that none of my attmepts at transcoding have worked out.

--jdiner

[AlphaWolf]

jdiner: faye was working on doing just this, and I believe she was saying its possible to do it with a directivo stream. She was converting the tystreams strait to a transport stream. I haven't heard from her in a long time though.

[abcdriver]

Why do people insist on having sig's longer than their posts?
:confused:

[RxMan]

Why does someone with 2 posts think he knows anything about one of the most respected members of this board?

Let's keep this on topic.

[stealthdave]

Originally posted by jdiner
You can't transcode the output of a DTivo stream. The SATivo should work but the DTivo won't.

It just so happens that I have an SA TiVo. :)

- Stealth Dave

[otakucode]

At what point in the timeline of development (feature wise, referring to your status update posts from the other thread talking about VOB generation and such) are you planning on releasing a version that processes a full stream? If I've missed a release that does this, please forgive my idiocy. The status update posts made it sound like a full version was out, but I couldn't find a release and then I saw your release of the version with the cutting ability that again only functioned on a small part, so I'm taking that as a hint that no version that runs for an entire stream has been made public yet... so if I'm right and no such version has been released, I was just wondering if that was going to be a "last step" or what? I know its really a choice of when you want to deal with the resetting PTS values and other bugs that are inevitably bound to crop up, so I'm just wondering when you've chosen to take up that sword ;)

OtakuCODE

[FredThompson]

Originally posted by jdiner
Ok. I can dump any number of frames to an external file. Perhaps down the road that is what should be done. Rather than doing any encoding myself. Dump the 5 frames that are required and then spawn off a call to an external compressor something with a nice API and then reload and mux using those new replacement frames and the original timestamp values.

Then we could get perfect seemless cuts. The audio is not a problem as it does not have any prediction going on.

Anyone know of there is an external API to tmpgenc or any of the other compressor tools?

--jdiner
Why don't you just take the reference code that's available and use it? Speed won't really be an issue because you're only talking about a few frames.

It's linked in the MPEG section of my sig. I'm INCREDIBLY interested in perfect edits (TLC and Discovery don't do long fades that often) so here 'tis:

ftp://mm-ftp.cs.berkeley.edu/pub/multimedia/mpeg2/software

I've stumbled on some other freely-usable source so let me know if you need something else. I'll happily go digging.

[FredThompson]

That link is a dog. Here's the source code.

[jdiner]

Originally posted by FredThompson
Why don't you just take the reference code that's available and use it? Speed won't really be an issue because you're only talking about a few frames.
I will have to have a look see at the source code. The reason I didn't want to use the reference source I have is that it is liscenced such that we could never, even with free tools use it. And it was crap. Not just big frames as the problem but bad frames. Very bad.

If this is better on both counts I have no issues with using it. I would also like perfect cuts. I really would. I am as much a fan of that as anyone here. But I also know how much work is involved in all of this. And comparing the work needed to do my own versus what we have already... I kind of have the feeling of "why bother" on doing better cuts. But that is just me.

--jdiner

[fredisdead]

Jdiner:

As I understand from your recent post, Temporal Reference errors are a result of bad data in the dtivo stream.

However, I have had two ty streams which generate Temoral Reference Errors in DVDMaestro when split with vsplit13c, but do not have the errors when split with the earlier vsplit13.

There are a few OOB's in the subject streams, but no holes reported (no vidbody off messages).

The subject streams play fine with no sync problems on the final burned DVD-R's.

[jdiner]

Originally posted by fredisdead
Jdiner:

As I understand from your recent post, Temporal Reference errors are a result of bad data in the dtivo stream.

However, I have had two ty streams which generate Temoral Reference Errors in DVDMaestro when split with vsplit13c, but do not have the errors when split with the earlier vsplit13.

There are a few OOB's in the subject streams, but no holes reported (no vidbody off messages).

The subject streams play fine with no sync problems on the final burned DVD-R's.

If you can find them in the stream cut them for me. I am not sure what is going on. But I have gone to far one way or the other in adding or removing what looked like bad data.

It should not be happening.

And again it is not bad data it is out of order data... However if I am cutting something I should not be then we would infact potentially be getting out of order data.

i.e. 1-1 1-2 1-3 [cut goes here] 2-5 2-6

and this looks likes out of order data...

--jdiner

[jdiner]

I mentioned again recently in response to questions and comments from others the VFR nature of the DTivo/DTV video stream.

To try and help clarify once again what is going on attached is a text file that shows the video frame count and the data sizes segmented by seconds. 1 line per second.

As an example here are the first few lines from the file:

29 nodes && (488848 bytes)
31 nodes && (519388 bytes)
28 nodes && (289136 bytes)
25 nodes && (277036 bytes)


First line means: 29 frames in that seconds worth of video with a total of 488k for just the video data.

Second line means: 31 frames for the 2nd second of video with a total of 519k of data for the 31 frames.

Third line means: 28 frames of video at 289k of data.

And so on...

The Variable Bit Rate (VBR)is quite obvious ranging from 519k to 277k in just the first 4 seconds of output.

The VFR is also obvious. 29 then 31 then 28 then 25. Look at the entire text file to see that the pattern just keeps going in that same vein. Now at every point along the way the output is reporting that it is 29.97 frames per second. If only... :)

This is again why custom mux'ing is needed. And why movies work and regular episodic TV rarely does. The movie clips I have report themselves as 24fps and the average, again cutting out the lead-in and lead-out segments, is 24. Almost perfectly so.

So there will be slight drift using another mux'er but not bad if all is well.

If anyone would like to check this out for themselves on their own clips grab the latest (1N) version of vsplit and run it with the -v (NOT -V) flag. For level 1 verbosity. Then grep for "MUXV:" in the output. It will generate for you the same file as I have attached here.

For movies check the lead-in. Probably fluctuating, then settles down the show itself.

For TV shows it will most likely fluctuate wildly from start to finish.

--jdiner

[jdiner]

To continue my last post for the SA Tivos...

Attached is an SA Tivo output from an episode of Lois and Clark.


MUXV: This much video in the Q: 21 nodes && (720015 bytes)
MUXV: This much video in the Q: 23 nodes && (788060 bytes)
MUXV: This much video in the Q: 23 nodes && (786240 bytes)
MUXV: This much video in the Q: 23 nodes && (787080 bytes)


Note the smaller first frame this is the cut of the 2 lead-in B-Frames from the previous GOP.

Then almost everything is 23's. The reason this is 23 and not 29-30 is that the code I am using just counts nodes not the frames in them. On the DTivo 1 frame per node. On the SA Tivo the B frame followed by the P frame is put into the same record on the Tivo.

But the output stays almost perfect all the way through. The imperfections are my display code rather than the stream.

Also noet the sizes. While not perfect this is CBR... 788, 786, 787 and so on. Nice and steady bit rate... :)

--jdiner

[jdiner]

Alright folks here it. Version 1N of the MUX'er.

This release has the following key things in it:

1- The mux'er has been further cleaned up and it is marginally faster now than before.

2- The system requires more memory. In dealing with holes things now take a bit more ram. Not substantial but the need is there.

3- The full stream is mux'ed out. Before the ending pieces were chopped off even with the length limits.

4- The temporary mux'ing buffers are now flushed to the file at the very end of the run.

5- These building buffers are now padded to be a full mpeg-2 sector in size. Amazing the kinds of fireworks this one caused. :)

6- The MUXA: and MUVX: lines were moved to show during the -v level of debugging output. These show how much data was collected for each second for both video and audio. Just a way to check the VFR and VBR...

7- Other sundry small fixes were added. Every time I go through the mux'ing I find something. :( Sadly this time it was if there was a full second of video and no audio (i.e. a hole) it caused vsplit to error out. This has been fixed.

8- The "cut" from 5-10 seconds has been removed.

9- It is still limited to 2000 chunks. But this phase is coming to and end. If there are not any major problems then it means that mux'ing is going as needed and we can deal with the last stages of the VOB and editing...

--jdiner

[jdiner]

Ok. I have the code built to create the editing I-Frame key file.

Any takers on building the GUI before I get started on it? There is other stuff for me to do so I am serious. If anyone wants to start the editing GUI that would be great with me.

--jdiner

[jdiner]

Don't you just hate it when you think what you are doing is the best way and they you get the thought bubbling up from the back brain that will make it all better? :)

I expect that I can make the mux'er 5-10% faster again. Was just kind of being dumb about buffer management.

--jdiner

[JasonJLee]

Jdiner,

I'll have a go at a gui..

I'm on a clients site rebuilding a mirrored server at the mo(taking forever, giving me time to checkup on this board), but I'll be around later.

I have trouble sleeping lately, so I have lots of time to code. (10am fri till 7am on saturday.. 21 Hours straight).

I'm only good for windows ATM though, that might be a problem.

PM me if your interested.

If not I do have one idea, which I have stated before, but I'm not sure if it helps, as I don't know in detail how your code works.

If we know that add breaks are at least 60 seconds long for example, we only need to take stills every 59 seconds on so...

Which means 1 still per minute rather than per iframe.

Then zero in on the iframes for that minute, and get the cut closer.

Then we have a cutlist.

Anyway, Servers at 90% so I'm packing up. Hope this helps!

J

[AlphaWolf]

IMO you should write a front end spec and just distribute a command line only binary, this way it would be easy to have GUIs written for either windows or linux, and us command line jockeys don't have to run X servers where they aren't necessary :D

[jdiner]

Originally posted by AlphaWolf
IMO you should write a front end spec and just distribute a command line only binary, this way it would be easy to have GUIs written for either windows or linux, and us command line jockeys don't have to run X servers where they aren't necessary :D
Exactly my plan. I wasn't clear enough but I was just getting started...

What I have right now:

1- A version of vsplit that can be run through a TyStream file. It outputs the actual time stamp and each I-Frame from within the file. Thus correct timestamps for a cutlist are easy to generate.

2- At some point this will be run on the Tivo but for now you have to have a downloaded TyStream file. But having gotten this far the rest of the work to put it on the Tivo is a piece of cake...

3- For right now I am assuming a process of "starting at the first timestamp then up to but not including the second timestamp" for the cutlist. This way even the first I-frame in the cut is gone but the second will be there. Looks to be about right with how things are arranged inside the files.

4- The program also prints the timestamps to stdout like the rest of the debugging. This way you can get a timestamp list as a text file for manual cutting.

5- Then you can create a simple text file cutlist as was mentioned before. 2 timestamps per line...

6- The -k is the command for this. It stands for "Key Frames file" rather than something like "Kut list"...

7- Then you re-run vsplit with the -c option for listing the cutlist file as an argument. It then cuts the unwanted data from the file.


As for a front-end attachment to the system I am not sure of file structure yet. I am trying to decide on the best way to do it. Fundamentally what is needed is the timestamp and then the frames data. But we need to package it for scanning back and forth. File positions would make that faster, flags that could be locked onto would make it faster, sizes of the data chunk that follows would make it faster. I just haven't decided yet. If anyone has any suggestions on that then please feel free to chime in.

But I do like the idea of others being able to do the front ends. I have enough work. And there are some here that are better at GUIs than I am and some that can actually do a UNIX gui. I haven't in years and if I had to again I would want to do it in Java as it would be the least painful. Not a problem if you run X and have java installed. But anyway I digress...

--jdiner

[jdiner]

Ha! Ok the edit list file stuff is cleaned up and finished. So it is now possible to provide a cut list with as many entries as desired and have it cut them all. When cutting on a specific GOP boundry all seems to work well with no visual issues at all. (Being on a GOP boundry SHOULD do that... :)

Also as I was playing with things I got 1 step further. I have a way to do visual editing in a very manual way. :)

Here is the process I am doing:

1- vsplit it normally with no mux'ing or other options and save the output files.

2- vsplit with -k to get the textual only edit list. The lines look like:


Type = 1 (I) Size = 21212 - PTS = (280532029) 00:51:57.022
Type = 1 (I) Size = 18544 - PTS = (280623621) 00:51:58.040
Type = 1 (I) Size = 18800 - PTS = (280713711) 00:51:59.041
Type = 1 (I) Size = 20832 - PTS = (280803801) 00:52:00.042
Type = 1 (I) Size = 37344 - PTS = (280895392) 00:52:01.059
Type = 1 (I) Size = 40768 - PTS = (281003500) 00:52:02.261


But as they come out in the standard output with other information (the ... 100... and so on) I used grep, for win32, to get just the lines above.

3- Load up the .m2v output from step1 with DVD2AVI.

4- Using the right and left arrow keys jumps by I frames. (See where this is going? :)

5- Count the steps for now and each right or left step directly relates to one of the above.

6- Build a cut list with the times from above, from this one to this one, and save it out.

7- Mux the fix with the -m and new -c options thus:

vsplit1o -c cut.txt -m file.ty output.m2v junk.m2a > junk.txt

and you get what appears to be a perfectly edited, on GOP boundries, MPEG-2 file.


Now this is not that easy to do! Sad but true. Counting the stupid arrow key clicks from the DVD2AVI main window is a royal pain! I got lost so many times trying this out and I wanted to use key repeat etc...

So the good news for this part too. I am almost done with an altered version of DVD2Avi that will show in the display window (at the side when previewing etc...) the current count of the I-Frame you are on. Then key-repeats allow for fast fast editing through the file.

This is still not direct or perfectly easy as you have to make a cut list manually and then go through the output list to get the timstamps. But a simple program to pull out timestamps from the output list is easy enough to write. And work could be done to do other things with the DVD2Avi app itself. Have it save the "counts" directly to a file to generate the edit list.

This really only helps the Windows people but it was a nice thing to note.

--jdiner

[Fugg]

vsplitmux-1n working fine here! no problems...

[Generiq]

I was just wondering if there is any info on how to test these releases out and what kind of clips you want them tested on?

[RxMan]

I can't believe the progress that has been made! For a while it was one step foward and two steps back. Great work JDiner!! This has got to be a gratifying point in the project for you.

[rpongett]

Originally posted by jdiner
I am almost done with an altered version of DVD2Avi that will show in the display window (at the side when previewing etc...) the current count of the I-Frame you are on. Then key-repeats allow for fast fast editing through the file.

This is still not direct or perfectly easy as you have to make a cut list manually and then go through the output list to get the timstamps. But a simple program to pull out timestamps from the output list is easy enough to write. And work could be done to do other things with the DVD2Avi app itself.
--jdiner

Wow. Where it is now (DVD2AVI to get I-frames for cut list) isn't even that big of a pain. There are usually only 3-4 commercial breaks in a half hour show counting the beginning and end.

Also, the commercial cutting feature is probably much bigger for those of burning animated shows, like the Simpsons, Southpark, The Critic, etc . . Most Southparks are only around 350MB for a full half hour. That means that the non-animated commercial breaks likely take up well over half of the space. So I can probably get twice or more the number of Southparks on a DVD with no ads as I could if I had ads.

[FredThompson]

Originally posted by rpongett
Most Southparks are only around 350MB for a full half hour. That means that the non-animated commercial breaks likely take up well over half of the space. So I can probably get twice or more the number of Southparks on a DVD with no ads as I could if I had ads.

Well, if you're willing to transcode you might be able to squeeze even more without any quality loss IF you use Donald Graft's dupe filter for AVISynth. It's specifically created for anime and if sequential frames are close enough to each other, the first replaces the second. Thus, when encoding, there is NO change between the frames so you get an even tighter encdoes. You can tweak it all you want for size of blocks compared and that kind of thing.

Hmmm...wonder if there's a way to do similar filtering such that "black" doesn't swim (like during credits on a lot of movies.)

[jdiner]

Originally posted by Generiq
I was just wondering if there is any info on how to test these releases out and what kind of clips you want them tested on?
Yeah. Read back in this thread and in the one with the same name but without the "#2" in the title. It is in this same forum.

Make some small clips using TyFileSplit that showcase things that can be a/v sync'ed and then start processing them.

Best bet would be to search for messages from me. There are a lot but you need to read them to catch up.

--jdiner

[jdiner]

Originally posted by RxMan
I can't believe the progress that has been made! For a while it was one step foward and two steps back. Great work JDiner!! This has got to be a gratifying point in the project for you.
Yeah. I was thinking the same thing earlier today. Things are finally beginning to fall into place. Still a ways to go but the end is in sight. Some clean ups, some better code for SA Tivo support, and a few other things and it is ready to roll.

--jdiner

[alunj]

mux1n ok on SA UK tivo
Sync Good.
Alun

[edpuffmonster]

Originally posted by FredThompson
Hmmm...wonder if there's a way to do similar filtering such that "black" doesn't swim (like during credits on a lot of movies.)
Sure, increase the bandwidth. :)

I was actually hoping the echostar/DTV merger would happen and they could combine resources and effectively double bandwidth for each channel.

Seriously though, sure you can force everything below a particular threshold to solid black. This can easily be done on playback and most DVD players have the feature built in. "enhanced blacklevel" is Toshiba's name for it.

Unfortunately, the same kind of artifacting happens with any solid color. Black, White, Blue, whatever...

Fixing the mpeg itself would require re-encoding. "Smart" encoders like Nandub SBC can be tweaked to automatically prevent that kind of artifacting... I don't know of any software mpeg-2 encoders with similar features.

I believe DTV has different encoder profiles for different channels. Obviously some are given more bandwidth than others... but if you look at cartoon network, for example, they've done a pretty good job of taking care of that type of artifacting. Look at anything animated / solid black / solid white on many of the other channels and it looks like shit. :)

Edit: Sorry, I've gotten completely off-topic here. If you have comments, please reply via PM and not in this thread.

[jdiner]

Alright most recent flame and responses deleted. That is the rule for this forum. Some did reply. No concern to me. I mean I wanted to as well. But we really need to avoid it...

I am not going to apologize for way I do things.

For those that want to wait it out, go away and do something else and the be surprised when it is all done. For those that want to be here as it happens I will continue to post comments on the progress being made.

I have spent the day going over the editing mechanism. DVD2AVI while a great program has issues all it's own and a ton of code written to do things in a certain way that DO NOT work well for what we were trying.

So I have spent the day building a new program based off of some free mpeg-2 decoding source (well several actually) so that we can do just what we want with it all. So far so good. I was coming here to post good news when I caught the flame. Amazing isn't how one ***** can dampen the mood for everyone. :(

--jdiner

[Pr.Sinister]

Originally posted by jdiner
I was coming here to post good news when I caught the flame. Amazing isn't how one ***** can dampen the mood for everyone. :(

--jdiner

Don't let that stop you man... i live for the little arrow icon indicating new posts in the thread. And when it says last post
was from jdiner, then i'm down right giddy! :)

I am salivating and can't wait to be able to edit properly!

In fact, i smell the end is closer and closer so i am finally taking
the plunge and buying a Sony DRU-500A on Thursday (payday!).

Keep up the good work... even if it is a bit excruciating sometimes ;)

-Pr.

[RxMan]

Don't let it bother you. Keep the updates coming. I like to follow along and test when I can (only I have yet to get a stream that has failed).

[jdiner]

Alright. The good news is that I have a working custom Win32 program for doing the editing from my own custom key frame files.

The bad news is that I had to abandon the changes to DVD2AVI. The author of that tool and myself just think differently. While I am convinced his is the better stuff than the free code I used as a basis for what I am doing now I was getting hung up majorly on his buffer manipulation.

Besides this is designed to be simple. And while the video output is kind of sad compared to the output from DVD2AVI you can visually see what is going on to do the editing.

I am just putting the finishing touches to the GUI for the creation of the edit points. Then a mass release of everything as it stands will be made. After some quick testing I expect things will move very quickly.

--jdiner

[sirfergy]

This sounds so exciting! I can't wait! Thank you so much for your hard work.

[rd001]

Originally posted by jdiner I am not going to apologize for way I do things. Nor should you.

Keep pressing to the finish line. Then just let the flamers argue with success.

Eagerly awaiting your final version and resisting the urge to patch...

[Rotten]

For a job well done. I will vote for you.

[bato]

Originally posted by Pr.Sinister
Don't let that stop you man... i live for the little arrow icon indicating new posts in the thread. And when it says last post
was from jdiner, then i'm down right giddy! :)

I am salivating and can't wait to be able to edit properly!

In fact, i smell the end is closer and closer so i am finally taking
the plunge and buying a Sony DRU-500A on Thursday (payday!).

Keep up the good work... even if it is a bit excruciating sometimes ;)

-Pr.


jdiner: I agree completly, I'm rebuilding my system and got a SA to test other things, and I beleive it will be ready next week.

pr.sinister: First thing I did when I got the 500A was to upgrade to 1.0D, no problems, make sure you send the $50 MIR for it, and buy some discs with it 2X if you can at 1X you will end dry with all that salivating :D

[abcdriver]

I'm salivating for cuts too (maybe it's just the turkey...)

Happy American Thanksgiving JD, thanks for all your hard work!

[jdiner]

Sorry for no release. My family celebrated early. Wed... So many married people running around now that we had to change the date to be able to get everyone together.

Wow. Sorry it is late. I left what I had really intended to say out. I have been working on the visual system for the editing and it is coming along nicely. I just simply ran out of time when the party started to do any more. Going to a cousins house for T-Day tommorow. So it is going to be a bit before I get anymore out. I will try to make the release as soon as possible. But be patient and we will have something good, and please one and all enjoy the holidays. For those not in the US. Well, have a good weekend... :)

--jdiner

[Immo]

Originally posted by jdiner
But be patient and we will have something good, and please one and all enjoy the holidays. For those not in the US. Well, have a good weekend... :)

--jdiner


Same to you jdiner, have a great Turkey Day. I have a bunch of TY streams ready to slice and dice for my kids (well me too):

Tom & Jerry - classics
Spongebob Squarepants
Looney Toons - classics
Jimmy Neutron

The latest release looked good, no issues.

Immo

[Hi8]

Slightly a side topic -- from the trend of vsplit->mux->edit

but I have become so spoiled by how fast I can extract, and burn to DVD-RW for archiving; I wanted to offer up an idea for modifications to tyserver_mfs, to make it even easier!

Because I always leave a session open running tyserver_mfs running on my DTiVo - I can quickly launch TyTool5 and pick and choose my shows to extract. But I thought what a GREAT thing it would be if tyserver_mfs was alittle more robust, in the sense that it could keep track of what new shows were available for extraction. This way it could create a queue - and with the client side on the PC - handshake with an option to extract EVERY show recorded on a predetermined schedule or cycle.

This way the PC could proccess un-attended - If it was able (tyserver_mfs) to be able to be configured to say transfer only those shows that were scheduled to expire, or be deleted by a date (not save until I delete) you could setup your recordings that you wanted to watch on TiVo with Save Until Delete - the rest would end up on your PC is short order and get automatically deleted off Now Showing, once extracted.

Automating the vsplit, and muxing is quite simple on the PC side using commandline tools, I'm going to drop a hint to Derrow, (IFOEdit - author) to provide commandline options for DVDAuthoring - This way In theory; you could have a folder structure on your PC that was automatically being filled with DVD TitleSet folders ready to burn/archive to DVD.

I am now burning to DVD 3-5 shows a day, and the compression is so good off my DTivo I can fit 2-3 movies per 4.7g disc.

It would require a new, or modified version of TyTool5 - to handshake with the agent or tserver_mfs - to allow for this automatic extraction process.

Just thinking and dreaming out loud, don't want to disturb the path of development; but wanted to throw out my ideas for future consideration and feedback.

thanks again jdiner, and hope you and your family have a safe and happy hoilday season!

[JasonJLee]

I posted this in this thread on the 28th October.

Wish no.1
Hi Jdiner.

Nice to have you back!

My number 1 wish is probably possible via other routes, but I'd rather use tytool if you can add it.

I'd like to have extraction of certain shows to be automatic, and happen during the day. Say the software polls the Tivo every hour, looking for any show from a list of shows. So I come home, and the shows have already been transfered to my machine, and maybe deleted from the Tivo? (I have a machine running all the time recording security cameras, so it makes no diff to electricy bills!)

I played around with various bits and pieces before my HD died last week and took all the .net stuff I'd been doing with it.

I managed to send the commands from the pc front end to the tivo running ty_server and get the program list back, but I felt a little funny playing around without mentioning it, and I'm pretty sure that the next bit would be much more difficult and I don't want to reinvent the wheel.

I don't want to rewrite anything clever I just want to add scheduling to the existing program, but I couldn't see any way of doing that reliably. I certainly don't want to get into rewriting anything thats already been written.

I know it's minor, but it would save me 5 Minutes a day of hassle.. And that does add up.

J



I'm doing the same as you.. Jdiner said he would put it in the next tytool release in that same thread the next day.

However, no tytool release since then, as he's tied up in the muxing. Though he has mentioned improvements to the system.

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=16370&perpage=15&pagenumber=2

That thread is now locked? Not sure why.

Anyway, you can put your mind at rest. Not to put words in Jdiner's mouth, but I think that features on its way.

J

[bronco13]

and the compression is so good off my DTivo I can fit 2-3 movies per 4.7g disc.

Hi-8, you are able to put multiple titles onto one DVD? Are you using ifoedit? How are you doing this? I've searched through the forum (for ifoedit) a couple of times and the answer always seems to be 'not yet', or 'when it's done'.

Are you authoring with something else besides ifoedit?

[abcdriver]

This is getting off topic, go read the Mac OS X workflow, you can already put multiple DTivo titles on a single dvd without muxing using only TyTool5 to extract and split, then composing the dvd with DVD Studio Pro.

[fredisdead]

Jdiner,

Another vsplit13c wierdness I am seeing is that every file I have split shows an OOB at exactly block number 4090. Coincidence or bug? No problems using the resulting elementals though. Also, any progress on the Temporal Reference problem in vpslit13c (but not in vsplit13) I previously reported.

Do you want me to send you some of the ty files? If so where should I ftp them?

Finally, any chance of you releasing an updated unrestricted vsplit13d fixing some of the 13c bugs (unrelated to muxing) for those of us burning shows with maestro or other authoring tools?

[pbar]

Originally posted by fredisdead
Jdiner,

Another vsplit13c wierdness I am seeing is that every file I have split shows an OOB at exactly block number 4090. Coincidence or bug?

Pretty much every UK tivo TySream I've run through vsplit has found an OOB packet at approx every 4096 chunks. Each one is 'ignored' leading to a cumulative A/V sync error. This never seems to have been addressed and prevents building a decent mpg from any extracted TyStream (on my tivo).

I've been hoping that the multiplexed output would work around this problem, but it seems that most of the DVD authoring programs I have insist on demuxing files before transcoding and thus throw away all of the timestamps jdiner has gone to great lengths to preserve. :(

Still not managed to get a single program burned to DVD with correct lipsync ... even following JDiner's magic 14-step failsafe procedure. Pretty depressing really. :(

[drapatin]

ABC:

Yes, a great solution if you happen to be one of the minority 4.5% of computer users who use a Mac.

[Hi8]

Originally posted by bronco13
Hi-8, you are able to put multiple titles onto one DVD? Are you using ifoedit? How are you doing this? I've searched through the forum (for ifoedit) a couple of times and the answer always seems to be 'not yet', or 'when it's done'.

Are you authoring with something else besides ifoedit?

I'm using version .096B1 it is a release that was distributed I think to current contributors - I also donated to other projects and have been included on a mailist that shares pre-releases, or betas.

I'm not sure if the latest "stable" release has that feature or not, (multiple Title Set) I don't recall. I know it was just added a short time ago.

But itis along with TyTool5 & Vsplit the Holy Grail for my TiVo! I no longer use "save to VCR" DVD-RW is MUCH faster, and the quality is apples an oranges!

I always avoided the VCR feature where possible, but when space was an issue I had no choice But now it's so easy and fast it has become routine, and space is no longer a problem for me.:D

[Kythorn]

Originally posted by pbar
Pretty much every UK tivo TySream I've run through vsplit has found an OOB packet at approx every 4096 chunks. Each one is 'ignored' leading to a cumulative A/V sync error.


I believe that # of chunks exactly coincides with the FSID boundaries.

[Hi8]

Originally posted by bronco13
Hi-8, you are able to put multiple titles onto one DVD? Are you using ifoedit? How are you doing this? I've searched through the forum (for ifoedit) a couple of times and the answer always seems to be 'not yet', or 'when it's done'.

Are you authoring with something else besides ifoedit?

IFOEdit .096B1 - does it all. right from the vsplit files .m2v & .m2a

[FreydNot]

Looks like the newest version of IFOedit wild on the net is 0.95. I sure hope the author doesn't get too distracted with his new project (dvdxcopy) and forgets to release the newer IFOedit :(

[jdiner]

Originally posted by fredisdead
Jdiner,

Another vsplit13c wierdness I am seeing is that every file I have split shows an OOB at exactly block number 4090. Coincidence or bug? No problems using the resulting elementals though. Also, any progress on the Temporal Reference problem in vpslit13c (but not in vsplit13) I previously reported.

Do you want me to send you some of the ty files? If so where should I ftp them?

Finally, any chance of you releasing an updated unrestricted vsplit13d fixing some of the 13c bugs (unrelated to muxing) for those of us burning shows with maestro or other authoring tools?

Go back and re-read some of the early messages about vsplit. Part of why it works where everything else has issues is becuase it finds and ignores the bad Out-Of-Band data. (And yes I borrowed that term from networking...)

As someone else pointed out it is indeed a reference to the boundry between FSIDs. There is often bad data extracted from the Tivo. It is a good thing that VSplit is catching it. As was mentioned over and over before it is only "bad" if the error messages has the word "hole" in it. That indicates that something that should have been there was not. Not the VSPLIT cut it but it never came down from the tivo.

So no, while I appriciate the offer, I don't need copies of it. What you have is simply proof that vsplit is working.

--jdiner

[jdiner]

Originally posted by pbar
Pretty much every UK tivo TySream I've run through vsplit has found an OOB packet at approx every 4096 chunks. Each one is 'ignored' leading to a cumulative A/V sync error. This never seems to have been addressed and prevents building a decent mpg from any extracted TyStream (on my tivo).

I've been hoping that the multiplexed output would work around this problem, but it seems that most of the DVD authoring programs I have insist on demuxing files before transcoding and thus throw away all of the timestamps jdiner has gone to great lengths to preserve. :(

Still not managed to get a single program burned to DVD with correct lipsync ... even following JDiner's magic 14-step failsafe procedure. Pretty depressing really. :(

No. Unless the OOB chunk report has the word "hole" in it then all is well. At least well enough that that is not the cause.

The OOB chunks you see are part of the extraction and are EXTRA data that is not part of the real stream being extracted.

I can't tell you why your mux'ing with other tools fails. I would have to have written them or at least looked at the internals. And while something must be going on, VSplit doesn't arbitrarily start throwing things out in the hopes of messing things up for you.

Any number of things can cause drift. The tivo can get busy and lose chunks of data during the saving process. This happens alot on my DTivo if I am using tivoweb during a recording. When this happens you don't lose a chunk, you just lose 1 or 2 audio records.

Reguardless. Mux'ing the split files is problematic for all of the reasons I have specified in so many messages. A single lost audio record accounts for at least 24ms of sync loss and I can see that. I bet you can 2. Lose just 2 across a full stream and you are out of luck completely.

Now use my mux'ing to get the output and things stay perfect. At least as perfect as they were on the tivo.

Eventually it will be producing something that can be used as input directly to various DVD tools. Perhaps not yours but that means you have to go out and get a different tool. One that will accept the pre-built VOB files.

--jdiner

[pbar]

Originally posted by jdiner
Any number of things can cause drift. The tivo can get busy and lose chunks of data during the saving process. This happens alot on my DTivo if I am using tivoweb during a recording. When this happens you don't lose a chunk, you just lose 1 or 2 audio records.
...
A single lost audio record accounts for at least 24ms of sync loss and I can see that. I bet you can 2. Lose just 2 across a full stream and you are out of luck completely.

Agreed ... but seems like I'm missing video frames rather than audio given the direction of the drift. It also seems to happen in almost every recording, and this makes it impossible to use any of the standard tools which split the tystream into m2a and m2v.

Now use my mux'ing to get the output and things stay perfect. At least as perfect as they were on the tivo.

Yep - your muxed files play back fine for me on the PC. (thanks for all the hard work!) The problem comes for me with DVD authoring software that doesn't pay any attention to the presentation timestamps and just blindly assumes that frames come at 25fps and audio at 32kHz. Most of the 'frame-serving' Divx tools I've tried seem to have this assumption as well.

Maybe one way to work around this would be to throw away / insert audio samples during extraction to keep the audio in step? (I'm naively assuming here that it's possible when extracting/muxing to tell when a frame or audio record has been dropped? - it's a while since I last read the source code of tyc.c!) ... of course you may already be doing this for all I know!


Eventually it will be producing something that can be used as input directly to various DVD tools. Perhaps not yours but that means you have to go out and get a different tool. One that will accept the pre-built VOB files.

Straight to VOB would certainly be the holy grail... but that would at least require transcoding 32KHz audio on a UK TiVo - even assuming the video was recorded at a DVD-compatible resolution. At the moment there's no way for me to upsample the audio in a muxed file without screwing up the sync again.

[hancocks]

Folks,

Hi8 turned me onto IFOEdit's new beta (0.96B1), and so far (very early) it looks totally cool for taking video/audio from jdiner's tools and mux'ing them into VOBs, with multiple titles. Jdiner, have you played with this, and if so, what do you think? I've run about five different tests so far and haven't had sync problems. And the VOBs generated are DVD-R/RW-ready, complete with IFOs.

- Stu

[jdiner]

Originally posted by pbar

Straight to VOB would certainly be the holy grail... but that would at least require transcoding 32KHz audio on a UK TiVo - even assuming the video was recorded at a DVD-compatible resolution. At the moment there's no way for me to upsample the audio in a muxed file without screwing up the sync again.
Another forum member has been putting in time on doing just that and while I have not had time to go over it in any detail it appears that he has been complete successful. The code can do any change between 32, 44.1, and 48. I.e. from any one of them to any other.

So that is taken care of as well. It is just something that will get slipped in as I finish what I am working on most right now. Finishing the mux'ing/editing.

--jdiner

[jdiner]

Originally posted by hancocks
Hi8 turned me onto IFOEdit's new beta (0.96B1), and so far (very early) it looks totally cool for taking video/audio from jdiner's tools and mux'ing them into VOBs, with multiple titles. Jdiner, have you played with this, and if so, what do you think? I've run about five different tests so far and haven't had sync problems. And the VOBs generated are DVD-R/RW-ready, complete with IFOs.

I don't have the 96 beta. From what I ready on the web site and here and what not it was a limited released to early testers etc...

As of .95 the mux'ing was failing on my known VFR test clips. That is not to say that it is bad. Just that it was as unaware of the nature of things as the rest of the tools. I would be more than willing to take a look at it if I had it. But it would appear I will have to wait it out.

I have an idea that if other followed it might lead to better mux'ing from everything out there. It is possible that this is what derrow did and if so then his mux'ing should come out perfect. I would like to see it at some point to check out my theories.

--jdiner

[Hi8]

Originally posted by jdiner
I don't have the 96 beta. From what I ready on the web site and here and what not it was a limited released to early testers etc...

As of .95 the mux'ing was failing on my known VFR test clips. That is not to say that it is bad. Just that it was as unaware of the nature of things as the rest of the tools. I would be more than willing to take a look at it if I had it. But it would appear I will have to wait it out.

I have an idea that if other followed it might lead to better mux'ing from everything out there. It is possible that this is what derrow did and if so then his mux'ing should come out perfect. I would like to see it at some point to check out my theories.

--jdiner

jdiner;

check your email @best.com --

I have had only one failure with .096B1 --"Sweet Revenge" I'm going to record again via TiVo --- and see if it was just a fluke. The lipsync was off over 2-3 secs. Other than that it was a dead-ringer EVERYTIME!

I've taken FULL advantage of the FREE HBO Thanksgiving Weekend --- and archived over 15 recordings to DVD-RW to watch later... I also use the app. DVDBurn from Derrow, or Nero (latest version) haven't created a coaster yet!

I'm still wondering how DTiVo compresses things so well? I can fit more than 2hours on DVD-RW using this method. This is CHEAPER than VHS! and let's not mention the quality.... hands down!

[Fugg]

any idea when .096B1 will be in general release?

[digitalAir]

Originally posted by Hi8
I'm still wondering how DTiVo compresses things so well? I can fit more than 2hours on DVD-RW using this method. This is CHEAPER than VHS! and let's not mention the quality.... hands down!

that's easy...

1. SVCD resolution (480x480) instead of DVD (720x480) - 2/3 the resolution of DVD allows the ability to spend more bandwidth on motion. besides a standard tv can't really display more that about 500 pixels across anyhow...

2. average of 2.4 Mbps (maxing out around 15Mbps) - not much to explain there... :)

3. and the coup-de-gras... VFR... Steel bandwidth from low-action scenes by holding a frame 50% longer and all of a sudden you've got quite a bit of extra bandwidth to play with... :)

[bronco13]

any idea when .096B1 will be in general release?

Yeah, how do the rest of us get to check out .096B1?

[hancocks]

bronco13,

It's pretty easy. You log onto Derrow's website, click on the beta button, pay PayPay $5, and: 1) you are now a member (after you get your emailed password) and have full privileges on the site, including his early releases, and 2) have contributed (in a small way) to his finances as incentive to keep doing what he's doing. Which he seems to be doing very well. $5? Geez. I bet we all blow five bucks on stuff we never think about. Every day.
And for that, you get to try his stuff early!

- Stu

[Hi8]

Originally posted by hancocks
bronco13,

It's pretty easy. You log onto Derrow's website, click on the beta button, pay PayPay $5, and: 1) you are now a member (after you get your emailed password) and have full privileges on the site, including his early releases, and 2) have contributed (in a small way) to his finances as incentive to keep doing what he's doing. Which he seems to be doing very well. $5? Geez. I bet we all blow five bucks on stuff we never think about. Every day.
And for that, you get to try his stuff early!

- Stu

yeah... and if you are feeling lucky make it $10-15 it's christmas season! he can prabably use the extra.. besides it's a donation. Don't think $5 is the amount you NEED to send.

[bronco13]

Thanks guys...

I go to www.ifoedit.com and then click on the button that says "Beta Area" and I get a "404 - File not found"

Also, it looks like I've been redirected to ifoedit.wh.fr0zen.com

When I go to ifoedit.wh.fr0zen.com/prel.html I get the same thing...

I have several T's here at work so I have access to quite a few IP's and I've tried from several