PDA

View Full Version : MUX'ing, VSplit, and MPG2 files.



Pages : 1 2 3 [4] 5

Pr.Sinister
10-30-2002, 09:53 PM
Originally posted by Krokus
Is vsplitmux compatible with windowsXP. It just opens and shuts down instantly. TyFileSplit does the same thing. I have been using tytool5r2 with no problems. Does anybody else have this problem? Do I need a diferent OS for these?

Thanks for this fun stuff JDiner

Krokus

Are you just double-clicking on it? If so, it won't work. You have to run it from the command prompt.

Start | All Programs | Accessories | Command Prompt

Then change to a directory with your TY files and run it from there.

The vsplit exe should be in the same directory or a directory in your PATH.

tweaky
10-30-2002, 09:57 PM
All test clips run sucessfully and stay in sync with PowerDVD XP (Windows media player plays them fine, but they are inverted, haven't quite figured that one out yet). The only issue is that they only last 5 minutes or so <Grin>.

Anxiously awaiting the full blown version.

rpongett
10-30-2002, 10:31 PM
jdiner:

Is there an authoring program that accepts raw VOB files (rather than elemental streams) for its imported "movies"? I am not sure if IFO edit can create multiple new IFO files that mesh together multiple raw VOBs into playable fashioin and, even if it could, I don't know how it would create a selection menu.

If not, it seems that it may be problematic to put more than one of these newly created VOB files on a DVD such that they can each be accessible through a normal DVD player.

mikertx
10-30-2002, 10:46 PM
Originally posted by rpongett
jdiner:

Is there an authoring program that accepts raw VOB files (rather than elemental streams) for its imported "movies"? I am not sure if IFO edit can create multiple new IFO files that mesh together multiple raw VOBs into playable fashioin and, even if it could, I don't know how it would create a selection menu.

If not, it seems that it may be problematic to put more than one of these newly created VOB files on a DVD such that they can each be accessible through a normal DVD player.

In linux-land there is ifogen. It will take a vob or properly formatted mpeg2 file and create an ifo file. It's not too fancy but it works. I don't have a DVD recordable drive yet but I have made a dvd iso file and mounted it on a loopback device and pointed a DVD player program at it and it worked.

--mikertx

Pr.Sinister
10-30-2002, 11:03 PM
Originally posted by tweaky
All test clips run sucessfully and stay in sync with PowerDVD XP (Windows media player plays them fine, but they are inverted, haven't quite figured that one out yet). The only issue is that they only last 5 minutes or so <Grin>.

Anxiously awaiting the full blown version.

I just ran it through a bunch of Alias and Sopranos clips and
they play for the whole hour no problem. How you ask? I just
made a bunch of files of 1000 chunks. 1st one starts at chunk 0,
next at chunk 1000, then chunk 2000 etc... That way i can test
multiple parts of a whole hour show easily. Here is the script i
made to help out in doing this.

@Echo off
for %%l in (%1) do cd %%~dpl
for /L %%m in (0,1000,10000) do (
Echo Muxing file to SVCD starting at chunk %%m ...
for %%l in (%1) do vsplit -m -d -j%%m "%%~nxl" "%%~nl%%m.mpg" "%%~nl.m2a" >"%%~nl%%m.txt"
for %%l in (%1) do del "%%~nl.m2a"
)

rpongett
10-30-2002, 11:40 PM
Originally posted by mikertx
In linux-land there is ifogen. It will take a vob or properly formatted mpeg2 file and create an ifo file. It's not too fancy but it works. I don't have a DVD recordable drive yet but I have made a dvd iso file and mounted it on a loopback device and pointed a DVD player program at it and it worked.

--mikertx But will it create a menu to select diffferent VOB's for which it created IFOs when they are all burned on a single DVD?

For example, say I use jdiner's tool (when ready) to extract and generate separate VOB files for The Simpsons, Curb Your Enthusiasm and Strangers with Candy episodes (30 mins each). IFOgen then creates separate IFO files for each. How do I put them all on a DVD such that a conventional DVD player could select each individual one to play?

jdiner
10-31-2002, 01:06 AM
Originally posted by Krokus
Is vsplitmux compatible with windowsXP. It just opens and shuts down instantly. TyFileSplit does the same thing. I have been using tytool5r2 with no problems. Does anybody else have this problem? Do I need a diferent OS for these?

Fine I will repost it. The archive that was in the sticky thread, but is no longer there apparently, had both versions in it.

I will do it in a minute I am kind of in the middle of something right now.

--jdiner

jdiner
10-31-2002, 01:10 AM
As for how to make the IFO files. I DO NOT KNOW. Sorry. I am not there yet. No DVD burner ready to roll, no experience of any kind with DVD software.

My goal to this point has been to made a valid VOB file. Once I do that! Then other things are possible.

I have explained it before, but the bottom line is that thanks to the VFR nature of DTV streams there is no way a standard tool will work. Said standard tool will open the Elementary Stream for the video and read that it is 29.97 frames per second. And read 29.97 frames from the file and multiplex them. In doing it will get anywhere from too many up to and including WAY too many.

So... I have to make the VOB as I go so that it will stay sync'ed. Where we go from there? Again I do not know. Either a tool exists that makes a good IFO file for this, or I/we/someone will make one. That is the long and the short of it. I mean look at what has been figured out to get to this point? From nothing working in any way to clean good ES streams to mux'ing that works and stays sync'ed. We are honestly on a roll here.

--jdiner

jdiner
10-31-2002, 01:12 AM
Here it is... Both the Win32 and the Linux versions.

You know I was just thinking about it and the archives for vsplit #9-#12 in the other major thread I do believe had both of them in it. Anyway, here it is.

--jdiner

jdiner
10-31-2002, 01:16 AM
Originally posted by Pr.Sinister
I just ran it through a bunch of Alias and Sopranos clips and
they play for the whole hour no problem. How you ask? I just
made a bunch of files of 1000 chunks. 1st one starts at chunk 0,
next at chunk 1000, then chunk 2000 etc...
I was wondering how long it would take for this to happen. Longer than I expected actually. So at this point I ask 2 things:

1- Don't ask me any questions or make any bug reports about a stitched together mpeg using this script or 1 like it. I have nothing to do with that side of things. (Things will not line up perfectly at the cuts due to how it works...)

2- Please continue to test the versions as they come out with the appropriate clips.

As for the rest of it, do whatever puts wind in your sails. :)

--jdiner

Pr.Sinister
10-31-2002, 01:57 AM
Originally posted by jdiner
I was wondering how long it would take for this to happen. Longer than I expected actually. So at this point I ask 2 things:

1- Don't ask me any questions or make any bug reports about a stitched together mpeg using this script or 1 like it. I have nothing to do with that side of things. (Things will not line up perfectly at the cuts due to how it works...)

2- Please continue to test the versions as they come out with the appropriate clips.

As for the rest of it, do whatever puts wind in your sails. :)

--jdiner

Hehehe... Well i've been doing it for a while now... Ever since there was audio in the muxing code... It's just that nobody ever asked...

1- I wasn't planning on asking any questions on that... Obviously if you didn't code something where i could get a stitched file, i have no business asking questions about a non-existing feature :). Actually i use the files as-is. No stitching.

2- I always do test the versions you put out. I'm not gonna stop now. I am eager to help.

The only reason i made this script was to put shows that already have no commercials (HBO, Showtime etc...) on SVCD. That way i create an SVCD with 5-7 minute chapters and use no menu and put no pause in between them. I was running out of space on my computer... If you had told me back in 1988 that i would be complaining about running out of space on a computer that has over 200 gigs, i would have told you that you were out of your mind! :D

FreydNot
10-31-2002, 02:30 AM
Originally posted by jdiner
Then onto editing. I have been toying with ways of doing this and I can do it pretty easily in terms of the actual editing. The interface on the program is another matter. So what I am thinking right now is a 2 step process. 1 to identify the cuts and the second to actually do it.

What you are describing is very similar to how video editing studios do things. First you digitize in all of your master tapes at a low resolution so it's easy to shuttle through it. Using those clips, you make all of your editing decisions. Once you are done, your editing software writes a file called an Edit Decision List (often refered to as an EDL). You feed the EDL to the tape machine/switcher and it does everything using the master tapes at full resolution.

Lots of pro and semi-pro editing software will write out an EDL in a standard format. Adobe Premiere and M2-Edit are two that come to mind immediately.

Seems like it would be a good idea to use a standard EDL format to exchange the edit points between step 1 and step 2.

I'll post some more info on EDL's in a few minutes.

FreydNot
10-31-2002, 02:59 AM
Okay, EDL's can be more complex then we need (think multi sources and different types of transitions). There is a good document with all of the gory details here http://www.editware.com/Editware-DOCs/EDLformat.PDF


Here is an example EDL created by M2-edit 5.0:


# TYPE=EDL
# VERSION=3.0.0
# EDIT MODE=FRAME ACCURATE
# TCC FRAME RATE = 29.97
# filename tags
# 3c0 = D:\TivoVid\SyncClips\vsplitmux-1i\3c.mpg

3c0 VA C 00:38:02:22 00:39:22:21 00:00:00:00 00:01:19:29
3c0 VA C 00:40:15:23 00:41:03:23 00:01:19:29 00:02:07:29
3c0 VA C 00:41:24:19 00:41:58:21 00:02:07:29 00:02:42:01


This is an example of a Grass Valley EDL exported from Premiere 6.0


UNTITLED3
GVG SUPER EDIT V4.X - V7.X FROM ADOBE PREMIERE V6.0
SMPTE FRAME CODE

0001 UND001 A12V C 00:00:00:00 00:00:04:07 01:00:00:00 01:00:04:07
* REEL UND001 IS CLIP JET DESCRIPTION WITH AUDIO (SQU
0002 UND001 A12V C 00:00:00:00 00:00:08:23 01:00:04:07 01:00:13:00
* REEL UND001 IS CLIP JET DESCRIPTION WITH AUDIO (SQU
0003 UND001 A12V C 00:00:00:00 00:00:12:27 01:00:13:00 01:00:25:27
* REEL UND001 IS CLIP JET DESCRIPTION WITH AUDIO (SQU
0004 UND001 A12V C 00:00:00:00 00:00:15:22 01:00:25:27 01:00:41:19
* REEL UND001 IS CLIP JET DESCRIPTION WITH AUDIO (SQU


Looks like there is some differences in the "standard". In any event, if you structured your cutter program to take data in using this kind of number, we should be able to code up a quick format converter and use off the shelf programs :)

Wooly
10-31-2002, 07:47 AM
hehehe...My server here at home is JUST for storing .ty files waiting for the finale of vsplitmux, so that they can be edited and such. It has 1.4 TB (that's right, not GB, but TB), and it has 24 gigs free right now. I'm going out today to buy another 160 gigger to hold additional stuff.

I feel your pain.


Originally posted by Pr.Sinister
If you had told me back in 1988 that i would be complaining about running out of space on a computer that has over 200 gigs, i would have told you that you were out of your mind! :D

keither
10-31-2002, 10:17 AM
I really do think than an "offline" editing solution (like the pro-grade ones mentioned above) is best. I've tried several editors and found they get really bogged down on larger files.

Perhaps this is due to the fact that the mux was from TMPG and not perfect.

For me, I don't mind a "preview scan" step where key frames are extracted for quick seeking. Plus, that would allow the creation of a frame index into the file for very quick random access seeking. Most sound editors (Sound Forge especially) will go through the entire file when you first load it and generate peak data. This ensures quick seeks in the file.

There's some overhead when you first load up, but the quickness you acheive during seeking and editing makes up for it.

Also, how are "chapters" handled in SVCDs and DVDs? I'd love to go through, search for scene changes and add "chapter points" then have the .ty file automatically turned into 1 (or many) properly constructed file(s) that handle it correctly so I can scene skip.

rpongett
10-31-2002, 12:50 PM
If IFO and creating menus (or some other way) to select multiple VOBSs on a DVD is problematic, it may be a good idea to add in a feature in the new applicatoin that just outputs post-edited and demuxed elemental streams that can be later imported into Maestro, Spruce or Scenarist for burning the old-fashioned way.

In fact, if its easily done without any further work, it may be a good idea to throw in that feature anyway, for people that have to make menus for whatever reason (e.g., my sister guest starred in some sitcoms and commercials and she wants me to make a DVD with a few of them on it with pointers to chapters of her scenes).

edpuffmonster
10-31-2002, 02:04 PM
I, for one, couldn't care less about menus. I'd rather have the disc just play when you put it in. One "title" per episode and chapter stops at commercial breaks would be perfect.

Pr.Sinister
10-31-2002, 02:32 PM
Originally posted by rpongett
If IFO and creating menus (or some other way) to select multiple VOBSs on a DVD is problematic, it may be a good idea to add in a feature in the new applicatoin that just outputs post-edited and demuxed elemental streams that can be later imported into Maestro, Spruce or Scenarist for burning the old-fashioned way.

In fact, if its easily done without any further work, it may be a good idea to throw in that feature anyway, for people that have to make menus for whatever reason (e.g., my sister guest starred in some sitcoms and commercials and she wants me to make a DVD with a few of them on it with pointers to chapters of her scenes).

<endless babbling>
One thing i don't get here a lot of people are asking for VOB creation.
Maybe i'm wrong but i was under the impression that
using SpruceUp or DVD Maestro, you could create a perfectly
fine DVD that would play in newer DVD players. I don't think
there ever were issues with the sync when doing this and
adding chapter points.

Now seems to me that the REAL problem is that nobody has
been able to cut commercials out in an easy fashion. Yes
TMPGEnc works most of the times but it is horribly inaccurate
and has too many issues with seeking in the file while editing.

IMHO, when jdiner comes out with his own cutter, then everything
else becomes a piece of cake. You edit out the parts you want
and use each part as a chapter. This works for SVCD or DVD.
The resulting MPEG-2 files can be burnt with Nero for SVCD and
with Maestro for DVD.

Doesn't that make sense? I mean VOB as an output is fine and
dandy but i think most people would prefer it if the VOB's didn't
have commercials in them. Saves space and gives you a warm
fuzzy feeling ;)

Don't get me wrong here, i'm not demanding anything or ragging
on anyone. I just wanted to point out that making DVD's is very
feasible right now and making SVCD's is also feasible. The thing
we are all having problems with is making a DVD or SVCD that
is edited to our likeing.
</endless babbling>

Thanks for letting me pass the time while i install this 2.2gig
monster that is called Visual Studio .NET :D

jdiner
10-31-2002, 06:18 PM
Ok. What I have in mind is the following:

A better tserver program. One that includes an option to extract only the I-Frames from the FSIDs on the disk. Couple this with the checking code in VSplit so that it sends only the "correct" I-Frames. The nice thing is this is what is needed for GOP level editing and it uses way less bandwidth. 1 frame in 30 or so on average is an I-Frame. (Sometimes more on the DTivo.) So it should transfer much more quickly.

The server then in a new format sends these frames of data with appropriate timestamps to the client.

An editing window then moves from I-Frame to I-Frame, as that is all there is, letting you mark things for cutting.

You then grab the entire stream, mux'ing as you go, and the system automatically edits things that should not be there at the users decision.

It stands to reason that these 2 need not be put together. I.e. go through and "edit" a number of streams and then start the download of all of the desired streams. That way you could leave and come back in the morning with a number of edited output streams...

Also I can see it being possible to remove things before doing the network transfer, i.e. on the tivo, so that it moves across the network faster, but that would be way down the road.

--jdiner

jdiner
10-31-2002, 06:21 PM
Originally posted by edpuffmonster
I, for one, couldn't care less about menus. I'd rather have the disc just play when you put it in. One "title" per episode and chapter stops at commercial breaks would be perfect.

For the most part this is where my interest lays. Menus and such are nice, but I want it to playback correctly everytime. However I would like to put more than 1 episode per disk.

And in general, not sure if I was ever clear. But it if comes down to it I will redo the IFO work so that it supports this. I am just hoping that someone that already understands it has something like this already. If not welll then... we will see...

--jdiner

Pr.Sinister
10-31-2002, 06:30 PM
Originally posted by jdiner
Ok. What I have in mind is the following:

A better tserver program. One that includes an option to extract only the I-Frames from the FSIDs on the disk. Couple this with the checking code in VSplit so that it sends only the "correct" I-Frames. The nice thing is this is what is needed for GOP level editing and it uses way less bandwidth. 1 frame in 30 or so on average is an I-Frame. (Sometimes more on the DTivo.) So it should transfer much more quickly.

The server then in a new format sends these frames of data with appropriate timestamps to the client.

An editing window then moves from I-Frame to I-Frame, as that is all there is, letting you mark things for cutting.

You then grab the entire stream, mux'ing as you go, and the system automatically edits things that should not be there at the users decision.

It stands to reason that these 2 need not be put together. I.e. go through and "edit" a number of streams and then start the download of all of the desired streams. That way you could leave and come back in the morning with a number of edited output streams...

Also I can see it being possible to remove things before doing the network transfer, i.e. on the tivo, so that it moves across the network faster, but that would be way down the road.

--jdiner

WOW! This is awesome... you just answered all my questions! I was wondering how we were gonna know that when we cut, since it won't be frame accurate, we are not cutting anything from the actual show but only from the commercials. With your I-Frame thing, we know for sure...

Damn you! I am gonna have a lot of sleepless nights now just
dreaming about this app! :p

rpongett
10-31-2002, 07:36 PM
Originally posted by jdiner
For the most part this is where my interest lays. Menus and such are nice, but I want it to playback correctly everytime. However I would like to put more than 1 episode per disk.

And in general, not sure if I was ever clear. But it if comes down to it I will redo the IFO work so that it supports this. I am just hoping that someone that already understands it has something like this already. If not welll then... we will see...

--jdiner By menus, all I meant was some way (anything functional) to put multiple episodes on disk and choose which one to play when you slap a disk in a hardware DVD player. The optional feature of outputting de-muxed post-edited elemental streams that could be imported into a DVD authoriing program (for people who needed menus) would be nice, but obviously not something to waste time on if it involves significant effort (only something to think about if it was an easy few line change feature for the full VOB creation application).

That I-Frame extraction flip through editor you outlined sounds fantastic for getting a snipped VOB file.

Thanks again for all of your effort.

fredisdead
10-31-2002, 08:46 PM
DVD Menu's aren't really necessary to support episodes menus on a single DVD.

If chapter breaks are put at the beginning of the episode, you could treat a multiple episode, single Title Set DVD as sort of a super VCR, no menu, it would autoplay at the beginning (first ty file in the sequence) and you can chapter skip to the next.

It seems that IFOEDIT when you combine multiple VOBS/VTS's (actually using a companion utility called VOBedit) does this for you... This utility takes the vobs from multiple title sets and combines them into a new single set of VOBs. You then run this through ifoedit to update the IFO files to correct their content and pointer maps and bingo you have a single menuless dvd with multiple episodes. This only requires a single command from ifoedit (get vts sectors).

Another approach would be if the vobmux part of vsplit automatically put a chapter marker every 5 mins or 10 mins, or better yet offered a command line option for chapter mark spacing, and created a single VOB set (video title set?) for multiple input files (ie vsplit -VOB -CH 5 xx1.ty xx2.ty xx3.ty ...) to create a single Video Title Set (VOB's and IFO) with 5 min chapter spacing and a chapter break at the start of each new ty file mux'ed.

With chapter or file/episode breaks that would be a quick and dirty way to create a DVD with multiple episodes with limited navigation without requiring additional menu authoring...

edpuffmonster
10-31-2002, 09:00 PM
As I was saying above, it would make more sense to use a "Title" for each episode and chapters within that title at commercial breaks, etc. After all, that's what the title feature is for.

rpongett
10-31-2002, 09:19 PM
Doing a Google search (I am not at all familiar with DVD authoring software besides Maestro/Spruce), I found two DVD authoring programs which purport to be able to "import VOB" files (unlike Maestro and Spruce).

(1) NeoDVD 4.0 Standard (http://www.mediostream.com/support/faq4.html) In response to the question "Can I export DVD files captured with neoDVD? Can I edit DVD files?", the author's site claims: neoDVD captures directly to DVD format. DVD video is stored in .VOB files in a folder called Video_TS. It is possible to import .VOB files into neoDVD for further modification or to mix .VOB files from several projects. This product is only $29.

(2) GEAR.wrks Toolkit (http://www.gearsoftware.com/products/wks/index.cfm) claims it can "Import VOB (Video Object files) into a DVD Video project." I believe the cost of this software will be high, as the basic GEAR.wrks software appears to be around $750.

Has anyone used either of these (especially teh more affordable NeoDVD 4.0)? I am sure editing the VOBs would screw up sync somehow (the 29.97 fps problem still exists), but it may cut through the BS of DVD creation/menus, etc . . .

jdiner
11-01-2002, 12:40 AM
Ok. I have identified my own set of tools, some cheap and some not, that makes claims to deailing with VOBs directly. The reasons for looking for one that does so have been well stated in the past.

From what I have read, but not yet tested, it should be possible quitely easily so to take VOBs to some form of navigatable DVD. Some make claims to full motion menus, other to titles etc... So I am pretty confident now that it is possible and nothing something that I will have to write. Which is a good thing mainly as it would take time and none of us wants that.

So back to mux'ing. Amazing how trick-or-treaters can punch a hole in ones night. But I must say there were some really cute costumes tonight... :)

--jdiner

jdiner
11-01-2002, 01:21 AM
Well I need to run it through its paces but I have video packing the way I want. I did not close every packing need because of the need for a timestamp on every frame. (Since it is not really 29.97 FPS for a DTivo stream...)

So every so often we do get a 24 bytes of padding. But... having said that. A full 1hour stream now pads by about 28kbytes in totoal. That is more than good enough and I have no desire to try and reduce it any further... :)

So a bit more testing a new version will be here. Hopefully tomorrow morning sometime.

--jdiner

misato
11-01-2002, 04:51 AM
Jdiner, pardon if this is a regression to discussions of the past, but I couldn't quite bring myself to go through all 50 pages of this thread; I spot checked it only. I didn't see a similar question posed.

I'm about to proceed with dtivo mods (mine is untouched at the moment), and trying to determine if extraction is at the point I need for it to be useful. Right now I do analog->DV capture, edit, and re-encode to program streams. I'm addicted to full editing ability (replacing voiced-over credits with clean theme music, normalizing, adding black slugs and sometimes title clips, etc... stuff a straight cut edit won't do.) My goal is program streams; I have no need to produce DVDs from my generated files.

So I prevailed on a kind soul to send me a .ty file to play with (extracted just last night, if that makes any difference). My intention was to convert it to an editable AVI file with a low loss codec (MJPEG set to highest quality), edit, encode, see if that would produce higher quality output than realtime capture to DV AVI.

I ran it through tytools5 and remuxed it with TMpegEnc. The resulting file plays perfectly through both my hardware MPEG cards (Creative DXR2 and Sigma Designs Hollywood+) and with no detectible A/V sync problems, but does not play in MS Media Player, nor can it transcode to AVI through any tool I've tried. This seems odd, since I've never seen MSMP choke on any MPEG file before unless it were totally munged.

When I tried transcoding just the m2v file, it ran for a fraction of a second (probably through one GOP) then choked. Even using RAW output choked, so I'd say it's not a problem with the codec selected for output.

One possible clue is Procoder, when looking at the mpg or m2v files, shows 480 X 480 resolution (correct), but 00:00:00.00 duration.

Is this expected behavior? Can you think of anything else I should try to convert these files to editable AVI? Can/should I send you the files for examination? (I'm assuming there's nothing new in them, but rather there's an incompatibility problem with Direct Show.)

Thanks!

qarlo
11-01-2002, 06:29 AM
jdiner,

Too cool, bro!!

Q

misato
11-01-2002, 07:12 AM
Now this is odd: Looking in more detail at the generated AVI file, AviInfo says it's 24 fps (I thought an extracted stream would be at 29.97?) and it shows up as 480 X 368 resolution. Windows explorer properties says the same. Did I do something wrong, or do you get similar results? Any thoughts?

skorous
11-01-2002, 11:27 AM
Originally posted by jdiner
So every so often we do get a 24 bytes of padding. But... having said that. A full 1hour stream now pads by about 28kbytes in totoal. That is more than good enough and I have no desire to try and reduce it any further... :)

--jdiner

And to think I called you a perfectionist. ;)

Looking forward to the next version.

Skorous

jdiner
11-01-2002, 12:54 PM
Originally posted by skorous
And to think I called you a perfectionist. ;)

Looking forward to the next version.

Skorous
Hey there is only so much that I can do. We have to have the PTS/DTS timestamps since the frame rate is not constant. (Which is a real bummer) and so at the very end of a frame and the start of the next if we get within 23 bytes (19 for the header, 3 for the potential lead-in, and 1 for the start of the 00 00 01 00 4-byte Frame header) we have to pad things out. With out doing so the frame gets the wrong timestamp...

So technically it is perfect for the scenario, just not completely padding free. And to be honest I might be able to come up with a better technique. Reset the timestamp in the PES header at the top of "the end of the previous frames bytes" and then just stick the new start in. Which is kind of what I did with the audio. But it just makes things ugly to do. And there is no real need for he level of padding we are seeing now.

So maybe I am not a perfectionist? Just close. :)

--jdiner

keith721
11-01-2002, 01:52 PM
jdiner originally said:
A full 1-hour stream now pads by about 28 Kbytes in total. That is more than good enough and I have no desire to try and reduce it any further... :)
and then skouros beat me to the punch:
And to think I called you a perfectionist. ;)
you just gotta respect a person who recognizes when he's done something well enough.
kinda restores your faith in humanity's capacity to be intelligent, sometimes :p :p

way to go, jdiner !! can't wait to try it out...

jalex9
11-01-2002, 02:22 PM
I have a couple of recordings that I only have the .m2a and .m2v, and don't have the .ty any more.

Will there be any method to use your muxing code to put them into a synced .mpeg? Or is the all the syncing data lost in their creation?

If there was a way to mux from .m2a and .m2v with your muxer would there be any way to upsample my SA recordings and get a sync'd .mpeg?

Just curious

skorous
11-01-2002, 03:30 PM
Originally posted by jdiner
Hey there is only so much that I can do. We have to have the PTS/DTS timestamps since the frame rate is not constant. (Which is a real bummer) and so at the very end of a frame and the start of the next if we get within 23 bytes (19 for the header, 3 for the potential lead-in, and 1 for the start of the 00 00 01 00 4-byte Frame header) we have to pad things out. With out doing so the frame gets the wrong timestamp...

So maybe I am not a perfectionist? Just close. :)

--jdiner

<MOCK INDIGNATION> Hey, hey, hey, _MY_ frame rate is rock solid. I won't have anybody defaming my frame rate. :-) < /MOCK INDIGNATION >


For the official record, I'm with Keith721.

Skorous

Immo
11-01-2002, 04:44 PM
jdliner,

I have read this thread several times and this is my first post in this forum. I have in two months acquired and completely hacked my new DSR6000 from hard drive upgrades, turbonet install, Kraven's upgrade, and boot299 improvements. I have never had to post before but feel compelled to now.

I am so impressed by the work you have done. I have tested each one of your beta releases and followed each of your tweaks. When you release the final version that you are happy with I would love to send you some beer money via Paypal.

Immo

misato
11-01-2002, 10:11 PM
Hm. It is confusing. Can you suggest specific threads to look at for this stuff?

skorous
11-01-2002, 10:47 PM
Riley,

I thought the DirecTivo's always output their stuff at 480x480?

Misato,

The DirecTivo's are variable frame rate unlike the Stand Alone Tivo's. JDiner has a big post back about six or eight pages where he talks all about it. If that's what you're looking for.

EDIT: Wow, it's aways farther back than I thought. It's on 10/12 at 3:00 in the morning. It's should be the mid thirties for those of you using a standard page size.

Skorous

skorous
11-01-2002, 11:01 PM
Huh, I would've sworn I read that DTV always sent 480x480. Sorry to have doubted you. :) I do remember correctly that you guys can't change your resolution/quality right?

Skorous

keither
11-02-2002, 12:20 PM
Hey JDiner --

I know this might be a bit premature, but if you've got video mux padding to a few K over a gig of data, would it the right time to release a mux version that supports more than 5 minutes?

BTW, with mux1j and the Matrox DVD player software (Cinemaster codec), sync stays true through 5 minutes.

It seems like this is a good point for us to go to larger clips.

mpauley
11-02-2002, 12:37 PM
Has anyone seen the new guide for IfoEdit (http://www.doom9.org/mpg/ifoedit-dvdauthor.htm)? The new version of IfoEdit will allow you to import an audio and video file, allow you specify an audio delay and to set chapters for DVD creation...

digitalAir
11-02-2002, 04:12 PM
It seems like this is a good point for us to go to larger clips.

Don't you really mean, "I'm too impatient to wait for you already established high standards for quality assurance, so please take the 1000 chunk limit out..."


Damn people, jdiner has stated on numerous occasions that the limitation is in there specifically to keep people from using muxing outside of a testing environment.

Hell, he's 1 man who has created enough work for himself just fixing the bugs that crop up in a controlled environment. Not to mention the ridiculous number of selfish requests, accusations, and demands that he is continuously flooded with everytime somebody doesn't agree with the decisions he makes on his own program...

Until all the bugs are flushed out, all the optimizations are implemented, and all the desired features are added there is no reason to move beyond the 1000 chunk limit. The last thing jdiner needs is to be hit with the, "my clip blows up at 90 minutes everytime" problem before he is confident that he can rule basic computation problems.

In short you need to be able to reduce the problem to "it's either the size of the clip or its a bad clip." And, in order to do that, he needs to have almost 100% confidence that the util works correctly for all small clips. Otherwise he could spend all his time investigating a 90 minute clip not knowing if a short 5 minute clip may or may not produce the same error.

1000 chunks is big enough to catch most known overflow conditions; yet small enough to be managable for testing.

When he's ready to release his muxing tool, he'll release it. And, if past performance is any indication, he won't release it a moment sooner than that... But, it will be right when he does so...

digitalAir
11-02-2002, 04:19 PM
sorry for the rant.

But, working in the software industry, I have experienced that imaginary deadlines, rushed products to market, and the ever present "I would buy, but I need this feature..." all lead to one thing...

a patch... (ver 1.01, ver 3.1.01, etc)

The man is creating a wonderful utility for us... Let's not rush him.

keither
11-02-2002, 05:31 PM
Actually, I meant "I have some 10 minute clips I'd like to mux".

By no means was my request meant to rush Josh. I'm extremely grateful for all the work he's done. I wish I had the time to invest that he does.

However, from what I've been reading, we are at the stage where we're going to a new featureset (i.e. VOB generation or editing). I thought now might be a good time to try longer streams.

Obviously if Josh thinks it's best to wait on that, then you won't hear me complain (I haven't yet and I've been following the development of this since before Josh really got started).

No need to bust my balls. ;)

misato
11-02-2002, 05:36 PM
After reading most of this entire thread (months!), I am sooo tired! (Glad jdiner's doing the real heavy lifting! <g>).

I found some discussions of the oddities of the dtivo stream, so I understand the basic problem: VBR (not terribly difficult), VFR (definitely a problem) and partway through the thread someone popped in with a statement that dtivo streams contain intermixed interlaced and progressive frames (gack!). One day (not right now) I'd like to hear jdiner explain just what he actually does about all of that; I'd be tempted to adapt an open source mpeg player to decode every frame into a buffer, then emulate the double buffered architecture the hardware playback chips probably use to cope with such abuse. If files aren't growing wildly through split/mux, he must be using a slicker approach that that, though.

In the mean time, (and I solicit further recommendations of specific threads or search terms to look at if that would help), I'm wondering if there's anything that can be done to solve the problem of muxed streams being read (by some of my commercial packages) as being of zero seconds duration? jdiner: Does anything come to mind immediately as a likely (and hopefully easily fixable) culprit for that behavior?

The closest I've come to getting a usable AVI from a muxed stream derived from a suggestion a few posts back to use mpeg mediator, but that utility seems confused by 4:3 aspect ratio. It feels compelled to change the actual vertical resolution to obey that flag, so I suppose its design assumption is square pixels. Since mediator's not open source, I can't do anything about that, so I'm hoping a muxed stream can be made palatable to other processing utilities. If muxed streams didn't look like they were of zero length to my favorite conversion apps, then maybe they'd be capable of the format conversions I'd like to perfrm.

skorous
11-02-2002, 09:14 PM
Originally posted by digitalAir
Don't you really mean, "I'm too impatient to wait for you already established high standards for quality assurance, so please take the 1000 chunk limit out..."


Damn people, jdiner has stated on numerous occasions that the limitation is in there specifically to keep people from using muxing outside of a testing environment.

Hell, he's 1 man who has created enough work for himself just fixing the bugs that crop up in a controlled environment. Not to mention the ridiculous number of selfish requests, accusations, and demands that he is continuously flooded with everytime somebody doesn't agree with the decisions he makes on his own program...

Until all the bugs are flushed out, all the optimizations are implemented, and all the desired features are added there is no reason to move beyond the 1000 chunk limit. The last thing jdiner needs is to be hit with the, "my clip blows up at 90 minutes everytime" problem before he is confident that he can rule basic computation problems.

In short you need to be able to reduce the problem to "it's either the size of the clip or its a bad clip." And, in order to do that, he needs to have almost 100% confidence that the util works correctly for all small clips. Otherwise he could spend all his time investigating a 90 minute clip not knowing if a short 5 minute clip may or may not produce the same error.

1000 chunks is big enough to catch most known overflow conditions; yet small enough to be managable for testing.

When he's ready to release his muxing tool, he'll release it. And, if past performance is any indication, he won't release it a moment sooner than that... But, it will be right when he does so...

DA,

I understand a little bit of work stress relief but I think there are a few extra factors involved. First, a workaround for the chunk limit was already posted in this section. JD already saw it, said that he figured it would be breached, and told everyone that he just won't answer any questions about people patching things together. Secondly one of the reasons he quoted for not releasing a longer running version is that he wanted all the padding out before he did. It's a logical question to ask if he's going to release a non-limited version now. Keither was properly deferential saying that he left it up to JD (of course) but that he thought this might be a logical time to test longer clips. If he'd done a Midnight Marauder and demanded a longer version or if he bitches about it when JD says no or ignores him I'd agree with you but in this situation I'd say it's a valid question. As always, just my $.02 worth.

Skorous

jdiner
11-03-2002, 03:33 AM
Originally posted by misato
I ran it through tytools5 and remuxed it with TMpegEnc. The resulting file plays perfectly through both my hardware MPEG cards (Creative DXR2 and Sigma Designs Hollywood+) and with no detectible A/V sync problems, but does not play in MS Media Player, nor can it transcode to AVI through any tool I've tried. This seems odd, since I've never seen MSMP choke on any MPEG file before unless it were totally munged.


Transcoding is really outside of the scope of this particular thread. Even as big as it is.

I have posted over over my method for transcoding it into an AVI. I successfully did hundreds until my RAID system died and I lost the TY's and the AVIs.

Do a better search on it in the TyTool thread and you will find what you are looking for.

--jdiner

jdiner
11-03-2002, 03:38 AM
Originally posted by jalex9
I have a couple of recordings that I only have the .m2a and .m2v, and don't have the .ty any more.

Will there be any method to use your muxing code to put them into a synced .mpeg? Or is the all the syncing data lost in their creation?

If there was a way to mux from .m2a and .m2v with your muxer would there be any way to upsample my SA recordings and get a sync'd .mpeg?


I had thought so. But now I think NO.

It work for the SATivo as that form has a solid frame rate that can be counted on. And that I could mux successfully. But then so do many many other programs really. (I am not talking after editing sync I am talking about just 1 mux'ing pass)

But for a DTivo stream with the VFR nightmares there is no hope. Without the information about frame rate it is just not going to work. Look back at the info I posted before where it fluctatued so wildly. I can tell what goes where with the whole TyStream of information, but with just 2 ES there is no way to piece it back together... Well technically there "is a way" but it is so ugly I am not going to touch it.

I can explain this other technique if someone else wants to try it. But like I said, I am not going to be working on it.

--jdiner

jdiner
11-03-2002, 03:44 AM
Originally posted by rc3105
the mpeg data in the tivo & extracts is pretty wierd, but it's still mpeg and usually it's actually 352 *480. storing at any higher res is just a waste of space. depending on the show & sat signal strength it may be 24 fps, or 29.97, or somewhere in-between.

Dude. This is good information. I had never looked that closely at it. Though I really should have.

The headers in the TyStream clear indicate that it is 480x480. Which is not a real DVD resolution. But 352x480 IS VALID! Which means bottom line really if I add an option reset this header value as processing is run through then most DVD programs would accept the output... And potentially even more players would like the output.

I just wonder if it is really always 352x480 or if it is just some streams. If you think about it, this kind of a resolution shift is indeed one of the very best ways to save bandwidth. 128 bytes per line at 480 lines means 60kbytes exactly less data to compress and then transmit. Makes sense.

But then the HQ movies and that demo channel for stores that were mentioned might quite litterally be 480x480 in a desire to give the best possible looking output.

Humm food for thought at the least and I can see a reason to look into this in detail down the road a bit.

--jdiner

jdiner
11-03-2002, 04:07 AM
Originally posted by keither
Actually, I meant "I have some 10 minute clips I'd like to mux".

By no means was my request meant to rush Josh. I'm extremely grateful for all the work he's done. I wish I had the time to invest that he does.

However, from what I've been reading, we are at the stage where we're going to a new featureset (i.e. VOB generation or editing). I thought now might be a good time to try longer streams.

Obviously if Josh thinks it's best to wait on that, then you won't hear me complain (I haven't yet and I've been following the development of this since before Josh really got started).


Yeah I am something of a new comer to this whole thing. I have had friends that had tivos for years before I got one. I am just a bit more successful with the software than others. :)

The benefits of having 2 parents both of whom teach the Computer Science department of the local University. The indoctrination started young and was quite thorough I might add...

I actually taught at the same university for 7 years. CS as well BTW. But then I realized I could make just a bit more money in the industry than teaching so off I went. Got sucked in by HP. Had a great offer from 3Dfx for an insane amount of money to take over dev on some low level video stuff for them... But I digress.

This is interesting and I don't like to leave things partially done. So I stick around and keep heading towards that I really want out of this thing.

I know that my methods are frustrating to many and other understand them. With ever release I get a flood of failure reports. Some few are real, but most are not. Many are related codec issues. You might notice that I have not mentioned Media Player in my test list in a very long time. There are many reasons for this. The mpeg-2 codecs available for it are by and large crap. I have figured out enough now that I could write a bullet proof one. But I have ZERO interrest in doing so unless someone wants to through serious money at me. Why you might ask? Because WinDVD and PowerDVD work like champs. Why re-invent the wheel just so we can use M$ lame interface... :)

Back to the bugs and what not. Every time in the past that I have made releases, I am thinking of VSplit pre #13 and TyTool. The first thing the majority of the "testers" seem to want to do is run out and get the ugliest stream possible. You would not believe the sheer number of comments I got about the olympics. 4+ hour long shows recorded at Best quality and over 7gig in size. And it became my responsibility to find a way to make it work to download them onto a machine with a FileSystem not capable of more than 2gig.

So I limited this one to 5 minutes. With the way mux'ed sync used to work this is more than enough to see if we start aligned and sync'ed and are staying that way. I can obviously do more as I have posted my results. But I have not put up a program that would do that.

In an attempt to be crystal clear here is why:
1- As soon as people think it is "good enough" they are going to try and use it for real.

2- I am still working on video padding reduction. It will make the stream 8-10% smaller.

3- Someone would ask how to/demand a way to "re-mux" what they did and got rid of the TyStream for, with the new less padding method.

4- I have not put in the VOB option to go to DVD. This is primarily now just adding 1 more feature to the PES output list.

5- Someone would ask... with the same results. A flame fest going both ways about what I should and should not do and the order to do it in.

6- There is at present "no clean end to a stream" being made. There is a PES header for the "this is the end" that I am not making. I am currently not flushing and padding out the end of the stream at the end of everything. I should be. If there is less that 1 second of audio and video at program shut down whatever is in there is just tossed. Without this "end" streams occasionally crash the player when played full to the very end.

I have been called a liar here in the forum, PMs and in email for saying that I get demands for the source and features and ports and... on a regular basis. Believe what you will but I got 3 today. 1- a BSD user that wants to put it into a FreeBSD /usr/ports/.2- a Debian linux user that wants to make a package for it. 3- a person who bluntly pointed out that not having a MAC version was costing me money. What money? I don't charge for it. :)

So happy or not I will continue as is. I have tested a ton of 2+ hour movie streams and they stay sync'ed in both players I test. I can move it up to 2000 chunks if it will make people feel better. That will be 10-14 minutes in size in general. The real problem is that we have a lot coming up that could shake things up. I just want to minimize the fall out of doing so.

--jdiner

jdiner
11-03-2002, 04:09 AM
Originally posted by misato

I found some discussions of the oddities of the dtivo stream, so I understand the basic problem: VBR (not terribly difficult), VFR (definitely a problem) and partway through the thread someone popped in with a statement that dtivo streams contain intermixed interlaced and progressive frames (gack!).
Ok the short quick answer... What the progressive vs interlaced affects is the frame counting mechanism. I have solved it and the solution works perfectly now for all 3 types of Tivos.

In terms of playback the effect is almost non-existant. What is affected is the number and types of players that will play it back. A good one will, a cheap/bad one will not.

--jdiner

jdiner
11-03-2002, 04:14 AM
Just to clear in case people don't read that last big one. One of main reasons I have kept stuff so short is that I have tested a ton of long streams myself. And had a friend do even more. 2+ hours movies mux in minutes and keep sync throughout. Nothing to worry about length wise. But if it fails in the first 5 then I have a bug I need to look for...

Also sorry for the delay in the video release. I have been busy with a wireless network problem for work. :( By and large the answer is this... NEVER EVER EVER use a Belkin piece of crap, I mean equipment. What a pain. 1 wireless device works great. 2 and they work so-so add a 3rd to the infrastructure and you are dead meat. But luckily it supports 64 of them so... I went and got a different WAP and it was up in mere minutes.

I will be getting back to polishing the video piece in the morning. And hopefully a release will be forth coming soon after.

--jdiner

jdiner
11-03-2002, 04:16 AM
Ok. So this is way off topic. But I hope you will cut me a little slack... Check this one out. Proof positive of the "bigger *****" method of business...

http://www.origamiboulder.com/...

--jdiner

misato
11-03-2002, 04:53 AM
Originally posted by rc3105
Jdiner's commented on how he's handling things LOTS. He's stated it would be quicker finish the program than to write a thesus on the whole issue and/or release his source & deal with the flood of inevitible questions.

Yes, I've seen a lot of comments around the edges of what he's doing, though without the fundamental context, they're only vaguely illuminating. I do understand his desire to code now and explain in great detail at some future time. My comment was that I look forward to that future time when he is ready to explain in more detail what he's been doing all along.


open source development is moving ahead with the exact same transport stream issues in the hdtv communities though. if you want to take a look at analysis & code with an eye to writing some utils youself look there.

Interesting. That's another area for me to dig into, then. Considering the (early) deployment status of HDTV, I wasn't aware anyone was even working on these issues in an open source context.


power dvd & win dvd report that extracted/muxed mpeg from the dtivo is 352v*480h, tmpgenc seems to agree. mpeg created at that res imports into the dtivo and plays nicely. other res's get wierd. mpeg mediator encodes it to 368v resolution because of a quirk in the multiple of 4 boundry requirement in the divx codec I use.

This I find confusing. Right now my serial cable is built, I intend to apply the HD massages in the next few days, and a turbonet card is on order. So I don't have a bunch of .ty files of my own to look over; I'm experimenting on one .ty file someone sent me for the purpose. Because of your comment, I looked at it again in Power DVD, and it displays the follwoing description:


Video Attribute :
Video compression mode : MPEG-2
TV system : 525/60 (NTSC)
Aspect Ratio : 4:3
Display Mode : reserved
Source picture resolution : 480x480
Frame Rate : 29.97
Source picture letterboxed : Not letterboxed
Bitrate : 15.00Mbps

I'm looking, as I said, at only one sample, and I *think* it's a PPV clip, if that makes a difference. But I don't know how to explain the difference in our results: my one sample showing up at 480 x 480 vs your experience of seeing 352 x 480. Any thoughts?

Mediator converts this to 480 X 368 regardless which codec I choose for output, and also produces output marked 23.976, so I don't think that's an effect of the output codec used. How it decides on that frame rate for this clip, I don't know. Maybe it's correct, but its methods are mysterious to me at this point. Do you know if it discards scan lines, or scales, or ...? Edit: in fact I just converted 100 frames to RAW output. (52MB! wow!) It too is 480 x 368.

Is mediator the tool you routinely use for conversion of muxed files to AVI?

If so, does the modified version you mention do anything functionally different from the released binary? (ie did you change anything in its actual handling of resolutions, frame rates, etc? Or are your changes purely to the executive behaviors, eg the automatic execution you mentioned?)

As for mediator being open source, and my remebering it otherwise, all I can say is "doh!". I committed an error of memory as most occasionally do. :/


... my personal varient does automatic tytool-like extraction & compression to avi whenever a new show is recorded. adding auto-extract & compress isn't hard & has been covered before also.

Cool! Does it run on windows and periodically query your tivo for new files, or have you ported it to run inside the tivo and do conversions at low priority inside there? Or something else? Is that something you'd be willing to describe and share with the rest of us? Or have you already, and I just need to find the post?

misato
11-03-2002, 04:56 AM
Originally posted by jdiner
Do a better search on it in the TyTool thread and you will find what you are looking for.

--jdiner

Okay, I'll look there.

Wooly
11-03-2002, 09:00 AM
hehehe...I learned this lesson out the hard way also. I have 2 words for you that I have had great success with:

Linksys
SMC

They're both great, and the encryption standards they use are inter-operable (unlike Dell's Laptops, who use bizarro standards that only work with THEIR AP's). I have had 40+ Linksys Wireless devices working off one SMC AP (and visa-versa) in the field with no problems and respectable bandwidth per device (bearing in mind what it is, what the max per AP is, and the budgets involved).


Originally posted by jdiner
Also sorry for the delay in the video release. I have been busy with a wireless network problem for work. :( By and large the answer is this... NEVER EVER EVER use a Belkin piece of crap, I mean equipment. What a pain. 1 wireless device works great. 2 and they work so-so add a 3rd to the infrastructure and you are dead meat. But luckily it supports 64 of them so... I went and got a different WAP and it was up in mere minutes.

--jdiner

scarabus
11-03-2002, 12:09 PM
Originally posted by jdiner
I was wondering how long it would take for this to happen. Longer than I expected actually. So at this point I ask 2 things:

1- Don't ask me any questions or make any bug reports about a stitched together mpeg using this script or 1 like it. I have nothing to do with that side of things. (Things will not line up perfectly at the cuts due to how it works...)

Understood. However having said that here's an interesting observation. It may be useful; feel free to ignore me.

The idea of cutting and stitching has occurred to me, but I haven't had time to try it until today. So I chopped up an hour long program into overlapping chunks - 1000 blocks, overlapping by 50 at each end - and edited the output with m2Edit.

One of the chunks failed to split properly. It seems that I'd picked a bad block to start on, so I shifted the start for that chunk and it worked OK.

What I found fascinating was that when I loaded each chunk into m2Edit it actually had the start and end time for the larger stream; ie a chunk might be 10:00 to 15:00 rather than all of them being 0:00 to 5:00

But the key fact was that the frames at a given timestamp didn't match between the two streams. So let's say I have two clips; one is from 0:00 to 5:00 and the other is from 4:00 to 9:00
If there's a cut from one shot to another in the first clip it may be at 4:45:00, but in the second it was always a few frames later, say 4:45:10
Now maybe this is what you meant by Things will not line up perfectly at the cuts due to how it works but I thought that it was worthg mentioning.

jdiner
11-03-2002, 12:21 PM
That is exactly what I meant by things not lining up. It is not as if you would be spot on. You would be close. i.e. within seconds but not perfect. To my way of thinking it is not a good solution. It is just a way to check the whole file. Anyway, like I said do it if you want, but it will almost never be a perfectly aligned cut.

As for why the timestamps appears to be somewhat close?!!? Even after it has been cut. That was because of what is going on with the "alternate" method I mentioned for the mux'ing. There is 1 more "timestamp" found in the elementary sream itself. But this other timstamps granularity is crap. It can be used... Sort of... And is in evidence here but it not good enough for real use unless you absolutely have to...

--jdiner

jdiner
11-03-2002, 12:28 PM
Originally posted by misato
Yes, I've seen a lot of comments around the edges of what he's doing, though without the fundamental context, they're only vaguely illuminating.
My posts on what I find are detailed right up the point of code. As in I say what I do, but not how as that is pointless... A good developer could rebuild what I had done from what I have posted with little trouble, in some cases they have... And if you are having trouble figuring it out, go back to the first of 4 monster threads I have been involved in and read them carefully. Everything is there.

But that doesn't matter. Insults, only perceived or otherwise, aside this thread IS NOT the place to ask your questions. I was OK'ed by the moderators to put some rules on this thread to help things move along. This one is for mux'ing and the tools to do it with, mine and others, ONLY! Occasionally a few other things traipse in from myself and others, and that is ok. But you have added many many off topic posts and the simple truth is that transcoding is a different discussion as has been mentioned time and again.

So please take what you are asking elsewhere. I know this is a high traffic thread which makes it seem the best place to get an answer, but please start your own thread, or jump back into 1 on transcoding.

Let's all get back to the task at hand...

--jdiner

misato
11-03-2002, 01:08 PM
Sorry, no insult was intended, and I originally asked because the muxed output showed up strangely in other tools: as zero length. That's what I intended initially to ask about, and the topics kindof smeared together. As is, I've found workarounds (rather elaborate, but seemingly working as far as I can tell), so I'll quit asking why muxed files show as zero length.

scarabus
11-03-2002, 02:28 PM
Originally posted by jdiner
That is exactly what I meant by things not lining up.

I thought as much.

I investigated a bit more with the edited clip - sync does drift. Video ends up ahead of audio. I demuxed with TMPGenc and Maestro reports the video stream to be 13s longer than the audio. Spruce won't read the muxed stream; it craps out at 13%. So as you rightly say, at the present time it's just a curio with no practical application.

That's OK, I can wait.

fredisdead
11-03-2002, 08:07 PM
jdiner:

I found another bug for you in vsplit13c.

When vsplitting a tyfile (Star Trek Voyager Caretaker Part 2) the extracted ty file is 1.3gb. vsplit13c only extracts 3/4 of the file (~900mb of video) reports normal termination with oob packets. Importing file m2v and mpa into maestro, got a temporal reference error and when inspecting the imported video it ends about 15 mins early.

Repeated the vsplit with v13 not 13c and got an error report of finding a 'hole' right about where the 13c terminated. But continued and wrote the full video and audio files.

Imported them into maestro and no problem...

NET: vsplit13c truncated the file where vsplit 13 found a 'hole' but proceeded to completion.

what would you like me to do?

NicholFD
11-04-2002, 05:42 AM
I PM'd jdiner, and uploaded some samples to an FTP site, demonstrating the same problem as fredisdead. I'm not sure if jdiner has had a chance to look at the problem.....

Pr.Sinister
11-04-2002, 01:24 PM
Had the same problem with a Fastlane episode.

Unfortunately, i deleted the TY file before i noticed the problem.

-Pr.

edpuffmonster
11-04-2002, 02:25 PM
Sorry if this has already been said, but...

Some channels are 352x480

Some channels are 480x480

Most of the local channels are 352x480. Almost every other channel (100 through 800) are 480x480.

mattdb
11-04-2002, 02:31 PM
Well as far as tesing is going, I really haven't done a lot of testing with the new versions. I am still using the old one to make my dvd's.

I did run a few tests to see how it was working and it worked great for the 5 minute clip. I am now waiting for the "final beta" so that I can start archiving away.....Straight to DVD would be nice but not a necessity as I want/need to do menus for multiple shows on one dvd.

Thanks to Jdiner for all his hard work.

As far as supporting Jdiner, I have already donated instead of waiting til he was done.

Matt

fredisdead
11-04-2002, 04:36 PM
Further to my last post, I have had a similar problem with another ty file. This time Star Trek Voyager 'learning curve'.

Vsplit 13c finishes (showing normal termination outputting only about 10% of the file, vsplit13 works fine. I am pasting the log output of the vsplit13 run


C:\tivohack\vsplit>vsplit13 "I:\tivofiles\vg 01x16 learning curve.ty" lc.mpv l
mpa
Processing 'I:\tivofiles\vg 01x16 learning curve.ty': (10 chunks per tick)
Detected Tivo Type: DTivo
Detected Audio Stream Type: MPEG Layer II
Final standardAudioSize = 492
Final standardFrameLength = 480
Final standardAudioDiff = 2160 or 00:00:00.024
First Video PTS: 00:00:41.905
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
......... 1100......... 1200......... 1300......... 1400......... 1500
......... 1600......... 1700......... 1800......... 1900......... 2000
......... 2100......... 2200......... 2300......... 2400...Found an OOB packet
. The Audio Diff is: 00:00:00.096
Is it in sequence??? It is OFF by exactly 4.000000 frames.
It is in the right range. Let's see if it lines up.
It is in sequence. Starting back up after the 'hole'...

THIS IS WHERE vsplit13c terminated...... vspit13 continues..

skorous
11-04-2002, 04:54 PM
Originally posted by edpuffmonster
Sorry if this has already been said, but...

Some channels are 352x480

Some channels are 480x480

Most of the local channels are 352x480. Almost every other channel (100 through 800) are 480x480.

Ed,

I'm confused. If there are 1000 channels and channels between 100 and 800 are 480x480 then aren't most 480x480 (80%). ;)

All kidding aside, is it simple like movie channels are 480x480 or something?

Skorous

jdiner
11-04-2002, 05:07 PM
I have not yet had a chance to go through them. #13c should just have extended what was considered good. Not killed things. So either I did something wrong, quite possible, or there is a side-effect I never noticed. I will go through it tonight. I just have my hands full with everything...

I got called out of town last night and won't be back until this evening.

--jdiner

edpuffmonster
11-04-2002, 06:05 PM
Originally posted by skorous
Ed,

I'm confused. If there are 1000 channels and channels between 100 and 800 are 480x480 then aren't most 480x480 (80%). ;)

All kidding aside, is it simple like movie channels are 480x480 or something?

Skorous

Yes Indeed. Most _LOCAL_ channels are 352x480. 0-99, like NBC, CBS, ABC, WB, FOX, etc :)

Except for the FOX, ABC, etc in the 380's. Those are 480x480.

Although I haven't exactly checked every channel, it appears to me that it's like this:

0-99 are 352x480 (locals at new location)
100-800 are 480x480
800-999 are 352x480 (locals at old location)

(There are some 'exceptions' like 900, 998, 999 which are 480x480)

I wouldn't be surprised to see DTV eventually switch more channels to 352x480 in order to cut down bandwidth and add more channels.

fredisdead
11-04-2002, 08:17 PM
If the locals are 352x480 (which is dvd compliant) how come the headers are set to 480x480 when split by vsplit? Bug?

skorous
11-04-2002, 10:11 PM
Fred,

Though I have no idea, I'd bet it's a case of DTV lying in the stream. I remember JDiner saying that their framerate flag is always 29.97 even though it blatantly varies. Given the choice of JD making a mistake vs. DTV lying in the stream it's not a hard choice. :-)

Skorous

skorous
11-04-2002, 10:19 PM
Ed,

hee hee. I actually took local to be the exact opposite of what you meant. I thought local as in DirecTV specific and network channels as "foreign". Where was my head at today? Duh....

Skorous

fredisdead
11-05-2002, 01:51 AM
jdiner:

Per my previous posts, I now have about 5 ty files which exhibit the same symptom, with vsplit13c truncating the files early (without abnormal termination). I can get some of those to split properly with vsplit13, but some files result in the dreaded temporal reference error. Any thoughts? Would you like me to sent/ftp one of the files? If so where?

fredisdead
11-05-2002, 01:53 AM
Yep! The spot beams are all in 352x480, which actually is great since that is a legal dvd resolution... No more dvd patcher to import the files into authoring programs.... On closer inspection, vsplit does set the headers correctly to 352x480..

jdiner
11-05-2002, 04:35 PM
Originally posted by fredisdead
If the locals are 352x480 (which is dvd compliant) how come the headers are set to 480x480 when split by vsplit? Bug?

No. I don't set/reset those values. They are part of the data that is just copied over. Part of the ES and not the mux'er output. As for why it works? They have a captive audience in their own hardware. They ignore/use data as is needed. Which is why the VFR plays for them and we are having such trouble with it.

--jdiner

ibsleepn
11-05-2002, 08:25 PM
Originally posted by edpuffmonster
Yes Indeed. Most _LOCAL_ channels are 352x480. 0-99, like NBC, CBS, ABC, WB, FOX, etc :)

Except for the FOX, ABC, etc in the 380's. Those are 480x480.

Although I haven't exactly checked every channel, it appears to me that it's like this:

0-99 are 352x480 (locals at new location)
100-800 are 480x480
800-999 are 352x480 (locals at old location)

(There are some 'exceptions' like 900, 998, 999 which are 480x480)

I wouldn't be surprised to see DTV eventually switch more channels to 352x480 in order to cut down bandwidth and add more channels.

The Locals off the 119 bird are 480x480 from what i have seen.

jdiner
11-06-2002, 12:54 AM
Originally posted by fredisdead
The Audio Diff is: 00:00:00.096
Is it in sequence??? It is OFF by exactly 4.000000 frames.
It is in the right range. Let's see if it lines up.
It is in sequence. Starting back up after the 'hole'...

THIS IS WHERE vsplit13c terminated...... vspit13 continues..

In response to this one and from an earlier post. Right now the mux'ing requires everything to be perfect in the stream. Holes are not handled yet. They will be soon. But for now they must line up perfectly in all respects.

--jdiner

fredisdead
11-06-2002, 12:21 PM
jdiner

I am not using the mux function yet, just looking for elemental streams. Will vsplit handle the holes and prepare muxable elementals?

dlang
11-06-2002, 12:37 PM
no becouse tehre is info in the .ty file that is not in the elementals that is nessasary to do a good mux in the presense of the variable frame rate signals from the dtivo

in other words, hang onto the .ty files until this gets worked out.

jdiner
11-06-2002, 01:21 PM
Originally posted by fredisdead
I am not using the mux function yet, just looking for elemental streams. Will vsplit handle the holes and prepare muxable elementals?

Yeah. That is what it does now. Holes are "correctly" handled now. As in processing continues. But the lost data makes the ES output files unmux'able with sync. Just not going to happen. Think about it.

With my own mux'ing these holes will be completely and correctly handled and sync will be maintained. That is what the mux'ing is for.

--jdiner

jdiner
11-06-2002, 01:21 PM
Originally posted by dlang
no becouse tehre is info in the .ty file that is not in the elementals that is nessasary to do a good mux in the presense of the variable frame rate signals from the dtivo

in other words, hang onto the .ty files until this gets worked out.

You are absolutely right. And well put. :)

--jdiner

jdiner
11-06-2002, 01:25 PM
Ok. I know what was is causing the early termination problems. I was right about it what it was. See what happens when I take code or ideas from someone else? Problems. That's what.

Basically a line that was supposed to be something like:



0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
===============================================
# 0 ( 0): 0 0 0 0 0 0 0 0 0 0 a 18 a c0 db 14


In the output was a "we are done flag" according to others. So I added a stop to it. And it does indeed work for stopping at the end of the file. But on the DTivo for some unknown reason there are potentially tons of these in the stream. I have one test stream now that has 4 of them... :( It almost looked like there was one at the end of each FSID but that is not the case... :(

Until I can get a better look at what is going on I will be removing this short cut to let things stop earlier and better, and we will just process the whole file like it used to. Since I cut bad data already this not really a problem it just means that the trailing "OOB chunks" display will return...

--jdiner

jdiner
11-06-2002, 01:35 PM
Ok. Here it is. A patched version of vsplit #13c Mux 1j to not have the early stopping problem. Nothing else in this version has been changed.

Both windows and Linux versions are included.

--jdiner

snoopy
11-06-2002, 06:01 PM
I use TY Tool to pull my files off the Tivo. They are already split into audio and video via TY Tool's built-in mechanism for doing that. I understand they are not split well and that Jdiner's vsplit splits faster and better than what is incorporated into TY Tool. Is vsplit a separate program that has to run outside of TY Tool (If so, I cannot figure out how to execute the file in Windows XP), or can it be incorporated into TY Tool to provide a one step solution similar to what TY Tool tries to do on it's own?

dlang
11-06-2002, 08:55 PM
Snoopy, TYtool includes an old version of vsplit, at the time it was thought to be complete, but the recent discoveries have shown that it's not a good idea to use it.

I'm sure that jdiner will update tytool once he gets vsplitmux where he is happy with it, but in the meantime you need to change tytool to not split the files, just transfer them and wait until vsplitmux is complete and then run it against them (I think the versio just released is still time limited)

jdiner
11-06-2002, 11:12 PM
Originally posted by dlang
Snoopy, TYtool includes an old version of vsplit, at the time it was thought to be complete, but the recent discoveries have shown that it's not a good idea to use it.

What recent discoveries? If there are still things that need to be fixed in the splitting engine I need to know ASAP.

--jdiner

dlang
11-06-2002, 11:40 PM
I thought that Tytool was based on vsplit 13, well before you discovered the joy of VFR and all the fun that goes with that.

'recent' is a relative term :)

bato
11-07-2002, 12:18 AM
Snoopy, tytool is a great software to take out you program from the tivo, it can work in 2 modes, one is extracting one file (.ty) and the other method is to split in video and audio (this is default), you change it using File - Vsplitmode or File - TyStreammode. You better have TyStream files if you plan to later using vsplit to generate a perfect (sync) muxed mpg file.

If I remember right the vsplit code in TyTool5r2 is not the one with vsplit13c. The vsplit software in this thread is a command line software, you go to a DOS window and execute the .exe, normaly if you double click the .exe you will only see a window flashing open and close.


jdiner, I think dlang is talking about the problem with vsplit #13c so no worries, you just patched it :)

GITM
11-07-2002, 01:08 AM
I tried the latest with my SA ty streams. It works perfect on every one. I'm excited to see it work on the entire stream soon.

-GITM

jdiner
11-07-2002, 02:25 PM
Alright. Lots of nice things cleaned up. I am going to let it run on the testbed for a little while and baring any problems. Expect the next release with video packing and correct exit this afternoon.

Then it is time to finish the DVD Private Stream 2 and VOB creation should be done. Hopefully in the next day or 2, and then if all is well. A full run-length release.

--jdiner

keith721
11-07-2002, 03:41 PM
.

RxMan
11-07-2002, 04:08 PM
I LIKE THIS KIND OF PARTY!!!

snoopy
11-07-2002, 06:09 PM
very cool stuff

qarlo
11-07-2002, 06:58 PM
GITM,

Yo bro - please check your PM.

Q

jdiner
11-07-2002, 07:18 PM
Ok. Got hung up on a simple problem at work. Didn't get back to this until just now.

More to come this evening.

--jdiner

qarlo
11-08-2002, 05:04 PM
Sweet!!

Fugg
11-09-2002, 08:01 AM
Dude!
__________
whew!!

:D

chris0583
11-09-2002, 10:06 AM
DUDE is right! Cant wait!

snoopy
11-09-2002, 12:15 PM
Where's my car!

dynodeano
11-10-2002, 10:04 AM
I gave the vsplitmux-1j2.zip posted earlier a spin and it stopped processing at 1000 chunks. The same tystream can be split with vsplit13 without problem.

I'm very excited about the built-in muxing option. The small portion ( 1000 chunks ) I was able to create as an MPG looked very good at first inspection. This

mattdb
11-10-2002, 11:48 AM
As any used the broadq player with a ps2 on the muxed output.

Well I just got one of the network adapters for the ps2 and the broadq qcast and qtuner software. This stuff rocks. Plays DIVX etx on the ps2. Looks really good. Anyway back to muxing. I downloaded some ty's off of my dtivo. use 1j2 to make a mpg. The broadq plays the file for awhile then the video stops and the audio keeps going. Sometimes when I let the 5 minute muxed file run through the video will come back. Also the video seems to be a little "jerky" at times on the ps2.

Of note. None of the mpg's or divx files I have downloaded exhibit this problem.

It would be really nice to beable to determine what is causing the stoppage in video and jerky play back.

www.broadq.com (http://) for information on the qcast tuner software for the ps2.

koreth
11-10-2002, 11:54 AM
The answer is on BroadQ's site:


Bitrate Support:

Raw VOB/high-resolution/bitrate MPEG2 is not yet supported. We recommend converting these files to DivX, or resizing the files to 512x384 or 320x240 with a bitrate less than 4mbit/second.

mrblack51
11-10-2002, 12:02 PM
dynodeano: all the current versions stop at 1000 chunks on purpose. the current versions are being tested to find bugs, and once its reasonably stable, the 1000 chunk limit will be removed AFAIK

dynodeano
11-10-2002, 01:39 PM
I see. I didn't see any more information as to that effect. The only hint I noticed was the 'prepare run-time length version' in one of the posts.

I'm anxious for the next rev as I guess everyone else is.
:)

jdiner
11-10-2002, 03:01 PM
Alright. I took something of a break for the last few days. Many have sent me messages wondering at why.

The bottom line is that I am tired of being a target. Recently someone new showed up making demands of me, trying to tell me what to do and even trying to guilt me into things. I don't like it. I don't know anyone that does. :)

So I have been trying to decide what to do. I contemplated releasing my source code and then just going off to do my own thing on this and leave off on the forum. Let everyone else do what they want and just be done.

But that isn't really fair. Things are only as far as they are because of the many contributions in many forms from the people here. Those that give time to test and provide bug reports. Those that have contributed cash to hardware for me. Those that provide network access and server space.

So that leaves me right back where I started. I will continue as I have been. I will work towards what I and many others here want. We are so close now I can almost taste it. I will continue to ask for help in testing and get problem clips when they arise, and so on.

And to those demanding my source, and my help, and my information for other things. My answer is simple. Leave me alone. I am not interested in the same fight every few months. I am not interested in the angry words posted here and in email. From here on out I intend to do my best to just ignore all such comments...

So let's get back to it.

--jdiner

jdiner
11-10-2002, 03:19 PM
Alright. I have the video compaction up and running. So far all is good. Really good. With what is going on.

Version 1i was pre-packing on either audio or video.
Version 1k has both Audio and Video packing at full strength.

A simple set of sizes for you:



My 7+ minutes obscura clip:
1i - 171745280 Nov 10, 2002 ob-1i.m2v
1k - 130330624 Nov 10, 2002 junk.m2v

Savings in just junk padding space: 41414656 (39.5 meg)

Again that is just for a 7 minute clip. Not bad. Not bad at all. If things remained the same for a full 60 minute clip the savigs would be roughly 339meg. Really adds up...

My 5 minute Dead Zone clip:

1i - Processed in 2:28 - 165554176 Nov 10, 2002 wheel-1i.m2v
1k - Processed in 1:28 - 141944832 Nov 10, 2002 wheel-1k.m2v

Savings in just junk padding space: 23609344 (22.5meg) And it was what 35% or so faster at doing it... :)



Rather than release it as is right now, like I have done in the past. I am going to take a little while and start cleaning up the code a bit. Make it a bit faster at what it does than it is now.

--jdiner

koreth
11-10-2002, 03:24 PM
Way to go. As someone who's developed a number of free software packages over the years (some popular, some not) I can sympathize. There's nothing like giving someone an inch to make them demand a mile -- even though 99% of the users are perfectly polite and many are downright helpful, the remaining 1% can leave a bad taste in your mouth if you let them.

Once you're satisfied enough with the result to release your source code, you'll at least be able to say "Don't like it? Fix it yourself!" to such people, which is my usual response to rude demands about my free software when I respond at all.

[edit:] I see that second paragraph might come off as a "release the source" request, which wasn't the intent -- more of an "at some point you'll have a good retort." On the contrary, keep your source to yourself until it's at a point where more eyes looking at it would do more good than harm. The only thing worse than a bogus bug report from a clueless user is a bogus bug report from a power user who misunderstands your code.

sirfergy
11-10-2002, 03:53 PM
I for one am very grateful for what you have done. Do not let the negative people get you down, otherwise I'll never get all the Quantum Leaps for my wife. :)

Kythorn
11-10-2002, 03:54 PM
Originally posted by jdiner


So let's get back to it.

--jdiner


Bring it on, I say, and to hell with the naysayers. If certain other parties want to try their hand at it, fine, and I truly wish them the best of luck. But nobody has any right to make demands on you, especially certain newcomer members who have only been around these parts since the end of august.

I can understand the appeal of an open source project. We've all heard certain people stating how much easier this would be if you'd released your source months ago. I also understand the frustration people have experienced with conservative approach you've been taking.

On the other hand, we simply cannot ignore the fact that nobody else has had near the level of success you've had. Most people seem content to get it to about 85% and then vanish, or never share their tools at all. These are the reasons I detest design by committee. All other projects, binary only, or source included have failed to produce anything approaching the level of usefulness of the current incarnations of vsplit.

That same conserative approach that annoys some people is the reason we're as far as we are right now. It's all too easy to become drunk with your own success and ignoring all the corner cases and other petty issues.

To all those who have some sort halfbaked idea that source code is the answer to everything, here:

http://tivo.samba.org/download/tridge/
http://minnie.tuhs.org/Programs/Playitsam/
http://arbor.ee.ntu.edu.tw/~jackei/dvd2avi/

There's some source, knock yourselves out. Astonish us all with your astounding powers of software engineering, or if you can't, stop bashing on the guy who's made it possible us to get where we are.

I'm not attempting to be unduly harsh here. I admire anyone who attempts this project. I wish certain individuals the best of luck in their own endeavos. If they manages to come up with something better, my hat will be off to them too, after all we're all in this for the best video extraction possible.

I don't presume to speak for anyone else, but I'm behind you all the way, and I know there are plenty of others in this for the long haul who haven't wavered in their support since you started publically working on various issues of video extraction at the beginning of the year. Keep up the great work, and don't let them get you down.

snoopy
11-10-2002, 04:30 PM
GO Jdiner GO

How dare they complain about you. I second the forementioned comments.

Tell em to put a sock in it!

Jdiner RULES (as we would have said in the 80's :)

zqfmbg
11-10-2002, 04:36 PM
Far too many people these days see free or open source software as "Oh! I get to do with it as I please! It must be mine!" and try to dictate what happens to it as such. It's disheartening because practically ALL projects in this realm delegate copyright and ownership TO THE ORIGINAL AUTHOR(S). It's just that the license states you can do whatever you want to it (as long as you adhere to some restrictions depending on license, like, oh, "maintain the authorship list").

Once JD lets go his source, do you think it's no longer his? No way. He gets to tell us what we can do with it. It's just that in this case he's saying we can change it and modify it and rerelease it.

jdiner
11-10-2002, 05:21 PM
Sweet. On this Dolby Digital Audio clip from Koreth... I got 982 bytes of mid-stream padding in a 131477504 byte (125.39meg) clip. Now that is what I am talking about... :)

On the aforementioned Smallville clip I get a padding count of 1824 bytes. 1 and a half K out of 130meg.

I dare say the packing can be considered done...

--jdiner

snoopy
11-10-2002, 05:27 PM
great i tell ya

jdiner
11-10-2002, 06:00 PM
Ok. Yet more stats the impatient and even the interested out there. I just ran the latest version, still cleaning things up..., and got the following results from a full 1 hour Smallville-Obscura TyStream.



The output junk.m2v was this size.
1068388352 Nov 10, 2002 junk.m2v

The input TyStream was this size:
1073741824 Sep 11, 2002 Smallville-Obscura.ty


I finally got what I wanted. The mux'ed output is smaller than the input. :)

--jdiner

newlooper
11-10-2002, 06:11 PM
Nice going. I like to see people stick to the granular details until they are where they like them.:D

mattdb
11-10-2002, 07:47 PM
You are da man. Thank you for all your hard work!!!

jdiner
11-10-2002, 09:17 PM
Ok. If this works for everyone then it marks the end of the series 1 releases and we will move onto mux 2.

This release has many many changes in it:

1- The mux'er now packs the output as much as is reasonable. Average padding added for a 1 hour show is 14k. Gotta love it.

2- The mux'er is now about 3 times faster than it was. I worked and re-worked it. There is more that could be done, some simple stuff that at some point I might do to make it faster but it would be minor gains. But for now it is about as fast as it is going to get.

3- The debug printing that had gotten so insane has been removed. It is now activated by flags just like before. So there is no more reason to dump to /dev/nul or nul: depending on OS. Infact it is important again to start watching it looking for other problems.

4- There was a real problem with the Dolby audio still. Something subtle that I had marked to do and then forgotten about. It has been finished and the problem solved. It had to do with the 4 byte sub-header for PrivateStream1 and the new packing mechanism.

5- A few other sundry fixes have been added. Not problem per se, i.e. if a stream segment failed before odds are it still will, but problems with the mux'ing and byte counting that should never really have happened. But could at times.

6- The 1000 chunk limit has been relaxed... Since so many where asking for it the 1000 chunk limit has been relaxed to 2000 chunks. This roughly doubles the play time of a chunk. That way people can test longer clips and re-assure themselves that it will indeed work.

7- Check the very last line of the output for the amount of padding added by the mux'er to the output file.

What comes next:

Ok. Right now everything must be perfect for the mux'ing to work. 2 common problems with DTivo show case the failures of the current code.

1- A hole. For some unknown reason there at times are a 1-4 second hole in the audio. Right now the mux'er would choke on that because it expects 1 second of video and 1 second of audio to show up in the Q at roughly the same time. And there would then in essence be 1-3 seconds too much video.

2- A reset of the PTS/DTS timestamps. Every now and again this happens. Apparently most often in the show BattleStar Galactica... :) The problem the math calculating the PTS/DTS values does not expect this. The end result? It would then be a very negative number and way out of the timeframe.

These are next on the list of fixes as that goes. The good news, the first 1 is easy to fix, a simple if/switch construct to allow the mux'ing of video only (which we did before during a testing phase, when there is a full second of video and no corresponding audio) would put us back in sync without fail.

For the second one it is a bit harder. But the same general process. I will have to add a routine that checks to see if the PTS is now < than the last one and try my best to determine what to do with it. The real problem is that when it resets it could potentially be off by different amounts from the video to the audio streams. :(

As for the next new feature it will be time for the DVD PrivateStream 2. Once that is in the code will be making a correct VOB and we should be able to move onto DVD creation.

I am not sure yet which will come first. The fixes or the new features. But one is sure to follow soon.

The released archive contains both the Win32 and Linux versions of the binary.

--jdiner

Fugg
11-10-2002, 09:41 PM
Looks great so far!
:D

mpauley
11-10-2002, 10:25 PM
Working great over here... 2000 chunks @ 480x480 has 2363 bytes of padding...

--mp

GITM
11-11-2002, 12:08 AM
Works great for my SA streams as well. They are all padded by about 4k.

Thanks for all the hard work Jdiner. There are always a couple *******s in a group. I believe most of us respect your right to do whatever makes you happy. We are happy with whatever you are willing to put out.

-GITM

dynodeano
11-11-2002, 12:29 AM
I ran one thru the mux and got 3490 bytes of pad.

They output looked good on first inspection.

I may setup a testbed tomorrow for processing a section of all of my ty files on disk. Would this be helpful or is your confidence level pretty good?:)

artships
11-11-2002, 02:06 AM
For the first time ever, tytool failed me. And in an episode of Andromeda, too. When Tyr returns from training, as he explains his mal-performance, he... ghosts and rainbows. Thereafter the audio is off 700ms. Happens at least once more in the episode.

But vsplit1k... Plowed right through (it's FAST), reproducing the visual glitches, but maintaining synch.

Ya did it, Josh. Thanks.

John
Sony 2K SA.

jdiner
11-11-2002, 03:17 AM
Originally posted by artships
For the first time ever, tytool failed me. And in an episode of Andromeda, too. When Tyr returns from training, as he explains his mal-performance, he... ghosts and rainbows. Thereafter the audio is off 700ms. Happens at least once more in the episode.


It sounds like there is a hole in the data. That would throw off the true splitting feature of TyTool and older VSplit versions, but the new one would maintain sync during mux'ing.

--jdiner

jdiner
11-11-2002, 03:20 AM
As for setting up major testbeds. Probably not worth it at this point. What I did today and over the last weeks or 2 is clean up what was already going on. I have verified that the byte-count remains perfect, that the PTS/DTS values remain in the correct range and never get to far out.

I have reworked the dolby audio just a touch and it is now as it should be in all aspects.

So a major testbed while nice is not what is really needed.

What I need at this point is for people to identify some problem streams. That way we can test the new... recovery code I guess would be the best name. But a way to test and see what is going on when the stream is not perfect. It should work around such problems and in some cases already does. But not in every case. And finding those where it has problems will be the best idea.

I have listed the 2 most common of these. But you never know what it will fail on. There are so many corner cases odds are I have missed one or 2 in the design of the mux'ing code.

--jdiner

jdiner
11-11-2002, 03:23 AM
Also, I just wanted to be clear on something. You still need to hang onto your TyStream files. While mux'ing is closing in on the finish line, it is not really finished yet.

I need to add the Program Stream finished packet. I just realized I forgot that one.

Also we need the vob information. Not everyone wants it. But until it is there, and if you are planning on DVD support don't get rid of them. If you do, there will be no easy clear cut way to get them added down the road.

So everyone hang onto your TyStream files for a bit longer.

--jdiner

tweaky
11-11-2002, 05:59 AM
All my clips play fine.. Thanks for sharing YOUR program....

Interesting sidenote: If you import the mpg into Tmpeg and go to the mpeg tools and try to merge and cut, it doesn't work...

newlooper
11-11-2002, 07:23 AM
Looking good here!

dynodeano
11-11-2002, 09:13 AM
Wouldn't it be easier to find more bad streams with a 'run-length' release. How about just adding a message at the beginning of the vsplit run indicating that the release is not ready for prime time or that is for testing only.

The errors which I see typically are in the middle 50% of the tystream and also towards the very end.

I have been doing software development for nearly twenty years in C and now Java.

I would be glad to assist with some more testing.

durian
11-11-2002, 10:22 AM
jdiner,
A 256MB Dtivo tyfile produces a 94MB mpg file. Here are some of the debug printout:

Found an OOB packet... The Video Diff is: 00:55:41.119
33333 -> + old Vid ByteCount = 68 91 D0 to 6A 7A F8
33333 -> + new Vid ByteCount = 6A 7A F8 to 6C 66 CC
33333 -> + WE ARE ALIGNED on the Video so it is good!!!
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
Unknown PES packet. Fix it dear henry...
type = 0x33
Unknown PES packet. Fix it dear henry...
type = 0xEF
Unknown PES packet. Fix it dear henry...
type = 0x5B
Unknown PES packet. Fix it dear henry...
type = 0x5E
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
Error. We did not find an embedded audio header!
..... 800......... 900......... 1000
......... 1100.......
Found an OOB packet... The Video Diff is: 00:01:57.233
33333 -> - old Vid ByteCount = 6A 7A F8 to 6C 66 CC
33333 -> - new Vid ByteCount = 61 39 14 to 63 19 0
33333 -> - NOT ALIGNED on the Video... It is an OOB chunk!
Found an OOB packet... The Audio Diff is: 00:01:57.144
Is it in sequence??? It is OFF by exactly 4881.000000 frames.


#1 55555 -> + old Aud ByteCount = 6F 53 94 to 6F 64 D8
#1 55555 -> + new Aud ByteCount = 6E B7 B0 to 6E D2 88
55555 -> - NOT ALIGNED on the Audio... It is an OOB chunk!
Nope... Not in sequence... Skipping it...


Is this the sign of a hole and/or are you interested in the clip?

Fugg
11-11-2002, 10:33 AM
It works great on all the files I have. No problem streams here!

... but I'm dying to find out if it will hold sync end-to-end across a 3gig multiple fsid sa high quality ty I have. It was extracted using the latest tytool and gradually looses sync, and gets worse the longer I let it run, as if it's loosing a bit of audio at every fsid transistion. It works great a thousand chunks at a time, but so did vsplit.

From a guy who is just looking for good, synced mpeg's, it looks like your there! I think your moving towards vob creation, right? Can you split off a version that just outputs good mpg's? That's the holy grail for me right now!

edit, clarification:
I'm finding the mpeg's created by your latest test muxer are, for my needs, Perfect! Almost 100% of what I'm recording contains commercials, so I'm dealing with blocks of less than 2000 chunks anyway to edit them out(in most cases). I'm using Tmpegenc to make gop edits, and with the latest test muxer, I'm getting pretty much uniform cuts every time. I'm playing back through windvd 3 and 4 as well as hardware mpeg cards with no problems playing or seeking, especially now with the latest version! I'm getting very little or sometimes not even a detectable delay in resuming playing after a seek.

Please don't take this as anything other than a progress report from someone who has not had one problem (and hasn't for a while!) with the files and was just feeling overly chatty this morning.;)

Thanks so much for all you've done. Your work is bloomin' amazin' and appreciated!!:D

sorry 'bout the spelling...

skorous
11-11-2002, 11:01 AM
Fugg,

I'm pretty sure that this came up once before (i.e. people that only want mpeg files not .vob's) and JD was going to add in a switch for that. Not sure if it's still in the works since it's been a while since it's been talked about but I'm bettin' it is.
< joke >If not, JD bribes pretty easy. < /joke >

As for the sync drift. I can tell you that I've seen the new mux code correct some really _ugly_ sync problems with a couple streams I have that are actually broken. The new code doesn't rely on perfect timing of the two streams to mux. It actually reads the PTS/DTS (sorry if terminology is wrong) time stamps to mux them. I had a stream that is missing pieces and as a result got off by like a second. It looked like a bad kung fu movie for about two seconds before it corrected itself. It's impressive.

Skorous

skorous
11-11-2002, 11:10 AM
Originally posted by dynodeano
Wouldn't it be easier to find more bad streams with a 'run-length' release. How about just adding a message at the beginning of the vsplit run indicating that the release is not ready for prime time or that is for testing only.

The errors which I see typically are in the middle 50% of the tystream and also towards the very end.



Dyno,

Yeah. It will be easier probably to find a lot more trouble streams once a full run version becomes available. Besides the obvious reason of being able to test a whole stream at one time there is a secondary reason. I think we all know as soon as a full run length version appears people (myself included) are going to start using it for production stuff. Yeah, JD could just ignore them but (IMHO) it's easier to ignore people that are just complaining that it's not fully functional. :-) All joking aside, JD does things on his own schedule. He's already said he's gonna release a full version as soon as he's ready.

Skorous

------------------------------------------------------------------------------
Since people have been pretty tetchy lately, I'm not trying to be a prick only expressing an opinion. Feel free to ignore it.
------------------------------------------------------------------------------

jdiner
11-11-2002, 11:45 AM
Originally posted by tweaky
All my clips play fine.. Thanks for sharing YOUR program....

Interesting sidenote: If you import the mpg into Tmpeg and go to the mpeg tools and try to merge and cut, it doesn't work...

Not to me sadly... I spent a lot of time trying other program to decipher things... Any cut or edit step has to remux things. Garanteed, no exceptions.

It is how it is remuxed that is the problem. tmpgenc rebuilds every PTS/DTS/SCR value in the whole stream and bingo everything is off just as it was before. the VFR is lost. It does tend to work pretty well for the SA Tivo stuff as that is not really VFR.

--jdiner

jdiner
11-11-2002, 11:49 AM
Originally posted by durian
jdiner,
A 256MB Dtivo tyfile produces a 94MB mpg file. Here are some of the debug printout:

..... 800......... 900......... 1000

Is this the sign of a hole and/or are you interested in the clip?

Yeah. Grab TyFileSplit and do a:

TyFileSplit inp.ty out.ty 700 50

That should be more than enough to figure out what is going on in that clip.

--jdiner

jdiner
11-11-2002, 12:03 PM
Ok. Sorry obviously I was clear enough. I meant to be but hey I knew what I wast talking about... :)

Ok. To find a hole, and I used that terminology on purpose. Do the following (this is just like before from way way way back when):

vsplit -n input.ty junk.m2v junk.m2a > junk.txt

Spefically use the first version of the #13 release that was pre-mux'ing. I did leave this part out. Sorry... Then edit the junk.txt file. It will be small so even note pad can be used.

The in junk.txt look for the word "hole" in the text. It will be part of an OOB line. A simple search for hole in the file will be good enough to find it.

the -n says no output for audio or video files, which makes it faster to un. The non mux mode, i.e. no -m flag, makes it ignore the mux'er step (but that needed version did not contain it anyway...) and run the whole thing from start to finish.

It will also ignore the reseting PTS/DTS fix that went into #13c I think. So I guess #13 from the sticky thread really is the best one to try.

Once you identify a problem either write it down for now, or actually use TyFileSplit to cut the segment of the file that contains the problem. 20-50 chunks is pretty much what is needed for things to run.

Do keep in mind that the result of a TyFileSplit call can be used as input to vsplit so it is easy to test it and see if what was cut contains the same problem.

Look for anything that is not a standard process. But keep in mind:

- Not every OOB is bad, it has to have the word "hole" in it.
- If it stops splitting and gives nothing but OOB reports to the end of the file (from more than the very end like always...) then you have found a problem.
- If you are not certain send me the short simple .txt output file and I will tell you what you have found.

--jdiner

Changed. To specifically reference the version #13 from the sticky thread...

Hi8
11-11-2002, 12:47 PM
I'm wondering as I look at my perfectly in-sync streams , dreaming about the day that the "chunk restriction" will be lifted...

Right now I have some great streams, and play great on my PC, but will the audio upsample take place during the mux process? I'm assuming the DTiVo audio is 32khz and must be upsampled for compliance for either DVD or VCD/SVCD.

Is this just a non-issue? I'm all for "direct2VOB" for those Showtime or HBO movies but what about those 1/2hour shows that fit prefectly on a SVCD? I guess I could put 5-6 of them on a DVD-R ?

anyway ... just wonder'in GREAT work jdiner! You are the guy that is making all this possible!

bato
11-11-2002, 02:07 PM
DTivo audio is 48khz
SA tivos audio is 32khz

SVCD is 44.1
DVD is 48

if target is DVD and you are using DTivo not need to change the audio.

durian
11-11-2002, 02:08 PM
Done, but not a hole, by your definition.



Originally posted by jdiner
Yeah. Grab TyFileSplit and do a:

TyFileSplit inp.ty out.ty 700 50

That should be more than enough to figure out what is going on in that clip.

--jdiner

alunj
11-11-2002, 02:49 PM
Hi Joel, you said the debug had been cut right back is the following expected? This is from a UK 2.5 SA stream
Plays just fine in Power DVD sync is good to end but there seem to be some clicks in the audio.

.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
$$$ The Aud Body is off! 2D 12 40 -> 2E BB 80 diff == 0xFFFE56C0 (-108864)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
2000


A/V Sync Offset: 13ms (i.e. plays 13ms early!)
MUX: Padded the output by 860 bytes...

sirfergy
11-11-2002, 03:49 PM
I second the SA Tivo Audio and resolution thing. I would love to have it as one step from Ty -> VOB, but the audio is at the wrong frequency and the resolution for all my current Tys are 480x480.

AlphaWolf
11-11-2002, 04:21 PM
I know pretty much null about mpeg whatnots, but just curious, what causes a buffer underrun? Is this what causes our audio to lose sync when we attempt to modify the mpeg file with any other programs?

mattdb
11-11-2002, 06:43 PM
Here are two files I did off my DTiVo

This one has AC3 "SpyGame"

Processing 'spygame.ty': (10 chunks per tick)
Detected Tivo Type: DTivo
Detected Audio Stream Type: Dolby Digital 5.1
Final standardAudioSize = 1552

Final standardFrameLength = 1536

Final standardAudioDiff = 2880 or 00:00:00.032
First Video PTS: 00:00:19.398
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
......... 1100......... 1200......... 1300......... 1400......... 1500
......... 1600......... 1700......... 1800......... 1900......... 2000


A/V Sync Offset: 10ms (i.e. plays 10ms early!)
MUX: Padded the output by 0 bytes...

This one is just normal. "Birds of Prey"

Processing 'birds.ty': (10 chunks per tick)
Detected Tivo Type: DTivo
Detected Audio Stream Type: MPEG Layer II
Final standardAudioSize = 492

Final standardFrameLength = 480

Final standardAudioDiff = 2160 or 00:00:00.024
First Video PTS: 00:02:28.989
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
......... 1100......... 1200......... 1300......... 1400......... 1500
......... 1600......... 1700......... 1800......... 1900......... 2000


A/V Sync Offset: 5ms (i.e. plays 5ms early!)
MUX: Padded the output by 0 bytes...

Matt

DeTer
11-11-2002, 07:14 PM
Excuse me for a very elementary question but I havn't been able to find the answer on my own so here goes. ......just exactly what is the command line to run vsplit1k. ???

Deter


PS: i think i got it with vsplit1k -m 20.ty junk.m2v junk.m2a

Rowan
11-11-2002, 07:25 PM
Originally posted by DeTer
Excuse me for a very elementary question but I havn't been able to find the answer on my own so here goes. ......just exactly what is the command line to run vsplit1k. ???

Deter


PS: i think i got it with vsplit1k -m 20.ty junk.m2v junk.m2a

type in "vsplit1k -h" without quotes and it will give you all of the avaliable options.

Rowan

DeTer
11-11-2002, 07:36 PM
Originally posted by Rowan
type in "vsplit1k -h" without quotes and it will give you all of the avaliable options.

Rowan

Rowan


Thanks for the reply, i have used the -h for help. I have used vsplit before but just can't figure out the muxing and what the file out put file should be. I ran the command line I posted above and ended up with and m2a & m2v......only the video had bytes.
Thx again
DeTer <-------kinda stupid......but not totally :) just new

Rowan
11-11-2002, 07:41 PM
Originally posted by DeTer
I posted above and ended up with and m2a & m2v......only the video had bytes.
[/B]

Yes, that is kinda confusing but total expected. The commandline interface still needs the m2a (or whatever you called it) file name there but it is not used. It is created but as you see it has zero bytes, the real muxed output is in the m2v file. Just ignore the other zero byte file when using the muxing option.

Rowan

NutMonkey
11-11-2002, 08:04 PM
Originally posted by Hi8
Right now I have some great streams, and play great on my PC, but will the audio upsample take place during the mux process? I'm assuming the DTiVo audio is 32khz and must be upsampled for compliance for either DVD or VCD/SVCD.

Audio is at 48khz. I guess the audio for SVCD's is supposed to be 44.1khz, but just for giggles I took the mux'ed mpeg file from vsplit1k and ran it through VCDImager, and it played on my friend's DVD player, though the audio was off by a second (which was not the case when I viewed the mpg file from my PC). I was going to run it through Nero or some other SVCD generating program to see if that would help.. It would be great to be able to just vsplit and burn, 5 minute SVCD's...

Kythorn
11-11-2002, 08:16 PM
48 KHZ is fine for DVD. The problem as I understand it is that SA tivo's record at 32 KHZ (I don't have one).

The only thing I'm positive requires 44.1 khz is redbook audio. DVD format is definetely 48 khz, and I think SVCD is also. I don't really know about VCD though.

jdiner
11-11-2002, 08:23 PM
Originally posted by Hi8
Right now I have some great streams, and play great on my PC, but will the audio upsample take place during the mux process? I'm assuming the DTiVo audio is 32khz and must be upsampled for compliance for either DVD or VCD/SVCD.
Someone else here on the forum has been helping me with the transcoding of the audio segment. It is not that hard a process just nothing something I have even had time to look at in anyway.

He is working over some code for me that will allow the users of vsplit/tytool to automatically change the bit-rate of the audio stream from 32 to 44.1 or 48 depending on what is needed and from 48 to 44.1 if needed.

Once that is in place the SVCD mode (2324 output chunks) output should run in just about any SVCD enabled dvd player.

Someone mentioned that audio is off by ~1 second. As near as I can tell from what I have read the problem is that you are out of the defined spec and it just doesn't like that.

When I tried it here the APEX I have did the same thing to me.

--jdiner

jpeden
11-11-2002, 10:16 PM
First off, gotta say "Great work, jdiner" -- I've been playing with vsplit for only a couple of weeks, and so far the progress made has been amazing!

That said, I've got a couple of anecdotes to share:

1) So far, on the streams I've tested, audio seems fully in sync. These are all from a 312 SA Tivo, recording at "good" quality (352x480x32kHz)

2) Seeking in software playback does not appear to work. At All. Tried Windows Media, WinAMP, ATI MMC... Have not yet attempted PowerDVD. Also CyberLink PowerDirector Pro 2.5 does not appreciate the files as much as it used to.

3) Being adventurous, I've been burning most output to DVD+RW using either the PowerDirector Wizard or ULead's MovieFactory... so far my Sony DVP-S300 has been handling OK. Output from vs1k which I tested this morning worked as follows:
* Took a 1GB .ty, chunked into four mpegs (2000 chunks each)
* Burned each of these as a separate chapter
* First chapter plays fine, sync looks very good -- Seek works for first couple of minutes UNTIL: About 5 minutes into the playback, I notice the play counter jumps backwards. At this point, seeking within the DVD fails
* The other chunks play video only -- no audio (but the audio plays fine on the PC). I'm not surprised much by this, since it's beyond the intended use of the latest version... but it had to be tried.


Eagerly awaiting the next release...

jdiner
11-12-2002, 01:50 AM
Hummm. Well on closer inspection and what little information I can find out there on the net. The answer to the remaining dvd NAV packet is to write one that just has all 0's in it.

Apparently Brent Beyeler figured this out. And it is from what I have read what bbmpeg does and what mjpeg does with it. But that makes little sense to me at the moment.

There are some pretty complex values needed. Makes me wonder if another tool is used to patch these values up after the fact. I just have no idea right now. Anyone got any more info of any kind?

--jdiner

jdiner
11-12-2002, 02:01 AM
Yeah it is obvious that there is a tool/technique I am missing here. Something to take the DVD formatted output from bbmpeg to something that can actually be used on a DVD. Anyone know? Anyone ever done it? Without more information I am a bit stumped... :(

--jdiner

FreydNot
11-12-2002, 02:15 AM
Originally posted by jdiner
He is working over some code for me that will allow the users of vsplit/tytool to automatically change the bit-rate of the audio stream from 32 to 44.1 or 48 depending on what is needed and from 48 to 44.1 if needed.


Josh, you've made my day! In-mux audio transcoding was the last piece I needed to get from SA to DVD with perfect audio. I'm thrilled to know its on the todo list somewhere.

I will wait patently for as long as it takes. I am very greatful for the code you have written (and are yet to write). Please don't interperate my words as pressure to hurry the code along. My intention is to do the opposite - give you words of encouragement and let things take their natural course.

I will continue to run my test clips against the test versions you post. Once testing includes 32khz to 48khz audio conversion, I'll have several more gigs of files to test :)

You da man!

Hi8
11-12-2002, 04:55 AM
Originally posted by jdiner
Hummm. Well on closer inspection and what little information I can find out there on the net. The answer to the remaining dvd NAV packet is to write one that just has all 0's in it.

Apparently Brent Beyeler figured this out. And it is from what I have read what bbmpeg does and what mjpeg does with it. But that makes little sense to me at the moment.

There are some pretty complex values needed. Makes me wonder if another tool is used to patch these values up after the fact. I just have no idea right now. Anyone got any more info of any kind?

--jdiner

jdiner;

you may want to wander over to Derrow's IFOEdit Forum (http://forum.doom9.org/forumdisplay.php?s=&forumid=50) he may not have wrote the book on DVD structure/spec but he sure did READ it a few times!

He's doing some pretty amazing stuff lately; The latest version of IFOEdit (.096 beta 1) does actually import m2v and m2a files directly from your vsplit - goes right to VOB creates a DVD TitleSet - and he's also starting to get Burning built in.

He's created a new application, called DVDBurn that uses the Nero SDK - which hooks anyone that has Nero 5.5.9.17 installed into burning right from that TitleSet.

He's un-available for the next week or so, but he might be worth shooting an email off to, when he returns.


just a note; I was able to burn DVD-Rs using this method perfectly, I'm now running into some lip-sync issues. I'm not sure what is causing it. Because Derrow has released quite afew changes in the last week, and I'm always working with 'new' streams , I haven't located the step that is making the difference yet. I'll be trying a few DTivo streams soon, I've only been playing with my SA streams at this point.

Pr.Sinister
11-12-2002, 12:30 PM
Hi JD,

I was trying to take a TY Stream and split it with the latest
version of Vsplitmux you posted. I was under the impression
that only the muxing was limited to 2000 chunks. The splitting
is also limited to 2000 chunks. Is this expected behavior?

Also, i was splitting the TY so that i could then use TMPGEnc
to mux the whole thing so my cousin could watch the last
3 Episodes of Alias with PowerDVD and that's when i noticed
the problem with the 2000 chunks. TMPGEnc would not
accept the elementary streams and i went to look at the
log.

Now even if there is a 2000 chunk limit, is it normal that TMPGenc
will not accept the files to mux together?

Just a heads up in case something is broken...

-Pr.

jdiner
11-12-2002, 12:49 PM
Yeah. It limits the splitting too. Basically it just sets the system to only be able to un 2000 total chunks. So you will need an older version to split it successfully.

I would fall back to #13c the original pre-mux version. Or perhaps the real #13 from the sticky thread would be better. That C version had the problems with the "should stop here" issue.

--jdiner

artships
11-12-2002, 01:16 PM
Originally posted by Pr.Sinister
TMPGEnc would not accept the elementary streams

Yes, TMPGEnc will accept a frame-served video.

Unmuxed:
Drop the ".m2v" file into DVD2AVI, select File->Save Project, creating a ".d2v" file. Feed TMPGEnc the ".d2v" and ".m2a" files.

Muxed:
Again drop the ".m2v" file into DVD2AVI, select File->Save Project, creating both a ".d2v" file and a file named something like "T01 DELAY 0ms.m2a". Feed these two files to TMPGEnc. But, don't bother to excise adverts. As Josh mentioned, the lovely, beautiful synch goes right out the window when you try to edit, rendering, for me, the long-awaited mux code utterly worthless. Oh, well.

John,
Who's remembering Josh talking about an GOP editor a long time ago.

snoopy
11-12-2002, 01:45 PM
It is confusing to me now, what your new release is supposed to encompass?

My understanding is that you are building an all-in-one tool that will extract, split, combine, and convert the ty streams, all in one simple step, to a format that can either be burned to a DVD or VCD, using a burning tool such as Nero, or any number of other packages that exist for such a task.

Is this a correct interpretation?

alunj
11-12-2002, 01:50 PM
Hi JD
did you catch my post a page back. the old vspit does not generate any errors on this stream (other than from the very 1st block) . The latest release generated a load of those errors, are they expected ?
Do you need a chunk of the stream? Am I worrying about nothing ?

Alun:)

sirfergy
11-12-2002, 02:19 PM
I was wondering if you would post a linux version of vsplit13c?

Thanks!

jdiner
11-12-2002, 03:46 PM
Originally posted by Pr.Sinister
Just a heads up in case something is broken...

Right you are. I knew that actually but had forgotten until now. I made a mistake in some of the code and it dumps out part of the mux'ing info even when mux'ing is not the goal. My bad... Soon to follow a fixed version that does not do that.

--jdiner

jdiner
11-12-2002, 03:54 PM
Ok. It is called 1l but it is just 1k with the just splitting, not mux'ing fix.

--jdiner

scarabus
11-12-2002, 04:39 PM
From my limited testing of 1K everything seems to be good. Even after editing in m2Edit the clips held sync. I was able to do frame-accurate commercial removal and maintain sync! :)

So the next stage is getting the DVD authored. From what jdiner has written I believe that he's looking to create VOBs from the MPEGs, but that doesn't help me because there isn't a frame-accurate VOB editor available (or at least none that I know of).

I want to author with Maestro since it gives you far more flexibility in terms of menus and controls, but only SpruceUp can import muxed MPEGs and so far I haven't found a demuxer capable of handling the VFR stream correctly.

So can anyone suggest a demuxing tool that may be able to cope with a VFR input stream?

snoopy
11-12-2002, 04:42 PM
It is confusing to me now, what your new release is supposed to encompass?

My understanding is that you are building an all-in-one tool that will extract, split, combine, and convert the ty streams, all in one simple step, to a format that can either be burned to a DVD or VCD, using a burning tool such as Nero, or any number of other packages that exist for such a task.

Is this a correct interpretation?

scarabus
11-12-2002, 04:54 PM
Originally posted by snoopy
It is confusing to me now, what your new release is supposed to encompass?

My understanding is that you are building an all-in-one tool that will extract, split, combine, and convert the ty streams, all in one simple step, to a format that can either be burned to a DVD or VCD, using a burning tool such as Nero, or any number of other packages that exist for such a task.

Is this a correct interpretation?

You missed out Editing Commercials.

The stated goal was to allow you to edit out sections of the stream, but only at the I-frame (GOP) level. Frame-accurate editing would require that you re-encode the GOP where the edit takes place, and so far that's not on the menu, so far as I understand it.

snoopy
11-12-2002, 05:19 PM
It is confusing to me now, what Jdiner's new release is supposed to encompass?

My understanding is he is building an all-in-one tool that will extract, split, combine, and convert the ty streams, all in one simple step, to a format that can either be burned to a DVD or VCD, using a burning tool such as Nero, or any number of other packages that exist for such a task.

Is this a correct interpretation?

laserfan
11-12-2002, 05:45 PM
Originally posted by rc3105
good grief people, read back a few pages instead of asking the same basic questions for the 50th time!...I agree w/Riley and am reminded of a couple things:

1. "Lead, follow, or get out of the way"

2. I recall being in an Engineering meeting once, in a foreign country which shall remain nameless, wherein I was the one to bring-up a question that had been discussed before but I had not understood, or couldn't recall the discussion. I was given a dirty look and my question was ignored.

In this forum we can't give each other dirty looks (and shouldn't make nasty replies), but Ignoring Some Posts might be the best way to handle them, methinks.

snoopy
11-12-2002, 06:03 PM
Well I cannot get good clarification from anyone, but I get the point. I will stop asking. Sorry.

Pr.Sinister
11-12-2002, 06:20 PM
Originally posted by laserfan
I agree w/Riley and am reminded of a couple things:

1. "Lead, follow, or get out of the way"

2. I recall being in an Engineering meeting once, in a foreign country which shall remain nameless, wherein I was the one to bring-up a question that had been discussed before but I had not understood, or couldn't recall the discussion. I was given a dirty look and my question was ignored.

In this forum we can't give each other dirty looks (and shouldn't make nasty replies), but Ignoring Some Posts might be the best way to handle them, methinks.

Ya maybe... but sometimes people don't notice that someone
isn't asking a dumb question and it is something that hasn't
been asked before. It is also in the interest of the project that
we might ask a question instead of telling jdiner there is a bug.

It's not a bug if it's intended to be that way. Now obviously i
don't know if one of these messages was aimed at me but
i was just trying to point out that the resulting streams
couldn't be imported into TMPGenc when the ones from
earlier VSplits could.

As for the 2000 chunk limit, i was just under the impression it
was only for the muxing and not the splitting. Now i have to
admit that if the hex editing of the vsplit binary was discussed
in this particular thread, i missed it. I never wanted to do that
anyway since i have my own method of testing muxing on
a whole TY file.

Sorry if i annoy people with my lame observations or comments
but i really am only trying to help. :confused:

-Pr.

AlphaWolf
11-12-2002, 07:29 PM
Originally posted by rc3105

to remove the 2,000 chunk limit in version k hexedit offset 0x53EF from D007 to FFFF


pladjurizer :p that doesn't remove the limit though, just sets it to 65535.

fredisdead
11-12-2002, 07:49 PM
Scarabus:

If you want to author in maestro, why don't you just use the elemental streams (video and audio) produced by vsplit? Maestro is intended to work with them. That's what I do.

The biggest problem is still the 'holes' which seem to cause the dreaded "temporal reference error" which will keep maestro from importing the video file.

I notice that vsplit1l is limited to 2k chunks on non-muxed output as well as muxing, this is a first..

jdiner: are there any improvements in the demuxing/hole processing over vsplit13c in the 1l release? If so, can you remove the limits for simple stream output?

scarabus
11-12-2002, 08:09 PM
Originally posted by fredisdead
If you want to author in maestro, why don't you just use the elemental streams (video and audio) produced by vsplit?

I do that for things that don't have commercials. But my ultimate goal is to get the whole series of Max Headroom on DVD with the commercial breaks edited out cleanly.

(Actually my goal is to have a copy that doesn't have the TECHTV logo and other crap, but that's too much to ask for. If they released it on DVD I'd be at the front of the queue.)

So far I've used Maestro to set chapter points before and after and automatically skip the commercials. And that works, but it's not clean. You get snippets before/after the chapters - they are only small but they are annoying.

I don't think I'm alone in believing that MUXing is not the ultimate solution, it's just the next step on the road to frame-accurate editing.

Pr.Sinister
11-12-2002, 08:25 PM
Originally posted by scarabus
I don't think I'm alone in believing that MUXing is not the ultimate solution, it's just the next step on the road to frame-accurate editing.

I'm with you there... I used to Video Capture all my shows
and put them to VCD/SVCD but i would edit out the commercials
1st and it was frame accurate. It looked very professional.

Once you are used to "perfection" it's hard to settle for less ;)

jdiner
11-12-2002, 11:28 PM
Originally posted by alunj
Hi Joel,

For the record. My name is Joshua. Not joel. :) Although I do have a neighbot named Joel.

Also for others... jdliner, note the L, is also incorrect. Just plain jdiner... :)

--jdiner

jdiner
11-12-2002, 11:48 PM
Oh man I hate Migraines. Right now I swear even my hair hurts. I have some information I want to post and some things I want to answer but for now I really gotta take a break.

If it doesn't let up soon I will just take some excedrin with caffiene(sp) and just give up on sleep tonight. But man that stuff can KO a headache for me like nothing else.

--jdiner

jdiner
11-12-2002, 11:51 PM
Originally posted by alunj

.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
$$$ The Aud Body is off! 2D 12 40 -> 2E BB 80 diff == 0xFFFE56C0 (-108864)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
2000


I should have taken a better look at this before. This is just my byte-counting code missing the fact that this is a reset in the byte-counting code. the 20 50 40 is a common reset value so it should have caught it. The 2d 12 40 is a wierd point to reset at though. Humm....

But this is informational only. It in no way impacts the splitting the or the mux'ing.

--jdiner

jdiner
11-12-2002, 11:55 PM
Originally posted by snoopy
It is confusing to me now, what your new release is supposed to encompass?

My new release is a bug fix release (1L). There was a recent release where I added the final packing of the Video in the mux'ing itself. Think of it as a shoe box. I was putting things in shoe boxes and finally reached a point where I needed to put only a little bit into a shoe box, the tail end of an audio or video Presentation Unit (in technical mpeg-2 terms) or frame. I was then adding according to the spec "padding bytes" These are bytes that are not data for playback. They are just placeholders that are later tossed. These things were originally taking up of 20-40% of the output file. Now they take up less than 20k. Meaning you can put more real shows on a single cd/dvd/hd.

That is all that is going on right now. There are many more new features to come. Just takes time. And no headaches...

--jdiner

jdiner
11-12-2002, 11:59 PM
Originally posted by sirfergy
I was wondering if you would post a linux version of vsplit13c?


Probably not. I might have that version of the source somewhere. But to be honest I would have to look for it. I would rather trying to figure out the new "bugs" that seem to have cropped up for some.

What was fixed in the 12c release was the reseting of the PTS values in the file. Every now and again the playback timestamp which I use to figure out what to keep and what to toss (the OOB data) is reset. From that point everything is in the wrong place...

As for the rest of it I broke something in 12c with the "stream stops here" check. It looked right but it is not. Not sure why. But we get them in the start middle and end of streams. So it can't be used...

So unless you have a reset'ing PTS problem in a stream the plain old vanila 12 will be enough.

Oh man. I knew I was not doing well. I said 12 everywhere but 13 is the version to be dealing with and what I meant.

--jdiner

jdiner
11-13-2002, 12:37 AM
I MEAN IT... EVERYONE STOP AND READ THIS ONE! READ IT CAREFULLY I WON'T REPEAT THIS AGAIN...

OK. I will try to be as clear as possible here. About where I think we are, why we are here, and where I intend to go next. Keep in mind that much of this is directed at DTivo users and not SA Tivo. Due to it's very nature the SA tivo stuff is much much easier to deal with it.

1- Mux'ing is not the end all be all... For some it is the end-point, being able to play with perfect sync on their PCs. This IS NOT the case for me personally.

It is merely a step on the road, albeit a very very valuable one. It was necessary to get here to figure out what was going on. Why nothing else was working right. Some of the problems I alone tested were over $3k a piece to purchase. How on earth where they getting it wrong?

It became clear. The DTV people are doing it techically "wrong". As long as it works for them they are happy and it does indeed. But for the rest of the world and the mpeg-2 specs it is plain old wrong.

We now know that, and we know how and why and what to do to fix it.

For those who wanted PC playback with the best packing we now have, GET THE HENCE... you have what you need.

2- DVD is my ultimate goal... I can't be more plain about this. I don't want a disk filled with mpeg-2 PS files that I need a computer to play. I want them to play in a DVD player. Even if it has to be a particular one I am fine with that personally. Most of the rest of you are 2. As long as it works "everytime" in your new/old/whatever DVD player.

Now again mux'ing was invaluable in getting this working. Don't get me wrong I am not done here yet. Not even close. But without what we now know we were never going to get there.

Why does sync suffer when you edit? What does it suffer when you try to go to DVD? What does it bite the dust even worse when trying to do both?

Variable Frame Rate! That is the long and short of it. Let me be perfectly clear. This is a DTV issue. By default there IS NO MEACHANISM in the MPEG-2 specs for dealing with VFR. It is not even legal. The fact that anything plays this is a marvel in and of itself. But it does so... SWEET!

The DTV stream gets around this problem by NOT having a linear frame rate setting for the timestamps. Literally there are 90,000 clock ticks per second. At the frame rate put in the file being sent by DTV, 29.97 frames per second, there should be 3003 clock ticks before the next frame is shown. 29.97 * 3003 = 89999.91 total ticks. Math and rounding and well you can see why it is off but it rounds nicely. :)

So why do these other tools get off? They read the 29.97 that is in the data, the SEQUENCE HEADER (one of the PES headers in the DTV data) and see that is is 29.97 and then just start grabbing 29.97 frames for each second.

Think back to when I was going over all of this, or read back if you are just tuning in now, due to the VFR we get 29, 27, 24 and a ton of other rates. If we have just 60 frames go by before they "should" we are a full 2 seconds off in the sync between audio and video at the end of the clip. If we lose 1 per second on a 42 minute show, 1 hour ep with commercials cut we lose 2520 frames, for a full 84 seconds off.

So anything that ignores the PTS values from a DTivo stream is garanteed for failure. Editors that just remux by looking at the 29.97 are doomed. DVD tools that take elementary streams and do the same are doomed. etc... etc... etc...

3- Why bother with making VOB files... When there are so many good tools why bother to do this? First off a VOB is just a Program Stream (PS) with 2 extra packets in it... So it is just a mux'ed file file DVD Private Streams 1 and 2. Think back the mux'ing discussion in segment #2. We need one that mux'ed correctly to have proper sync and playability.

None of the DVD tools out there are TyStream format aware, so none of them are going to do it right.

So why am I bothering? Because if I add just 2 more small pieces of data, 1 of which is the Dolby audio I have already done BTW, we get something that can be used on a DVD to play in a DVD player with perfect sync from start to finish.

That is all why I am bothering. I would love not to have to do it. I have no DVD specs. What little information I do have appears to be only partially correct.

But I do hope that everyone can see why I worked so hard on mux'ing. And what I hope to use it as a spring board to accomplish.

Continued...

jdiner
11-13-2002, 12:38 AM
4- But what about menu's and my favorite tool _____ for DVDs... I personally couldn't care less about menus. I want playback. Plain and simple. There are other tools that can be used to create menus. From what I have read the ifoedit tool from derrow is working hard at accomplishing this.

If you want Maestro or Spruce up or DVDit or ... perhaps none of this will help you. But IMHO this is the only path to take that does not include re-encoding.

If you re-encode you will make it. Plain and simple. By doing so you get rid of the VFR and other DTV'isms. But you lose quality and you spend more time.

If you do it my way, or at least the way I am trying like mad to build, then you will get the same quality as when you play from your DTivo but you might just have to give up fancy menu's etc...

5- Why build VOB's first... people can see where things are going. So they can test it and see if I am doing everything right. So they can try the various title, chapter, menu building tools to see what works best and we can lay down a path to make it slick for the masses as possible.

That might just mean getting some direct help from Derrow, olaf_sc, or people like them that are DVD wizards to add special hooks in my tools or theirs to get something that can be used as easily as possible to get the most possible etc... But that takes time. And the sooner we know the sooner we get there...

6- What about the hole issue people keep talking about... It is well known to me. I posted it first. Said it would not do it. I know why. I know what to do to fix it. But right now I wanted to make sure that the packed output with so little padding was working. Once that is done, and we seem to be there, we move on.

What is next are the special caese. The reset'ing PTS values. And the "holes" in the data. Both need to be addressed. My best guess is it will take me about 4 hours for each and once all is known about 50 lines of C source to have both problems solved.

So it is coming, but I have tried to be clear that I am doing this 1 step at a time.

7- Editing... This is without a doubt on the list of things to do. I want this one personally very badly! I intend to work on this while others are working on checling tools to do menu's, titles, etc...

I have put a great deal of thought into what is going on and hope that at some point soon it will be time to start this.

The mux'er has been built to deal with editing quite nicely. The plain old splitting engine can do nothing with it because of VBR and the fact that the audio and video have different starting clock values.

Hope this helps to clarify why I "wasted" so much time on the mux'er and where I plan to go next.

--jdiner

jdiner
11-13-2002, 12:41 AM
Wow. Posting here must have medicinal value. My headache is nearly gone. Of course the 4 advil might have something to do with it, but my vote is for the forum... :)

--jdiner

zqfmbg
11-13-2002, 01:00 AM
And I was going to say "Go get some rest already!" :rolleyes:

Anyway, I've noticed streams that tend to go horribly out of sync if you let stuff free-run (i.e. Space Ghost from Cartoon Planet) are just great with the latest mux code. I wasn't sure if I was seeing audio just a bit ahead of video but there really isn't any way for me to tell at the moment, besides going to MTV...

snoopy
11-13-2002, 01:13 AM
Get some rest already :)

Thanks for being such a good sport with all of us demanding leachers. Have a fantastic night/day etc!

jdiner
11-13-2002, 01:45 AM
Nah... The pain killers I took will make sleep a difficult commodity to come by tonight.

I figure out the rest of the VOB stuff. It is a piece of cake. :)

Now while what I have is a VOB file in layout the vob spefic entries are empty. Something has to be used on them. ifoedit? Anyone know?

That was surprisingly easy to add. I figure I will add an option to not do them as they expand the output size needlessly if you are not going to a VOB and then we can pretty much consider the VOB stuff done. Always nice when things work out quickly.

--jdiner

jdiner
11-13-2002, 01:54 AM
Ok. Keep in mind that I have never used it or seen it directly.

But I was just thinking about PlayItAgainSam. It is a Tivo based interface for the editing of streams. You use your remote and your TV and you edit the streams visually. Then the next time you play the stream you get only the parts you wanted.

Or at least this is what the docs seem to indicate.

However, currently using is makes it so that my programs see major holes everywhere and don't extract nicely at all.

But I was thinking about that. How many here would like to see editing as part of all of this done that way instead of as a program on the PC/Mac/Unix machine you download things to.

It would mean some major changes to it but all in call could be very very nice. It would depend on how easy PlayItAgainSam is to really use?!!? What I was thinking was rather than having it make real changes to the MFS instead it would save to a place on the /var/ drive (since it is writable) a list of "edits". Then using TyTool to extract and mux and "edit" during download (all options at that point) you would get a single mux'ed file that had the editing on it... What do you all think?

So anyone ever used it? And what do you peoples think? Client side editing or Tivo side?

--jdiner

jdiner
11-13-2002, 02:01 AM
Ok. I was thinking about why tmpgenc has as much success as it does with Dtivo output and here is what I am thinking. The mux'er in tmpgenc, as long as no cutting takes place, seems to work about 80% of the time. I was wondering why so I took a look. I believe I know what data they are looking at and that it can indeed be used to mux and get close. But it is not perfect... It is nowhere near as good as what we have right now. Why you might ask?

There is another time value in the file. It deals with GOPs themselves and the pictures within it. This value by no means has a good granularity, instead of in miliseconds it appears to be in seconds at it's finest. But it has a pretty close gap on that second boundry.

So you can use it figure out what is supposed to be "in" a seconds worth of video. It works. It really does. But it means that within each second, some times a little more and sometimes a little less, you "float" back forth around sync. All in all not a bad technique but a pain the butt to figure out. And I could still be wrong.

I am not sure that is what they are doing but it is the last of what I can "figure" they are doing. There is just no further data that could be being used. And using it would help fix "bad" Elementary Streams and tmpgenc definately seems to be doing that.

--jdiner

jdiner
11-13-2002, 02:05 AM
I would like to verify my stuff. But I need to know something about the tools people have been using todate.

The "dvd output" from bbmpeg. Are any of you using it and if so with what dvd tools?

What tools are people using, besides spruce up, for dealing with make dvds.

More specifically I am looking for something that will make a VOB file I can directly access.

I am open to any suggestions.

--jdiner

FreydNot
11-13-2002, 02:35 AM
Originally posted by jdiner
Ok. Keep in mind that I have never used it or seen it directly.

But I was just thinking about PlayItAgainSam. It is a Tivo based interface for the editing of streams. You use your remote and your TV and you edit the streams visually. Then the next time you play the stream you get only the parts you wanted.


The last time I looked at PlayItAgainSam, I get the feeling it won't work on my 3.0 Tivo. As a concept, its interesting though. There is something to be said about doing the edits using the already stored source device. However, there is also something to be said for leaving the Tivo free for family use while editing on the computer.

For me, I would go for whatever is easiest and/or most practical.

FreydNot
11-13-2002, 02:41 AM
Originally posted by jdiner
I would like to verify my stuff. But I need to know something about the tools people have been using todate.

The "dvd output" from bbmpeg. Are any of you using it and if so with what dvd tools?

What tools are people using, besides spruce up, for dealing with make dvds.

More specifically I am looking for something that will make a VOB file I can directly access.


I haven't tried the DVD output from bbMpeg. In fact, I haven't used anything that outputs VOB files at all. Most of the time I've been working in DVD Maestro.

Lately I've been trying out the new IFOedit's Create DVD mode. I suppose that creates VOB files.

IWantMyDTV
11-13-2002, 02:43 AM
Josh,

I know that this is a soft spot for you right now because of certain "forces" within the community attempting to rush you, but is the release of your code still a part of your project plan?

SirTrini
11-13-2002, 08:06 AM
It doesn't seem like you could get that accurate cut points on the DTivo.

laserfan
11-13-2002, 08:06 AM
To jdiner:

If I understand the PlayItSam concept (edit on the Tivo) correctly; not only would editing be in a relatively comfortable environment (one's sofa and the Tivo remote!) but it would theoretically reduce the time it takes to export/transport the file(s) to the PC. Fine, great even. But my take at this point is that it is a distraction--if you need a distraction, well, fine, but I might have guessed you would want to get some kind of a new release out-the-door that would get all of <us> off your back for a while at least...

Re: DVD tools, I have only ever used SpruceUp to produce, and sometimes Nero to burn. I have Maestro in reserve--my sense is that many others here use these as they are readily available for "free".

But I am 100% with you that I don't want to monkey w/Menus at all. I just want to get my programs to DVD and maybe have chapter markers for my Twilight Zone/Austin City Limits discs. That's all.

You know what you're doing and what you want to achieve. I'm not sure why you're as "invitational" as you are re: getting feedback but...well, you should do what gives you satisfaction. As Groucho Marx once said when asked if he was enjoying himself, "I always enjoy myself; it's others I have trouble enjoying!" So do what you want, when you want to do it.

keither
11-13-2002, 10:04 AM
Josh asked:


But I was thinking about that. How many here would like to see editing as part of all of this done that way instead of as a program on the PC/Mac/Unix machine you download things to.

I'd 100% request that editing NOT be done on the Tivo. I'd prefer to work on video at my PC.

I don't mind waiting for the TYStream to extract if I can work with muxing and splitting and editing on a machine that's 10x or more faster than the Tivo.

Plus, I think on a PC it's a lot easier to get repeatable results, especially down to the frame level. With the Tivo, isolating individual frames seems difficult at best. What I've found is that "standalone" solutions (CD recorders, video recorders, etc.) are never as full-featured or easy to use for detail work as a PC based solution.

I'm also trying to keep my Tivo as "unedited" as possible. All I do is load network drivers and noscramble. Adding in a whole bunch of other software that has to run on my Tivo doesn't appeal to me in the slightest. :)

Jellybean
11-13-2002, 10:34 AM
Originally posted by jdiner

What tools are people using, besides spruce up, for dealing with make dvds.

More specifically I am looking for something that will make a VOB file I can directly access.

I am open to any suggestions.

--jdiner

I use IFOedit to create a .VOB. Click on Dvd Auther->Author New DVD and it muxs 1 set of m2a and m2v files to 1 VOB. That file is playable via the Software DVD players on the PC. If you want to play it on a hardware player there is another step or two.

JB

mattdb
11-13-2002, 10:35 AM
1. I prefer to do my editing on my PC. The wife might wanna be on the TiVo.....

2. The only program that I have used to date that worked well making dvd's with the mpgs is Spruce Up. You have to patch the header and do the crash of spruce up, but you get a nice dvd with menus when done. "Pootie Tang" was my first and it plays just fine in my PS2.

3. Thanks for all the hard work jdiner. I really do appreciate it.

Matt

Hi8
11-13-2002, 10:38 AM
I would be happy to have anything to chop out un-wanted content .. But my 2-cents ... I recall playing with a WEBTV box for a week or so, and that is NOT a way to surf the www, nor a good email client. I'm thinking that anology may not be to far off, using a IR remote to edit video.

I've never used or seen PlayItSam perhaps I should explore in more detail before I pass judgement, but my gut votes for PC editting.

scarabus
11-13-2002, 11:21 AM
And what do you peoples think? Client side editing or Tivo side?

You're never going to get frame-accurate edits on the TiVo since it involves partially transcoding two GOP. So I'll always do my final editing Client Side.

keith721
11-13-2002, 11:28 AM
mattdb recently said:
"Pootie Tang" was my first
I still LMAO at that movie. Glad to see another TiVo user thought it worth saving.


and jdiner earlier said:
The pain killers I took will make sleep a difficult commodity to come by tonight.
Have you ever tried 'Percogesic' over-the-counter pain-killers? No caffeine, and great for when you just want to get some rest and relief from excruciating :( headaches.

Fugg
11-13-2002, 12:02 PM
editing?

PC side. The less the stress on my poor old overloaded tivo, the better.
:)

Rotten
11-13-2002, 12:55 PM
I have been playing with the new mpgs in DVD WS 1.3 and am able to produce a DVD with menu very easily, the one thing missing is the ability to have chapters. I am still working on this.

1. Mux with JDiner's vsplitmux L
2. import into DVD WS. Chapters will be erased if you make them
3. make menu with pointers to the titles first chapter
4. Save
5. Use DVD Patcher to change first header to 720X480 Variable (Max. 8000 kbps)

6. you will be asked if you want to relink files in DVD WS. Say no.

7. Create template as follows in DVD WS.

MPEG files
24 Bits, 720 x 480, 29.97 fps
(DVD-NTSC)
Video data rate: Variable (Max. 8000 kbps)
LPCM Audio, 48 KHz, Stereo

8. Make disc having Do Not Convert Compliant Files (on)
9. Thats it. it does not reencode the files and sync looks great.

I would like to have chapters for commercials but not too sure if it is possible yet.

jdiner
11-13-2002, 02:14 PM
Originally posted by rc3105
selective extraction would be a trivial addition to tserver_mfs5.

just document the interface calls 'n let somebody with experience writing video editors write the app, be it pc/mac/*nix or in-tivo.

That was exactly my suggestion a few weeks ago. I just had the other thought last night and I thought I would see what others thought of it.

--jdiner

jdiner
11-13-2002, 02:16 PM
Originally posted by Rotten
I have been playing with the new mpgs in DVD WS 1.3 and am able to produce a DVD with menu very easily, the one thing missing is the ability to have chapters. I am still working on this.

It all sounds great. But where does one get the DVD WS tool? Who makes it? Cost? URL? :)

--jdiner

edpuffmonster
11-13-2002, 02:16 PM
I know I'm repeating myself here, but it seems like adding "no change" P/B frames would be a reasonable thing to do -- then you have a fixed framerate mpeg that can be edited with anything. It should cause only a very minimal size increase and requre no re-encoding.

But I really don't know enough about the mpeg structure to know how difficult this would be... I could really make a mess of things if you need to add a frame in the middle of a bunch of B frames, but if the mpeg structures are right, it would deal with it.

Jdiner, Is this even possible?

bato
11-13-2002, 02:25 PM
Originally posted by Rotten
I have been playing with the new mpgs in DVD WS 1.3 and am able to produce a DVD with menu very easily, the one thing missing is the ability to have chapters. I am still working on this.

1. Mux with JDiner's vsplitmux L
2. import into DVD WS. Chapters will be erased if you make them
3. make menu with pointers to the titles first chapter
4. Save
5. Use DVD Patcher to change first header to 720X480 Variable (Max. 8000 kbps)

6. you will be asked if you want to relink files in DVD WS. Say no.

7. Create template as follows in DVD WS.

MPEG files
24 Bits, 720 x 480, 29.97 fps
(DVD-NTSC)
Video data rate: Variable (Max. 8000 kbps)
LPCM Audio, 48 KHz, Stereo

8. Make disc having Do Not Convert Compliant Files (on)
9. Thats it. it does not reencode the files and sync looks great.

I would like to have chapters for commercials but not too sure if it is possible yet.

So you keep it patched at 720x480?

jdiner: Ulead sell the product around 299 I beleive. Just visited the site and only show ver 1.2, maybe they have a patch to 1.3 EDIT: yep, ver 1.3 link right in the buy product page.

alunj
11-13-2002, 02:26 PM
Hi jdiner
here is a link for dvd ws trail
http://www.ulead.com/dws/trial.htm

About $220

1.3 update is on the buy page
http://www.ulead.com/tech/dws/dws_ftp.htm

Alun

jdiner
11-13-2002, 02:38 PM
Originally posted by edpuffmonster
I know I'm repeating myself here, but it seems like adding "no change" P/B frames would be a reasonable thing to do -- then you have a fixed framerate mpeg that can be edited with anything. It should cause only a very minimal size increase and requre no re-encoding.

But I really don't know enough about the mpeg structure to know how difficult this would be... I could really make a mess of things if you need to add a frame in the middle of a bunch of B frames, but if the mpeg structures are right, it would deal with it.

Jdiner, Is this even possible?

Ok. I looked into this but I must never have posted my findings. The simple answer is no, this won't work.

The reason/problem is this:

The frame order is correct. The frame types are correct. The frame presentation duration is what is wrong.

Meaning it really does go IBBPBBPBBP in frame order. Which is correct and required. It just does

1 1 1.5 1 1.5 1.5 1 1 1 1 1.5

etc... in duration of how long to play each frame. So if we "padded" frames into the mix. We would get a duration list of

1 1 1 1 1 1 1 1 1 1

which would be great, But we would get

IBBBPBBBBPBBPBBPBB

and that is just wrong. What that really messes up is the prediction. A frame does not look for the "right" frame when predicting, it does a simple count. Just forward 1, back 1 and back to the previous I frame etc...

I am not saying it could not be done. I am just saying that it would be a literal nightmare to make it work. Truely re-encoding would be a better/simpler idea but would have time and quality issues. :(

What is really needed is an intelligent re-mux'er. Which we might be able to work with Derrow to accomplish. I could do it, if I could just find out what really needs to go into the new Private Stream 2 PES packets.

What it needs to do as all of this is being built is to take the information from the current mux and use it in the output. Rather than just re-calculating PTS/DTS timestamps and the SCR it needs to use what is in the file to a large extent and then just add what is new (the private stream 2 stuff). If someone/something is doing that then we are there. But as near as I can tell no one does.

--jdiner

alunj
11-13-2002, 03:05 PM
Aplology time.
I keep calling jdiner joel ... I dont know why :eek: I seem to have a mental block .... Sorry
But its jdiner from now on !

Alun

jdiner
11-13-2002, 03:27 PM
Alun,

if you are aroud right now let me in and I wll fix that UK issue.

--jdiner

Rotten
11-13-2002, 05:20 PM
Originally posted by bato
So you keep it patched at 720x480?

jdiner: Ulead sell the product around 299 I beleive. Just visited the site and only show ver 1.2, maybe they have a patch to 1.3 EDIT: yep, ver 1.3 link right in the buy product page.


Exactly, works pretty good just wish I could get chapter points. Anyone know a way to put them in after a title set is made.

jdiner
11-13-2002, 05:57 PM
Just to be clear did this work with an SA Tivo stream or a DTivo stream.

I never intended that particular distinction to become so unclear in this thread but it kind of has.

For an SA Tivo stream I am not surprised that it works, for a DTivo stream it would mean that they did it right and that would be truly great news after all.

--jdiner

Rotten
11-13-2002, 06:06 PM
Originally posted by jdiner
Just to be clear did this work with an SA Tivo stream or a DTivo stream.

I never intended that particular distinction to become so unclear in this thread but it kind of has.

For an SA Tivo stream I am not surprised that it works, for a DTivo stream it would mean that they did it right and that would be truly great news after all.

--jdiner

It was a DTivo stream.

I would like to try it with a 2 hour stream can you send me a copy without the restriction of time.

mikertx
11-13-2002, 06:19 PM
Originally posted by jdiner
I would like to verify my stuff. But I need to know something about the tools people have been using todate.
...
--jdiner

I've used the dvdauthor tools on linux. See
http://dvdauthor.sourceforge.net/
"ifogen" needs an mpeg2 file with VOBU's but they can be invalid or empty (sounds like you've got this already) and it will fix them up and generate the IFO file. Plus it's open source so you can easily see what the author is doing.

I finally got a DVD-R/W drive last week (oem Pioneer A04 for $199 at Fry's). When you are ready to post a version with the VOB holes I'll give it a try.

--mikertx

mattdb
11-13-2002, 09:00 PM
Rotten,


Originally posted by Rotten

8. Make disc having Do Not Convert Compliant Files (on)
9. Thats it. it does not reencode the files and sync looks great.

I would like to have chapters for commercials but not too sure if it is possible yet. [/B]

When I try to use this program it doesn't like the mpg file.

I get an error that states that the file is not compatible with the program.

Matt

JasonJLee
11-14-2002, 10:05 AM
Hi All,

Just thought I'd second what someone else has already said on another thread...

Are Threads that have over 1000 posts really a good idea?

Jdiner, would you consider creating a new thread stating the curent position, along with any ideas that people have mentioned in this thread that you are working with..?

Just might make it easier on new guys rather than trying to wade thru 1000 posts for the important points?

Just my 2p's worth..

J

IWantMyDTV
11-14-2002, 10:14 AM
The problem is, due to the nature of this project, there would be 1000 small threads to look through rather than 5 or 6 (yes this this thread has been continued in a new thread several times before this one) large threads to peruse.

It's the lesser of two evils. :)

jdiner
11-14-2002, 01:30 PM
I happen to agree with the last sentiment. I tried at first to have several small ones dealing with different issues of tyTool etc... And no one was seeing everything. Only bits and pieces. So I changed my methods and put as much as possible into 1 place. That way people could read it all.

It didn't work like I had planned. People just hit the last page button and never seem to read anything, but at least they know where to look when they decide to. I don't know that there is a better way to do it all.

--jdiner

icculus
11-14-2002, 04:32 PM
Hello... I've been lurking for a bit, playing around with the various extraction tools. First of all, it's obvious that jdiner is doing some great work, and thank you!

I am just trying to sort out the current Best Thing to do with my archived shows.

I've got most of Greg the Bunny and The Tick still sitting on my SA Tivo. I'd like to get them off and free some space.

My long term goal, of course, is to get them onto DVD, but I'm not (yet) having much luck, and am perfectly willing to wait for the tools to mature a bit more.

But do y'all think extracting them in .ty format will be sufficient for future archiving onto DVD? I want to make sure I've got a complete copy of the files before I wipe them from my Tivo.

What tool is currently considered the best for this? I've played around with both TyTool5 (and the included tserver_mfs5), and with TivoApp (and included ExtractStream).

I'm sorry if this has been answered recently, but I've been digging through this thread for a couple of days and my head is still spinning.

thanks!

Hi8
11-14-2002, 04:56 PM
Originally posted by icculus
Hello... I've been lurking for a bit, playing around with the various extraction tools. First of all, it's obvious that jdiner is doing some great work, and thank you!

I am just trying to sort out the current Best Thing to do with my archived shows.

I've got most of Greg the Bunny and The Tick still sitting on my SA Tivo. I'd like to get them off and free some space.

My long term goal, of course, is to get them onto DVD, but I'm not (yet) having much luck, and am perfectly willing to wait for the tools to mature a bit more.

But do y'all think extracting them in .ty format will be sufficient for future archiving onto DVD? I want to make sure I've got a complete copy of the files before I wipe them from my Tivo.

What tool is currently considered the best for this? I've played around with both TyTool5 (and the included tserver_mfs5), and with TivoApp (and included ExtractStream).

I'm sorry if this has been answered recently, but I've been digging through this thread for a couple of days and my head is still spinning.

thanks!


can't answer for sure for each persons preference .. but ... I have both an SA and an DTivo :-) I can now extract via TyTool5 (making sure I use the Resource Editor's mod recording format to 720x480 5.8m -- and can go use vsplit13c Besweet to upsample audio to 48khz. and IFOEdit .096B1 to burn right to DVD.

I also burn to SVCD Seinfeld episodes... daily.. 2X from superstation. --- VERY good quality .. 1 disc per episode.


good luck.. I'm burning a DVD from my DTivo extract .. now I should know soon if it has a lipsync problem... I'm not expecting it to work.

icculus
11-14-2002, 04:59 PM
Originally posted by Hi8
I can now extract via TyTool5 (making sure I use the Resource Editor's mod recording format to 720x480 5.8m -- and can go use vsplit13c Besweet to upsample audio to 48khz. and IFOEdit .096B1 to burn right to DVD.

Well, all of these episodes were recorded long before I had Tivonet or anything on my box... and also long before 3.0. I think the oldest one is over a year old.

Hopefully I can find a way...

Hi8
11-14-2002, 05:37 PM
Originally posted by Hi8
I'm burning a DVD from my DTivo extract .. now I should know soon if it has a lipsync problem... I'm not expecting it to work.

well, I'm Happy to report that my burn is looking GREAT!

I've previewed in PowerDVD audio sync looks good, and Nero is burning as I type.

I really can't believe it is working. I have used the same .bat file I used for my SA TYstreams --

it even went thru the upsample audio to 48khz, which I was told it already is. But I was too lazy to make a new .bat file to skip.


my DVD has commericals... but "I don't Care!" I never saw Scarface, (I love Pacino) I'll hopefully FF through them ...!

more to come...

Pr.Sinister
11-14-2002, 06:56 PM
Originally posted by Hi8
my DVD has commericals... but "I don't Care!" I never saw Scarface, (I love Pacino) I'll hopefully FF through them ...!

more to come...

Please tell me you're kidding when you say you never saw
Scarface and are planning on watching an "Edited for Television"
version!

Noooooooo! Don't do it man! Much too good of a movie to
ruin it by watching it from a network feed.

If you had TiVo'ed it from a Premium Movie Channel then i would
say go ahead and enjoy it, but network TV??? Come on man
buy the original! Or wait for it to show on Encore o something!

Can't wait that long? Then if you are a US Resident, go to
Netflix.com and rent it man!

If you still don't want to do that, i'll FedEx you my store bought
DVD, you watch it, then you send it back to me...

Hehehehe...

-Pr.

Hi8
11-14-2002, 06:58 PM
ok ... I'm surprized as the next guy...

I have just burned a DVD-R with my A04 from my SAT T-60 using Tytool5r2->vsplit13c-Upsample=48khz-via BeSweet->scarface.m2v & scarface.mp2 -> IFOEdit.096B1-Nero5.5.9.17(BURN-BABY!)

it is perfect! DTiVo2DVD-R

jdiner (PM me if you are interested in a copy of the DVD-R!)

I owe you that much and much MORE!

Hi8
11-14-2002, 07:08 PM
Originally posted by Pr.Sinister
Please tell me you're kidding when you say you never saw
Scarface and are planning on watching an "Edited for Television"
version!

Noooooooo! Don't do it man! Much too good of a movie to
ruin it by watching it from a network feed.


NEVER! can't wait ... I can't lie ... I will probably watch it from TiVo - FF works so much better than my DVD FF.

but no , never saw it, and it came on TNT , almost bought the special edition off Ebay .. did'nt want to spend the $24.95.

I just wanted something to extract and BURN....

I still can't belive it!

it looks(and sounds) pretty GREAT!

(oh yeah forgot to mention .. I'm a NETFLIX sub'er it's NOT available ... now LOOONG wait!) it's on my list!

jdiner
11-15-2002, 02:18 AM
Originally posted by Hi8
ok ... I'm surprized as the next guy...

I have just burned a DVD-R with my A04 from my SAT T-60 using Tytool5r2->vsplit13c-Upsample=48khz-via BeSweet->scarface.m2v & scarface.mp2 -> IFOEdit.096B1-Nero5.5.9.17(BURN-BABY!)

it is perfect! DTiVo2DVD-R

jdiner (PM me if you are interested in a copy of the DVD-R!)

I owe you that much and much MORE!

Dude! Hang onto your files. There are a few things I would like you to check in the next little while. The in-line audio upsample code etc... It would just be really interesting to see how things line up without using the BeSweet program.

But as for scarface. I never saw it. What is it? Movie? Episode of something? Horror flick? (I never watch those... :)

--jdiner

jdiner
11-15-2002, 02:24 AM
Originally posted by IWantMyDTV
Josh,

I know that this is a soft spot for you right now because of certain "forces" within the community attempting to rush you, but is the release of your code still a part of your project plan?

To be honest. I don't know. I planned on it. But I have gotten so tired of being harrassed about it. That in anger I made a statement about never releasing it.

Then I calmed down and figure at the very least when I am truly done I will release the source.

Then rc3105, alphawolf, and a few others made so reasonable cases for the release of my source to help other projects out. And I have been thinking about that alot lately.

I have just not decided yet what I want to do. I tell you what. When I have decided I will make a public announcement on the forum detailing my plans.

If there is something specific that you want to know or a reason you want to provide one way or the other then feel free to PM me and I promise to read it. :)

--jdiner

jdiner
11-15-2002, 02:25 AM
Originally posted by SirTrini
It doesn't seem like you could get that accurate cut points on the DTivo.

From what I have read it GOP level accurate which ties in perfectly with what I am working on.

--jdiner

jdiner
11-15-2002, 02:27 AM
Originally posted by Jellybean
I use IFOedit to create a .VOB. Click on Dvd Auther->Author New DVD and it muxs 1 set of m2a and m2v files to 1 VOB. That file is playable via the Software DVD players on the PC. If you want to play it on a hardware player there is another step or two.

The problem is that once again this is an external mux'er not aware of what is going on within the tivo stuff itself. For the SAtivo this will work most of the time. For the DTivo it will almost never work.

I was and am looking for tools that deal with already made, even though only partially correct, VOB files. Not another way to make them.

--jdiner

jdiner
11-15-2002, 02:34 AM
Ok. Well I will assume from the silence that the latest mux'er works as expected. And further that the editing votes are in.

It was pretty unanimous to do it PC side and leave the Tivo alone. I am fine with that. I think it benefits from being PC side as well. I just thought that I would mention my thoughts and see what other thought.

There was a suggestion to use the visual/editing facilities of VirtualDub and have a frame server option to do it with. The only problem with that is the cost of doing. The main cost being limiting the program to Windows... There are many here who do this all with Linux/unix and a good custom program could have a simply GUI wrapped around it pretty easily for other platforms.

So we will go with something that is all custom so that hopefully everyone can get in on it. Any volunteer to port/rebuild what I will have to other OSs? :)

I have been working on the next stage with the mux'er closing things out better and adding support for holes and reseting PTS values. Some of these are easy to deal with. Some of parts of them are easy to deal with and some are quite difficult.

--jdiner

jdiner
11-15-2002, 02:44 AM
The reset'ing PTS value in 1 clip does this:



End of chunk 27:
PTS CHECK: Video == (445506605) 01:22:30.073
PTS CHECK: Video == (445500598) 01:22:30.006

Start of chunk 28:
PTS CHECK: Video == (108964) 00:00:01.210
PTS CHECK: Video == (99954) 00:00:01.110

And the audio in chunk 28 does this:
PTS CHECK: Audio == (445500346) 01:22:30.003
PTS CHECK: Audio == (445503586) 01:22:30.039
PTS CHECK: Audio == (97542) 00:00:01.083
PTS CHECK: Audio == (100782) 00:00:01.119


The problem... What to do with the PTS values. The simple math I was using to relocate every PTS value no longer works.

Other examples I have do the same with the video value. I.e the PTS values are reset in the same block rather than on a boundry.

The splitting code has been updated to work on these in terms of detecting that it is real data. But as I mentioned it is a mess.

If the 01.083 is assumed to be in reality 1:22:30.075 then this could be used to determine what the reset value offset really was. Hummm. The regularity of the audio step may be the key to calculating this value. Of course that leaves us making the assumption that the reset does not impact the step value between audio packet PTSs...

I will look into this some more but at least now I have an idea about how to proceed on this.

--jidner

jdiner
11-15-2002, 02:46 AM
The hole is easier. I have many fewer samples of this. But one I got from someone here is a clip from Charmed that had an almost 4 second audio gap.

The fix here is to just check and if there is 1 full second of video and it all starts and ends before the first audio record in the queue then we have a hole and we mux around it.

This one is a piece of cake compared to the other problem.

--jdiner

daa
11-15-2002, 03:41 AM
Originally posted by jdiner
The problem is that once again this is an external mux'er not aware of what is going on within the tivo stuff itself. For the SAtivo this will work most of the time. For the DTivo it will almost never work.

I was and am looking for tools that deal with already made, even though only partially correct, VOB files. Not another way to make them.

--jdiner

I've been using tytool in vsplit mode to get separate files from my DTivo which are muxed by my DVD authoring program, DVD Studio Pro on my Mac. This seems to be 100% reliable in producing DVDs that play back in sync on my Pioneer player. Am I just lucky that this works as well as it does? If you'd like some sample VOB files, let me know.

Doug

Hi8
11-15-2002, 04:26 AM
Originally posted by jdiner
Dude! Hang onto your files. There are a few things I would like you to check in the next little while. The in-line audio upsample code etc... It would just be really interesting to see how things line up without using the BeSweet program.

But as for scarface. I never saw it. What is it? Movie? Episode of something? Horror flick? (I never watch those... :)

--jdiner

well; I did delete my files from my PC, disc space is ALWAYS an issue for me. BUT I do still have it on my SAT T-60, so I can extract again with TyTool.

BeSweet, upsample , yes I did use it - as I stated in my earlier post. I just tried another stream, without it and there is an audio sync problem with that TitleSet.

I have a few more streams to play around with. So we shall see.

Scarface is NOT a horror flick, I can't watch them either..
"A Cuban refugee (Al Pacino) builds a crime empire in Florida."


(edit)

just wanted to follow-up on my success, my original "perfect" burn, I have found has the same video problem on my Sony S7700 that the disc sI burn using the DVDPatcher proceedure. I'm assuming it is the 352x480 thing. It just doesn't scale properly on my Sony. However it plays properly on my Apex, Nextbase, Koss, and PC without a problem.

I'm not sure why , if it is true that 352x480 IS really a valid DVD compliant value. ( I think that's what somebody said it was)

(/edit)

Hi8
11-15-2002, 04:52 AM
Well I've authored another couple of TitleSets, using the same proceedure, and it is too early to tell, but the two streams that were perfectly in-sync were from the superstation. "Scarface" and "CopyCat" another I tried was WAY outta sync, that was off Showtime "Shadows and Fog" - seems strange perhaps it makes sense to someone else? I heard mention that each channel on DTV uses different compression rates, perhaps that is effecting the way TyTool save the files?:confused:

(edit - update)

well that theory didn't seem to pan out... I just did two more TitleSets with IFOEdit .096B1 - and they are all perfectly sync'd.

1 from AMC, 1 from SHO. These sets were created from files NOT upsampled with BeSweet. One confusing thing is all the software I have reflects a 480X480 video, where I thought DTivo created 352x480 ??

skorous
11-15-2002, 08:27 AM
Originally posted by Hi8
One confusing thing is all the software I have reflects a 480X480 video, where I thought DTivo created 352x480 ?? [/B]

I think it depends on the channels. I thought the opposite, that it was always 480x480 , until EdPuffMonster set me straight. Not that I remember which are which mind you, but then again, I don't have DTV... :D

Skorous

Jellybean
11-15-2002, 10:21 AM
Originally posted by jdiner
The problem is that once again this is an external mux'er not aware of what is going on within the tivo stuff itself. For the SAtivo this will work most of the time. For the DTivo it will almost never work.

I was and am looking for tools that deal with already made, even though only partially correct, VOB files. Not another way to make them.

--jdiner

Hi,

The other steps have to do with making the VOB perfect so that it can be burned to DVD-R and you can do all the normal Hardware DVD player things and are all done within IFOEdit.

Did I not understand your original question?

jdiner
11-15-2002, 12:19 PM
Originally posted by Jellybean
Hi,

The other steps have to do with making the VOB perfect so that it can be burned to DVD-R and you can do all the normal Hardware DVD player things and are all done within IFOEdit.

Did I not understand your original question?
Yes.

ifoedit does the mux step. It is external. Some one was refering before to a mac program that works most if not all of the time. But I have used ifoedit myself. IT IS NOT AWARE OF THE VFR NATURE OF DTIVO... So in my tests it was off. Way off. I was looking for the list of steps using manual tools at each stage. i.e.

1- You get a raw incomplete vob.
2- You run ifogen, ifoedit, whatever to clean up and fill in the the vobu "blanks" in the vob file.
3- You ... the vob file using .... and the end result is a ready to roll vob file.
4- You then take the finished vob file and build an ifo file for it using...
5- You then build/burn/whatever the whole mess to disk using...

That way I can begin to see how well what I am working on now actually well works.

I don't need something that makes vobs in 1 step all internally. I need to figure out how they get made correctly. And a finished output doesn't help.

--jdiner

jdiner
11-15-2002, 12:27 PM
You know. It is really sad when you get stuck in a mind set and look for ways to smash what you see into that mind set. I had a realization this morning that I had done exact that.

It does with the "hard" way of dealing with a reset PTS value. Here is what came to me this morning. It doesn't matter if they line up.

Think about when your dvd player switches layers. Mine... stumbles for a split second. Almost a real pause and then continues. Annoying? Not really. Not anymore anyway. I am so used to it that I rarely even notice.

Even if the calculation for the new time stamps don't line up with the old. They line up with each other. So we get a slight pause while things re-adjust to what it was reset to. From there on out the playback will line up. So with 1 little pop, everything else will be fine.

Since the reset of the PTS value seems to be so rare I figure that for now that is good enough... :) And the hardest one just became easy.

--jdiner

sirfergy
11-15-2002, 12:37 PM
I have run into an odd problem with the muxing. I was a bad boy and changed the hex value to FFFF, and on one of my shows, after about halfway through, it starts eating memory until it finishes. Unfortunately, when it starts that, it is no longer writing to disk, so I have an incomplete mpeg. Have you had a similar issue? And when I say eating memory, it's as if it's writing the results to memory instead of to disk, so it ends up using like 500mb of memory.

BTW, it does in on both windows and linux versions.

JasonJLee
11-15-2002, 12:47 PM
Just a quick question..

I'm using vsplit1k on a westwing episode at high quailty from a UK SA Tivo. (edited the size limit out as people have previously mentioned)

The file is about 3 Gig.

This is the start of the output....


Processing 'd:\thewestwing-thetwobartlets.ty': (10 chunks per tick)
Jumping 1 chunks into input (131072 total bytes)
Detected Tivo Type: Standalone
Detected Audio Stream Type: MPEG Layer II
Final standardAudioSize = 880
Final standardFrameLength = 864
Final standardAudioDiff = 3240 or 00:00:00.036
First Video PTS: 00:00:01.210
MUX1 Found the B3-B8 header block and the 1st PES header block I1-frame
MUX1 Video Embedded PES Header: (108966) 00:00:01.210
MUX2 Now we have the almost full first packet (lead-in missing), and the header
for the B2-Frame...
MUX2 lead-in skip == 2
MUX2 PTS = 108966 == 00:00:01.210
MUX2 Video Embedded PES Header: (101765) 00:00:01.130
MUX Found the 3rd PES header block (B3-frame)
MUX3 there are 3 lead-in bytes for the P frame.
MUX2 Video Embedded PES Header: (105364) 00:00:01.170
MUX Found the 4th PES header block (P4-frame) - We DO NOTHING with the data!
MUX2 Video Embedded PES Header: (112565) 00:00:01.250
MUX Have the body for P4-frame and Header for B5-Frame.
MUX5 Video Embedded PES Header: (116165) 00:00:01.290
MUX-add: Audio PTS: (110485) 00:00:01.227
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
$$$ The Aud Body is off! 2D 12 40 -> 2E BB 80 diff == 0xFFFE56C0 (-108864)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.$$$ The Aud Body is off! 2D 12 40 -> 2E BB 80 diff == 0xFFFE56C0 (-108864)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)
.### The Vid Body is off! 20 50 40 -> 2D 12 40 diff == 0xFFF33E00 (-836096)

Not sure if all these messages mean something is actually wrong or not..? I did see someone else post something that I think looked like this a few pages back, and Jdiner said it looked a little odd.

Well, this post may at least confirm the UK Tivo differences.

The final output is Padded by 11066 bytes and works fine with everything I've tried it with. Nice job jdiner.

Takes a while to process....I haven't timed it or anything, but someone here mentioned its really quick. Didn't seem that quick to me, but its a while since I've extracted anything. But it could be these messages aren't normal, and slowing things down?

Anyway, nice job jdiner..

J

dlang
11-15-2002, 12:52 PM
any chance you are running into a 2G filesystem limit?


Originally posted by sirfergy
I have run into an odd problem with the muxing. I was a bad boy and changed the hex value to FFFF, and on one of my shows, after about halfway through, it starts eating memory until it finishes. Unfortunately, when it starts that, it is no longer writing to disk, so I have an incomplete mpeg. Have you had a similar issue? And when I say eating memory, it's as if it's writing the results to memory instead of to disk, so it ends up using like 500mb of memory.

BTW, it does in on both windows and linux versions.

sirfergy
11-15-2002, 01:38 PM
Um, not for both windows and linux. :) The file ends up being only 800mb in size.

koreth
11-15-2002, 01:56 PM
My bet is that you've caused a counter to wrap around to negative somewhere, in a way that won't happen when the time limit is removed completely.

I would hope jdiner won't waste even an instant looking at this problem, since it's not a bug in the code he released, but rather one you probably introduced the moment you took a hex editor to his executable. Unless you really know what that change actually does to the code, as in you've disassembled the binary and walked through it enough to understand what side effects changing that value might have. In which case, my hat's off to you.

sirfergy
11-15-2002, 01:59 PM
I concur, was just posting mostly a heads-up.

gadgetboy
11-15-2002, 02:51 PM
[a madman's rant, I apologize.]

Fugg
11-15-2002, 03:31 PM
Originally posted by JasonJLee
Just a quick question..
...
J

Have you tried to process the file using an unmodified vsplit1k?
The weirdness seems to start right at the beginning, so if it's the ty that's hosed, you should see the prob just like the section you posted.

If it runs fine with the origional vsplit1k, the probs in your modded file.

How big was the input ty and how big was the resultant mpeg? jooc....

i reserve my rights to deliver a spanking for looking for support for modded files at a later date
;)

JasonJLee
11-15-2002, 04:25 PM
Fugg

It does this kind of output on an UNMODIFIED binary as well.

I should have mentioned that in my last post. Sorry.

So I guess its not that.

Gadgetboy, see above. So I think the spanking is off. (This is meant in good humour, just as it was suggested).

I'm just trying to highlight whats happening here to see if its a bug.

And as jdiner had posted earlier that he had no bug reports as though this code is about to have the limit removed and he's mentioned in the tytool thread that he's building a new version with the new vsplit built in.

I have posted the info. If it's an issue, he now has the info to work from. If it isn't he will say so. Great with me either way.

I could have just pm'ed Jdiner, but I sometimes when others see posts, they chime in and we all gain.

It could be an issue with my Tivo, but my Tivo functions perfectly in terms of user interaction and recording.

J

[edited to tone down whining, knee **** response]

jdiner
11-15-2002, 06:56 PM
As I said before. I am not answering any questions of comments on hex edited versions of what is going on. If you really think it is bad then save it. But this is early test version of software. Nothing more.

--jdiner

drapatin
11-15-2002, 07:00 PM
Jdiner,

IMHO I think you should keep your source closed.

We all know very well that the minute you release your source under the guise of "open source" it will be stolen and incorporated into something that someone ELSE makes money off of. How many times have we seen this before?

I would bet serious money that there is more than one person lurking on this board and thread for just that purpose.

How many hours have you put in?
How much flak have you taken?
How much money (in time) have you lost due to this project?

I say, release the final for $5-10.00 a pop and a month or so later, release the source. Make some cash to make up for your effort.
I'll be the first in line at paypal.

Think about this...
Get with a few other guys who know what they are doing (GUI, DVD writing, Menu Creation and make a REAL commercial release for editing/creating DVD's (along with Tivo) It certainly seems like you know enough to do that now.

Then you can vacation in Fiji !!

BubbleLamp
11-15-2002, 07:07 PM
Originally posted by drapatin
I say, release the final for $5-10.00 a pop and a month or so later, release the source. Make some cash to make up for your effort.

Sell it to Tivo or Sony for a few hundred grand! :cool: