tyremux a GPLed TyStream demuxer and to be remuxer

[racingclub]

Will try the IFOEdit route you suggest, and see how it gets on.

I have now created a few SVCD's with the following method:

1)author a DVD with IfoEdit using the .m2a & .m2v from tydemux
2)feed the VOB's into DVD2AVI (maybe do some clipping) and save the project
3)feed the project in VFAPI and convert to avi
4)feed the .avi into CinemaCraft Encoder and encode an ES MPEG2 .mpv
5)use BBMPEG to mux up the .mpv from CCE and the .mpa from DVD2AVI

voila one SVCD!

(much the same as DVD2SVCD but done by hand!)

[MrBassMan]

I have at last burned a DVD from a UK stream that stays in sync the whole way through. It's a 2.1GB 1h40Min movie.

The process I used was:
tydemux 0.4.0 to get seperate audio and video files
IFOedit (96b) 'Create DVD' to produce an VIDEO_TS directory
Nero to burn this to DVD (with an empty AUDIO_TS directory)

Works a treat playing on my PC but my DVD player doesn't like it at all, possibly due to the lack of menus.

Next I will try creating the DVD-R with Sonic MyDVD which I have successfully used in the past to create DVDs for my player. I can also use this to cut off the start and end of the stream which contains the usual broadcaster's 'coming soon' advertising.

If anyone has any hints on making IFOEdit's output more compatible with DVD players I would love to hear from them.

Olaf: There have been some requests here to accept multiple .ty input files. I think multiple output files would also be useful to make editing easier. i.e. Output until the video file reaches roughly 1GB, close the files and create new ones with _2 appended to the filenames etc.

Have you detected any major changes in streams when the adverts start? If so, this could also be a cue to start new files.

[mpauley]

MrBassMan -

Is your audio @ 48kHz? I know the SA TiVo encode @ 32kHz. That would stop it from being read.

--mp

[MrBassMan]

Originally posted by mpauley
MrBassMan -

Is your audio @ 48kHz? I know the SA TiVo encode @ 32kHz. That would stop it from being read.

--mp

Opening the IFO file produced by IFOEdit shows a line that says:

Audio 1: English (Mpeg-1 ) 2ch 48Kbs 16bps (ID: 0xc0)

So the answer would APPEAR to be yes it is at 48KHz

[mpauley]

And not to throw out all the obvious, but did you encode your video? I didnt see that in you procedure.... I thought DTiVos only did 480x480 and DVDs have to be 720x480 or 352x480.

[moshmothma]

Yes, that's correct. Your DVD would not be compliant at 480x480. However, you may not need to reencode. You can use software called DVDPatcher to 'fool' your player into thinking the video is compliant. The only prob is not all players are able to accept this. Go to vcdhelp.com to see a list of players that accept this type of video.

[racingclub]

thought DTiVos only did 480x480 and DVDs have to be 720x480 or 352x480.

He's using a SA tho....

[olaf_sc]

I'm quite sure this is not a bug in tydemux triggering a bug in TMPG. Beside it can actually be TMPG tiggering a TMPG bug.

I also very sure that the mpeg that is come out of tydemux is very valid. The mpeg video stream is more or less trippel checked by tydemux.

Cheers Olaf

Originally posted by racingclub
Hi All,

I've now tested a few more UK streams - and they seem to cause TMPG to fail (crash).

I'm trying to re-encode the .m2v and .m2a as SVCD's using the simple TMPG project wizard with no 'frills'. I seem to be able to encode small sections with no problems.

I can use the TMPG MPEG tools to mux the 2 files together without any problems - but when I run the resulting .mpg through the TMPG SVCD wizard I get a crash in the same place.

I've also tried authoring a DVD with IfoEdit then running the VOBs through DVD2SVCD (which uses CCE) - this works fine and gives me a 'good' SVCD with proper sync.

Is anyone else (maybe just in the UK) trying SVCD creation?

[Nugget]

I doubt its a tydemux bug, but is it possible its a TIVO encoder bug, (given that the TIVO encoder was put together before MPEG2 spec was complete?) and that although the mpeg structure data, of GOPs etc is valid, that the mpeg data within the frames in the GOPs may be what is causing the problem.

I assume this as muxing/demuxing isnt going to care too much about the content of each bit of mpeg data, it seems pretty much the payload is just transferred as-is from one stream to the next. But if the payload itself contained errors, then when tmpgenc came to decode it, it falls over in a heap.
Other players [powerdvd/dvd2avi/set-top] might not care about the correctness of the data.

Ive just completed my first frameserve approach with output split using tydemux, fed thru DVD2AVI, loading the d2v from DVD2AVI and the m2a from tydemux. This produced a good SVCD mpeg [as others indicated it would].

Both using the frameserving approach, and using the MPEG2 plugin for tmpgenc from vcdhelp.com allow tmpgenc to process the data without invoking its own mpeg2 decoding routines. So it does look like its got a problem decoding the tivo data. The fault could lie either with the data, or the decoder..

To find out which, is there any way to 'validate' the payload data within an mpeg stream? [Im assuming here that the frames that are deltas are some form of modification instruction for the preceeding frames, and that this modification could be invalid. (I dimly recall hearing something in the past of something about motion vectors initating/terminating beyond the edge of the frame causing problems on some other project, I cant recall which tho)]

Cheers,
Nugget

[MrBassMan]

Originally posted by Nugget
To find out which, is there any way to 'validate' the payload data within an mpeg stream? [Im assuming here that the frames that are deltas are some form of modification instruction for the preceeding frames, and that this modification could be invalid. (I dimly recall hearing something in the past of something about motion vectors initating/terminating beyond the edge of the frame causing problems on some other project, I cant recall which tho)]

Cheers,
Nugget

In the last couple of days I saw a download link to an official MPEG stream validation program released by the MPEG group. I'm sorry but I cannot remember where I saw it but that may be what you are looking for.

[olaf_sc]

Originally posted by MrBassMan
Olaf: There have been some requests here to accept multiple .ty input files. I think multiple output files would also be useful to make editing easier. i.e. Output until the video file reaches roughly 1GB, close the files and create new ones with _2 appended to the filenames etc.


Yes, that one would be nice to implement - it would be instyle with what e.g. mkisofs uses when producing dvd images on systems not supporting 2GB and bigger files.

There is also no reason to why we should not be able to make DVD's under a OS that doesn't support large files. Since the maximum size of a vob file is just 1024MB.



Have you detected any major changes in streams when the adverts start? If so, this could also be a cue to start new files.

There are a few but that is mostly on DirecTV mpeg streams, nothing solid in SA/UK streams. It's a very interesting subject although and something that is definitly worth investigating. However my main point in adv detection is that has to be done in the compressed space - decompression of the mpeg stream in order to detect adv is a big no no in my view. Why? It simply takes to much time to process even on a fast computer.

Cheers Olaf

[olaf_sc]

That one is intersting, I didn't check what audio it was when I did tested UK Tystreams.

Cheers Olaf

Originally posted by MrBassMan
Opening the IFO file produced by IFOEdit shows a line that says:

Audio 1: English (Mpeg-1 ) 2ch 48Kbs 16bps (ID: 0xc0)

So the answer would APPEAR to be yes it is at 48KHz

[Nugget]

My UK SA Tivo streams have 32khz/192kbps audio, at least the ones I've checked have. Didn't think it was known how to tell Tivo to change it to another value. However, the SVCD transcoded output from tmpgenc had 44.1khz/224kpbs output.

I just tried using MPEG2VCR's 'GOP Fix' feature on a tydemux'd m2v, and then passing the 'fixed' result to tmpgenc. Still crashes. Think this points nicely to whatever the problem is, not being tydemux.

You want the real fun test? Feed an mpeg2 CREATED by tmpgenc (I used the output from my frameserved approach to create an svcd mpeg) back to tmpgenc and ask it to transcode it. You would expect the mpeg2 it created would be valid. Still crashes.

I guess tmpgenc's builtin mpeg2 decoder has some 'issues'.

Guess I'm stuck with the frameserving approach. In which case, anyone know if DVD2AVI supports commandline options to create the d2v file? I found some pages mentioning /preview and /saveavi but no obvious docs on /makemead2vfilefromthis ;-)

Cheers
Nugget

[Nugget]

Swiftly answering my own question.

A commandline variant of DVD2AVI can be found over at DVD2SVCD Project Homepage (http://www.dvd2svcd.org/)
Where its available on the bottom left of the page, complete with source.

Can use that version within a batchfile with tydemux to create the d2v to feed to tmpgenc.

I read somewhere tmpgenc can be started with an argument to make it process a batch list, which you can generate beforehand. If I can get this working then can have a windows batch file to convert .ty to svcd / vcd compatible mpeg streams.

I worry about audio offsets though ? maybe its better to let besweet convert the audio, tmpgenc just convert video, and then use mplex to mux with the offset. Anyone tried this?

Cheers,
Nugget

[olaf_sc]

Part 5 of ISO-13818 standard includes the software simulation of ISO-13818 part 1, 2 and 3 (system, video and audio). I have attached it in my post.

However note that some of this is outdated and newer versions of the software (parts) is avalible at www.mpeg.org under mssg.

Cheers Olaf

Originally posted by MrBassMan
In the last couple of days I saw a download link to an official MPEG stream validation program released by the MPEG group. I'm sorry but I cannot remember where I saw it but that may be what you are looking for.

[tungsten2k]

Originally posted by olaf_sc
[re: allow multiple input files] Yes, that one would be nice to implement - it would be instyle with what e.g. mkisofs uses when producing dvd images on systems not supporting 2GB and bigger files.
i was going to wait and look at the code to see if this is feasible but since you're considering playing around with the input routines, another option would be to allow cutoff points as arguments for the input streams in much the same way that bassman's tychopper does now. there is less total output and more importantly, we can ditch any of the stream we don't want right off the bat, rather than having to keep all the stream and cut it out with editing tools down the road.

problem i see with this is that i'm not sure how easy it is to detect the chunk:time ratio with enough accuracy to be able to take time offsets (from begining of first input ty) and cut up accordingly. obviously wouldn't need to be exact, but being able to cut with +/- 1 minute accuracy would be necessary to make it worthwhile. obviously taking chunks as a parameter would be easy but we don't really have the chunk handy while we're watching our recordings at hit a 15-minute commercial marathon ;)

ps- i chopped that stream with +/- 50 chunks around the bad spot and ran the 20MB file through tydemux040w32 again... same result. filename is tungsten2k01.ty - have fun with it :D

[olaf_sc]

Hello Folks

Found a nice little app "Moonlight MPEG Stream Explorer" it's a demo but it worked fine for me on a small random mpeg file I had at my disposal at work.

Use this url to download it http://www.elecard.com/ftp/pub/mpeg/StrExpl/MMSExplorer.zip.

Cheers Olaf

PS: Yes it's not much of a analyzer but it lets your step trough the stream in both decoder and display order and it shows all frames etc...

[MrBassMan]

Originally posted by tungsten2k
i was going to wait and look at the code to see if this is feasible but since you're considering playing around with the input routines, another option would be to allow cutoff points as arguments for the input streams in much the same way that bassman's tychopper does now. there is less total output and more importantly, we can ditch any of the stream we don't want right off the bat, rather than having to keep all the stream and cut it out with editing tools down the road.

problem i see with this is that i'm not sure how easy it is to detect the chunk:time ratio with enough accuracy to be able to take time offsets (from begining of first input ty) and cut up accordingly. obviously wouldn't need to be exact, but being able to cut with +/- 1 minute accuracy would be necessary to make it worthwhile. obviously taking chunks as a parameter would be easy but we don't really have the chunk handy while we're watching our recordings at hit a 15-minute commercial marathon ;)


In theory you can do this now with tychopper. If you can work out how long a chunk plays (prabably very variable due to compression) you could calculate the chunks to chop out and keep. It's more like using a blunt axe than a surgeons scaple but it would sort of do what you want.

[olaf_sc]

Choping chunks is not such a good idea, there are several limitations to it.

Both video and audio data can spaw over more than one chunk hence you will end up with half frames (video/audio). Tydemux naturally detects that but that is not a guarntie that things will work.

There are also times when a chunk is mainly video or audio hence cuting out a chunk can reder in a long loss of audio or video (audio loss is definitly bad).

One way to do it would be to have a app to display I frames (we have to cut at gop boundary anyways which is tied to the I frame). We could very easily scan the whole tystream and and build up a list of I frames or build a list of where the iframes are located in the stream.

Now when playing back and forth we could extract those I frames. The user can review them and add/delet cut points.
Tydemux is numbering every seq/gop with a sequence number. This sequence number is also attached to the I frame (which is in the same container as the gop and seq header). A cut start will therefore be tied to one seq number and the stop to another. When extracting the data from the stream we can simply skip those seq/gops we don't want. We will naturally need to skip the same amount of audio data.

Anyideas?

Cheers Olaf



Originally posted by MrBassMan
In theory you can do this now with tychopper. If you can work out how long a chunk plays (prabably very variable due to compression) you could calculate the chunks to chop out and keep. It's more like using a blunt axe than a surgeons scaple but it would sort of do what you want.

[mpauley]

Originally posted by MrBassMan
Opening the IFO file produced by IFOEdit shows a line that says:

Audio 1: English (Mpeg-1 ) 2ch 48Kbs 16bps (ID: 0xc0)

So the answer would APPEAR to be yes it is at 48KHz

MrBassMan -

I pulled a medium stream from my Philps SA and verified that the stream was 480x480 and the audio was 32Kbps. After I authored a DVD in IfoEdit, I checked the IFO info and it showed me:


Video: MPEG-2 352x240 (NTSC) blah blah blah...
Audio 1: English (Mpeg-1) 2ch 48Kbps 16pbs (ID:0xc0)


So, I'm guessing Derrow didn't put in any stream checks in this tool, to make sure it would be DVD compliant.

When I record a stream on my Philips SA @ 720x480 (Setting Zero in resource editor), resample the audio to 48Kbps and author with IfoEdit, it plays perfect on all my DVD players.

-mp

[TheSaint]

Works for me for >2Gb.

For those still struggling with the audio upsample to get it to DVD compliance I use some programs in batch mode to achieve this.

BeSweet.exe -core( -input "%1\%2.m2a" -output "%1\out32.wav" -2ch )

ssrc.exe --rate 48000 --twopass "%1\out32.wav" "%1\out48.wav"

toolame.exe -m s -b 224 -e "%1\out48.wav" "%1\out48.mp2"

I know there are some direct conversions, but those always caused my audio to "tick". The combination of these programs seems to do the trick for me.

Olaf,

Could you explain the difference between a closed and open GOP ? I've heard reference that cutting video is preferable on closed GOPs as it makes for a smoother cut I note that my tydemux video outputs generally don't have closed GOPs except GOP #1. Is that normal ?

TheSaint

[laserfan]

Originally posted by mpauley
When I record a stream on my Philips SA @ 720x480 (Setting Zero in resource editor), resample the audio to 48Kbps and author with IfoEdit, it plays perfect on all my DVD players.mp what do you use to resample audio??

[artships]

Well, the MrBassMan's new Win binary (Thanks!) works better, but it abends with 4,794,885KB of video where it should have generated 5,141,098KB.

John,
Working on tystreams tytooled from a Sony 2k SA to a WinXP NTFS disk.

[FreydNot]

Originally posted by TheSaint
For those still struggling with the audio upsample to get it to DVD compliance I use some programs in batch mode to achieve this.

BeSweet.exe -core( -input "%1\%2.m2a" -output "%1\out32.wav" -2ch )

ssrc.exe --rate 48000 --twopass "%1\out32.wav" "%1\out48.wav"

toolame.exe -m s -b 224 -e "%1\out48.wav" "%1\out48.mp2"


I strugled with a similar problem (clicks in the audio). I found using the latest beta from the besweet home page fixed the clicks. Here is the command line I'm using in my batch file...

besweet -core( -input "%1.mpa" -output "%1.wav" -2ch ) -ota( -norm 0.98 ) -ssrc( --rate 48000 )


I then feed the wave file into virtual dub to cut commercials and save out a new wave file (post cuts). My final step converts the wave into ac3 using ac3machine. I prefer a 2 channel ac3 since it is more compliant with the DVD spec then mp2 is.

Here is my ac3machine command line:

"BeSweet.exe" -core( -input "&1.wav" -output "&1.ac3" ) -ac3enc( -b 224 ) -profile( The AC3Machine v0.4 )

[olaf_sc]

Sounds like a good idea, note put into todo list.

Cheers Olaf

Originally posted by rc3105
just a suggestion for the multiple files support

since the maximum size of a tivo tystream is 4096 chunks, which is 512 meg, why not use that as the "packet" size

it's small enough to fit on a cd, fsid's can be processed individually, (which will simplify accessing tystream metadata like cc info later) it ties in more simply with the tytar format and anything cabaple of running tydemux can access 512 meg at a chunk easily

tydemux can probably safely free memory & reinit itself for each fsid

--
Riley

[olaf_sc]

A quick one (dead tired 1:am here).

Mpeg has three different frame types

I Frame - this is a independent compleate picture
P Frame - this is a picture that is dependent on a previous P or I frame inorder to be decoded to a compleate picture
B Frame - this is a picture that is dependent on a previous I/P frame and a I/P frame a head of it in order to be decoded to a compleate picture.

Okay so far?

Now Mpeg comes as a stream of frames like this

2I 0B 1B 5P 3B 4B 8P 6B 7B

The number in front of the frame (IPB) is the order they shoud be display in. But they come in this order in the stream since that is the decoding order.

The example above is a open gop (a gop resets the frame numbering and always starts with a I frame). As you can see Frame 0 and 1 is depening on the previous gop's P or I frame inorder to be decoded properly. Hence if you cut infront of the gop you will not be able to decode the 0 and 1. You will see blocky pictures. In other words a open gop is dependent on the previous gop in order to decode all frames properly.

Now a closed gop is a gop that is not dependent on the previous gop. The start of such a gop can look like this

0I 3P 1B 2B 6P 4B 5B

As you can see we now have a I/P both behind and in front of the first B frames.

Now you mentioned that the first gop is closed in the video that tydemux produce. That is actually "me" fixing the video stream. What I do is simply dropping the first two B frames and renumber all the frames so the display order is proper. This is also the way to properly cut a video steam.

Hope things are clearer now.

Cheers Olaf





Olaf,

Could you explain the difference between a closed and open GOP ? I've heard reference that cutting video is preferable on closed GOPs as it makes for a smoother cut I note that my tydemux video outputs generally don't have closed GOPs except GOP #1. Is that normal ?

TheSaint

[olaf_sc]

Ohh, yes fixed all bugs that has been reported now. So stay tuned for a new release of tydemux tomorrow (i.e. in 24 hours).

Cheers Olaf

[MrBassMan]

Originally posted by artships
Well, the MrBassMan's new Win binary (Thanks!) works better, but it abends with 4,794,885KB of video where it should have generated 5,141,098KB.

John,
Working on tystreams tytooled from a Sony 2k SA to a WinXP NTFS disk.

There are instances of bad tystream data that can cause tydemux to crash - I suspect this is one of those instances.

The way to prove/disprove this is to run tychopper on the file to extract well before and after the point it crashes. Then run tydemux on the smaller chopped file.

If it crashes on the smaller stream, keep chopping until you get a file small enough to send to Olaf for analysis.

Alternatively - wait another day or two (or longer - I assume Olaf has a liitle bit of social life occasionally) when another version of tydemux will come out with some fixes for this type of problem.

[mpauley]

Originally posted by laserfan
mp what do you use to resample audio??

I found that BeSweet works the best for me. I also found that burning the DVD with Nero in ISO/UDF format is the best way to go for my players.

[laserfan]

Originally posted by mpauley
I found that BeSweet works the best for me.Hmmm, I use BeSweet and Nero as well. Maybe I simply have to take the next step and create the VIDEO_TS from SpruceUp (it was SU's preview mode that caused me to believe the vid/aud were way off). In theory it's this simple, right: tydemux to mpv and mpa files, BeSweet the mpa to 48kHz, create & burn VIDEO_TS? Thanks.

[olaf_sc]

Hello Folks,

Sorry but you have to wait another 24 hours for tydemux 0.4.1. I working with a really wacked stream from RC3105 that he sent me this evening. It makes "tydemux" loose sync which is kind of bad :(. Anyhow I have a solution in mind and will release 0.4.1 late tomorrow evening (PST).

Cheers Olaf

[TheSaint]

FreydNot,

Thanks for the pointer on the BeSweet beta. It works much better now. No pops or clicks at all.



[QUOTE]Originally posted by FreydNot
[B]I strugled with a similar problem (clicks in the audio). I found using the latest beta from the besweet home page fixed the clicks. Here is the command line I'm using in my batch file...
[code]
besweet -core( -input "%1.mpa" -output "%1.wav" -2ch ) -ota( -norm 0.98 ) -ssrc( --rate 48000 )

[artships]

Originally posted by MrBassMan
There are instances of bad tystream data that can cause tydemux to crash - I suspect this is one of those instances.

The way to prove/disprove this is to run tychopper on the file to extract well before and after the point it crashes. Then run tydemux on the smaller chopped file.
Excellent suggestion, but I went with option B, re-install WinXP to fix some OS stuff I'd buggered. Good news, tydemux worked through the whole 2-hour show, creating components which, when "simple muxed" by TMPGEnc, did not, this time, complain about missing frames in one or the other stream. Bad news, the audio stream contained something that killed toolame. Oh, and my modem's dead (though not in WinME, on another disk. Go figure).

I'm going to try using your program to skip past the first minute or so of my show, hoping to get past the audio anomaly. T'would be nice if tydemux had a "skip some stuff up front" option, but here I am, finally getting a pony, and complain that the bridle doesn't have tassels.

[olaf_sc]

Originally posted by artships
stream. Bad news, the audio stream contained something that killed toolame. Oh, and my modem's dead (though not in WinME, on another disk. Go figure).


Maybe your disk has bad blocks sounds something is broken.

Cheers Olaf

[MrBassMan]

Originally posted by artships
I'm going to try using your program to skip past the first minute or so of my show, hoping to get past the audio anomaly. T'would be nice if tydemux had a "skip some stuff up front" option, but here I am, finally getting a pony, and complain that the bridle doesn't have tassels.

Olaf is way ahead of you - the release due any day now has that feature.

tydemux - tassles ARE included, bells and whistles are coming shortly.

[du1jec]

I just created 3 DVD's on my Mac using tydemux to split the tyfile and Sizzle to create a dvd image and burned it using disk copy. AV sync was great! Played great on both my Samsung and my Bose 321. My Bose 321 is pretty picky about dvds too.

It wont let you create menus and advanced stuff, but if all you want is to get your movies on a dvd this is really easy.

I have attached a link to sizzle for you guys who may be interested.

http://www.versiontracker.com/dyn/moreinfo/macos/17331

[olaf_sc]

Thansk for the info

Some more info about Sizzle ;).

Sizzle is a nice gui frontend to dvdauthor (0.3.1 included in latest Sizzle). Hence if you are a MacOS X guy and want to script you can do so directly with dvdauthor.

Dvdauthor is a cli unix prog that should be able to compile on most unix platforms.

Cheers Olaf

Originally posted by du1jec
I just created 3 DVD's on my Mac using tydemux to split the tyfile and Sizzle to create a dvd image and burned it using disk copy. AV sync was great! Played great on both my Samsung and my Bose 321. My Bose 321 is pretty picky about dvds too.

It wont let you create menus and advanced stuff, but if all you want is to get your movies on a dvd this is really easy.

I have attached a link to sizzle for you guys who may be interested.

http://www.versiontracker.com/dyn/moreinfo/macos/17331

[olaf_sc]

Ahh, some more info.

Apperently they also use mkisofs with the DVD-Video support code that I wrote :).

Cheers Olaf

[tungsten2k]

Originally posted by olaf_sc
Some more info about Sizzle ;).

Sizzle is a nice gui frontend to dvdauthor (0.3.1 included in latest Sizzle). Hence if you are a MacOS X guy and want to script you can do so directly with dvdauthor.

Dvdauthor is a cli unix prog that should be able to compile on most unix platforms. wow, where have i been ? out 10 days and i didn't know about it ? lol. well, sizzle, even before 0.0.2r2, works great... it used to use ifogen, mplex, and mkisofs to generate a quite nice disk image that burns like a charm with Apples Disk Copy and any A03/4 (my A05 has some small issues with the 1.0 firmware... haven't upgraded to 1.2.1 yet but it will prob fix it... Toast 5.2 works great tho.)

New toy to play with... hehe. Looks like support for multi-track has been added in the latest release, which was the only thing lacking in the previous version (nothing that couldn't be got around by just using the actual tools) fyi, for those that don't know... any OSX app written in the standard gui wrapper sample app (like Sizzle, ffmpegX, etc) you can find the actual binaries in the "/Applications/[application name].app/Contents/Resources" directory. oh, and one last item... whatever you do, don't use the built-in ffmpegX "Installer" it sucks major ass. oops, no pun intended. major4, the developer is a very responsive guy and really cares about his app... unfortunately he seems pretty stuborn on doing things his own way too. i suggested he create an installer for the required binaries and have them put into the aformentioned directory, or even "/usr/local/bin" and he goes and insists on requiring them in the OS tree in "/usr/bin" (*shakes finger... shame shame shame). whao, really off topic now...

[rc3105]

as we all know, mfsstream & tytool export all the segments of a show as one large tyfile. the file has the name of the show but the rest of the info is discarded.

there is a tivoweb module called tarserver that tivodvlpr did that extracts the show complete with all the nowshowing database info.

http://alt.org/forum/index.php?t=msg&th=18&prevloaded=1&rid=24&S=516e91240c3c16007803cf57f85dea77&rev=&reveal=&start=140&count=20

the tarball is a regular tar file. the first file is showing.xml and each fsid follows thereafter as a seperate file with the naming convention part00.ty, part01.ty, part03.ty, etc

if you plan to archive or burn dvd to this is a great way to extract as eventually there will be utilities to automatically use the xml data to set title & episode info and perhaps even menu structure for dvd's with multiple shows.

to request a show you type in the address of your tivo + showname or fsid from a browser

http://192.168.0.2/mfsstream/Stargate SG-1.tar

or

http://192.168.0.2/mfsstream/778857.tar

if you use show name, it matches the first show in the nowshowing list and may not grab the one you intended, if you use fsid it works fine, but it's the fsid of the recording object (not anything tytool will show you) so getting the number is a nuisance, also, untarring the segments and catting them back together is clunky.

a few posts down in the alt.org thread, I posted a tweaked version of mfsstreamweb.98 that automatically invokes the tytar server to make extraction with info a bit simpler.

http://alt.org/forum/index.php?t=getfile&id=49&rid=24&S=516e91240c3c16007803cf57f85dea77

the next post after that is a little script that will take a tarball and extract the showing.xml file and generate a single mfsstream style tystream.

the tydetar.tcl.zip is the tcl script,

tydetar.zip is a windows executable (just change the name from .zip to .exe) freewrap runtime library that includes the tcl script and will execute as though it were a regular compiled binary.

http://alt.org/forum/index.php?t=getfile&id=51&rid=24&S=516e91240c3c16007803cf57f85dea77

freewrap uses the executable zip packing method so DO NOT OPEN IT WITH WINZIP. (unless you want to look through the runtime lib) just change it from .zip to exe

eventually tydemux will parse tarballs directly (I'll add that if olaf doesn't feel like it) but in the meantime I hope you find these usefull.

--
Riley

[AlphaWolf]

There are so many reasons why the tar format should become the new tystream archiving "community standard" (especialy for those of you who like to make DVD's from tivo streams), globbing all of the parts together is just plain messy. Now if only we could come up with a name for it. :D

[Hi8]

Originally posted by AlphaWolf
There are so many reasons why the tar format should become the new tystream archiving "community standard" (especialy for those of you who like to make DVD's from tivo streams), globbing all of the parts together is just plain messy. Now if only we could come up with a name for it. :D

how-about ... 'tyball'

[olaf_sc]

It's time for a new release of tydemux 0.4.1 I hope you will find it useful!

This is mainly a bug fix release but two new features are added.

During the probe of the stream tydemux will now search for the start of the chunk inside the tystream (see below) this will enable people how hasompensates for missing audio frames, this is most of the time a single a tystream that has alignment error to still demux it propely.

tydemux demux will now check if we have any audio missing. You might see a warning while demuxing that tydemux is compensating for missing audio frames.

Large file support is now permanetly fixed in windows.

About the seek, as said it's only during the probe due to a number of reasons we aren't seeking during the actual demuxing. Anyhow If you see warnings like this during the probe:


Probing TyStream .....
WARNING: Chunk 0 is out of alignment with 1024 bytes
WARNING: Chunk 10 is out of alignment with 1024 bytes
[snip]
WARNING: Chunk 90 is out of alignment with 1024 bytes
WARNING: Chunk 100 is out of alignment with 1024 bytes

Your stream is not aligned - please use -b to specify byte offset


Then you will need to use the new -b to specify a byte offset in this case it will look something like this:

tydemux -b 1024 -i /usr/local/samples/UK/MrBassman_Bad4.ty -v out.v -a out.a


As usually there will be bugs even if I tried my best to iron out the once I have got reports about. Anyways some of the code is more or less impossible to test unless you actually have TyStreams that has that particular error. Now this doesn't mean that tydemux isn't useful for a wider audience it only means that you should report bugs as soon as you find them. As usually I have a ftp site (66.121.15.35) where you can upload TyStreams that triggers bugs or other peculiarities in tydemux. Please email me (olaf_sc@yahoo.com) if you have really big files (1GB and above) that you want to upload.



The current features of tydemux:
--------------------------------


Demultiplex of video and audio (both MPEG and AC3/a52) from all Tivo types made - SA Tivo Series 1, SA TIVO Series 2, DTivo Series 1 and UK Tivo using software revision 1.3 and up. NOTE: DTivo Series 2 should in principle be supported but I have not yet be able to obtain a TyStream from it.

Tivo Type/Series probe you don't need to worry about what type your TyStream is tydemux will detect it and use the right engine to demux the audio and video.

Check and correction when we have a overflow or underflow of frames.
- Overflow - This is more or less always a video sequence (gop) header and corresponding I frame that is actually missing. What happens is that tydemux doesn't catch the seq/gop hence when we count frames for the seq/gop it looks like a over flow. Tydemux repairs this problem by adding both seq, gop and I frame at the right location in the stream.

- Underflow - This is when one or several B or P frames is missing from the video sequence (gop). Tydemux will repair this by inserting a B or a P frame at the appropriate location in the stream.

Compensates for missing audio frames, this is most of the time a single audio frames that is missing. But sometimes larger parts can be missing.

Correction of A/V sync drift in TyStream, tydemux will correct gradua drift on A/V sync. Depending on frame rate tydemux will mend A/V drifts of as little as 8ms (max 20ms).

Correction of temporal reference i.e. frames out of order. tydemux will detect any error in the temporal reference in a TyStream and mend all types of temporal reference errors that can happen.

Audio probe, you don't need to worry about what type of audio a recordings has. tydemux will probe the whole TyStream, determine audio type and skip to the first chunk that has that type of audio.

TyStream repair will repair the TyStream if it detects holes/gaps in the TyStream. Up on detection of a hole/gap tydemux will align video and audio in such way that there is no or very little loss of sync. Further more tydemux will fix the start and end of the hole/gap in such way that Video and Audio doesn't deteriorate in quality.

tydemux is minimizing the Audio/Video sync offset. At the end of the execution it will tell you the offset in milli seconds.

2GB file limit removed on platforms supporting large files.




The tydemux_W_0.4.1.zip holds Windows binaries and sources/project files from MS Visiual Studio 6.x.

The tydemux_L_0.4.1.zip holds Linux x86 binaries and source/make files for gcc.

The tydemux_M_0.4.1.zip holds MacOS 10.2 binaries and source/make files for gcc.


While all of them has the "exact" same codebase I still think it's easier to post to different packages.

Cheers Olaf

[olaf_sc]

Linux sources and binaries

[olaf_sc]

And finally the mac binaries and sources

[rc3105]

I had 50+ gig of problem streams the choked 3.1 & vsplit (as well as my other ty2mpeg progs) even 4.0 couldn't repair a few of them.

4.1 has fixed/split those now such that the corrupted segments play back better as pc mpeg than the original streams did in the tivo.

WOOOOHO!!!!

--
Riley

[Nugget]

Hi.. 0.4.1 seems to have regressed the tydemux.c/misc.c file, exit codes of below 1000 now also mean error rather than audio delay, and the return from tydemux is now fixed at 0 for a good return..

Looks like the audio delay return bit was intentionally disabled, any ideas of if it may come back next revision?

Nice new message during demux though...

Starting demux process
......Hmm is this really like this
... 100.....


By the way, does anyone know where I can find a large file compatible windows version of mplex, or an alternate (command line) multiplexer?

Cheers,
Nugget

[rc3105]

well we're not completely out of the woods yet.

the repair routines can be VERY memory intensive and w98 memory management sucks.

olaf's running 2 gig of ram under linux and I have 256/win 128/mac 128/linux.

streams that process fine with 128 under os-x / rh can thrash themselves to death in w98 / 256. (not terribly suprising)

my problem collection is going back through the wringer. with valgrind, win memory monitors, a lot of head scratching on my part and the ocassional eureka from olaf we inch foreward... :D

in the meantime, typical streams process fine and produce editable mpeg with sync that's dead on.

for foo/bar streams under os-x / rh you probably need 196+ ram and/or to shut down anything nonessential while repairing.

under windows (can't speak to nt/xp issues) you DEFINATLY want 96m+ of ram and to run tydemux as the only major process with at least 1 gig or 10% (whichever's larger) free space on your boot/swap drive.

this is just for the worst of the worst. if you have a real computer you're good to go. if your pc came in a box of cracker jacks as many of mine did don't worry, I'll keep bugging olaf till it runs in the tivo itself ;)

--
Riley

[Hi8]

Originally posted by Nugget


By the way, does anyone know where I can find a large file compatible windows version of mplex, or an alternate (command line) multiplexer?

Cheers,
Nugget

I've been searching for the same ... It would be GREAT if there was a 'tyremux' (hint, hint)

[newlooper]

Olaf,

Congrats for making so much progress!:) I am having a problem though:( I seem to be getting a lot of temporal errors. It says it corrects them and that they are caused by an frame underflow. when I move to mplex. Mplex bails out with

**ERROR: [mplex] Too many frame drops -exiting

Would you have any suggestions?

-newlooper

[Nugget]

Originally posted by newlooper

**ERROR: [mplex] Too many frame drops -exiting

Im just guessing that you aren't using -f 3 on mplex, and that you are trying to use mplex to make SVCD.

Ive seen if I have an mpeg with a high bitrate and I try mplex -f 4 I get errors similar to yours.

Try with -f 3 and see if that helps ?

Cheers,
Nugget

[olaf_sc]

Hello

I will actually integrate the latest CVS version of mplex (mjpeg tools) into tydemux in version 0.5.0 hence tydemux will become tyremux as of that version.

It also means that I will make a native port of mplex to win32 and yes it means that I will look into fixing large file support in mplex under windows.

I will naturally fwd my changes to mplex to Andrews hence hopefuly we will later on have official versions of mplex that has large file support under windows.

Cheers Olaf


Originally posted by Hi8
I've been searching for the same ... It would be GREAT if there was a 'tyremux' (hint, hint)

[olaf_sc]

Hello

Please see my inline comments,

Originally posted by rc3105
well we're not completely out of the woods yet.

the repair routines can be VERY memory intensive and w98 memory management sucks.


Well normaly a repair isn't that memory intensive around 20MB of RAM or so. However when we fail to repair tydemux will start caching all as many chunks as it can. If you have a big stream this mean a huge amount of ram will be eaten up.

I'm very interested in streams that behaves like that. Simply a repair shoud not fail tydemux should in principle always be able to repair the stream!

If you are demuxing and it looks something like this:

......... 5100......... 5200......... 5300......... 5400......... 5500
......... 5600...
Will try to revert and repair the TyStream better with a gap than nothig
get_audio: Warning! Audio size 43599 while pes says size 480

Repairing TyStream
Failed to repair tystream
****.**********.**********.**********


Then it means that the repair faild and tydemux started to cache chunks - PLEASE HIT CTRL-C. It's no use to continue tydemux will almost never be able to repair the stream.

Now it's probably a good idea to have some sort of upper limit for this caching and I will implement that in 0.4.2.

However if you see this error I really want you to send me the stream.

You don't need to send me the whole stream you can either use dd (Linux/Mac) or MrBassMans tychopper (windows). You can find tychopper* here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=78453#post78453) .

What you need to do is to chop out a number of chunks before and after this happens and send to me. In the example above it probably best to chop out chunk 5500 to say 5900.

MrBassMan/John - simply forgot to include tychopper in tydemux to dead tired. Will be there in 0.4.2.



olaf's running 2 gig of ram under linux and I have 256/win 128/mac 128/linux.


Yea, more or less nothing will take down my box ;). Although it's 4GB or ram I have.


Cheers Olaf

[olaf_sc]

Hello Newlooper

It's probably has nugget says try to multiplex with -f3 (I assume that you have the latest version of mplex from mjpegtools).

If tydemux is reporting temporal reference errors well then it is a temporal reference error or a bug in my temporal reference error check. However when it says underflow in frames I'm more or less 100% sure that we actually have a under flow.

However I'm very interested of your stream If you could please upload it to me it would be very nice - it's not until then I can really figure out what is happening.

Cheers Olaf

Originally posted by newlooper
]Olaf,

Congrats for making so much progress!:) I am having a problem though:( I seem to be getting a lot of temporal errors. It says it corrects them and that they are caused by an frame underflow. when I move to mplex. Mplex bails out with

**ERROR: [mplex] Too many frame drops -exiting

Would you have any suggestions?

-newlooper

[AlphaWolf]

Olaf_SC: thought I would prod you a little with a suggestion for the status output of tydemux ;)

I think it would look neater to show a percentage of completion in a fixed spot, and below it, print the errors that occur, each on a separate line, and the exact chunk number that they are occuring in. This would make it much easier to tell where to split the files apart rather than having to count the little dots around all of that text. :)

[olaf_sc]

Sure thing - I'm any how working on a new debug subsystem so I can as well do that too. Dunno if I will do it in 0.4.2 (forked the code for the upcomming 0.5.0).

Cheers Olaf

Originally posted by AlphaWolf
Olaf_SC: thought I would prod you a little with a suggestion for the status output of tydemux ;)

I think it would look neater to show a percentage of completion in a fixed spot, and below it, print the errors that occur, each on a separate line, and the exact chunk number that they are occuring in. This would make it much easier to tell where to split the files apart rather than having to count the little dots around all of that text. :)

[MrBassMan]

Originally posted by olaf_sc
Yea, more or less nothing will take down my box ;). Although it's 4GB or ram I have.
Cheers Olaf
That's what I thought about my server which has 2GB memory - Last Thursday night the boot disk died, the RAID recovery failed (thank-you Compaq for documenting this, it does not help finding out AFTER a failure though!). To top it all, a full restore from tape crashes the system. Agghhh!

Anyway - that is why I have been quiet and not answering emails the last couple of days - Normal operation is nearly restored and so I can get back to helping with tydemux.

[psxjunky]

I am getting the following error with both ver 0.4.0 and 0.4.1 --



tydemux -i show.ty -a show.m2a -v show.m2v

TyStream is to small to demux
We need at least 40 chunks


The show.ty file is > 2GB (3.15 GB or 3,387,555,840 bytes) ... but I thought the issue with > 2GB files was fixed.

Anyone know what's wrong ?

[MrBassMan]

Originally posted by psxjunky
I am getting the following error with both ver 0.4.0 and 0.4.1 --



tydemux -i show.ty -a show.m2a -v show.m2v

TyStream is to small to demux
We need at least 40 chunks


The show.ty file is > 2GB (3.15 GB or 3,387,555,840 bytes) ... but I thought the issue with > 2GB files was fixed.

Anyone know what's wrong ?

What O/S are you using? Large file support is only on linux, Win 2K and WinXP. I don't know if 0.4.1 has Mac support, 0.4.0 did not.

[rc3105]

Originally posted by Nugget
Im just guessing that you aren't using -f 3 on mplex, and that you are trying to use mplex to make SVCD.

Ive seen if I have an mpeg with a high bitrate and I try mplex -f 4 I get errors similar to yours.

Try with -f 3 and see if that helps ?

Cheers,
Nugget

I run mplex with -8, then use other mpeg tools to massage the file to fit wherever it's going.

--
Riley

[Rowan]

Originally posted by psxjunky
I am getting the following error with both ver 0.4.0 and 0.4.1 --



tydemux -i show.ty -a show.m2a -v show.m2v

TyStream is to small to demux
We need at least 40 chunks


The show.ty file is > 2GB (3.15 GB or 3,387,555,840 bytes) ... but I thought the issue with > 2GB files was fixed.

Anyone know what's wrong ?

I have had this happen to me before; it happened when I was tying to run the program for the second time and already had the m2a m2v file. If this is your case just delete the m2a m2v output files and run the program again. This seemed to solve the problem for me.

[psxjunky]

Originally posted by MrBassMan
What O/S are you using? Large file support is only on linux, Win 2K and WinXP. I don't know if 0.4.1 has Mac support, 0.4.0 did not.

Thanks Rowan and MrBassMan !

I am using WinXP Pro. And I do NOT already have the target m2a/m2v files in the directory.

I can use vsplit13c fine to split most .ty files ... but unfortunately vsplit13c chokes on this particular .ty file.

[olaf_sc]

MrBrassMan

Can you verify that large file support is working with the binary I buildt? I have no way of verifying it since I don't have any streams that big on my small laptop.

Cheers Olaf

Originally posted by MrBassMan
What O/S are you using? Large file support is only on linux, Win 2K and WinXP. I don't know if 0.4.1 has Mac support, 0.4.0 did not.

[Rowan]

Originally posted by olaf_sc
MrBrassMan

Can you verify that large file support is working with the binary I buildt? I have no way of verifying it since I don't have any streams that big on my small laptop.

Cheers Olaf

I can verify it, I just did a > 5 GB ty stream and it worked just fine.

[psxjunky]

I tried with another show which is 3.82GB and got the same error.

Unfortunately I don't have anything smaller than 3.15GB at the moment. I'll extract a smaller show and try with that.

I am using a Series 1 SA Tivo, if that matters.

[Rowan]

Originally posted by psxjunky
I tried with another show which is 3.82GB and got the same error.

Unfortunately I don't have anything smaller than 3.15GB at the moment. I'll extract a smaller show and try with that.

I am using a Series 1 SA Tivo, if that matters.

For the input stream place '-i "d:\movie\stream.ty"', make sure you use everything around the ' including the ",s. I am also using windows XP pro and have done over 10 ty streams with any problems.

[psxjunky]

Nope ... that didn't work either :(


Here's what I get ...

C:\>dir show.ty
Volume in drive C is WXP_C_DRIVE
Volume Serial Number is 6C87-7A33

Directory of C:\

11/20/2002 11:32 PM 3,387,555,840 show.ty
1 File(s) 3,387,555,840 bytes
0 Dir(s) 14,099,705,856 bytes free

C:\>tydemux -i "c:\show.ty" -a "c:\show.m2a" -v "c:\show.m2v"
TyStream is to small to demux
We need at least 40 chunks

C:\>

Should I try to build tydemux myself ? I have Visual Studio installed on this machine.

[Rowan]

Originally posted by psxjunky
Nope ... that didn't work either :(


Here's what I get ...

C:\>dir show.ty
Volume in drive C is WXP_C_DRIVE
Volume Serial Number is 6C87-7A33

Directory of C:\

11/20/2002 11:32 PM 3,387,555,840 show.ty
1 File(s) 3,387,555,840 bytes
0 Dir(s) 14,099,705,856 bytes free

C:\>tydemux -i "c:\show.ty" -a "c:\show.m2a" -v "c:\show.m2v"
TyStream is to small to demux
We need at least 40 chunks

C:\>

Should I try to build tydemux myself ? I have Visual Studio installed on this machine.

open a cmd window change into the 'c:\show' directory

then type:

tydemux -i show.ty -a show.m2a -v show.m2v

[psxjunky]

There is no c:\show directory ... the file show.ty is copied to the root directory of the C drive.

I have tried both the following commands and they both fail. I have tried three other Ty files as well (all bigger than 3 GB less than 4 GB) and they all failed with the same error.


tydemux -i "c:\show.ty" -a "c:\show.m2a" -v "c:\show.m2v"

and

tydemux -i show.ty -a show.m2a -v show.m2v


Since this is working for other people, I am sure it is something in my environment ... I just don't know what it is :(

Rowan ... could you please send me the binary of TyDemux that is working for you. I need just the tydemux.exe file, nothing else.

You can e-mail me here (sudiptoc@netzero.net)

Thanks !

[Enigma]

Looks like I have a nastily corrupted stream here :)

Starting demux process
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
Check Chunk: Had big gap - 41020167 ticks -> and then every thing was badrow skiped
Will try to revert and repair the TyStream better with a gap than nothig

Repairing TyStream
Failed to repair tystream
*.**********.**********.**********.**********.**********.**********.********** 1400**********.******

You really should bail after awhile on these errors, the process was up to 800MB of memory before I noticed the server was getting sluggish. Since I have the tystreams->tyfile->tydemux->mplex process totally automated, if I didn't happen to be watching the box because I had just installed the new version it could have taken the machine down with its memory consumption (at least until the linux OOM killer caught it [and probably init too :| ]).

Since everybody else is posting their wishlist for this app, I thought I would as well. Since I am using this app to distribute video extracted from my Tivo to various HTPCs throughout my home, it would make the process 100% easier if tydemux had the ability to read the tystream from STDIN and write the resulting MPEG (as of 0.5.0?) to STDOUT. Is it stands now my script has to:

1) Put the 512MB tystreams together
2) Invoke tydemux to split the stream
3) Invoke mplex (with the offset from tydemux) to create the final file to serve to the client.

This creates a huge amount of filesystem and CPU overhead that could easily be avoided. If I could write to tydemux from STDIN I could skip step 1 and just write the individual tystreams to STDOUT one by one. Once the application is truly tyremux with input from STDIN and output to STDOUT, I could pipe the stream extracting from my TiVo to tyremux and stream the output to the client. It would be an almost lagless operation compared to now where the client has to wait quite some time between the request to export the program and the final mpg being available.

Olaf, if you want a chunk of the stream that failed above, please repost your FTP IP address and let me know how large of a file you would like and from what byte offset.

[rung]

Originally posted by olaf_sc

The tydemux_W_0.4.1.zip holds Windows binaries and sources/project files from MS Visiual Studio 6.x.



Thanks Olaf! I found the following batch file very helpful in using your program in Windows. Just put this batch code tydemux.bat in the same directory as tydemux, make a shortcut on the desktop, and then just drag&drop a "ty" file on it.


cd %~d0%~p0%
tydemux -i "%~d1%~p1%~n1.ty" -a "%~d1%~p1%~n1.m2a" -v "%~d1%~p1%~n1.m2v"
pause

[olaf_sc]

Hello

Please see my inline comments:

Originally posted by Enigma
Looks like I have a nastily corrupted stream here :)

Starting demux process
......... 100......... 200......... 300.........
[snip]
to repair tystream
*.**********.**********.**********.**********.**********.**********.********** 1400**********.******


You really should bail after awhile on these errors, the process was up to 800MB of memory before I noticed the server was getting sluggish.

Yes, I will implement that in 0.4.2 ;) - didn't think to much abouit. Beside I think I will get more bug reports this way (grin).



Since everybody else is posting their wishlist for this app, I thought I
easily be avoided. If I could write to tydemux from STDIN I could skip step 1 and just write the individual tystreams to STDOUT one by one. Once the application is truly
[snip]
tyremux with input from STDIN and output to STDOUT, I could pipe the stream extracting from my TiVo to tyremux and stream the output to the client. It would be an almost lagless operation compared to now where the client has to wait quite some time between the request to export the program and the final mpg being available.


It's on my wish list too, although I can't really write to STDOUT since it's two files I'm writing ;). But FIFO handling would be nice to have. However 0.5.0 will have STDIN/OUT since it will have a integated muxer.



Olaf, if you want a chunk of the stream that failed above, please repost your FTP IP address and let me know how large of a file you would like and from what byte offset.

Yes, please upload it to my ftp server (66.121.15.35) put your nick in the filenames so I know who it is from.

It looks like if faild just after chunk 1000 so I need at least chunk 900 to 1150 but the optimal would be 750 to 1250. One chunk is 131072 bytes so the byte offset for 750 would be 98304000 (if I'm not totally wrong).

Cheers and thanks in adv for uploading the stream.

Olaf

[olaf_sc]

Ahh a nice one, please can you send me that one in a email (olaf_sc@yahoo.com).

Cheers Olaf

Originally posted by rung
Thanks Olaf! I found the following batch file very helpful in using your program in Windows. Just put this batch code tydemux.bat in the same directory as tydemux, make a shortcut on the desktop, and then just drag&drop a "ty" file on it.


cd %~d0%~p0%
tydemux -i "%~d1%~p1%~n1.ty" -a "%~d1%~p1%~n1.m2a" -v "%~d1%~p1%~n1.m2v"
pause

[olaf_sc]

Hello psxjunky

Sorry I can't really help you and it looks like other people doesn't have the same problem as you have.

Comming from Unix I can only ask the std questions - have you rebooted the box, have you reinstalled windows, etc grin. Well to be honest I hope some of the windows programmers here can help me with this one. I will try to take a look at it but I can't promise anything.

Cheers Olaf

Originally posted by psxjunky
There is no c:\show directory ... the file show.ty is copied to the root directory of the C drive.

I have tried both the following commands and they both fail. I have tried three other Ty files as well (all bigger than 3 GB less than 4 GB) and they all failed with the same error.


tydemux -i "c:\show.ty" -a "c:\show.m2a" -v "c:\show.m2v"

and

tydemux -i show.ty -a show.m2a -v show.m2v


Since this is working for other people, I am sure it is something in my environment ... I just don't know what it is :(

Rowan ... could you please send me the binary of TyDemux that is working for you. I need just the tydemux.exe file, nothing else.

You can e-mail me here (sudiptoc@netzero.net)

Thanks !

[olaf_sc]

Hello Folks,

I got a email bug report (nice) about a stream that played will with other split programs. Sure it did and tydemux found a errror very strange in my view.

Well, here is what happend (yes it will be fixed in the next bug fix release).

When I check that a chunk is valid I make a check that it only has valid records. I have a table of valid records but apperently I found one that is valid but that I classify as invalid.

Further more I found a bug in the main read_chunk function.

It's to serius bugs :(, and I therefore need to put out a little warning for 0.4.1. It will not loose sync or so but it can degrade video and audio quality. Where the steam actually has perfect video and audio present.

Cheers Olaf

[olaf_sc]

Hello Folks,

Bugs fixed, will release 0.4.2 tomorrow. It would although be very nice if anybody with repair failure uploaded thier streams to me so I can fix that bug too.

Cheers Olaf

[AlphaWolf]

olaf_sc: Just when everything was almost fully covered, now the DirecTivo Series 2 streams are ready to be prodded. :)

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=21142

[psxjunky]

Hey What do you know !!! Rowan was kind enough to send me a copy of the TyDemux Windows binary he is using (THANKS VERY MUCH ROWAN !!!) and using that I am being able to split the show perfectly :D (note that this show earlier choked both vsplit and TivoApp).

Thanks again Rowan !

[olaf_sc]

Hello

Tydemux can actually demux S2 tystreams already - S2 SA that is. Posted a post in the thread you refered to and asked if I could get one of his I assume S2 DTivo streams.

Cheers Olaf

Originally posted by AlphaWolf
olaf_sc: Just when everything was almost fully covered, now the DirecTivo Series 2 streams are ready to be prodded. :)

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=21142

[Enigma]

Originally posted by olaf_sc
Hello
It's on my wish list too, although I can't really write to STDOUT since it's two files I'm writing ;). But FIFO handling would be nice to have. However 0.5.0 will have STDIN/OUT since it will have a integated muxer.

Yeah, I have tried the FIFO route for the output but had mixed results. I also had to read the FIFO from another application for tydemux to start the read. It is a trivial change to read from STDIN rather than an input file (I already had to add it to TivoDvlpr's mfs_import as well as adding STDOUT support to Tridge's mfs_export), if you have not already done it when you release 0.4.2 I will hack it up and send you a diff.


Yes, please upload it to my ftp server (66.121.15.35) put your nick in the filenames so I know who it is from.

It looks like if faild just after chunk 1000 so I need at least chunk 900 to 1150 but the optimal would be 750 to 1250. One chunk is 131072 bytes so the byte offset for 750 would be 98304000 (if I'm not totally wrong).

Olaf

I am uploading chunks 750-1250 (64000k) as I post this. Filename is Enigma2175_Unrepairable_stream_segment.ty.

[MrBassMan]

Originally posted by olaf_sc
MrBrassMan

Can you verify that large file support is working with the binary I buildt? I have no way of verifying it since I don't have any streams that big on my small laptop.

Cheers Olaf

Afraid not at the moment - My ty streams are on a backup tape (I hope!) and I haven't got round to restoring them yet. - Can anyone else confirm it's OK?

[racingclub]

I tried a stream >2GB yesterday - worked OK!

[Rowan]

Originally posted by MrBassMan
Afraid not at the moment - My ty streams are on a backup tape (I hope!) and I haven't got round to restoring them yet. - Can anyone else confirm it's OK?

I have tested with many streams that are over 2g and one that was over 5g and everything works just fine.

olaf_sc, I got this message on one of my streams, it may just be a comment, if not do you want me to send you the first part that generates this comment?

------------------------------------
Probing TyStream .....

Tystream recorded on:
SA Tivo Series 1
Software rev 2.0 or higher

Tystream recoding audio stats:
MPEG Layer II audio
Average tyrecord (audio) size: 880
Audio frame size: 864
Audio frame time: 3240 (ticks)

Tystream recoding video stats:
Frame rate: 29.97 frames/sec

Seeking TyStream start of MPEG Layer II audio
Found start of MPEG Layer II audio
Skipping to chunk 1 - reseting chunk numbering

Starting demux process
.........Hmm is this really like this
100......... 200......... 300......... 400......... 500

[jbb]

Hello all,
I just discovered this forum and downloaded tydemux 0.4.1 (running on window XP pro). I got program data from my UK tivo using tivoweb and a thing called mfsstream to get the .ty file for a program. Then tried this :-

C:\mux>tydemux\tydemux -i gp.ty -v gp.v -a gp.a

Probing TyStream .....

Tystream recorded on:
SA Tivo Series 1
Software rev 1.3

Tystream recoding audio stats:
MPEG Layer II audio
Average tyrecord (audio) size: 864
Audio frame size: 864
Audio frame time: 3240 (ticks)

Tystream recoding video stats:
Frame rate: 29.97 frames/sec

Seeking TyStream start of MPEG Layer II audio
Found start of MPEG Layer II audio
Skipping to chunk 2 - reseting chunk numbering

Starting demux process
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
....PLEASE SEND A BUG REPORT - SAVE THE TYSTREAM - KNOW TYPE - UNKNOWN CONT
Check Chunk: chunk 1041 record: 0 - junk record type: e06 - size is 1029102
DATA:
b8 07 79 96 5a 10 8e 77 0e 92 58 61 09 98 9a 84
..... 1100......... 1200......... 1300......... 1400......... 1500
......... 1600......... 1700......... 1800......... 1900......... 2000
......... 2100......... 2200......... 2300......PLEASE SEND A BUG REPORT - SA
VE THE TYSTREAM - KNOW TYPE - UNKNOWN CONT
Check Chunk: chunk 2360 record: 0 - junk record type: e06 - size is 247168
DATA:
00 01 03 c0 30 00 dd 80 00 00 00 cc 8e aa 8b 27
... 2400......... 2500
......... 2600......... 2700......... 2800......... 2900......... 3000
......... 3100......... 3200......... 3300......... 3400......... 3500
......... 3600.......PLEASE SEND A BUG REPORT - SAVE THE TYSTREAM - KNOW TYPE -
UNKNOWN CONT
Check Chunk: chunk 3676 record: 0 - junk record type: e06 - size is 187065
DATA:
13 24 89 1f a1 e3 ae 21 03 0b 3c e3 f6 1f e0 38
.. 3700......... 3800......... 3900......... 4000
......... 4100........PLEASE SEND A BUG REPORT - SAVE THE TYSTREAM - KNOW TYPE
- UNKNOWN CONT
Check Chunk: chunk 4189 record: 0 - junk record type: e06 - size is 72285
DATA:
00 00 4a e0 30 00 d5 ec 00 00 01 6a cd 70 4f f1
. 4200......... 4300......... 4400......... 4500
......... 4600......... 4700......... 4800......
Demux process finished

A/V Sync Offset: 0ms (The -O option is not needed in mplex)
Return value 0

At the end I just get zero byte .a and .v files.
Am I doing something wrong?

[olaf_sc]

Hello

Please see my inline comments,

Originally posted by Enigma
Yeah, I have tried the FIFO route for the output but had mixed results. I also had to read the FIFO from another application for tydemux to start the read. It is a trivial change to read from STDIN rather than an input file (I already had to add it to TivoDvlpr's mfs_import as well as adding STDOUT support to Tridge's mfs_export), if you have not already done it when you release 0.4.2 I will hack it up and send you a diff.


If you could do so it would be very nice (seen me a diff I mean), my email address is olaf_sc@yahoo.com. I will however most probably not include it in the 0.4.x series of tydemux (unless it's a very clean patch).


I am uploading chunks 750-1250 (64000k) as I post this. Filename is Enigma2175_Unrepairable_stream_segment.ty.

Great, I'm loooking fwd to see what it is that is causing tydemux to repairing a stream.

Cheers Olaf

[olaf_sc]

Hello

Please see my inline comments,

Originally posted by Rowan
I have tested with many streams that are over 2g and one that was over 5g and everything works just fine.


Thanks, I was afraid that I had to upgrade to XP/2000/NT inorder to compile properly.

Anyhow I'm actually putting together a Athlon 2100 box etc... to host WinXP. So I can do some more serious programmimg work under Windowz.


olaf_sc, I got this message on one of my streams, it may just be a comment, if not do you want me to send you the first part that generates this comment?

Found start of MPEG Layer II audio
Skipping to chunk 1 - reseting chunk numbering

Starting demux process
.........Hmm is this really like this
100......... 200......... 300......... 400......... 500

You are right it's actualy just a comment of me that I should have removed in the release version. It happens when a SA has PES packet that has a B Frame, P Frame and then a SEQ header followd by a GOP header. Unusual it is but totally okay.

Cheers Olaf

[olaf_sc]

Hello

It looks like your stream triggered a bug in tydemux. Most likely it's a error during the probe - since I guess that the stream you are extracting is rather new and not a old 1.3/1.5 stream.

If you could upload chunk 0 to chunk 300 plus chunk 900 to chunk 1100 it would be really great. My ftp server is ftp://66.121.15.35/UK in your case, please put your nick in the file name so I know how it is from. This is basically the only way I can debug your problem. You can use tychopper to extract the individual chunks (a chunk is 131072 bytes). You can find tychopper* here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=78453#post78453) .

Cheers Olaf


Originally posted by jbb
Hello all,
I just discovered this forum and downloaded tydemux 0.4.1 (running on window XP pro). I got program data from my UK tivo using tivoweb and a thing called mfsstream to get the .ty file for a program. Then tried this :-

C:\mux>tydemux\tydemux -i gp.ty -v gp.v -a gp.a

Probing TyStream .....

[snip]
Tystream recoding video stats:
Frame rate: 29.97 frames/sec

Seeking TyStream start of MPEG Layer II audio
Found start of MPEG Layer II audio
Skipping to chunk 2 - reseting chunk numbering

Starting demux process
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
....PLEASE SEND A BUG REPORT - SAVE THE TYSTREAM - KNOW TYPE - UNKNOWN CONT
Check Chunk: chunk 1041 record: 0 - junk record type: e06 - size is 1029102
DATA:
b8 07 79 96 5a 10 8e 77 0e 92 58 61 09 98 9a 84
..... 1100......... 1200......... 1300......... 1400......... 1500
......... 1600.........
[snip]
Am I doing something wrong?

[olaf_sc]

Hello

Both your tries to upload has been successful :). Hence I have two files to play around with. Anyways big thanks for uploading your stream.

Originally posted by Enigma

I am uploading chunks 750-1250 (64000k) as I post this. Filename is Enigma2175_Unrepairable_stream_segment.ty.

[jbb]

Originally posted by olaf_sc
Hello

It looks like your stream triggered a bug in tydemux. Most likely it's a error during the probe - since I guess that the stream you are extracting is rather new and not a old 1.3/1.5 stream.

If you could upload chunk 0 to chunk 300 plus chunk 900 to chunk 1100 it would be really great. My ftp server is ftp://66.121.15.35/UK in your case, please put your nick in the file name so I know how it is from. This is basically the only way I can debug your problem. You can use tychopper to extract the individual chunks (a chunk is 131072 bytes). You can find tychopper* here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=78453#post78453) .

Cheers Olaf

Yes it's a new one. It's not anything I particularly wanted but it was the shortest thing I had recorded so seems like a good one to try it out with.

I'll upload soon.
Thanks for the reply

[Enigma]

Originally posted by olaf_sc
Hello

Both your tries to upload has been successful :). Hence I have two files to play around with. Anyways big thanks for uploading your stream.

Heh, my ftp client (command line Linux FTP client) is acting funny. It just hangs once it has written all the data. Perhaps it assumes greater permissions on the directory than it is granted. I killed the first transfer after it hung at the end all night, I did the second one just to be sure ;)

[olaf_sc]

Hello Enigma,

I'm terribly sorry but I don't get any error demuxing your files. However the files you upload was choped a bit funny and I had to open them in a hex editor inorder to find the start which I did 114688 bytes into the file. Something tells me that you got the wrong chunks sent to me. If possible just upload the whole thing :). I dunno what you used to chop the files. Here is how to use dd in your case

dd if=your_wacked_stream.ty of=send_to_olaf.ty bs=131072 skip=750 count=500

Cheers Olaf


Originally posted by Enigma

Yeah, I have tried the FIFO route for the output but had mixed results. I also had to read the FIFO from another application for tydemux to start the read. It is a trivial change to read from STDIN rather than an input file (I already had to add it to TivoDvlpr's mfs_import as well as adding STDOUT support to Tridge's mfs_export), if you have not already done it when you release 0.4.2 I will hack it up and send you a diff.



I am uploading chunks 750-1250 (64000k) as I post this. Filename is Enigma2175_Unrepairable_stream_segment.ty.

[artships]

Originally posted by psxjunky
C:\>tydemux -i "c:\show.ty" -a "c:\show.m2a" -v "c:\show.m2v"
TyStream is to small to demux
We need at least 40 chunks


With me, this was caused by having two versions of tydemux on my system. Version 4.0 extracted to c:\windows\tydemux.exe and generates that error. I downloaded 4.1 and extracted it, BUT it went to c:\windows\tydemux\tydemux.exe. Move this guy out of the tydemux directory and you should be ok. Note that the correct version number shows-up when you execute it.

[Enigma]

Originally posted by olaf_sc
Hello Enigma,

I'm terribly sorry but I don't get any error demuxing your files. However the files you upload was choped a bit funny and I had to open them in a hex editor inorder to find the start which I did 114688 bytes into the file. Something tells me that you got the wrong chunks sent to me. If possible just upload the whole thing :). I dunno what you used to chop the files. Here is how to use dd in your case

dd if=your_wacked_stream.ty of=send_to_olaf.ty bs=131072 skip=750 count=500

Cheers Olaf

Hmm, that's odd. I used dd but I used

dd if=BROKEN-STREAM.tyxx of=Enigma2175_Unrepairable_stream_segment.ty bs=1024 skip=90000 count=64000
Using the ty chunk size as the bs is clever, I will do that and re-send the stream.

On another note, I modified tydemux.c to accept input from STDIN, but the application experiences problems when attempting to probe the stream. Sometimes it would exit with:

Probing TyStream .....
Error in probe - exit

and sometimes it gets into a nasty loop that goes like this:

Probing TyStream .....
WARNING: Chunk 0 is out of alignment with 652 bytes
WARNING: Chunk 0 is out of alignment with 1866 bytes
WARNING: Chunk 0 is out of alignment with 892 bytes
WARNING: Chunk 0 is out of alignment with 1700 bytes
WARNING: Chunk 0 is out of alignment with 1551 bytes
WARNING: Chunk 0 is out of alignment with 1617 bytes
WARNING: Chunk 0 is out of alignment with 1732 bytes
WARNING: Chunk 0 is out of alignment with 1624 bytes
WARNING: Chunk 0 is out of alignment with 1708 bytes
WARNING: Chunk 0 is out of alignment with 357 bytes
WARNING: Chunk 0 is out of alignment with 269 bytes
WARNING: Chunk 0 is out of alignment with 44 bytes
WARNING: Chunk 0 is out of alignment with 1332 bytes
WARNING: Chunk 0 is out of alignment with 998 bytes
WARNING: Chunk 0 is out of alignment with 1860 bytes
WARNING: Chunk 0 is out of alignment with 923 bytes
WARNING: Chunk 0 is out of alignment with 770 bytes
WARNING: Chunk 0 is out of alignment with 1212 bytes
WARNING: Chunk 0 is out of alignment with 26 bytes
WARNING: Chunk 0 is out of alignment with 718 bytes
WARNING: Chunk 0 is out of alignment with 248 bytes
WARNING: Chunk 0 is out of alignment with 1239 bytes
WARNING: Chunk 0 is out of alignment with 47 bytes
WARNING: Chunk 0 is out of alignment with 1287 bytes
WARNING: Chunk 0 is out of alignment with 292 bytes
WARNING: Chunk 0 is out of alignment with 535 bytes
WARNING: Chunk 0 is out of alignment with 336 bytes
WARNING: Chunk 0 is out of alignment with 87 bytes
WARNING: Chunk 0 is out of alignment with 1136 bytes
WARNING: Chunk 0 is out of alignment with 40 bytes
WARNING: Chunk 0 is out of alignment with 1809 bytes
WARNING: Chunk 0 is out of alignment with 272 bytes
WARNING: Chunk 0 is out of alignment with 575 bytes
WARNING: Chunk 0 is out of alignment with 279 bytes
WARNING: Chunk 0 is out of alignment with 680 bytes
WARNING: Chunk 0 is out of alignment with 1114 bytes
WARNING: Chunk 0 is out of alignment with 287 bytes
WARNING: Chunk 0 is out of alignment with 90 bytes
WARNING: Chunk 0 is out of alignment with 1457 bytes
WARNING: Chunk 0 is out of alignment with 1567 bytes
WARNING: Chunk 0 is out of alignment with 3 bytes
WARNING: Chunk 0 is out of alignment with 1159 bytes
WARNING: Chunk 0 is out of alignment with 981 bytes
Terminated

That "terminated" at the end is the Linux OOM killer saving my system because the process is getting HUGE, fast! I haven't delved into the probing code, but I suspect it is far beyond my meager C abilities to code a fix for this. I will look at it and see if I can at least identify the problem. :(
edit: Can't spell

[Enigma]

Looking at std_probe_tystream() there are a boatload of seeks that I don't think are too compatible with STDIN (it's hard to seek on a stream). I guess I will have to wait until there is an official effort to make the app stream compatible for my STDIN support.

[olaf_sc]

Yes, that is one of the problems. Basically if you are feeding from stdin you will need to bypass the probe and set all the values that the probe is setting by hand (i.e. switches on the command line).

Cheers Olaf

Originally posted by Enigma
Looking at std_probe_tystream() there are a boatload of seeks that I don't think are too compatible with STDIN (it's hard to seek on a stream). I guess I will have to wait until there is an official effort to make the app stream compatible for my STDIN support.

[jbb]

Originally posted by olaf_sc
Hello

It looks like your stream triggered a bug in tydemux. Most likely it's a error during the probe - since I guess that the stream you are extracting is rather new and not a old 1.3/1.5 stream.

If you could upload chunk 0 to chunk 300 plus chunk 900 to chunk 1100 it would be really great. My ftp server is ftp://66.121.15.35/UK in your case, please put your nick in the file name so I know how it is from. This is basically the only way I can debug your problem. You can use tychopper to extract the individual chunks (a chunk is 131072 bytes). You can find tychopper* here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=78453#post78453) .

Cheers Olaf

Done and uploaded :-)

[olaf_sc]

Hello

Still terribly sorry I don't get the third file you send me to fail it just demuxes like sharm.

Sorry to say but it looks like you have to send me the whole stream is that possible for you? It would be really great if you did - big thanks in adv.

Cheers Olaf



Originally posted by Enigma
Looks like I have a nastily corrupted stream here :)

Starting demux process
......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
Check Chunk: Had big gap - 41020167 ticks -> and then every thing was badrow skiped
Will try to revert and repair the TyStream better with a gap than nothig

Repairing TyStream
Failed to repair tystream
*.**********.**********.**********.**********.**********.**********.********** 1400**********.******

You really should bail after awhile on these errors, the process was up to 800MB of memory before I noticed the server was getting sluggish. Since I have the tystreams->tyfile->tydemux->mplex process totally automated, if I didn't happen to be watching the box because I had just installed the new version it could have taken the machine down with its memory consumption (at least until the linux OOM killer caught it [and probably init too :| ]).

Since everybody else is posting their wishlist for this app, I thought I would as well. Since I am using this app to distribute video extracted from my Tivo to various HTPCs throughout my home, it would make the process 100% easier if tydemux had the ability to read the tystream from STDIN and write the resulting MPEG (as of 0.5.0?) to STDOUT. Is it stands now my script has to:

1) Put the 512MB tystreams together
2) Invoke tydemux to split the stream
3) Invoke mplex (with the offset from tydemux) to create the final file to serve to the client.

This creates a huge amount of filesystem and CPU overhead that could easily be avoided. If I could write to tydemux from STDIN I could skip step 1 and just write the individual tystreams to STDOUT one by one. Once the application is truly tyremux with input from STDIN and output to STDOUT, I could pipe the stream extracting from my TiVo to tyremux and stream the output to the client. It would be an almost lagless operation compared to now where the client has to wait quite some time between the request to export the program and the final mpg being available.

Olaf, if you want a chunk of the stream that failed above, please repost your FTP IP address and let me know how large of a file you would like and from what byte offset.

[olaf_sc]

Found the error - your stream is off by 16 byts all zeros before that hence I think it's a v13. Bug fix will be in 0.4.2 which will be released when I get Enigmas stream to fail on me ;). Ahh yes I have to fix his bug too.

Cheers Olaf

PS: Yes I have now a limit to how much we try to repair before we abort :).

Originally posted by jbb
Done and uploaded :-)

[jbb]

Great - thanks for looking at it!

[und420_247]

Recorded the DD5.1/Letterbox Spiderman off of PPV.

Here's the output when attempting to split w/0.4.1:

C:\SpruceUpWorking>..\tydemux\tydemux.exe -i Spiderman.ty -a spiderman.ac3 -v spiderman.m2v

Probing TyStream .....

Tystream recorded on:
DTivo Series 1
Software rev 2.0 or higher

Tystream recoding audio stats:
AC3 (Dolby Digital) audio
Average tyrecord (audio) size: 1552
Audio frame size: 1536
Audio frame time: 2879 (ticks)

Tystream recoding video stats:
Frame rate: 29.97 frames/sec

Seeking TyStream start of AC3 (Dolby Digital) audio
Found start of AC3 (Dolby Digital) audio
Skipping to chunk 47 - reseting chunk numbering

Starting demux process
....Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
.Init of Tystream faild!!

This just goes on and on, with the occasional . or # (300) showing up.

[olaf_sc]

Hello

If you could send me chunk 0 to 500 of the stream if would be most helpful. I.e. I would like to have the first 64MB of the stream. It's more or less impossible for me to debug without the stream. Big thanks in adv if you could do this.

My ftp server is ftp://66.121.15.35/ please put your nick in the file name.

Hmm, some explanations to the bug we have seen. Tydemux tries it's best to probe the stream so you don't need to do anything. However it fails some times :(, especially if the chunks in the tystream is missalinged. I will provide command line switches to tydemux so don't need the probe but I will also try to fix the "bugs" in the probe mechanism.

Cheers Olaf



Originally posted by und420_247
Recorded the DD5.1/Letterbox Spiderman off of PPV.

Here's the output when attempting to split w/0.4.1:

C:\SpruceUpWorking>..\tydemux\tydemux.exe -i Spiderman.ty -a spiderman.ac3 -v spiderman.m2v

Probing TyStream .....

Tystream recorded on:
DTivo Series 1
Software rev 2.0 or higher

Tystream recoding audio stats:
AC3 (Dolby Digital) audio
Average tyrecord (audio) size: 1552
Audio frame size: 1536
Audio frame time: 2879 (ticks)

Tystream recoding video stats:
Frame rate: 29.97 frames/sec

Seeking TyStream start of AC3 (Dolby Digital) audio
Found start of AC3 (Dolby Digital) audio
Skipping to chunk 47 - reseting chunk numbering

Starting demux process
....Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
Init of Tystream faild!!
.Init of Tystream faild!!

This just goes on and on, with the occasional . or # (300) showing up.

[und420_247]

Happy to upload it, but I can't seem to find the instructions on extracting just a piece of a TYstream.

Can someone refresh my memory on which utility it is I can use to send olaf just the first 500 chunks from this file?

I could use 'dd' but I don't think that's the best way to do it :)

und420_247

[olaf_sc]

Thanks!

How do it -

Unix/MacOSX:

The right way to do it is dd: Here is how to do it in your case.

dd if=yourstream.ty of=stream_to_send_olaf.ty bs=131072 count=500

Win:

Use tychopper - you can get it here (http://www.dealdatabase.com/forum/showthread.php?s=&postid=78453#post78453) .

Originally posted by und420_247
Happy to upload it, but I can't seem to find the instructions on extracting just a piece of a TYstream.

Can someone refresh my memory on which utility it is I can use to send olaf just the first 500 chunks from this file?

I could use 'dd' but I don't think that's the best way to do it :)

und420_247

[Enigma]

Hmmm, looks like I can demux the extracted chunk as well, but the source stream still throws the error. I don't think I can upload the while stream, it is 1.5 GB and with my upstream speed it would take ~ 13.5 hours at maximum efficiency. I will try some more tests and see if I can identify the source of the problem.

Originally posted by olaf_sc
Hello

Still terribly sorry I don't get the third file you send me to fail it just demuxes like sharm.

Sorry to say but it looks like you have to send me the whole stream is that possible for you? It would be really great if you did - big thanks in adv.

Cheers Olaf

[nazrak]

I've got a tystream that using tydemux 0.4.1 on produces:

WARNING: Audio frame<s> missing - compensating

throughout the entire file, and ends upcreating an audio file *larger* than the video file, full of stuttering audio... any ideas? vsplit13 crashes immediately on the stream.

[Enigma]

Olaf-

I sent you chunks 900-1600 of that stream. I think the error is actually happening around 1200-1300 and the error line overwrites the progress output. Take a look at the stream and let me know if you find the problem.

[rc3105]

Originally posted by Enigma
Hmmm, looks like I can demux the extracted chunk as well, but the source stream still throws the error. I don't think I can upload the while stream, it is 1.5 GB and with my upstream speed it would take ~ 13.5 hours at maximum efficiency. I will try some more tests and see if I can identify the source of the problem.

log tydemux's output to a text file. you should be able to see when it hits a problem. add 100 chunks to that point and there's the end of your problem segment.

pick a point halfway between the start and end of that segment, chop from there to the end, try it. if it errors, repeat until it doesn't. then you know how much "run-up" to the problem you need to repeat it.

you 'n stdin... :p

--
Riley

[Enigma]

Originally posted by rc3105
log tydemux's output to a text file. you should be able to see when it hits a problem. add 100 chunks to that point and there's the end of your problem segment.

pick a point halfway between the start and end of that segment, chop from there to the end, try it. if it errors, repeat until it doesn't. then you know how much "run-up" to the problem you need to repeat it.


That's exactly what I do, but it looks like the error message overwrites the last line of the output

So the application prints

......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
......... 1100......... 1200......... 1300....<ERROR>

But then overwrites the third line so the final output looks like

......... 100......... 200......... 300......... 400......... 500
......... 600......... 700......... 800......... 900......... 1000
Check Chunk: OOS chunk - 81 chunk(s) in a row skiped k - 1 chunk(s) in a row skiped



you 'n stdin... :p

--
Riley
Yeah, I can't help myself. It is much more efficient than writing the file to disk. :)

[shades21]

Olaf,
First, thanks for all your work.

Now my request:
Would it be possible to start a new thread each time you make a major release (0.x) available? It's a real pain to read through 25 pages of posts just to find the latest release.

Thanks!

[Gruph]

Originally posted by shades21
Olaf,
First, thanks for all your work.

Now my request:
Would it be possible to start a new thread each time you make a major release (0.x) available? It's a real pain to read through 25 pages of posts just to find the latest release.

Thanks!


Keeping it in the same thread would be easiest for us people that like to follow it.. Instead of a new thread, how about updating the first posting to this thread with a link to the posting that has the latest release?

Keep up the good work Olaf!!!!

-Gruph

[TheSaint]

Olaf,

Here's a ty that outputs this message:

probe_audio_tick_size: Warning: can't determine std audio size PLEASE SEND A REPORT - SAVE THE TYSTREAM

This was made from a SA 3.x sytem. I'm uploading it to the SA_3x directory with my nick.

TheSaint

[Jellybean]

Hi,

I got really tired of typing huge ugly command lines and having to sit and watch demux's so I could start another one.

So I wrote a simple FE that will automate it.

My workflow is tydemux->ifoedit->burn to DVD-R since I have a DTivo. If people want improvements, let me know and we can start another thread. Otherwise enjoy.

Source is included.

If/When tydemux becomes DLL'able I'll update to that.

Its really .rar file, I don't have a way of making a zip file and you can't post rars

Thanks,
JB.

[mpauley]

The Destination Directory button doesnt work for me... Runs pretty nice tho...

--mp

[olaf_sc]

Hello

I always make

1: A link at page 1 of this thread with a link to the latest stable

2: Make a separate thread with a link to the latest release every time I release a new tydemux. I just a note thread I don't answer anything in that thread.

I hope that is sufficent.

Cheers Olaf

Originally posted by shades21
Olaf,
First, thanks for all your work.

Now my request:
Would it be possible to start a new thread each time you make a major release (0.x) available? It's a real pain to read through 25 pages of posts just to find the latest release.

Thanks!

[olaf_sc]

Thanks will take a look at it first thing in the morning (night here now in PST timezone).

Cheers Olaf

Originally posted by TheSaint
Olaf,

Here's a ty that outputs this message:

probe_audio_tick_size: Warning: can't determine std audio size PLEASE SEND A REPORT - SAVE THE TYSTREAM

This was made from a SA 3.x sytem. I'm uploading it to the SA_3x directory with my nick.

TheSaint

[olaf_sc]

Ahh, nice

Will make tydemux into a DLL as of version 0.5.0 cleaning up the sources in the middle of 0.4.x is simply not a good idea. My original idea was that 0.4.0 should been a DLL/lib but I didn't have the time at the moment.

Cheers Olaf

Originally posted by Jellybean
Hi,

I got really tired of typing huge ugly command lines and having to sit and watch demux's so I could start another one.

So I wrote a simple FE that will automate it.

My workflow is tydemux->ifoedit->burn to DVD-R since I have a DTivo. If people want improvements, let me know and we can start another thread. Otherwise enjoy.

Source is included.

If/When tydemux becomes DLL'able I'll update to that.

Its really .rar file, I don't have a way of making a zip file and you can't post rars

Thanks,
JB.

[olaf_sc]

Hello Folks,


Okay 0.4.2 will come up tomorrow (night time here at the moment). I simply had to take a family day today (or this afternoon / evening) - can't code 24/7 ;).

Anyways thanks all of you for uploading streams - it is definitly a big help for me when I debug tydemux.

Cheers and thanks

Olaf

[COLOR=skyblue][COLOR=skyblue][COLOR=skyblue]

[artships]

Slow-poke 850Athlon system, but with .75G of RAM. Running anything at the same time eventually provokes tydemux41 to pop-up:

Microsoft VisualC++ Debug Library

Debug Assertion Failed!
Program: C:/WINDOWS/tydemux.exe
File: dbgheap.c
Line: 622
Expression: _CrtIsValidHeapPointer(pUserData)

Doubtless something to be resolved in 4.2.

[Jellybean]

Originally posted by mpauley
The Destination Directory button doesnt work for me... Runs pretty nice tho...

--mp

[Jellybean]

Originally posted by mpauley
The Destination Directory button doesnt work for me... Runs pretty nice tho...

--mp

There is a new thread for this app with the archive which fixes that issue.

Sorry,
MarkV.

[cwingert]

Before I go and re-invent the wheel. Has anyone had success with creating a patch for mplayer to play ty streams. I was thinking of patching tydemux/mplex/mplayer all together, so I could play them natively without the conversion to mpg.

My main purpose is to live stream ty's from my TiVo to MythTV clients around the house. I have used playitagainsam to do this function, but found it to have some annoying popping issues.

Thanks

[Enigma]

Originally posted by cwingert
Before I go and re-invent the wheel. Has anyone had success with creating a patch for mplayer to play ty streams. I was thinking of patching tydemux/mplex/mplayer all together, so I could play them natively without the conversion to mpg.

My main purpose is to live stream ty's from my TiVo to MythTV clients around the house. I have used playitagainsam to do this function, but found it to have some annoying popping issues.

Thanks

I have been doing this by extracting the video, converting it on the server then streaming it to the client. Look a few messages back in the thread for discussion of using piped input and output from tydemux. Once the re-multiplex capability is built into tydemux, it will be much easier to pipe the MPEG stream to an arbitrary output. As for the input, I am working on it but suspect Olaf or someone better with C than I will do it first. Once we have piped input and output it shouldn't be too hard to modify mythtv to pipe the output to mplayer. I would be doing this on my server, because my clients are all pretty low-horsepower machines. This may be really easy to do with the coming split between the MythTV front- and back-end processes.

I see no reason why you couldn't do the ty conversion on a standalone MythTV client if you have enough CPU cycles to spare.

[olaf_sc]

It's time for a new release of tydemux - 0.4.2 I hope you will find it useful!

This is mainly a bug fix release two new "features" are however present.

You must use -s tivo_version where tivo_version is 1 for Tivo version 1.3/1.5 or where tivo_version is 2 for Tivo version 2.x and up.

Thanks to Kurt J. Lidl we also have a BSD port (source only download the Linux zip archive and recompile).

New is also that tydemux will exit when we fail to repair a TyStream - after extensive debugging it should only happen when there is a reset of the PTS timestamps in the TyStream (as in Enigmas stream). You will need to chop up the stream in two files in this case use dd (Unix) or tychopper Windows. NOTE: I'm still very interested in streams that fails to repair!! I will fix this PTS reset problem in 0.5.0.

Tychopper is now included in the windows archive - thanks Mr BassMan!! - there is also a simple DnD tydemux.bat in the Windows archive (see batch dir) - Thanks rung!


Ahh If you see warnings like this during the probe:



Probing TyStream .....
[snip]
WARNING: Chunk 100 is out of alignment with 1024 bytes

Your stream is not aligned - please use -b to specify byte offset


Then you will need to use the new -b to specify a byte offset in this case it will look something like this:

tydemux -b 1024 -i /usr/local/samples/UK/MrBassman_Bad4.ty -v out.v -a out.a


As usually there will be bugs even if I tried my best to iron out the once I have got reports about. Anyways some of the code is more or less impossible to test unless you actually have TyStreams that has that particular error. Now this doesn't mean that tydemux isn't useful for a wider audience it only means that you should report bugs as soon as you find them. As usually I have a ftp site (66.121.15.35) where you can upload TyStreams that triggers bugs or other peculiarities in tydemux. Please email me (olaf_sc@yahoo.com) if you have really big files (1GB and above) that you want to upload.



The current features of tydemux:
--------------------------------


Demultiplex of video and audio (both MPEG and AC3/a52) from all Tivo types made - SA Tivo Series 1, SA TIVO Series 2, DTivo Series 1 and UK Tivo using software revision 1.3 and up. NOTE: DTivo Series 2 should in principle be supported but I have not yet be able to obtain a TyStream from it.

Tivo Series probe you don't need to worry about what series your TyStream is tydemux will detect it and use the right engine to demux the audio and video.

Check and correction when we have a overflow or underflow of frames.
- Overflow - This is more or less always a video sequence (gop) header and corresponding I frame that is actually missing. What happens is that tydemux doesn't catch the seq/gop hence when we count frames for the seq/gop it looks like a over flow. Tydemux repairs this problem by adding both seq, gop and I frame at the right location in the stream.

- Underflow - This is when one or several B or P frames is missing from the video sequence (gop). Tydemux will repair this by inserting a B or a P frame at the appropriate location in the stream.

Compensates for missing audio frames, this is most of the time a single audio frames that is missing. But sometimes larger parts can be missing.

Correction of A/V sync drift in TyStream, tydemux will correct gradua drift on A/V sync. Depending on frame rate tydemux will mend A/V drifts of as little as 8ms (max 20ms).

Correction of temporal reference i.e. frames out of order. tydemux will detect any error in the temporal reference in a TyStream and mend all types of temporal reference errors that can happen.

Audio probe, you don't need to worry about what type of audio a recordings has. tydemux will probe the whole TyStream, determine audio type and skip to the first chunk that has that type of audio.

TyStream repair will repair the TyStream if it detects holes/gaps in the TyStream. Up on detection of a hole/gap tydemux will align video and audio in such way that there is no or very little loss of sync. Further more tydemux will fix the start and end of the hole/gap in such way that Video and Audio doesn't deteriorate in quality.

tydemux is minimizing the Audio/Video sync offset. At the end of the execution it will tell you the offset in milli seconds.

2GB file limit removed on platforms supporting large files.




The tydemux_W_0.4.2.zip holds Windows binaries and sources/project files from MS Visiual Studio 6.x.

The tydemux_L_0.4.2.zip holds Linux x86 binaries and source/make files for gcc.

The tydemux_M_0.4.2.zip holds MacOS 10.2 binaries and source/make files for gcc.


While all of them has the "exact" same codebase I still think it's easier to post to different packages.

Ahh I must also thanks all people uploading files!!

Cheers Olaf

[olaf_sc]

Linux sources and binaries

Cheers Olaf

[olaf_sc]

And finaly Mac sour