Instructions to convert extracted video to SVCD

[dopester]

Have you tried NOT using netmplex at all and setting up a samba or nfs mount to write the m2a/m2v files? So far splitting has not caused me any problems yet (Im up to about my 20th SVCD so far and none of them have been screwed up).

[willieb]

OK, so it sounds like maybe we are getting somewhere -- Still haven't had time to try and use the -p option -- prob won't until tomorrow -- but I am netcat'ing the m2a and m2v files over as ExtractStream creates them as described in the docs. DO you think that that could be creating some problem? Is it possible that using nfs or samba would fix it -- I can't imagine this making a diff, though. It sounds like the 3 of us are all going at it differently -- dopester, what do you use to mux the streams? Nothing but TMPGenc? And are you using mpeg tools or the long encoder? And are you using a win machine or a linux box for your muxing, etc? (I think you said but I forgot.)
I am using Windows. I netcat over. Netmplex drops too many frames for me to use it, so I just mux the streams in TMPGenc after netcating. Maybe it has something to do with our systems setup etc. Just random thoughts.

[dopester]

This is exactly how I rip the video.

1. I get the TYStream numbers from httpd

2. I run a script named riptivo on my tivo that is as follows:

#!/bin/sh
export VIDEO_OUT=/var/hack/videos/$1.m2v
export AUDIO_OUT=/var/hack/videos/$1.m2a
ExtractStream -p $2 $3 $4 $5 $6

/var/hack/videos is a nfs mount on my linux box.

3. From there I ftp them to my windows box (since I cannot get samba to work otherwise I would omit this step and write it straight to my windows box)

4. I open the 2 seperate files up in TMPGenc and follow the directiosn on the web. This muxes the files during the bitrate lowering phase.

If my memory serves me correctly using mplex from the mjpeg tools on the linux box and then using tmpgenc did screw up from time to time. But I tried so many things its hard to remember if I stopped using mplex on the linux box and opening the already mplexed file in TMPGenc because it was unneeded or if it was because it posed problems for the output.

When you ask about MPEGTools are you talking about the MPEG Tools present in TMPGenc or another set of tools.

I am begining to wonder if netcat is possibly posing a problem. Because every video I make has no problems whatsoever.

[willieb]

I am going to try killing myworld, running ES and saving to a dir ON the Tivo -- then I am going to FTP over the files and try muxing them -- I'll let you know.....

[dopester]

I got a desynched video! I am not sure what caused it, but made an SVCD that is slightly desynched, but its definatly enough to tell.

I am curious if it has anything to do with what ExtractStream creates or if its just a bug in TMPGEnc.. I get all kinds of strange errors when extracint video sometime like Bogus something.. Or Missing header, skipping block, etc..

[KenNashua]

willieb, using ExtractStream to export to m2v and m2a files either through netcat or smb mounts should be the same... both are reliable protocols and will correct any errors...

[dopester]

I THINK I figured out why the video is desynched. I ripped some video the yesterday that I encoded that was desynched. I used a different script and left out the -p option in ExtractStream. This includes PES headers (whatever those are). All the previous video I dumped I used the -p option and nothing was desynched. I will run more tests to prove or disprove this theory.

[dopester]

OK Test results so far conculde that using the directions that I wrote on digitalregime and extracting with the -p option there is no desyncing.

I am almost complete with a modified httpd.tcl script to extract the video by simply clicking on the stream. When I get it complete I will post instructions for instalation, usage, and of course the code.

[willieb]

Hey dopester -- any more word on tests? are you *sure you have no sync problems --- some people say they dont have sync problems when what they really mean is that it is tolerable after they adjust and tweak it. Are you saying that using the -p option that everything works fine -- even after processing in tmpgenc?

I am recording on my tivo right now, and I still have not had time to try with the -p option, but I just wanted to check. If you are saying you have no problems over lots of SVCDs ripped them I think you may have figured it out.

First thin gin the AM I am goin gto start using -p and I will post my results. THanks again for your work on this -

I'd be interested if Ken could try with -p as well and post to make sure your case isn't isolated..

WillieB

[dopester]

So far everything encoded with TMPGenc when the video was extracted using the -p option is fine..

Almost everything I extracted without the -p option was screwed up.. Some a small bit that I had to look very closely, others were severly screwed up and were off by a few words..

It makes sense if you think about it.. The -P option includes some sort of headers, and if not present TMPGEnc just has to assume how to stick the audio and video together.. With these headers its possible it gives TMPGEnc some sort of refrence points so it doesnt get screwed up.

[willieb]

Ok, dopester -- I am using -p to extract and so far so good -- I am getting synced audio! I have not done anything longer than a 30 min show yet, but so far I think it is working.

Is Ken still reading this thread? If so, have you tried using -p to extract and are you getting sync when you do it? I am recording a show at High Quality right now and I am going to extract it when it is done at 7:00 -- it is an hour long and my goal is to mux it, get it in sync and then make an SVCD -- it'll be my first one so I will keep you posted --

I am still curious though -- the ExtractStream docs say that -p isn't necessary anymore, etc etc, but everyone I can see is complaining of sync problems. Whoever is doing the docs should update them.

Or if anyone is reading this and actually can explain the PES headers to us and what they do and why the docs say they are no longer necessary that owuld be good too. Or why we might be getting sync when we include them.

For that matter, can anyone explain where the "bogus" errors come from and what result it has on extraction, I would be appreciative too.

THanks again dopester.

[KenNashua]

I'm still reading. I haven't gotten a chance to run any netcat extractions as I need to finish porting it to the mac.

Has anyone subsequently tried cutting out commercials from the resulting mpeg-2 multiplexed stream or downsampling them to a lower bitrate?

I'd be interested in seeing someone:

1) ExtractStream -p through netcat
2) TMPGEnc merge the streams
3) Reencode to a lower bitrate
4) check resulting sync
5) cut out commercials
6) check resulting sync

[willieb]

Have either of you tried the splitstream util form the yahoo board? I asm getting pretty good results -- it seems a little finicky but when it works, it works well.

WB

[KenNashua]

No I haven't. My preference is to actually fix the existing tools so they work for everyone. Especially since I'm on a Mac, all these different tools for doing the exact same thing mean I just need to split my time porting different apps.

I think my next task will be to sit down and try to understand ExtractStream/ConvertStream/SplitStream and update ExtractStream so that it works. Having all these multiple tools instead of one that runs on the TiVo means extra work for every platform.

Besides, I have currently don't have any confidence (until I hear otherwise) that any of these tools, when fed into TMPGEnc to create MPEG-2 streams will successfully pass the test I outlined above. I don't know whether it's because of non-compliant MPEG-2 Tivo streams, bugs in TMPGEnc, or what...

Since I desire to archive to SVCD, anything else is sort of moot if I can't accomplish the job end-to-end.

[zaphod]

Dopester,
Any luck on updating the httpd tcl script? Which tcl script are you
working from? Lightn's TivoWeb or the original script?

I'm willing to help tweek & tickle the tcl...

Thx, and keep up the good work, nice instruction page.