View Full Version : Instructions to convert extracted video to SVCD
09-30-2001, 02:36 AM
I have written up some directions to convert the extracted video from a stand alone tivo to SVCD, since I have found nothing on the net about how to do it.
If you have any questions send me a private message on this forum.
10-02-2001, 04:18 AM
I'm glad to see old friends here doing great things. This is one of the neatest extraction techniques I have seen so far.
Keep up the good work :)
10-02-2001, 11:19 AM
How do you overcome the TMPGenc time limit restrictions - Only 30 days of MPEG2 processing are allowed. Do you do the date switch trick?
10-03-2001, 11:28 AM
Thanks dopester for the good how-to -- the best part I thought was explainig why High quality is best for SVCDs. I like it when people tell the WHY behind something in their how-tos.
I am not really asking for help, I was just wondering if you could verify for the group that the audio still does get out of sync sometimes. As far as I can tell, this is an ExtractStream prob that everyone encounters. Some people say they never have sync problems, but I just can't believe it. My latest attempt was ALMOST in sync until I cut out the commercials like you explined in your howto, but it messed up the sync even worse.
As far as I am concerned, this is the only remaining problem -- I agree with you that the video quality is quite acceptable using your method. Thanks again.
10-04-2001, 07:26 PM
I don't believe the audio sync issues are ExtractStream issues.
Why? Because I'm using ExtractStream and netmplex and the resulting multiplexed mpeg-2 stream plays PERFECTLY.
If you go over the the TMPGEnc fourms, it's pretty clear its a TMPGEnc bug. See my posting there... link (http://www.tmpgenc.com/cgi-bin/ebbs/board.cgi?board=tmpgenc#topic3969)
So my question is, how are you getting the audio and video to be in sync? The url at digitalregime above glosses over this part of the process.
10-04-2001, 11:00 PM
Hey ken -- thanks for the reply and the pointer to the other forum -- Well, I would be happier if it is a TMPGenc problem and not an ES problem, because I feel certain TMPGenc people will fix it sooner than ES people.
I was getting mine almost synced by using the audio offset as described in that post in the TMPGenc forum. By the end though, it is usally about .5 - 1 second out of sync.
By the way, I am assuming you have a linux box you are using for netmplex? I go tthe windows port of it and when I try to use it, it drops frames like crazy -- get about a frame a second -- if you happen to use windows, could you give my a set of instructions with it that you know works? THanks.
10-05-2001, 10:36 AM
I'm running netmplex on the Mac (I ported it).
As for the windows version, my guess is it's the buffering code which I rewrote on the Mac. I don't think it really worked...
I'll see if I can get it building on the PC and release a new version...
Oh, and for TMPGEnc fixing it...I wouldn't hold my breath. If you look at the support forum, this bug seems to have been there forever and the author doesn't seem to be active in the forum.
10-06-2001, 12:03 AM
Ken -- I have a win compiled version but I don't know if I have the source -- I'll check in the AM and send it to you if you want -- I wish I knew more about programming, but unfortunately I am pretty weak at it. I'd love for some more help with things. ...
10-06-2001, 12:05 PM
There are major differences between the versions of TMPGenc. I have tried some versions that wont open the files extracted from the tivo. I have some versions that will crash when converting them. I have some versions that work great.
I think the sync problems you guys are having may be related to the version of TMPGenc you are using.
ExtractStream -p $2 $3 $4 $5 $6
That is exactly how I rip video from the tivo. Its a script i have on the tivo. I include the -p headers option. I dont know if this helps out any, but so far with the few shows i've saved so far the sound doesnt appear to be off, but I will check and make sure since I had already seen these shows, and wasnt watching very closely.
10-06-2001, 01:23 PM
Actually, it's with all versions of TMPGEnc. The difference is it seems to be limited to multiplexed streams...
Seems I'll have to try just the netcat extraction from TiVo.
10-06-2001, 02:08 PM
What are you using to play newly created SVCD mpeg with? I have sent examples to friends who say they are horribly desynched, and it always turns out to be the software they use to play it.
I use WinDVD v3.00 to play the SVCDs and mpegs on my computer, and eveything ive created is not deschyned.
When played on my Apex AD-5131 DVD Player they are not
desychned either. If you have an example that is "desycnhed" feel free to send it over and we can see about eliminating the problem.
I noticed on your post on the TMPGenc forum you are using version 2.00. I am using Beta 12i. Some other versions I have did screw up the audio sync.
When you extract the video from the Tivo do you use the -P option? I dont know enough about mpeg, but its possible if you dont use that option some headers wont be included in the stream, and when re-encoding them sync is harder to be detemined, and thats why everything goes to hell after a while.
10-06-2001, 04:10 PM
I'm about to head out to see a movie, but when I get back later, I will try again with the -p option and see what I get -- I have WinDVD ver 2.6 and it is out of sync. I am using Bete 12j -- I'll get i and try it too --- I'll see what I get and then maybe post a file if it still isn't working -- I'd love to figure this out -- last obstacle for me......
10-06-2001, 09:30 PM
I finally think I have a grip on what is going on here.. I just converted a mpeg from tivo to svcd. I was playing it with WinDVD and as I skipped around and through the MPEG I noticed that it was out of sync at one point. Then I skipped ahead some more, and it was back in sync. Then I went back to where I thought it was out of sync previously, and it was fine. I think most of these problems are player related. I am about to burn the cut up mpegs to an SVCD and see how they play in my DVD player.. I will keep you guys posted..
10-06-2001, 09:36 PM
Nope. I think the difference is you're taking the two separate streams and running them through TMPGEnc. I'm taking a multiplexed stream from netmplex and running it through.
Playing my multiplexed stream through both Elecard AND WinDVD play fine. Playing my multiplexed stream which has been reencoded at a lower bitrate through TMPGEnc fails on BOTH Elecard and WinDVD.
From reading the TMPGEnc forums, it would appear that this may be related to the problem with people using TMPGEnc to cut commercials from MPEG-2 streams...
I'll be modifying netmplex to also allow outputting the separate streams to see if I can avoid the problem (though removing any commercials will just bring back the sync problem).
10-06-2001, 09:47 PM
Have you tried NOT using netmplex at all and setting up a samba or nfs mount to write the m2a/m2v files? So far splitting has not caused me any problems yet (Im up to about my 20th SVCD so far and none of them have been screwed up).
10-06-2001, 11:13 PM
OK, so it sounds like maybe we are getting somewhere -- Still haven't had time to try and use the -p option -- prob won't until tomorrow -- but I am netcat'ing the m2a and m2v files over as ExtractStream creates them as described in the docs. DO you think that that could be creating some problem? Is it possible that using nfs or samba would fix it -- I can't imagine this making a diff, though. It sounds like the 3 of us are all going at it differently -- dopester, what do you use to mux the streams? Nothing but TMPGenc? And are you using mpeg tools or the long encoder? And are you using a win machine or a linux box for your muxing, etc? (I think you said but I forgot.)
I am using Windows. I netcat over. Netmplex drops too many frames for me to use it, so I just mux the streams in TMPGenc after netcating. Maybe it has something to do with our systems setup etc. Just random thoughts.
10-06-2001, 11:41 PM
This is exactly how I rip the video.
1. I get the TYStream numbers from httpd
2. I run a script named riptivo on my tivo that is as follows:
ExtractStream -p $2 $3 $4 $5 $6
/var/hack/videos is a nfs mount on my linux box.
3. From there I ftp them to my windows box (since I cannot get samba to work otherwise I would omit this step and write it straight to my windows box)
4. I open the 2 seperate files up in TMPGenc and follow the directiosn on the web. This muxes the files during the bitrate lowering phase.
If my memory serves me correctly using mplex from the mjpeg tools on the linux box and then using tmpgenc did screw up from time to time. But I tried so many things its hard to remember if I stopped using mplex on the linux box and opening the already mplexed file in TMPGenc because it was unneeded or if it was because it posed problems for the output.
When you ask about MPEGTools are you talking about the MPEG Tools present in TMPGenc or another set of tools.
I am begining to wonder if netcat is possibly posing a problem. Because every video I make has no problems whatsoever.
10-07-2001, 12:30 PM
I am going to try killing myworld, running ES and saving to a dir ON the Tivo -- then I am going to FTP over the files and try muxing them -- I'll let you know.....
10-07-2001, 12:43 PM
I got a desynched video! I am not sure what caused it, but made an SVCD that is slightly desynched, but its definatly enough to tell.
I am curious if it has anything to do with what ExtractStream creates or if its just a bug in TMPGEnc.. I get all kinds of strange errors when extracint video sometime like Bogus something.. Or Missing header, skipping block, etc..
10-07-2001, 10:28 PM
willieb, using ExtractStream to export to m2v and m2a files either through netcat or smb mounts should be the same... both are reliable protocols and will correct any errors...
10-08-2001, 01:34 AM
I THINK I figured out why the video is desynched. I ripped some video the yesterday that I encoded that was desynched. I used a different script and left out the -p option in ExtractStream. This includes PES headers (whatever those are). All the previous video I dumped I used the -p option and nothing was desynched. I will run more tests to prove or disprove this theory.
10-08-2001, 05:16 PM
OK Test results so far conculde that using the directions that I wrote on digitalregime and extracting with the -p option there is no desyncing.
I am almost complete with a modified httpd.tcl script to extract the video by simply clicking on the stream. When I get it complete I will post instructions for instalation, usage, and of course the code.
10-10-2001, 09:47 PM
Hey dopester -- any more word on tests? are you *sure you have no sync problems --- some people say they dont have sync problems when what they really mean is that it is tolerable after they adjust and tweak it. Are you saying that using the -p option that everything works fine -- even after processing in tmpgenc?
I am recording on my tivo right now, and I still have not had time to try with the -p option, but I just wanted to check. If you are saying you have no problems over lots of SVCDs ripped them I think you may have figured it out.
First thin gin the AM I am goin gto start using -p and I will post my results. THanks again for your work on this -
I'd be interested if Ken could try with -p as well and post to make sure your case isn't isolated..
10-11-2001, 01:36 PM
So far everything encoded with TMPGenc when the video was extracted using the -p option is fine..
Almost everything I extracted without the -p option was screwed up.. Some a small bit that I had to look very closely, others were severly screwed up and were off by a few words..
It makes sense if you think about it.. The -P option includes some sort of headers, and if not present TMPGEnc just has to assume how to stick the audio and video together.. With these headers its possible it gives TMPGEnc some sort of refrence points so it doesnt get screwed up.
10-11-2001, 07:48 PM
Ok, dopester -- I am using -p to extract and so far so good -- I am getting synced audio! I have not done anything longer than a 30 min show yet, but so far I think it is working.
Is Ken still reading this thread? If so, have you tried using -p to extract and are you getting sync when you do it? I am recording a show at High Quality right now and I am going to extract it when it is done at 7:00 -- it is an hour long and my goal is to mux it, get it in sync and then make an SVCD -- it'll be my first one so I will keep you posted --
I am still curious though -- the ExtractStream docs say that -p isn't necessary anymore, etc etc, but everyone I can see is complaining of sync problems. Whoever is doing the docs should update them.
Or if anyone is reading this and actually can explain the PES headers to us and what they do and why the docs say they are no longer necessary that owuld be good too. Or why we might be getting sync when we include them.
For that matter, can anyone explain where the "bogus" errors come from and what result it has on extraction, I would be appreciative too.
THanks again dopester.
10-11-2001, 08:37 PM
I'm still reading. I haven't gotten a chance to run any netcat extractions as I need to finish porting it to the mac.
Has anyone subsequently tried cutting out commercials from the resulting mpeg-2 multiplexed stream or downsampling them to a lower bitrate?
I'd be interested in seeing someone:
1) ExtractStream -p through netcat
2) TMPGEnc merge the streams
3) Reencode to a lower bitrate
4) check resulting sync
5) cut out commercials
6) check resulting sync
10-15-2001, 09:35 PM
Have either of you tried the splitstream util form the yahoo board? I asm getting pretty good results -- it seems a little finicky but when it works, it works well.
10-16-2001, 11:59 AM
No I haven't. My preference is to actually fix the existing tools so they work for everyone. Especially since I'm on a Mac, all these different tools for doing the exact same thing mean I just need to split my time porting different apps.
I think my next task will be to sit down and try to understand ExtractStream/ConvertStream/SplitStream and update ExtractStream so that it works. Having all these multiple tools instead of one that runs on the TiVo means extra work for every platform.
Besides, I have currently don't have any confidence (until I hear otherwise) that any of these tools, when fed into TMPGEnc to create MPEG-2 streams will successfully pass the test I outlined above. I don't know whether it's because of non-compliant MPEG-2 Tivo streams, bugs in TMPGEnc, or what...
Since I desire to archive to SVCD, anything else is sort of moot if I can't accomplish the job end-to-end.
10-16-2001, 09:57 PM
Any luck on updating the httpd tcl script? Which tcl script are you
working from? Lightn's TivoWeb or the original script?
I'm willing to help tweek & tickle the tcl... :)
Thx, and keep up the good work, nice instruction page.
10-17-2001, 02:11 AM
I am sorry.. I was on Fall Break from school this weekend and up to now.. I have it pretty much done.. I will post it on my web site soon, most likely tomorrow evening.
I modded the original script. There are two local variables in the "Tystream" section that need to be changed for your specific
needs, and for people like Ken who dont extract to an NFS/Samba mount it can easiy be modified.
Right now it extracts 4 tystreams into one file, and then does another 4 with one overlaping. This keeps all files under the 2 gig limit, and makes it easy to cut between commercials.
Before I post it I am going to make it so you can choose to either extract all streams to one file, or split them into files slightly under 2 gigs. I am sure not everyone has the 2gb problem.
Basicly all it does is create a temp file so you can only extract one show at a time, and write a script to extract everything and then executes the script. When its done it deletes the temp file, so you can extract more video.
I just fixed my PC (I had some bad ram) so now I can start re-compressing more video to prove/disprove weather TMPGenc is posing a problem.
10-17-2001, 02:23 AM
this is kinda off the subject, sorry to do that, but i must say you REALLY seem like you know your stuff with this mpeg extraction dopester. I was just curious if this was even feasable to be done on a DirecTivo. I cant really say i understand alot about how the video was encoded and stored on the DirecTivo units, or even the format. And I was just curious if you had any insight on this. Thanks
10-17-2001, 04:20 PM
I have played with extracting video from the DirecTivo some. I havent really had time to mess with it too much, but this is what I found.
All video is stored as 480x480 VBR.
The Bitrate is still way to high for SVCD so would require still using TMPGenc to encode it to SVCD.
ExtractSream would not work when trying to extract the mpegs, but did work for tystreams when extracting the raw stream.
With a raw extracted stream I was able to play the audio under linux using mpg123, but it would play audio then noise then audio then noise..
The same holds true for the video.. It would play a few frames, and then pixelate (Like the file is corrupted.. It looked much like a fading digital satellite signal).
My hunch is that there are some diescrpencies in the tystreams stored on the DirecTivo that makes them not 100% mpeg, but I think you could easily patch the files to be mpeg again..
I havent had the time to mess with it, and it doesnt help that the MPEG file/stream formatting is only avaliable from ISO, and costs money..
I would love to look into ripping the video from the DirecTivo.. You might want to investigate the Transport IC on the DirecTivo (Its the same one as on the regular DirecTV recievers). If you can get some info about that chip you might get your answer.. I am almost certain that the output from the Transport gets streamed right to the harddrive..
I am not sure why people are not looking into this.. It could be that since it could be considered a DirecTV hack (In a twisted way) people are paranoid, but I think it wouldn't take too much effort if someone were able to analyze the stream.
If you are going to keep this topic going I suggest we move it to the DirecTivo Forum..
10-18-2001, 11:41 PM
Anyone have any thoughts why I'm getting the following
bash-2.02# ./ExtractStream -p 1958
Attempting to locate tyStream with fsid 1958...
...tyStream located, sector map follows:
Failed to connect to Video codec.
10-19-2001, 12:50 PM
You need to read the ExtractStream documentation.. You are getting the video codec not connected error because you have not set the variables VIDEO_OUT, and AUDIO_OUT..
I also am not certain that is a valid TyStream FSID so you might want to check that as well.. All TyStreams I have extracted are way larger than 1958.. Most I extract are something more like 15032..
10-19-2001, 04:23 PM
I just updated my site with more information about tivo -> svcd. I also posted the modified httpd.tcl script, which works, but has a few bugs I would like to try to fix.
I also posted information about how to create menus using TSCV which I just discovered.
10-19-2001, 07:39 PM
Originally posted by dopester
I just updated my site with more information about tivo -> svcd. I also posted the modified httpd.tcl script, which works, but has a few bugs I would like to try to fix.
Found a bug ... I'm getting a
retrying after errTmActiveLockConflict ...
When I look at now showing. It seems to be failing in proc action_NowShowing ... looks lise something wrong in the ForeachMfsFile loop. Further info when I find it.
Plus I'm converting it into a module that will work with TivoWeb.
I will send you (dopester) the script when I finish, so you can do version control.
(UPDATE) I think the problem stems from having a partial show recorded....
Just doing my part to help out. :)
Cogito Tivo Sum!
(My apologies to the Romans....)
10-19-2001, 08:20 PM
That problem has existed from the old version of the script.. I didnt know there was another httpd script.. I'll try to check in later.. My frickin Windows died.. So im under linux now, but i'll end up spending the entire evening re-building windows again..Ack..
Maybe that ram wasnt the only thing wrong with my pc..
10-19-2001, 08:35 PM
Yeah, pretty nice modular one with a decent UI (with Cascading Style Sheet support)
I've got the module stripped out, just testing the results.
Trying to figure out why I'm getting a ..
INTERNAL SERVER ERROR
action_NowShowing '/' ''
can't open object (errDbNotFound)
I probably goofed something. I'll PM you what I have so far (work in progress)
EDIT: This sucks! I can't attach a file! Will have to e-mail you a URL
10-19-2001, 09:26 PM
Thats another Tivo to SVCD page I just found today.. It looks like a much longer process, but some of you guys might want to check it out..
I'm kind of worried about the 32k->44.1k upsampling.. Thought you guys might want to see..
10-19-2001, 10:42 PM
The error is occuring in the transaction statement on the line:
set obj [db $db openidconstruction $fsid 11]
... this is in action_NowShowing ...
Cogito Tivo Sum! - I think, therefore I am a TiVo! :)
10-22-2001, 07:46 PM
Below is a simple shell script for a linux machine to convert tivo high quality to svcd. I THINK its ok, but its going to take me forever to test. I thought you guys might want to take a look, and possibly try it out.
This code assumes that you have 2 m2* files each with the same name, but differing extention. Sorry for not wraping the code, but its a good deal of pipes, and thats just how it is.
cat $1.m2v | mpeg2dec -o YUV | yuvscaler -I USE_472x480+8+0 -M RATIO_472_472_480_480 -O SIZE_480x480 -O NOT_INTERLACED | mpeg2enc -b 2500 -m 2 -o test.$1.m2v
mplex -m 2 -f 4 -o $1.muxed.mpg test.$1.m2v $1.m2a
I am trying this out using the MJPeg tools to give some of you guys a non tmpgenc option for converting tivo to svcd. I am just testing this using some High Quality extracted video from the tivo. I am testing it on a P200, so it could take quite a while. If anyone wants to see what kind of results they get please do and post the results here.
10-23-2001, 12:40 AM
Well, I'm getting no answers elsewhere, so I'll try this here as well...
A week ago I munged my rc.sysinit file and after trying to boot on my 1.3 partition, I ultimately had to restore from a 2.0.1 backup which I made just prior to installing my TiVoNet adapter.
After I did my restore, I went to my Now Showing and deleted the two programs since my backup was minimal.
Now, I'm having a problem with the httpd web server. Now Showing is giving me the wrong streams for newly recorded programs. Everything plays fine and the TiVo otherwise operates fine.
Looking in the MFS directory I see the following:
Part = 612425/12 612425/14 612425/16 612425/17 which expands to the following fsids: 617797 617798 617816 617817
httpd.tcl's Now Showing gives me:
Title: The West Wing
Description: Flashbacks show the president refining his speech for a re-election announcement.
Play TyStream: /617797/617798/617816/617817
TiVo Web shows:
These all agree, but what gets extracted doesn't correspond to the show. Instead I get:
Part of Enterprise (deleted)
Part of West Wing
Part of The Practice (deleted)
Anyone have any ideas what's wrong?
10-23-2001, 01:41 AM
No clue man..
But initial findings so far with usnig the MJPeg tools under linux look to have yielded the best results for converting tivo to SVCD.. You dont have to de-interlace it, since the MJPeg tools take care of that.. I am considering buying a speedy processor and just dedicate it to converting video now..
I will keep testing, and hopefully get more conclusive results by the end of the week. Its also likely I will modify the httpd.tcl script more to spawn a conversion process on the "conversion machine".
If anyone has tried this on a faster machine that my whimpy P-200 with 32 megs of ram let me know your results.
Obviously this isnt a completely SVCD compliant stream because of the audio, but its also possible that the array of audio tools for linux can do a better job. I prefer to be non compliant and have the best quality out there..
Any suggestions/comments let me know.
10-23-2001, 10:05 PM
# Give brief synopsis with no command arguments
if [ "$1" = "" ]; then
echo "Simple Tivo -> (S)VCD Script by dopester"
echo "Usage: "
echo " $0 filename format"
echo " "
echo " filename is without any extenion, and it also assumes"
echo " that there is a matching .m2a file for the audio"
echo " format can be SVCD or VCD"
echo " "
# remind user that you must speficy a format
if [ "$2" = "" ]; then
echo "You must give a format either VCD or SVCD"
if [ "$2" = "VCD" ] ; then
echo "Converting to VCD..(Mpeg1)"
cat $1.m2v | mpeg2dec -o YUV | yuvscaler -I USE_472x480+8+0 -M RATIO_472_352_480_240 -O VCD | mpeg2enc -m 1 -o temp.$1.vcd.m2v
mplex -m 1 -f 1 temp.$1.vcd.m2v $1.m2a -o $1-vcd.mpg
if [ "$2" = "SVCD" ] ; then
echo "Converting to SVCD..(Mpeg2)"
cat $1.m2v | mpeg2dec -o YUV | yuvscaler -I USE_472x480+8+0 -M RATIO_472_472_480_480 -O SIZE_480x480 -O NOT_INTERLACED | mpeg2enc -b 2500 -F 3 -m 2 -o temp.$1.svcd.m2v
mplex -f 3 -m 2 temp.$1.svcd.m2v $1.m2a -o $1-svcd.mpg
echo "Unknown format request. Please select between VCD or SVCD"
There is what ive come up with that seems to work quite well for VCD or SVCD. This most likely produces better SVCDs than TMPGenc does since it accomodates for interlacing, rather than using a filter to remove it. If anyone wants to try it out, or cleanup my very quick job of a shell script please feel free to.
There are two files an m2v and an m2a file. This script assumes they have the same name, but differing extension. This script also assumes you have the mjpegtools installed, as well as mpeg2dec. This script requires that the extracted streams be recorded in High Quality mode. You can change it to work with any other setting, by playing with the USE and RATIO settings on YUVSCALER.
06-23-2002, 06:01 PM
Dopester, I get this error message
Converting to SVCD..(Mpeg2)
INFO: [yuvscaler] yuvscaler (version 1.6.0) is a general scaling utility for yuv frames
INFO: [yuvscaler] (C) 2001 Xavier Biquard <firstname.lastname@example.org>
INFO: [yuvscaler] yuvscaler -h for help, or man yuvscaler
yuvscaler: option requires an argument -- O
mpeg2dec-0.2.0 (C) 2000-2001 Aaron Holtzman <email@example.com>
Invalid video driver: YUV
usage: mpeg2dec [-o mode] [-s] file
-s use program stream demultiplexer
-o video output mode
**ERROR: [yuvscaler] Could'nt read YUV4MPEG header!
./tivocombine: SIZE_480x480: command not found
also does anyone know: if I change the recording options for the resolution and bitrate of say Basic on my tivo through TivoWeb, I'm guessing for SVCD I should choose 480x480 as the resolution, but what should my maxbitrate be if I want to get say 70minutes or so(66mins plus some buffer time) on an 800mb CDR?
that way I can just set TiVo to download the show to my computer and just vsplit and mplex without having to convert anything.
Powered by vBulletin® Version 4.2.2 Copyright © 2013 vBulletin Solutions, Inc. All rights reserved.