best way to edit out commercials?

[dpaschal]

When I extract and convert a stream, at best quality, it is over 2 gigs. I was trying to use Ulead Video Studio to go through and edit out the commercials to shrink the size down a bit to (hopefully!) make them fit onto a cdr so that I can archive them. It seems that when using UVS, it takes forever to go to a certain Timecode segment when I want to jump directly to it. I have over 600megs of ram on my computer, and can't figure out why this process is so slow. The hard drive's access light intermittantly blinks on and off which tells me that it is caching or something like that. Does anyone have any answers or can they point me to something better to use for editing? Thanks.

[FreydNot]

I prefer m2-edit for commercial editing. You can set up an EDL (Edit Decision List) for the program with the commercials cut out (which is frame accurate). Once you have the EDL defined, m2-edit can dump out an mpeg without transcoding the video.

This makes it pretty fast (5 or 10 minutes for a 1.5Gig file on my system) and you don't loose any video quality. I don't understand why more people don't do this.

M2-edit also has some cool tools to help you mark the commecial breaks. F3 will advance you to the next "cut". If you get close to a commercial, press F3 and you should be within a few frames of the cut.

M2-edit is a bit pricey. $300 or so if I recall. I'm using version 4.0 which does the trick for me perfectly. I tried the trial version of 5.0, but I didn't see anything I had to have.

[jemenake]

Funny you should mention this. I've currently been wrestling with Ulead Media Studio. I have a different problem from you. UMS will, after about 10 minutes of editing, start showing different frames in the thumbnails from what is being displayed in the preview window. To put it another way, I've told UMS to show thumbnails all across the entire clip (not just the ends) in the timeline. If I click at the beginning of a thumbnail, I *usually* see that exact frame in the preview. However, it sometimes gets a little confused and the preview gets shifted by about 2 seconds or so from the thumbnails. It doesn't happen with DV video, either... just mpegs.

Also, I've been getting loss of A/V sync when I generate the final movie. It used to happen only when I tried to output to MPEG-2, but now it's wacking out when I output to MPEG-1, too.

It sucks... because the muxed output from bbmpeg looks like it's immaculate. So, the extraction is happening perfectly. It's the editing which has me hitting a wall.

Adobe Premiere 6, Pinnacle Studio 7, and Vegas Video all complain that they can't read the mpeg file. Ulead is the only one that seems to be able to handle it, but then it chokes sometimes. Ugh!

I tried M2-Edit last night. It kinda sucks that I can't see a timeline full of thumbnails (so you have to have a good hunch about where the commercials are) but that's a small annoyance if it works reliably. The only problem I'm having with it is that it only generates the video. It didn't put any audio in.... (and, yes, I did have it in Video-and-Audio edit mode).

- Joe

[RedSunshine]

I think I may know why Premiere cannot read the file. Find and install the LSX MPEG-2 Pluggin. I haven't tried opening MPEG-2 files in premiere without it installed, but I only use premiere to edit out the commercials.

[IndieRockSteve]

anyone know of a program for linux that can do this? I've gotta play with mjpeg tools, I think it can, but just wanted to see if anyone has already found something.

thanks!

[FreydNot]

Originally posted by IndieRockSteve
anyone know of a program for linux that can do this? I've gotta play with mjpeg tools, I think it can, but just wanted to see if anyone has already found something.

M2-edit is available for Solaris.

http://www.mediawaresolutions.com/downloads.html

ftp://64.91.238.245/pub/m2editpro_sol_4.0.22.tar.gz

[jemenake]

Originally posted by IndieRockSteve
anyone know of a program for linux that can do this?

Did you try Cinelerra yet? http://heroinewarrior.com/cinelerra.php3

I'm tempted to install a copy of Linux on my video-processing machine just to try it....

- Joe

[Andy in NYC]

Originally posted by RedSunshine
I think I may know why Premiere cannot read the file. Find and install the LSX MPEG-2 Pluggin. I haven't tried opening MPEG-2 files in premiere without it installed, but I only use premiere to edit out the commercials.

So I need to spend $250 just to get the encoder/codec to work with the extracted files from my Tivo? (See: http://www.ligos.com/index.phtml?pi=5&n1=store/)

Can someone point me in a better/different direction?

Andy

[RedSunshine]

Well if you want to use premiere to edit, then you'll need an MPEG-2 codec. From what I understand about Premiere is it just wraps around the system codecs to open the files. It does not load it's own MPEG-2 codec.

Basicly if you can open the file with Windows Media Player, then you should be able to open it with Premiere. Now once you get it in Premeire, I would not use the LSX to reencode the files. The LSX encoder is absolute crap. I personally frameserve the exported clip and then encode with CinemaCraft 2.5.

[Andy in NYC]

Nah, when I pull files from the Tivo, Media Player is unable to play the video.

When I used TyApp, the video (brought out as Mpeg was not Media Player readable).

Starting with the .m2a and .m2v files, what is your advice?

Thanks

[IndieRockSteve]

Originally posted by jemenake


Did you try Cinelerra yet? http://heroinewarrior.com/cinelerra.php3

I'm tempted to install a copy of Linux on my video-processing machine just to try it....

- Joe

Haven't tried Cinelerra yet, but I've dabbled with Broadcast. But from what I understand, they both re-encode the mpeg.

I'm looking for something that will cut sections out of an mpeg(or cut the wanted sections out and then cat them back together in order and keeping the mpeg head and sync information et all intact.

thanks!

[jemenake]

Originally posted by IndieRockSteve

I'm looking for something that will cut sections out of an mpeg(or cut the wanted sections out and then cat them back together...


There are some issues surrounding this.

Not all MPEG frames are the same. There are "I" frames which are perfect frames that can be viewed without referring to any other frames, then there are P and B frames that depend upon 1 and 2 other frames, respectively.

Although there's some leeway in how these can be mixed (for example, you can often encode using "I-Frame only" which will give you each frame separately encoded without reference to any adjacent frame... much like MJPEG), but I don't think you're supposed to *change* it in the middle of a file.

So, now, let's say that your encoding pattern looks like this:

I - P - B - I - P - B - I - P - B etc...

A typical pattern doesn't look quite like this, but it serves for the illustration....

Now, suppose that you want to edit out a commercial. Let's say that this commercial is only two frames long (it's a SHORT commerical.... borderline subliminal, actually). So, you take out two frames and now your frame sequence isn't consistent anymore.

So, the only way to keep everything kosher is to make all cuts only where the pattern *starts*. Well.... a typical sequence involves about 15 frames or so. That's half of a second. So, if you want to make a cut, the editor would have to actually cut the video at the nearest half-second boundary. So you wouldn't have as much control over where the cuts were actually made.....

... if you wanted something to clip video without ever re-encoding, that is.

- Joe

[skyping]

The process I use to edit out commercials, while a bit complex, gives excellent results and uses free tools.

The Tools
TyTool -- thanks, jdiner, no more a/v sync issues!
WinAmp -- m2a to 44KHz WAV resampling
DVD2AVI -- m2v to avi, step 1 (d2v)
VFAPIConv -- m2v to avi, step 2 (avi)
VirtualDub -- simply, easy editing
TMPGEnc -- final MPEG transcoding
AVISynth/VirtualDub Script Reader (readavs.dll) -- enables frame server support in TMPGEnc

[B]The Process[B]
1. Extract mpa/mpv from TiVo using TyTool.
2. Resample the audio to 44KHz using WinAmp's diskwriter plug-in, force to 44KHz.
3. Generate the d2v hints file for VFAPIConv via DVD2AVI: open the m2v file and save a "project", producing a d2v file.
4. Generate the avi file using VFAPIConv: open the d2v file and convert to avi.
5. Open the avi and wav files in VirtualDub, edit out the commercials and start the frame server.
6. Open the "signpost" vdr file in TMPGEnc, select your transcoding options (VCD, SVCD, etc) and click start.

Several hours later, a final MPEG suitable for burning will be produced.

Note: you may want to start by transcoding just a subset to verify the final results. Select the subset of frames to transcode in TMPGEnc.

We've produced hundreds of VCDs of our favorite TV shows using this process to great satisfaction. Aside from the rather lengthy extraction and transcoding processes, a typical edit session takes about 10 minutes, with some shows taking slightly longer due to variance in commerical cueing.

Your mileage may vary. Much experimentation and online research was required to locate all the tools and validate the process.

Yes, we only do VCD, but nothing prevents one from doing SVCD (which does require paying for MPEG2-capable TMPGEnc). We are considering SVCD, but were not satisfied with past results nor transcoding times (even on a 1GHz Athlon).

Good luck.

[DoctorW]

What about Playitsam?

http://minnie.tuhs.org/Programs/Playitsam

This program allows you to play a video stream on your TiVo, to edit out unwanted sections (e.g ads), and to export the stream to an external storage area. From there, you can use Playitsam to play the stream back again.

[RedSunshine]

I use a program called CinemaCraft Encoder SP for all my MPEG1&2 encodings. If you have the means or can find a copy from a friend, I highly recommend trying it. It's the fastest I've seen and I've gotten only great results.

[dpaschal]

Originally posted by DoctorW
What about Playitsam?

http://minnie.tuhs.org/Programs/Playitsam

This program allows you to play a video stream on your TiVo, to edit out unwanted sections (e.g ads), and to export the stream to an external storage area. From there, you can use Playitsam to play the stream back again.

Great program, but you have to have a Linux box to do this. I currently have everything running on WinXP. When I buy my new machine, I will convert the current one into Linux, but until then, I suppose I will try to mess with Adobe Premiere.

[IndieRockSteve]

dpaschal,
this looks interesting. I'm gunna play around with it.
thanks for the post!

[Fugg]

best way? ...still looking...
(waiting for jdiner's muxer, lotta promise there)

i can edit commercials now using TMPGEnc's merge and cut under it's mpeg tools, but it cuts at gop frames. the cuts are within +/- .5 second from where i want them to be, but it's the only way i've found so far without re-encoding.

first, simple mux 'em together then use merge and cut.
quick and easy.

if you want to re-encode the m2v/m2a files, pick up the MPEG-2 Video VFAPI Plug-in for TMPGEnc (http://dealdatabase.com/forum/showthread.php?s=&threadid=13431) load 'em into TMPGEnc and the resulting re-encoded file should be editable with any good mpeg2 editing tool.

as usual, ymmv. ;)

[dpaschal]

Someone want to explain what muxing is to me?

[RedSunshine]

Muxing is combining the 2 streams into 1 file. The 2 separate audio and video streams are known as elementary streams and the combined file is called a program stream.

[dpaschal]

Originally posted by skyping
The process I use to edit out commercials, while a bit complex, gives excellent results and uses free tools.

[B]The Tools
TyTool -- thanks, jdiner, no more a/v sync issues!
WinAmp -- m2a to 44KHz WAV resampling
DVD2AVI -- m2v to avi, step 1 (d2v)
VFAPIConv -- m2v to avi, step 2 (avi)
VirtualDub -- simply, easy editing
TMPGEnc -- final MPEG transcoding
AVISynth/VirtualDub Script Reader (readavs.dll) -- enables frame server support in TMPGEnc

The Process[B]
1. Extract mpa/mpv from TiVo using TyTool.
2. Resample the audio to 44KHz using WinAmp's diskwriter plug-in, force to 44KHz.
3. Generate the d2v hints file for VFAPIConv via DVD2AVI: open the m2v file and save a "project", producing a d2v file.
4. Generate the avi file using VFAPIConv: open the d2v file and convert to avi.
5. Open the avi and wav files in VirtualDub, edit out the commercials and start the frame server.
6. Open the "signpost" vdr file in TMPGEnc, select your transcoding options (VCD, SVCD, etc) and click start.

Several hours later, a final MPEG suitable for burning will be produced.

Note: you may want to start by transcoding just a subset to verify the final results. Select the subset of frames to transcode in TMPGEnc.

We've produced hundreds of VCDs of our favorite TV shows using this process to great satisfaction. Aside from the rather lengthy extraction and transcoding processes, a typical edit session takes about 10 minutes, with some shows taking slightly longer due to variance in commerical cueing.

Your mileage may vary. Much experimentation and online research was required to locate all the tools and validate the process.

Yes, we only do VCD, but nothing prevents one from doing SVCD (which does require paying for MPEG2-capable TMPGEnc). We are considering SVCD, but were not satisfied with past results nor transcoding times (even on a 1GHz Athlon).

Good luck.

I got as far as step 5. When trying to do step 6, I am trying to open the vdr file in TMPGEnc and it doesn't understand the filetype. What am I doing wrong?

[dpaschal]

Originally posted by dpaschal


I got as far as step 5. When trying to do step 6, I am trying to open the vdr file in TMPGEnc and it doesn't understand the filetype. What am I doing wrong?

Ok, I figured out how to install the avisynth .dll file. Got that in there, but now, I am getting this message from TMPGEnc when trying to open the vdr file:

TMPGEnc: TMPGEnc.exe - Unable To Locate Component
This application has failed to start because MSVCR70.dll was not found. Re-installing the application may fix this problem.

What gives?

[skyping]

Originally posted by dpaschal


Ok, I figured out how to install the avisynth .dll file. Got that in there, but now, I am getting this message from TMPGEnc when trying to open the vdr file:

TMPGEnc: TMPGEnc.exe - Unable To Locate Component
This application has failed to start because MSVCR70.dll was not found. Re-installing the application may fix this problem.

What gives?

MSVCR70.DLL is the latest (.NUT Blamework) MicroSuck Visual C/C++ Run-Time DLL and it's not installed on your system. Unfortunately, a component in the chain requires it. Exactly which, I do not know. I avoid the problem by installing/running an older version of VirtualDub: 1.4.8. There is a mention of this issue on www.virtualdub.org -- might have a work-around available as well.

{SoapBox}
Yes, I have no love for Microsoft or Bill Gates and their continuing efforts to rip our computers from our hands. .NET Framework is yet another giant step, along with XP, to make Your computer Bill's computer. Win2K is the latest Microsoft OS I will install, and only because I like tools such as VirtualDub and games such as Anarchy Online.
{/SoapBox}

-- sky

[FreydNot]

Originally posted by rc3105
On a slightly related note, Windows Media Encoder works great for transcoding, if you have playback codecs than can handle the mpg2 audio and video.

Great information which I would like to try out. Can you give me more details on how you encode with Windows Media? I tried using M2-edit set to WM encoding, but it didn't give me any options to call out the Mpeg4 encoder.

I don't need step by step instructions. Just a general nudge towards the best tools will do just fine.

[IndieRockSteve]

For all you linux guys, I found a great program for chopping out sections of the mux'd mpeg2 files. it cuts along the GOP marks.
its called GOPchop and can be found at:

http://outflux.net/unix/software/GOPchop/

also, mpeg toolbox at

http://mpgtx.sourceforge.net/ (http://mpgtx.sourceforge.net/)

is a good program for combining and splitting large mpeg2 files into specific sizes(for putting movies, etc. onto cd)

[FreydNot]

Looks like GOPchop is a linux app. Good resource though.

rc3105: Thanks for the info. I was able to get it all running and made a 720x480 1.5Mbps version of "The Mole - Episode 201".

The only problem I had was using windows media encoder v7.01 and not v8.0. Version 8.0 appears to be a command line based encoder with no GUI.

For those who are interested, here is what I currenlty do.

Extract the file with TyTool. This gives me an m2v m2a pair when its done.

I use bbMpeg to mux the two files into a program stream (mpg file).

I load up the mpg file into M2-edit and cut out the commercials. I find with The Mole there are quite a few frames of black at the commercial boundries. This helps when M2-edit doesn't cut exactly where I tell it to.

Once I've marked all the commercials I dump out an mpeg of the edited piece. It comes out around 44 minutes. On my SA tivo recorded in high mode, the file is 1.17Gig.

Next I loaded up Windows Media Encoder v7.01. I did a whole bunch of stuff to configure up a profile for 720x480 1.5Mbs Mpeg4 encodings.

Then I just let it crunch on the edited mpeg file.

2+ hours later (on an AMD XP 2000+ system) I got a wmv file that was 447Megs. It looks great and will fit on a CDR disk so I'm pretty happy about that.

[logicTrAp]

Originally posted by IndieRockSteve
For all you linux guys, I found a great program for chopping out sections of the mux'd mpeg2 files. it cuts along the GOP marks.
its called GOPchop
Unfortunately, GOPchop tends to mangle mpeg files in ways such that they're not further manipulatable. It's probably OK if all you want to do is watch the files in Xine or vlc or whatnot, but if you want to demux, reencode, or make SVCDs from it, you'll probably find that GOPchop damages the stream beyond use in most cases.

mpgtx seems to do a better job, but its notion of "time" seems to be somewhat different from the other players I have, making it very frustrating to figure out where to split the stream to kill the commercials.

[IndieRockSteve]

how does GOPchop mangle the file? i thought the whole idea of cutting on GOP marks was to prevent this problem?

[keith721]

I think that GOPchop doesn't attempt to repair the PTS time stamp data, when 'chopping' out sections of the MPEG stream. This would leave time warps or jumps, which causes many utilities (most notably Windoze Media Player) to lock/freeze or abort.

[skyping]

Originally posted by rc3105
Been trying to figure this out... I understand the general concepts (mostly)

(snip! Riley's successful steps removed)

I'm dreading setting up a frameserver and transcoding, I would be perfectly happy (for the moment) with an edited mpeg file.

Can I set VDub video to directstream copy and then output the file from VirtualDub?

OR will I be better off just cutting and splicing in TmpgEnc?

maybe I just need more experience in the transcoding dept...


Riley,

Well, you're quite far along, so, the frameserver is actually quite easy. The problem is, if you do directstream copy and save, the resulting AVI file will be quite large. I believe the VFAPI "wrapper" codec presents a 24-bit RGB frame to the application, decoded from the original MPEG frame.

The advantage of the frameserver is avoiding the time/disk consumption of saving the edited video. Instead, each frame and audio sample is fed directly to TMPGEnc for encoding to the desired MPEG format.

BTW, MPEG editing at GOP boundaries sucks. You end up with audio pops and are not able to precisely specify, by frame, your edit points.

If you have specific questions on setting this up, send me a PM or reply here.

-- sky

[DoctorW]

Originally posted by dpaschal


Great program [playitsam], but you have to have a Linux box to do this. I currently have everything running on WinXP. When I buy my new machine, I will convert the current one into Linux, but until then, I suppose I will try to mess with Adobe Premiere.

There's a windows server, so you can run Playitsam
on the TiVo and the tystream server on your
windows box. No Linux required.