Commercial extraction?

[whisla14]

Hey all. I was searching the internet everywhere for a program that automatically or semi-automatically removed commercials from MPEG files (files extracted from the Tivo). I couldn't find anything at all.

I know it would be kind of difficult (or impossible) for a program to remove commercials entirely on its own. I was thinking of a program that checked each frame of a given MPEG file, and when the frame was completely black (which usually marks the beginning or end of commercials), it would ask the user if that is the beginning or end of a commercial, and cut out those parts. It wouldn't be difficult, and would make removing of commercials a much faster process.

My programming skills in VB6 and VB.NET are very good, but when it comes to encoding MPEG files and such I'm lost. If someone knows of a program that does do this, or think its a worthwhile project and wants to help out on getting something going, post a reply.

The main reason I'm interested in something like this is because I like to rip my football games off my Tivo and burn them to DVD. Using TyTools' GOP editor it takes me FOREVER to manually mark the dozens of commercials. Removing the commercials is a necessity because the file with the commercials goes past the capacity of a DVD (4.7 GB), and when the commercials are removed it usually takes about 2 GB off the filesize.

Thanks.

- Andrew

[NutKase]

Hey all. I was searching the internet everywhere for a program that automatically or semi-automatically removed commercials from MPEG files (files extracted from the Tivo).

Great idea.

I think ChunkEdit (http://www.dealdatabase.com/forum/showthread.php?t=31799&highlight=commercials) might be what you're looking for but I haven't tried it.

It might be fun when I have the time.

Let us know how it works for you.


NutKase

[rc3105]

chunkedit rebuilds streams for putting them back in the tivo w/o commercials, unfortunatly, tytool / tystudio can't convert a chunkedited ty into mpg


virtualdub & several other gpl utils handle mpeg & have automatic commercial detection built in, plenty of usable source there if you're inclined to poke through it


tystudio has the ablility to create cutlists prior to extracting - only takes a minute or two even without automated detection. updating it to produce gopeditor cutlists is pretty easy


edit: not that you couldn't write something else to do ty->mpg or generate gopeditor cutlists, converting between list formats is just the easiest (laziest) way with the tools that are allready available

[whisla14]

That is what I was thinking, to just make a program that generates CUT files and have tyStudio cut them out of the MPEG for me. I thought about averaging up all the pixels on a frame and if the R, G, and B are all below 20 then consider that frame to be black, and ask the user if this is the beginning or end of a commercial segment. The problem I have is how to go about reading frames out of a MPEG file.

I downloaded VirtualDub, and it wont even open the MPEG files from the Tivo. Says "No video frames found in MPEG file." You said this software had automatic commercial detection built in? I didn't see anything on the website about that feature.

The source to this program would help me out a lot if it was in VB. I know no C++ at all.

Thanks.

- Andrew

[rc3105]

tyserver generates an index file by analyzing the ty in mfs. tystudio-editor can request them from tyserver or contruct on the fly from tyfiles on the pc hd. the editor also decodes & displays I-frames which is probably the type of routines you're looking for.

there are several virtualdub forks with mpg support. doom9.org is a good source of mpg info, links to utils, howto's, etc


unfortunatly, if you can't learn at least enough c / c++ to read source you're sorta up the proverbial creek

[captain_video]

The only flaw in your thinking is that not all shows have blank frames (i.e., black) between the program material and the commercial breaks. There have been numerous attempts at creating consumer recording devices with this type of feature built in and they have all flopped for this very reason. One missed black frame and you'll end up cutting out the program instead of the commercial. I have seen hundreds of transitions between commercials and program material with no black frames at all, making it very tricky to make frame accurate cuts without trimming any of the program material or leaving in any of the commercial.

[whisla14]

Thanks for the replies. Does anyone know of a documentation on the KeyFrame file format that tyStudio generates for the GOP Editor? I was thinking it would be much easier to read frames from a KeyFrame file than it would a MPEG file. If I could figure out how to read a KeyFrame file, I would be able to make a program that detects commercials and generates a cut list file.

Thanks.

- Andrew

[whisla14]

Captain_Video:
Thanks for responding. The program I am wanting to create will not be 100% automatic. It will still require user intervention. Most of the time there is a black frame inbetween commercial breaks, and the program I'm wanting to make will simply seek to a point where it finds a black frame, and ask the user something like "Is this the beginning or end of a commercial break?" and the user will choose either one. It will simply speed up commercial extraction, considering you dont have to manually seek to where a commercial begins or ends. If my program seeks ahead more than a minute or two without finding a black frame, it will assume there was no black frame inbetween the show and the commercial and ask the user to seek manually to the beginning of the commercial.

tyStudio already reads the cut files generated by the GOP editor, so I was planning on my program just outputting a cut file and have tyStudio do the extraction. I was hoping to find some documentation on how to read the KeyFrame file format the GOP editor uses. It seems it would be much easier to read the frames from a KeyFrame file than it would to read them from a MPEG file.

Anymore help would be great. Thanks.

- Andrew

[rc3105]

It seems it would be much easier to read the frames from a KeyFrame file than it would to read them from a MPEG file.

not really, mpeg & ty are fairly well documented whereas tytool is completely closed source

[whisla14]

So there would be no documentation on how to read the KeyFrame files?

- Andrew

[rc3105]

you'd have to ask the author

fwiw, if he'd ever shared info before it wouldn't have been necessary to write tystudio

[whisla14]

I'll I want to know is how the KeyFrame file format is structured. That's it. What it tyTool's website so I can contact the author? Thanks.

- Andrew

[captain_video]

jdiner is the author of TyTools and is also a mod here. Look for his TyTools thread in the extraction support forum.

If you're trying to use TyStudio to locate your black frames you may be out of luck. Both TyTools and TyStudio identify the GOP headers and not individual frames when generating the keyfiles. TyTools can locate individual frames but that feature is lacking in TyStudio. The black frames you seek do not always fall on a GOP header so you may have some difficulty in keying on GOP headers to make your cuts.

[rung]

A couple of comments. First, the repair routines in tystudio have never worked well when encountering a cut in the middle of a recording. This still unfixed bug results in audio sync problems. You can avoid all audio sync problems by creating multiple cut files - one for each video segment. You then can concatenate the output mpeg files or use one for each DVD chapter.

Another comment I want to make is about chunkedit files not working with the extraction tools. If you are only cutting from the beginning and end (like the cut files that work with tystudio that I described above), then there is no problem. If you go that route (i.e creating a chunkedited ty for each video segment), you could use merge.tcl to combine the video segments into one muilt-part recording, but I never tried that.

Finally, you can make tystudio compatible cut files from your Tivo by using my tcscutfile module for TCS. The cut file format info can also be freely gleaned from that module - it was release under GPL (assuming you stay GPL).

Regards,
Rung