s3tots: Convert series 3 TY/TY+/TMF to mpeg2 transport stream format

[bcc]

s3tots will convert your unencrypted TY recordings into standard mpeg2 transport streams, in a single step.

s3tots works on all combinations of OTA, analog cable, digital cable, SD, HD, and even tivocast, but not Amazon unbox. (Unbox recordings seem to include extra macrovision(?) scrambling of the video&audio streams that I'm not interested in getting into.)

s3tots has a simple command line user interface just like tytompg. Typical usage is simply:
s3tots <show>.ty
Output is to <show>.ts by default. Some switches:-y Allow overwrite of output file without confirmation,
even if it already exists
-i <file> Specify input ty/ty+/tmf file
-o <file> Output .ts to specified file
-c start,duration Only extract .ts from start thru start+duration time.
Start and end are decimal numbers, units in seconds.
-s <number> Skip <number> chunks on input before processing.
Note: if input file is a .tmf file,
this option will not work correctly
-n <number> Process <number> chunks from input
-t Omit PAT/PMT tables in generated .ts file
(included by default)
Example:
% s3tots sell.ty
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 0.7, Source is sell.ty, dest is sell.ts
TY set video,audio pid: 7c0,7c1. Audio is AC3
12956960 transport stream bytes
Recording elapsed time: 6 seconds Rate: 16.00 Mbps
%

I've had success playing back transport streams with most media players, including xine, mplayer, vlc, and powerdvd. For some streams, xine&mplayer can screw up the a/v sync during playback, even though the sync is preserved during the decode process. powerdvd&vlc don't seem to share this problem.

If you're having trouble with a media player's ability to play .ts files, or with an authoring program's support of .ts, you may want to try converting the .ts to .mpg. I believe the best tool for this is videoredo as it also allows for frame accurate editing and can repair transmission problems in your stream, all without re-transcoding the streams.

s3tots tracks and reports on continuity errors, discontinuity flags, and warped PCR timestamps. You likely can safely ignore any such errors, so long as you process your resulting .ts file with a program such as videoredo. Otherwise, such errors likely are a sign that you have reception problems and you're best off re-recording your show.

Windows 32, linux-x86, linux-x86-64 binaries attached, as well as source.

Enjoy,

B.C.

[bcc]

<more usage/troubleshooting info goes here>

[tiver]

wait ..

Is this what I think it is ?

Does this mean I can take my recorded HD shows off of my s3 tivo and transfer them to my Unix fileserver, without DRM, and keep them forever, even if I throw away the tivo ?

[Roger Dylan]

wait ..

Is this what I think it is ?

Does this mean I can take my recorded HD shows off of my s3 tivo and transfer them to my Unix fileserver, without DRM, and keep them forever, even if I throw away the tivo ?If you mean can you use this to take your recorded HD shows off of your s3 tivo and transfer them to your Unix fileserver, without DRM, even if you throw away the tivo, in order to accomplish legitimate, legal time shifting, yes.

[tiver]

Well, I don't know the legality - I mean, I am _paying for_ the cable TV, and I don't see any option possible to _pay more_ for it...

This is HUGE! I am going to buy an s3 tivo _today_ ... previously it was irrelevant because the shows were locked into a closed, proprietary system and could not be moved arbitrarily to whatever unix-based device I wanted to ...

Does tivo know how many people are disqualified from using their products because their products are "defective by design" ?

Ok, last question - does this work on _both_ the original s3 and the new "HD" tivo ? No problems using cable cards as the inputs ?

[ronnythunder]

keep in mind that this will require a prom hack (hardware modification) and a software patch to disable encryption; shows recorded prior to this won't be extractable.

ronny

[Roger Dylan]

shows recorded prior to this won't be extractable.Tivodecode will do that except for flagged shows.

[tiver]

What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?

I am thinking of the recent iPhone situation where a number of third party apps stopped working due to a iPhone software update ...

I'd really like to buy a HD tivo and get this going (that is, the ability to actually archive shows, not just have them tenuously stored on a volatile, defective-by-design, closed, non-redundant system) but I don't want to invest the time and money if tivo will just break it next month...

comments ?

[Narf54321]

What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?

I am thinking of the recent iPhone situation where a number of third party apps stopped working due to a iPhone software update ...


As mentioned, you need a PROM mod to be able to change the boot kernel. But once that's done, everything else is software on the hard drive, and the hard drive is relatively easy to fix/replace if you botch it up.

If you ever get nervous about "patched" system software, you can always allow it to update to regular Tivo software.

[bcc]

What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?None of the updates so far have done anything to break s3tots (been going strong for almost a year now). For a longer track record, take a look at other tools such as hdemux, tytompg, which were around since 2004. No update actively broke these tools. There was one update in 6.x code that changed some assumptions about timestamps in TY headers. It was easy to update tytompg to address this. (Tytool broke at that point, simply because it hasn't been maintained).

As far as system software goes, with a little brain power one has so far been able to accept OS upgrades while preserving ones modifications. There are threads around that show how that's worked out. In summary there hasn't been an active countermeasure in that area either, but once in a while a few extra steps are required (such as extra changes to installSw.itcl).


I'd really like to buy a HD tivo and get this going (that is, the ability to actually archive shows, not just have them tenuously stored on a volatile, defective-by-design, closed, non-redundant system) but I don't want to invest the time and money if tivo will just break it next month...

comments ?Note that the .tivo file format encryption has been defeated for some time now as well, and 9.2 doesn't seem to do anything to harm this. See http://tivodecode.sourceforge.net/ If you don't mind the extra encryption&decryption steps involved (they slow transfers down certainly), you may be OK with just that solution.

[tiver]

Ok, I am sorry to beat this to death, but now I am really confused ... about 6-7 months ago I asked this question:

http://www.dealdatabase.com/forum/showthread.php?t=53492

(see third post in the thread - "what does this get me?")

And I was told that full-res, HD, non-DRM extraction from an S3/HD unit was not yet possible.

But you are saying that it has been around for over a year ... so either the answer I got in that thread was wrong, and full-HD, non-DRM extraction _was_ possible in early 2007

OR, s3tots is not what I think it is ...

I thought that the s3/HD encryption was different than previous models ?

The bottom line is, I want to offload (without DRM) full-res, HD copies of protected, premium cable TV channels (HDNet, MHD, HBO, etc.) ... am I understanding that that is possible now with an S3 or an HD tivo ?

[bcc]

I was told that full-res, HD, non-DRM extraction from an S3/HD unit was not yet possible.
So the answer you got was that there were no publicly released solutions, which strictly speaking was correct at the time. s3tots already existed at that point, but wasn't public.

It's a lot tivo CSRs claiming esata drive expansion wasn't possible until yesterday :)
But you are saying that it has been around for over a year ... so either the answer I got in that thread was wrong, and full-HD, non-DRM extraction _was_ possible in early 2007The answer you got was misleading, as it was possible since Feb. 07.

OR, s3tots is not what I think it is ...

I thought that the s3/HD encryption was different than previous models ?
s3tots works as advertised, and does what you're interested in I believe. Note that s3tots is NOT involved in any decryption. The decryption defeat that s1/s2 tools relied upon is effectively the same for the s3. Infact the tivoapp patch for 9.1 is the same for the s2 and s3 models.

The TY file format for s3s is new/incompatible from s1/s2' see my new development thread for an overview of that. The bottom line is, I want to offload (without DRM) full-res, HD copies of protected, premium cable TV channels (HDNet, MHD, HBO, etc.) ... am I understanding that that is possible now with an S3 or an HD tivo ?Yes, s3tots works with both s3 and HD models, and works on content that is flagged copy protected (just not unbox). Again I should clarify: s3tots is not defeating any protection. It is simply working on your already unprotected content. Any defeat is via prom mod & tivoapp patching as with older tivo models.

[Narf54321]

So the answer you got was that there were no publicly released solutions, which strictly speaking was correct at the time. s3tots already existed at that point, but wasn't public.
..
The answer you got was misleading, as it was possible since Feb. 07.


I wouldn't say misleading, there seems to be very very few folks who had your expertise and knowledge of what needed to be done with the S3 recordings.

Seeing as I was one of the respondents in that thread, in my own defense I certainly wasn't aware of your great progress on decrypting the S3 transport streams at the time that was written.

[bcc]

I wouldn't say misleading, there seems to be very very few folks who had your expertise and knowledge of what needed to be done with the S3 recordings.Generally speaking I think it's misleading to claim to authoritatively know what hasn't/can't be done on a topic without being authoritative. That being said, it happens all the time, (see tivocommunity for daily examples :), so don't worry about it!
Seeing as I was one of the respondents in that thread, in my own defense I certainly wasn't aware of your great progress on decrypting the S3 transport streams at the time that was written.Thanks, I did figure out the new container format and the differences were hardly trivial. But I didn't have anything to do with decrypting. Someone else deserves credit for any decrypting (the tivoapp changes).

[captain_video]

FYI - TyTools works great for extracting files from an S3 Tivo without the previous limitations on file sizes for HD content as was previously experienced with the HDTivo. I believe there used to be a maximum number of FSIDs TyTools could extract before it quit. This does not to appear to be an issue with the S3 Tivos since I've been able to extract movies exceeding 25GB with no problem. However, FSIDs used to be limited to 512MB, IIRC. I've had FSIDs from an S3 Tivo be well over 1.5GB and there doesn't seem to be a consistent size for them anymore.

[ronnythunder]

i've split off the discussion about the ethics/rights/wrongs/etc. of the extraction into this (http://www.dealdatabase.com/forum/showthread.php?t=57587) thread in the general discussions forum.

please keep this thread on the topic of s3tots in specific.

ronny

[laserfan]

I am confused about what value-added s3tots brings to the party. With the new TivoToGo and Tivo Desktop 2.5.1 we can transfer non-copy-protected HD programs to our PCs and run them in WMP or Media Player Classic or edit them in VideoReDo and convert them etc. etc.

What's this do that's special-or-necessary?

[jt1134]

What's this do that's special-or-necessary?
It works for unencrypted video. TTG transfers adds extra DRM.

[laserfan]

Hmmm, so if you have a hacked S3 w/direct access to .ty files, s3tots will process anything/everything including those files w/copy-protection?

OK I get it. I guess it doesn't help me then since I only have OTA (no copy-protected programs) anyway. Thanks.

Oh, and my S3 isn't hacked either!

[bcc]

Hmmm, so if you have a hacked S3 w/direct access to .ty files, s3tots will process anything/everything including those files w/copy-protection?Yes, everything but unbox videos.
OK I get it. I guess it doesn't help me then since I only have OTA (no copy-protected programs) anyway. Thanks.

Oh, and my S3 isn't hacked either!If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo subscription costs IMO.

[Jamie]

Yes, everything but unbox videos.
If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo subscription costs IMO.The hdhomerun (http://www.silicondust.com/wiki/products/hdhomerun) is pretty nice for that.

[buckster]

I am confused about what value-added s3tots brings to the party. With the new TivoToGo and Tivo Desktop 2.5.1 we can transfer non-copy-protected HD programs to our PCs and run them in WMP or Media Player Classic or edit them in VideoReDo and convert them etc. etc.

What's this do that's special-or-necessary?

I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?

[Jamie]

I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?I believe the restriction is implemented on the tivo side, not the PC side. The tivo won't send a recording if copy protection prevents it.

[bcc]

I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?If you connect to https://<tivo-ip>/nowplaying/index.html, you'll see that the content is flagged protected by the tivo itself. It's not a flag that the PC application should be adding.
I believe you can see the flags in the CopyProtectionInfo object associated with the FSID (alternatively the xml produced by mfs_ftp). But I haven't looked into that.

On hacked systems, it would be easy enough to try stripping the CopyProtectionInfo objects from a recording to see if that alone clears the flag.

[buckster]

If you connect to https://<tivo-ip>/nowplaying/index.html, you'll see that the content is flagged protected by the tivo itself. It's not a flag that the PC application should be adding.
I believe you can see the flags in the CopyProtectionInfo object associated with the FSID (alternatively the xml produced by mfs_ftp). But I haven't looked into that.

On hacked systems, it would be easy enough to try stripping the CopyProtectionInfo objects from a recording to see if that alone clears the flag.

Sigh - until there is a kernal hack, I won't be doing this. Removing the eprom gives me the willies.

[bcc]

Sigh - until there is a kernal hack, I won't be doing this. Removing the eprom gives me the willies.I'll be surprised if anyone publishes a software-only exploit.
Looks like the system has implemented good enough security to keep out most users.

[laserfan]

If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo...Well, I have all those too! ;) (a couple of tuner cards and HDHomeRun).

But the Tivo S3 beats 'em all for simple convenience and reliability. Glad to pay the $7 bucks a month for it!

[laserfan]

Removing the eprom gives me the willies.I am an ace with the solder-sucker and iron, but have no interest to rip-open my S3! At least not until the thrill is gone (and the warranty expires). It ain't broke, so I'm not gonna try to fix it!

[ScanMan]

Should I be concerned about the following warning? The ts seems fine; opened up in VideoRedo (although has that strange color band on the right). Edited the commercials, saved the mpeg. Plays in Media Player Classic fine (no band) but the audio is a bit out of sync...
J:\Movies>s3tots -i h:\TyProjects\AFC_Divisional.ty -o j:\Movies\AFC_Divisional.ts
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.0, Source is h:\TyProjects\AFC_Divisional.ty, dest is j:\Movies\AFC_Divisional.ts
TY set video,audio pid: 800,801. Audio is AC3
Warning: Timer wrapped or warped backwards, packet 60858029, stream 800
32520922064 transport stream bytes
Recording elapsed time: 14400 seconds Rate: 17.23 Mbps
Any way to fix it? Do I need to? Think I'm going to transcode it for DVD so I'm pretty sure I can sync it up in TMPGEnc.

[bcc]

Should I be concerned about the following warning? Warning: Timer wrapped or warped backwards, packet 60858029, stream 800
The ts seems fine; opened up in VideoRedo (although has that strange color band on the right). Edited the commercials, saved the mpeg. Plays in Media Player Classic fine (no band) but the audio is a bit out of sync...
Well that error may mean that the broadcaster reset their clock, or it might mean that there is a transmission error in your recording, or it may simply be the timestamp wrapping around harmlessly. I would review the recording at the time of the error to see if it plays smoothly across that point. TS stream packets are 188 bytes, so packet 60858029 is 60858029*188=11441309452 bytes. (11441309452/32520922064)*14400=5066 seconds aprox=1hr,24min,26sec. aprox. You can analyze the transport stream timestamps with something like vobedit to see if the PCR is in fact being warped or is wrapping.

I could fix the error reporting so that it distinguishes between warping & wrapping since the later is harmless (for most players).


Now, it doesn't make sense that you're having any audio sync issues, I assume you have those right from the beginning of the recording?
It may be that your editing with videoredo is introducing your sync problems. Is the a/v in sync if you play the transport stream directly (such as with powerdvd)?

[captain_video]

Well, I have all those too! ;) (a couple of tuner cards and HDHomeRun).

But the Tivo S3 beats 'em all for simple convenience and reliability. Glad to pay the $7 bucks a month for it!
I'd agree if you're using your PC for recording but if you have a dedicated HTPC to use as a DVR then the convenience factor is a dead heat, IMHO. I've got two S3 Tivos (one hacked and one virgin) in addition to my six ATSC tuner HTPC and they're all going through my HDMI switcher to my surround syetem and HDTV. I have my Harmony 880 remote set up to control everything, including various apps on the HTPC using EventGhost and a USB-UIRT IR receiver. Operating a program such as PowerDVD or BeyondTV is no different than using the S3 Tivos.

[bcc]

The new version of s3tots clarifies the recently mentioned warning about PCRs, fixes a problem in auto-detection of PIDs from TY streams, and cleans up visual studio compilation.

[bcc]

You can analyze the transport stream timestamps with something like vobeditScratch that, I was thinking program stream not transport stream for a moment.
Anyways, I've updated s3tots so that the error reporting distinguishes between a wrapped timer and one that has inexplicably has moved backwards. For a wrapped timer you'll see for exampleWarning: Timer wrapped, packet 78145, stream 800
If this is what you see, you can safely ignore it (unless you use a buggy player such as vlc which can't handle the wrap).

Given that you only saw 1 warning, and your recording is long (4 hours) this is the most likely case.

[ScanMan]

Anyways, I've updated s3tots so that the error reporting distinguishes between a wrapped timer and one that has inexplicably has moved backwards.<snip>
Given that you only saw 1 warning, and your recording is long (4 hours) this is the most likely case.Thanks bcc, next time at least I'll know up front. Turns out it was harmless wrapping; the sync turned out to be a non-issue as well. I edited in VideoRedo, saved a transport stream and fed that to HDTV2DVD (which I know is just a front-end to ffmpeg). Anyway, transcoded to a Dual Layer DVD and although there was some expected quality loss, it still looks pretty good. Thanks again bcc, 's3tots' is a great utility.

[ScanMan]

Hey bcc, have another one for you. Using the 1.1 version of s3tots, I got the following error message.H:\TyProjects>s3tots -i AFC_Championship.ty
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is AFC_Championship.ty, dest is AFC_Championship.ts
TY set video,audio pid: 800,801. Audio is AC3
Warning: Timer wrapped, packet 45655, stream 800
idx: 10a94 Len remain: f3d0
Assertion failed: FALSE, file s3tots.c, line 228

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.FWIW, I got the same message using 1.0 version as well. It writes over 90% of the file (which is all I need anyway) and the .ts plays fine. What do you think?

[bcc]

idx: 10a94 Len remain: f3d0
Assertion failed: FALSE, file s3tots.c, line 228So your recording has some garbaged data at this point. I've never seen this, so I wasn't sure which way to code recovery (1. linear search for next transport stream packet, 2. assume next transport stream packet will be OK after 188 bytes, or 3. assume rest of chunk is garbage).

You could send in the problem chunk and then I'd know which recovery method is best. Otherwise I'll just do the most conservative thing, #3, in the next release. On the other hand, that f3d0 length field suggests that #2 would work better.

I wonder what your tivo does when you try to play across this error point.

[ScanMan]

So your recording has some garbaged data at this point. I've never seen this, so I wasn't sure which way to code recovery (1. linear search for next transport stream packet, 2. assume next transport stream packet will be OK after 188 bytes, or 3. assume rest of chunk is garbage).

You could send in the problem chunk and then I'd know which recovery method is best. Otherwise I'll just do the most conservative thing, #3, in the next release. On the other hand, that f3d0 length field suggests that #2 would work better.

I wonder what your tivo does when you try to play across this error point.Couple of quick questions:
How do I calculate the location of the bad chunk (in H:M:S) from the error message?
How do I extract the offending chunk?

[bcc]

How do I calculate the location of the bad chunk (in H:M:S) from the error message?
How do I extract the offending chunk?You would estimate the chunk number based upon the file size of your .ts file. <ts file size in bytes>/131072=# of valid chunks.
Assuming you're working with a .ty file instead of .tmf you can use the -s and -n arguments to s3tots to home in on exactly which chunk is causing a problem.
Then, tychopper or dd to extract the sample.

In anticipation, I've already coded solution #2 mentioned above, if you'd rather just test drive a possible fix.

[lrhorer]

Hey bcc,

Great program!

I'm having a little problem, though. I want s3tots to automatically translate .ty files which show up in the appropriate directory, so I wrote a couple of bash scripts to handle the processing. The problem is, s3tots complains about the filenames I use, I think almost surely because they have spaces in them. I'm running Debian Etch under Linux kernel 2.6.18-5-amd64. I tried this to replace the spaces with "\ ":


videofilein=$(echo $1 | sed 's/.\///' | sed 's/ /\ /')
videofileout=$(echo $1 | sed 's/.\///' | sed 's/ - .ty//' | sed s'/.ty//' | sed 's/ /\ /')
videofileout=$videofileout.ts

sourcedir=/RAID/Server-Main/Movies/TiVo_HD/
targetdir=/RAID/Recordings/

s3tots -i $sourcedir$videofilein -o $targetdir$videofileout
and also this to enclose the whole string in quotes:


videofilein=$(echo $1 | sed 's/.\///')
videofileout=$(echo $1 | sed 's/.\///' | sed 's/ - .ty//' | sed s'/.ty//')
videofileout=$videofileout.ts\"

sourcedir=\"/RAID/Server-Main/Movies/TiVo_HD/
targetdir=\"/RAID/Recordings/

s3tots -i $sourcedir$videofilein -o $targetdir$videofileout

The command line is passed to the routine by find, so it is a fully qualified string, and I checked that the variables have the right form. How can I get s3tots to react properly to the filenames, all of which have spaces in them? It works fine from the Linux prompt, of course by enclosing the names in quotes.

[bcc]

It works fine from the Linux prompt, of course by enclosing the names in quotes.I don't understand what you're asking; if it works from the command line what more are you expecting s3tots to do? Works for me:% s3tots 'a b c.ty'
s3tots: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Version 1.2, Source is a b c.ty, dest is a b c.ts
TY set video,audio pid: 90,91. Audio is AC3
17213468 transport stream bytes
Recording elapsed time: 47 seconds Rate: 2.72 Mbps

[lrhorer]

It doesn't work from the script, which is what I want to handle the transcoding. I tried replacing " " with "\ " and also placing quotes around the entire string as I demonstrated above, but neither works. I confirmed the script works fine if the filenames have no spaces in them, but every file extracted from the TiVo has spaces in the filename.

[Jamie]

It doesn't work from the script, which is what I want to handle the transcoding. I tried replacing " " with "\ " and also placing quotes around the entire string as I demonstrated above, but neither works. I confirmed the script works fine if the filenames have no spaces in them, but every file extracted from the TiVo has spaces in the filename.This is a shell issue, not an application issue.

Put quotes around the outside of the variable expansion if there are spaces or other special characters inside the variable value.

For example,
% x="a b c"
% y="d e f"
% s3tots -i "$in" -o "$out"
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 0.6, Source is a b c, dest is x y z
...

[lrhorer]

Thanks, I just found that out. That's counter-intuitive. I would have thought the quotes force everything inside them to be taken as a literal, rather than forcing the contents of the variables inside the quotes to be taken as a literal. In any case, it's working!

[Jamie]

Thanks, I just found that out. That's counter-intuitive. I would have thought the quotes force everything inside them to be taken as a literal, rather than forcing the contents of the variables inside the quotes to be taken as a literal. In any case, it's working!Read up on the difference between single and double quotes.

[lrhorer]

I'd agree if you're using your PC for recording but if you have a dedicated HTPC to use as a DVR then the convenience factor is a dead heat, IMHO. I've got two S3 Tivos (one hacked and one virgin) in addition to my six ATSC tuner HTPC and they're all going through my HDMI switcher to my surround syetem and HDTV.
I disagree quite strongly. First of all, the last thing I would ever want to do is record anything for my theater from an ATSC tuner, with the exception of PBS. The networks had very little other than garbage on them when I was born (50 years ago), and they've gotten rather steadily worse over time.

Secondly, the Tivo has a host of important features which to the best of my knowledge are missing or poorly implemented in every other platform. The three biggies are wishlists, suggestions, and hyperfine combination search criteria. The entire list is quite long.

Operating a program such as PowerDVD or BeyondTV is no different than using the S3 Tivos.
Oh, yes it is. I cannot speak very much to Beyond TV, but PowerDVD certainly does not let me copy between any DVR and the PC from any DVR on the network. It won't show me program information like the actors, director, year, or program synopsis on the screen while selecting videos. I also have yet to find either a motherboard or video card which works well with either of my HD sets. Most won't work at all at 1080i (let alone 1080p), and those which I have tried that do look terrible at 1080i. I have also run into significant performance issues with PowerDVD, Nero Showtime, and every other application I've tried.

Beyond TV looks like it has some interesting features; especially its Beyond TV Link and SmartSkip (if it works well) features, but the fact it runs on Windows pretty much eliminates it at the outset. Not only that, but I am skeptical CableLabs will ever certify Beyond TV for use with CableCards as long as SmartSkip and Beyond TV Link are functional. As for SDV support, I'm pretty sure you can just forget about it entirely.

I'm also not sure what analogs to TiVoWebPlus, TyTool, MFS_FTP, etc. are available for Beyond TV. I do however look forward to the day TiVo enables MPEG-4 support on the S3 class units, which of course is supported by Beyond TV.

[bcc]

It [powerdvd] won't show me program information like the actors, director, year, or program synopsis on the screen while selecting videos. It'll do all that and more (cover art) in conjunction with media center. Please, most every feature of the tivo can be had in hardware/software from others at this point. Lets face it, tivo has let the competition catch up.
I also have yet to find either a motherboard or video card which works well with either of my HD sets.I recommend the MSI NX8500GT-MTD256EH video card ($70 at newegg). Does mpeg4 avc & vc1 decoding in hardware, so it works well for blu/hd-dvd even with low-end CPUs. I do however look forward to the day TiVo enables MPEG-4 support on the S3 class units, which of course is supported by Beyond TV.If tivo wakes up and does the development to support mpeg-4 playback, I'm sure they'll want to sell you a new box for their effort. I'll be surprised if they support it on "old" hardware given their track record.

Wait a minute, what's all this doing in an s3tots thread? Moderator?

[lgkahn]

is a .ts a streameable format I have been looking for a way to stream my .tmf files stored on my pc.. the .asx etc is not streameable as the whole file must be downloaded b4 playback begins.

[Jamie]

Mpeg transport streams are designed for streaming: link (http://en.wikipedia.org/wiki/MPEG_transport_stream).

[lgkahn]

ok thanks jamie cool. but just realized this doesn't work for series 2 and looking for some way to stream my series 2 shows.

thanks

[lrhorer]

I'm getting quite a few errors along with aborted conversions. Since I don't have any other way to try to convert to .ts, I'm stuck. Here are some entries from the logs:

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Ghostbusters - Sat-03-08-08-HDNETMV.ty, dest is /RAID/Recordings/Ghostbusters - Sat-03-08-08-HDNETMV.ts
TY set video,audio pid: d8,d9. Audio is AC3
Warning: Continuity error for stream d8, packet 48324028
Warning: Continuity error for stream d9, packet 48324030
Packet 81768523, audio pid now 21 (vid pid d8)
Packet 81768533, audio pid now 22 (vid pid d8)
Audio is now MPEG
Packet 81768541, audio pid now 21 (vid pid d8)
Packet 81768601, audio pid now 22 (vid pid d8)
Audio is now MPEG
Packet 81768605, audio pid now 21 (vid pid d8)
Packet 81768648, audio pid now 22 (vid pid d8)
Audio is now MPEG
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/The Bourne Supremacy - Sat-03-08-08-TNTHD.ty, dest is /RAID/Recordings/The Bourne Supremacy - Sat-03-08-08-TNTHD.ts
TY set video,audio pid: ad,ae. Audio is AC3
Warning: Continuity error for stream ad, packet 7186308
Warning: Continuity error for stream ae, packet 7186336
Warning: Continuity error for stream ae, packet 17866957
Warning: Discontinuity flag set for stream ae, packet 17866957
Packet 17867109, stream ae Time warped by 121
Warning: Continuity error for stream ae, packet 30977698
Warning: Discontinuity flag set for stream ae, packet 30977698
Packet 30977716, stream ae Time warped by 60
Warning: Continuity error for stream ae, packet 31999756
Warning: Discontinuity flag set for stream ae, packet 31999756
Packet 31999793, stream ae Time warped by 60
Warning: Continuity error for stream ae, packet 42017037
Warning: Discontinuity flag set for stream ae, packet 42017037
Packet 42017052, stream ae Time warped by 61
Warning: Continuity error for stream ae, packet 51224128
Warning: Discontinuity flag set for stream ae, packet 51224128
Packet 51224145, stream ae Time warped by 60
Packet 73718047, audio pid now 21 (vid pid ad)
Packet 73718059, audio pid now 22 (vid pid ad)
Audio is now MPEG
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Shrek - Sun-03-23-08-TNTHD.ty, dest is /RAID/Recordings/Shrek - Sun-03-23-08-TNTHD.ts
TY set video,audio pid: ad,ae. Audio is AC3
Warning: Continuity error for stream ae, packet 11761527
Warning: Discontinuity flag set for stream ae, packet 11761527
Packet 11761538, stream ae Time warped by 31
Unparsable chunk: 27411
s3tots: s3tots.c:844: parse_chunk_rec: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Back to the Future Part II - Mon-03-24-08-HBOHD.ty, dest is /RAID/Recordings/Back to the Future Part II - Mon-03-24-08-HBOHD.tsTY set video,audio pid: 209,20a. Audio is AC3
idx: 7c1c Len remain: 1818c
s3tots: s3tots.c:228: ts_sanity: Assertion `0' failed.
Exit Code: 134

[tiver]

bcc: does ffmpeg also fall into the category of players (like VLC) that cannot handle the timer wrapping ?

I have a relatively benign error output:

[root @ home]#s3tots -i pti.tmf -o /delete/pti.ts
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is pti.tmf, dest is /delete/pti.ts
TY set video,audio pid: 1001,1002. Audio is AC3
Warning: Timer wrapped, packet 16721498, stream 1001
6111766072 transport stream bytes
Recording elapsed time: 2701 seconds Rate: 17.26 Mbps
[root @ home]#

So, just one error ... but the resulting video, while playable, refuses to allow forward/backward skip, and won't thumbnail properly.

I am using XBMC for OSX (uses ffmpeg), and I have some other .tmf files that decoded (with one or more timer wrap errors) and do not display this strangeness.

So I think osxbmc/ffmpeg can handle the timer wraps ... sometimes. But when it can't, it can't parse through the file to move in skip ahead / reverse and it can't thumbnail.

Comments ?


Edit:


Ok, actually, the clip that successfully works after decoding has these, different, errors:

TY set video,audio pid: 42,43. Audio is AC3
Warning: Discontinuity flag set for stream 42, packet 12181440
Packet 12181440, stream 42 Time warped by 2
Packet 12181448, stream 42 Time warped by 95443
Warning: Discontinuity flag set for stream 42, packet 12185310
3916405096 transport stream bytes
Recording elapsed time: 1801 seconds Rate: 16.59 Mbps

So perhaps ffmpeg can handle "time warps" but not "time wraps" ?

[bcc]

bcc: does ffmpeg also fall into the category of players (like VLC) that cannot handle the timer wrapping ?I don't use ffmpeg for playback, but I just tried a simple ffmpeg -vcodec copy -acodec copy -i x.ts x.mpg on a .ts with a wrapped timer and ffmpeg handled it ok (and the resulting recording didn't have a warped timer).
So, just one error ... but the resulting video, while playable, refuses to allow forward/backward skip, and won't thumbnail properly.Those sound like limitations of your player, trick play operations are normally possible with mpeg2-ts.So perhaps ffmpeg can handle "time warps" but not "time wraps" ?For players I've tested that can't handle PCR wrapping, the result is a glitch in video decoding (massive macroblock errors) and some players hang as well until one skips over the wrap point. For your warped timer warnings, your stream had the discontinuity flag set at the same points and so those are probably just normal transitions between programs. I believe I should simply expect the timer to warp when the discontinuity flag is set. You could check your show to answer your own question on that one.

[bcc]

I'm getting quite a few errors along with aborted conversions. Since I don't have any other way to try to convert to .ts, I'm stuck. Here are some entries from the logs:For the first one, look like you ran into garbage at end of recording after 14GB into your show. What is the size of the actual feature? I bet the resulting .ts has the complete movie.

(You might try my mfs_uberexport, here (http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15) , to see if it addresses the garbage at end of this recording. If nothing else it'll save you some disk space).

Actually looks like the same answer for the rest of your examples except perhaps the last one. Not sure what is wrong with that last one, you'd probably have to upload a sample for me to be sure. I did get rid of that assertion check in version 1.2 however, I'll see if I can post an update before leaving town.

[tiver]

bcc:

Ok, I made an error - osxbmc does not use ffmpeg for mpeg decoding, it uses libmpeg2.

So when I go to the libmpeg2 people, I should tell them:

"libmpeg2 does not handle PCR wrapping properly"

and that is good information, or can I be more specific ?

[bcc]

bcc:

Ok, I made an error - osxbmc does not use ffmpeg for mpeg decoding, it uses libmpeg2.

So when I go to the libmpeg2 people, I should tell them:

"libmpeg2 does not handle PCR wrapping properly"
No, I don't think you've shown that. You haven't mentioned whether you're seeing symptoms of PCR wrap playback problems, have you?
You've indicated that you're having trouble with trick play operations, so you could ask the player authors why they don't handle trick play operations on transport streams (ff/rew)?

xine&mplayer use libmpeg2 and don't have such trick play problems.
Also, mplayer doesn't have a problem with PCR wrap whereas xine does. So libmpeg2 isn't even the right place to point the finger for PCR wrap playback issues, as far as I know.

[tiver]

well, the FF/REW, etc., works fine on other mpegs, as well as the s3tots-converted mpegs that have time warps, but not time wraps.

So yes, there could be some other third problem that is causing the inability to jump around and thumbnail these particular mpegs, but since these "trick play operations" are successful _in general_ in osxbmc, it would seem to be the time-wrapped mpegs.

ALSO, I am curious, where do these problems come from originally ? Why is tivo recording these streams in a way that produces these errors ?

[bcc]

well, the FF/REW, etc., works fine on other mpegs, as well as the s3tots-converted mpegs that have time warps, but not time wraps.Oh, I wish I realised you were making that distinction before. In that case, it'd seem your player has a timer wrap bug. Just guessing, but I bet it tries to compute the recording duration by looking at the PCR at the end of file and the PCR at the beginning of the file, in order to estimate skip points. If one is careless with their timer math and simply subtracts, it's pretty clear how that leads to nonsense values, and leaves one unable to compute any skip points.ALSO, I am curious, where do these problems come from originally ? Why is tivo recording these streams in a way that produces these errors ?It is not an error for mpeg2-ts streams to have PCR timers that wrap. In fact the spec specifically allows for this and requires an implementation to be able to handle the situation in order to be compliant. A broadcaster may reset the PCR between shows, or at will, in which case the timer may never wrap. Otherwise the timer is going to wrap every 13+ hours, and in that case your chances of seeing the wrap in a recording are ~10% (guesstimating recording length).

To answer your question, the problem originates in players that are not sufficiently careful with their timer arithmetic. Such players may get along just fine normally with freshly authored/remuxed/transcoded content.

You may want to search for a different player, one that is accustomed to playing content that is taken directly from OTA/DVB/tivo broadcast.

[lrhorer]

For the first one, look like you ran into garbage at end of recording after 14GB into your show. What is the size of the actual feature? I bet the resulting .ts has the complete movie.
I can't say one way or the other. The source file is long gone. I don't recall for certain whether I re-recorded the show, or the .ts was intact. I think the former.

(You might try my mfs_uberexport, here (http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15) , to see if it addresses the garbage at end of this recording. If nothing else it'll save you some disk space).
Not really, because the source .ty file is deleted as soon as it's processed. Errors beyond the end of the program are no big deal. As long as the .ts is whole, everything else is fine.

Actually looks like the same answer for the rest of your examples
If you mean it ran into garbage (or what it thinks is garbage), that's pretty obvious. If you mean the .ts is whole, not even close. The resulting .ts files only contain a fraction of the program. The .ts for Back to the Future Part II, for example, only contains 2.5 out of 11 Gigs, or about 22 minutes.

I tried re-running s3tots on the source file for Back to the Future Part II, and got precisely the same results. I tried re-running Shrek, and got different results, but still far from a complete recording.

except perhaps the last one. Not sure what is wrong with that last one, you'd probably have to upload a sample for me to be sure. I did get rid of that assertion check in version 1.2 however, I'll see if I can post an update before leaving town.

What's the best way for me to trim out a chunk from the .ty file which should contain the error?

[lgkahn]

this sounds like a really good solution to the problem of vserver not working with series 3 hd stream.. you could compile this for the tivo itself and use this to read stdin and convert the stream on the fly to a .ts anyone undertaken this... in this way you could have the view links back in tivowebplus for hd streams

thanks

[Jamie]

Take a look at this (http://www.dealdatabase.com/forum/showthread.php?t=58805).

[lgkahn]

getting close anyone here see anything wrong with my run line modified the tivoweb latest extract module for .ts files here is the log entry

bash-2.02# tail tivoweb.log
216.177.20.66 - - [20/May/2008 15:51:04] /extract
loading module extract (0)
-(MOD::action_extract)--(mfs_path=/Recording/NowShowingByClassic)
216.177.20.66 - - [20/May/2008 15:51:08] /{{Lord of War}{2005-01-01}{}{0525 AM W
ed Aug 17 2005}{TMCw}}{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw
}}{}{174509}.ts
-(action_.ty sock14 {{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw}
}{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw}}{}{174509}.ts} {hea
d_req 0 last_modified 0 noheader 0 ts 1})--(fsid=174509)
s3tots: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Version 1.3, Source is -, dest is (null)
TY set video,audio pid: 21,22. Audio is MPEG
Warning: Discontinuity flag set for stream 22, packet 0
Warning: Discontinuity flag set for stream 21, packet 44
bash-2.02#


looks like it is not finding the output

here is the execute line

if $tmf {
exec bash -c "${mfs_bin_dir}mfs_uberexport -t -x $part_files" <$xmlf >@$chan &
} elseif $ts {
# mfs_uberexport -v -R 174509 | s3tots - > 2.ts
exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid | s3tots -" >@$chan &
} else {
exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid ; cat $xmlf" >@$chan &
}
return 0
}


thanks

got it working will let you know how streaming works needed this for the run line extra -

exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid | s3tots - -" >@$chan &

[lgkahn]

got this working but it doesnt stream.. ie saves b4 starting to play.. assume I need a special mime header for streaming and cannot find out what it should be for a .ts stream any help would be appreciated.

[lrhorer]

I have a file which throws an assertion failure under both s3tots and tytompg at what I assume is the same spot (it certainly looks like it from the output file). Here is the output from s3tots:
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Wait Until Dark - Wed-09-24-08-HDNETMV.ty, dest is /RAID/Server-Main/Movies/TiVo_TS/Wait Until Dark - Wed-09-24-08-HDNETMV.ts
TY set video,audio pid: 1ea,1eb. Audio is AC3
Warning: Timer wrapped, packet 42326591, stream 1ea
Packet 70874202, audio pid now 1a3 (vid pid 1ea)
Packet 70874220, audio pid now 1a4 (vid pid 1ea)
Packet 70874221, audio pid now 1a3 (vid pid 1ea)
Packet 70874268, audio pid now 1a4 (vid pid 1ea)
Packet 70874269, audio pid now 1a3 (vid pid 1ea)
Packet 70874296, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
Packet 70874297, audio pid now 1a3 (vid pid 1ea)
Packet 70874305, audio pid now 1a4 (vid pid 1ea)
Packet 70874306, audio pid now 1a3 (vid pid 1ea)
Packet 70874325, audio pid now 1a4 (vid pid 1ea)
Packet 70874326, audio pid now 1a3 (vid pid 1ea)
Packet 70874331, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

And this from tytompg:
RAID-Server:/RAID/Server-Main/Movies/TiVo_TS_Error# ./tytompg -y -3 -s 10 "Wait Until Dark - Wed-09-24-08-HDNETMV.ty"
tytompg: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Multiplexer version 1.1, Demuxer version 1.0
Source is Wait Until Dark - Wed-09-24-08-HDNETMV.ty
TY set video,audio pid: 1ea,1eb. Audio is AC3
AC3 audio at 48kHz, 384 kbps, 1536 bytes/sync frame
Video frame size is 1920x1080 - high definition. Frame rate 29.970030
Video bit rate is 38810400 bits/sec., 38810 Kbit/sec.
TY set video,audio pid: 1ea,1eb. Audio is AC3
Warning: Timer wrapped, packet 35101919, stream 1ea
Packet 63649530, audio pid now 1a3 (vid pid 1ea)
Packet 63649548, audio pid now 1a4 (vid pid 1ea)
Packet 63649549, audio pid now 1a3 (vid pid 1ea)
Packet 63649596, audio pid now 1a4 (vid pid 1ea)
Packet 63649597, audio pid now 1a3 (vid pid 1ea)
Packet 63649624, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
Warning: Tossing ac3 audio frame with bad sync data at end of packet. length 3086. Chunk 102951
Packet 63649625, audio pid now 1a3 (vid pid 1ea)
Packet 63649633, audio pid now 1a4 (vid pid 1ea)
Packet 63649634, audio pid now 1a3 (vid pid 1ea)
Packet 63649653, audio pid now 1a4 (vid pid 1ea)
Packet 63649654, audio pid now 1a3 (vid pid 1ea)
Packet 63649659, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
tytompg: demux_ty.c:416: ts_parse_adaptation: Assertion `0' failed.
Aborted

I would really like to save this movie, but with the last 3 minutes or so missing, it's pretty much a goner. Does anyone have any suggestions?