PDA

View Full Version : s3tots: Convert series 3 TY/TY+/TMF to mpeg2 transport stream format



bcc
10-23-2007, 08:40 PM
s3tots will convert your unencrypted TY recordings into standard mpeg2 transport streams, in a single step.

s3tots works on all combinations of OTA, analog cable, digital cable, SD, HD, and even tivocast, but not Amazon unbox. (Unbox recordings seem to include extra macrovision(?) scrambling of the video&audio streams that I'm not interested in getting into.)

s3tots has a simple command line user interface just like tytompg. Typical usage is simply:
s3tots <show>.ty
Output is to <show>.ts by default. Some switches:
-y Allow overwrite of output file without confirmation,
even if it already exists
-i <file> Specify input ty/ty+/tmf file
-o <file> Output .ts to specified file
-c start,duration Only extract .ts from start thru start+duration time.
Start and end are decimal numbers, units in seconds.
-s <number> Skip <number> chunks on input before processing.
Note: if input file is a .tmf file,
this option will not work correctly
-n <number> Process <number> chunks from input
-t Omit PAT/PMT tables in generated .ts file
(included by default)
Example:

% s3tots sell.ty
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 0.7, Source is sell.ty, dest is sell.ts
TY set video,audio pid: 7c0,7c1. Audio is AC3
12956960 transport stream bytes
Recording elapsed time: 6 seconds Rate: 16.00 Mbps
%

I've had success playing back transport streams with most media players, including xine, mplayer, vlc, and powerdvd. For some streams, xine&mplayer can screw up the a/v sync during playback, even though the sync is preserved during the decode process. powerdvd&vlc don't seem to share this problem.

If you're having trouble with a media player's ability to play .ts files, or with an authoring program's support of .ts, you may want to try converting the .ts to .mpg. I believe the best tool for this is videoredo as it also allows for frame accurate editing and can repair transmission problems in your stream, all without re-transcoding the streams.

s3tots tracks and reports on continuity errors, discontinuity flags, and warped PCR timestamps. You likely can safely ignore any such errors, so long as you process your resulting .ts file with a program such as videoredo. Otherwise, such errors likely are a sign that you have reception problems and you're best off re-recording your show.

Windows 32, linux-x86, linux-x86-64 binaries attached, as well as source.
Update: 1.4 version (source code only; as change vs 1.3 is only source code portability for osx).

Enjoy,

B.C.

bcc
10-23-2007, 08:41 PM
<more usage/troubleshooting info goes here>

tiver
10-24-2007, 09:49 AM
wait ..

Is this what I think it is ?

Does this mean I can take my recorded HD shows off of my s3 tivo and transfer them to my Unix fileserver, without DRM, and keep them forever, even if I throw away the tivo ?

Roger Dylan
10-24-2007, 09:55 AM
wait ..

Is this what I think it is ?

Does this mean I can take my recorded HD shows off of my s3 tivo and transfer them to my Unix fileserver, without DRM, and keep them forever, even if I throw away the tivo ?If you mean can you use this to take your recorded HD shows off of your s3 tivo and transfer them to your Unix fileserver, without DRM, even if you throw away the tivo, in order to accomplish legitimate, legal time shifting, yes.

tiver
10-24-2007, 10:05 AM
Well, I don't know the legality - I mean, I am _paying for_ the cable TV, and I don't see any option possible to _pay more_ for it...

This is HUGE! I am going to buy an s3 tivo _today_ ... previously it was irrelevant because the shows were locked into a closed, proprietary system and could not be moved arbitrarily to whatever unix-based device I wanted to ...

Does tivo know how many people are disqualified from using their products because their products are "defective by design" ?

Ok, last question - does this work on _both_ the original s3 and the new "HD" tivo ? No problems using cable cards as the inputs ?

ronnythunder
10-24-2007, 10:48 AM
keep in mind that this will require a prom hack (hardware modification) and a software patch to disable encryption; shows recorded prior to this won't be extractable.

ronny

Roger Dylan
10-24-2007, 10:54 AM
shows recorded prior to this won't be extractable.Tivodecode will do that except for flagged shows.

tiver
10-24-2007, 11:29 AM
What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?

I am thinking of the recent iPhone situation where a number of third party apps stopped working due to a iPhone software update ...

I'd really like to buy a HD tivo and get this going (that is, the ability to actually archive shows, not just have them tenuously stored on a volatile, defective-by-design, closed, non-redundant system) but I don't want to invest the time and money if tivo will just break it next month...

comments ?

Narf54321
10-24-2007, 01:10 PM
What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?

I am thinking of the recent iPhone situation where a number of third party apps stopped working due to a iPhone software update ...


As mentioned, you need a PROM mod to be able to change the boot kernel. But once that's done, everything else is software on the hard drive, and the hard drive is relatively easy to fix/replace if you botch it up.

If you ever get nervous about "patched" system software, you can always allow it to update to regular Tivo software.

bcc
10-24-2007, 01:38 PM
What is the track record for future tivo software updates/upgrades breaking this setup, should I dive into it ?None of the updates so far have done anything to break s3tots (been going strong for almost a year now). For a longer track record, take a look at other tools such as hdemux, tytompg, which were around since 2004. No update actively broke these tools. There was one update in 6.x code that changed some assumptions about timestamps in TY headers. It was easy to update tytompg to address this. (Tytool broke at that point, simply because it hasn't been maintained).

As far as system software goes, with a little brain power one has so far been able to accept OS upgrades while preserving ones modifications. There are threads around that show how that's worked out. In summary there hasn't been an active countermeasure in that area either, but once in a while a few extra steps are required (such as extra changes to installSw.itcl).



I'd really like to buy a HD tivo and get this going (that is, the ability to actually archive shows, not just have them tenuously stored on a volatile, defective-by-design, closed, non-redundant system) but I don't want to invest the time and money if tivo will just break it next month...

comments ?Note that the .tivo file format encryption has been defeated for some time now as well, and 9.2 doesn't seem to do anything to harm this. See http://tivodecode.sourceforge.net/ If you don't mind the extra encryption&decryption steps involved (they slow transfers down certainly), you may be OK with just that solution.

tiver
10-24-2007, 03:59 PM
Ok, I am sorry to beat this to death, but now I am really confused ... about 6-7 months ago I asked this question:

http://www.dealdatabase.com/forum/showthread.php?t=53492

(see third post in the thread - "what does this get me?")

And I was told that full-res, HD, non-DRM extraction from an S3/HD unit was not yet possible.

But you are saying that it has been around for over a year ... so either the answer I got in that thread was wrong, and full-HD, non-DRM extraction _was_ possible in early 2007

OR, s3tots is not what I think it is ...

I thought that the s3/HD encryption was different than previous models ?

The bottom line is, I want to offload (without DRM) full-res, HD copies of protected, premium cable TV channels (HDNet, MHD, HBO, etc.) ... am I understanding that that is possible now with an S3 or an HD tivo ?

bcc
10-24-2007, 04:26 PM
I was told that full-res, HD, non-DRM extraction from an S3/HD unit was not yet possible.
So the answer you got was that there were no publicly released solutions, which strictly speaking was correct at the time. s3tots already existed at that point, but wasn't public.

It's a lot tivo CSRs claiming esata drive expansion wasn't possible until yesterday :)

But you are saying that it has been around for over a year ... so either the answer I got in that thread was wrong, and full-HD, non-DRM extraction _was_ possible in early 2007The answer you got was misleading, as it was possible since Feb. 07.


OR, s3tots is not what I think it is ...

I thought that the s3/HD encryption was different than previous models ?
s3tots works as advertised, and does what you're interested in I believe. Note that s3tots is NOT involved in any decryption. The decryption defeat that s1/s2 tools relied upon is effectively the same for the s3. Infact the tivoapp patch for 9.1 is the same for the s2 and s3 models.

The TY file format for s3s is new/incompatible from s1/s2' see my new development thread for an overview of that.
The bottom line is, I want to offload (without DRM) full-res, HD copies of protected, premium cable TV channels (HDNet, MHD, HBO, etc.) ... am I understanding that that is possible now with an S3 or an HD tivo ?Yes, s3tots works with both s3 and HD models, and works on content that is flagged copy protected (just not unbox). Again I should clarify: s3tots is not defeating any protection. It is simply working on your already unprotected content. Any defeat is via prom mod & tivoapp patching as with older tivo models.

Narf54321
10-24-2007, 08:27 PM
So the answer you got was that there were no publicly released solutions, which strictly speaking was correct at the time. s3tots already existed at that point, but wasn't public.
..
The answer you got was misleading, as it was possible since Feb. 07.


I wouldn't say misleading, there seems to be very very few folks who had your expertise and knowledge of what needed to be done with the S3 recordings.

Seeing as I was one of the respondents in that thread, in my own defense I certainly wasn't aware of your great progress on decrypting the S3 transport streams at the time that was written.

bcc
10-24-2007, 08:54 PM
I wouldn't say misleading, there seems to be very very few folks who had your expertise and knowledge of what needed to be done with the S3 recordings.Generally speaking I think it's misleading to claim to authoritatively know what hasn't/can't be done on a topic without being authoritative. That being said, it happens all the time, (see tivocommunity for daily examples :), so don't worry about it!

Seeing as I was one of the respondents in that thread, in my own defense I certainly wasn't aware of your great progress on decrypting the S3 transport streams at the time that was written.Thanks, I did figure out the new container format and the differences were hardly trivial. But I didn't have anything to do with decrypting. Someone else deserves credit for any decrypting (the tivoapp changes).

captain_video
10-25-2007, 07:11 AM
FYI - TyTools works great for extracting files from an S3 Tivo without the previous limitations on file sizes for HD content as was previously experienced with the HDTivo. I believe there used to be a maximum number of FSIDs TyTools could extract before it quit. This does not to appear to be an issue with the S3 Tivos since I've been able to extract movies exceeding 25GB with no problem. However, FSIDs used to be limited to 512MB, IIRC. I've had FSIDs from an S3 Tivo be well over 1.5GB and there doesn't seem to be a consistent size for them anymore.

ronnythunder
10-25-2007, 02:33 PM
i've split off the discussion about the ethics/rights/wrongs/etc. of the extraction into this (http://www.dealdatabase.com/forum/showthread.php?t=57587) thread in the general discussions forum.

please keep this thread on the topic of s3tots in specific.

ronny

laserfan
11-01-2007, 09:49 PM
I am confused about what value-added s3tots brings to the party. With the new TivoToGo and Tivo Desktop 2.5.1 we can transfer non-copy-protected HD programs to our PCs and run them in WMP or Media Player Classic or edit them in VideoReDo and convert them etc. etc.

What's this do that's special-or-necessary?

jt1134
11-01-2007, 10:04 PM
What's this do that's special-or-necessary?
It works for unencrypted video. TTG transfers adds extra DRM.

laserfan
11-01-2007, 10:09 PM
Hmmm, so if you have a hacked S3 w/direct access to .ty files, s3tots will process anything/everything including those files w/copy-protection?

OK I get it. I guess it doesn't help me then since I only have OTA (no copy-protected programs) anyway. Thanks.

Oh, and my S3 isn't hacked either!

bcc
11-01-2007, 10:38 PM
Hmmm, so if you have a hacked S3 w/direct access to .ty files, s3tots will process anything/everything including those files w/copy-protection?Yes, everything but unbox videos.

OK I get it. I guess it doesn't help me then since I only have OTA (no copy-protected programs) anyway. Thanks.

Oh, and my S3 isn't hacked either!If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo subscription costs IMO.

Jamie
11-01-2007, 11:04 PM
Yes, everything but unbox videos.
If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo subscription costs IMO.The hdhomerun (http://www.silicondust.com/wiki/products/hdhomerun) is pretty nice for that.

buckster
11-01-2007, 11:16 PM
I am confused about what value-added s3tots brings to the party. With the new TivoToGo and Tivo Desktop 2.5.1 we can transfer non-copy-protected HD programs to our PCs and run them in WMP or Media Player Classic or edit them in VideoReDo and convert them etc. etc.

What's this do that's special-or-necessary?

I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?

Jamie
11-01-2007, 11:25 PM
I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?I believe the restriction is implemented on the tivo side, not the PC side. The tivo won't send a recording if copy protection prevents it.

bcc
11-01-2007, 11:26 PM
I hate to be such a bloody newb, but it seems to me that the copy protected content flag in "tivotogo" should be easily removed in the tivotogo program to allow these "copy protected" programs to be downloaded to our computers. Has anyone done this, or am I missing something very basic here?If you connect to https://<tivo-ip>/nowplaying/index.html, you'll see that the content is flagged protected by the tivo itself. It's not a flag that the PC application should be adding.
I believe you can see the flags in the CopyProtectionInfo object associated with the FSID (alternatively the xml produced by mfs_ftp). But I haven't looked into that.

On hacked systems, it would be easy enough to try stripping the CopyProtectionInfo objects from a recording to see if that alone clears the flag.

buckster
11-02-2007, 07:20 AM
If you connect to https://<tivo-ip>/nowplaying/index.html, you'll see that the content is flagged protected by the tivo itself. It's not a flag that the PC application should be adding.
I believe you can see the flags in the CopyProtectionInfo object associated with the FSID (alternatively the xml produced by mfs_ftp). But I haven't looked into that.

On hacked systems, it would be easy enough to try stripping the CopyProtectionInfo objects from a recording to see if that alone clears the flag.

Sigh - until there is a kernal hack, I won't be doing this. Removing the eprom gives me the willies.

bcc
11-02-2007, 02:05 PM
Sigh - until there is a kernal hack, I won't be doing this. Removing the eprom gives me the willies.I'll be surprised if anyone publishes a software-only exploit.
Looks like the system has implemented good enough security to keep out most users.

laserfan
11-02-2007, 04:45 PM
If you only care about OTA, and you're interested in extracting to a PC, you might as well just go with a PC tuner card and skip the tivo...Well, I have all those too! ;) (a couple of tuner cards and HDHomeRun).

But the Tivo S3 beats 'em all for simple convenience and reliability. Glad to pay the $7 bucks a month for it!

laserfan
11-02-2007, 04:48 PM
Removing the eprom gives me the willies.I am an ace with the solder-sucker and iron, but have no interest to rip-open my S3! At least not until the thrill is gone (and the warranty expires). It ain't broke, so I'm not gonna try to fix it!

ScanMan
01-14-2008, 11:01 PM
Should I be concerned about the following warning? The ts seems fine; opened up in VideoRedo (although has that strange color band on the right). Edited the commercials, saved the mpeg. Plays in Media Player Classic fine (no band) but the audio is a bit out of sync...

J:\Movies>s3tots -i h:\TyProjects\AFC_Divisional.ty -o j:\Movies\AFC_Divisional.ts
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.0, Source is h:\TyProjects\AFC_Divisional.ty, dest is j:\Movies\AFC_Divisional.ts
TY set video,audio pid: 800,801. Audio is AC3
Warning: Timer wrapped or warped backwards, packet 60858029, stream 800
32520922064 transport stream bytes
Recording elapsed time: 14400 seconds Rate: 17.23 Mbps
Any way to fix it? Do I need to? Think I'm going to transcode it for DVD so I'm pretty sure I can sync it up in TMPGEnc.

bcc
01-15-2008, 02:53 PM
Should I be concerned about the following warning? Warning: Timer wrapped or warped backwards, packet 60858029, stream 800
The ts seems fine; opened up in VideoRedo (although has that strange color band on the right). Edited the commercials, saved the mpeg. Plays in Media Player Classic fine (no band) but the audio is a bit out of sync...
Well that error may mean that the broadcaster reset their clock, or it might mean that there is a transmission error in your recording, or it may simply be the timestamp wrapping around harmlessly. I would review the recording at the time of the error to see if it plays smoothly across that point. TS stream packets are 188 bytes, so packet 60858029 is 60858029*188=11441309452 bytes. (11441309452/32520922064)*14400=5066 seconds aprox=1hr,24min,26sec. aprox. You can analyze the transport stream timestamps with something like vobedit to see if the PCR is in fact being warped or is wrapping.

I could fix the error reporting so that it distinguishes between warping & wrapping since the later is harmless (for most players).


Now, it doesn't make sense that you're having any audio sync issues, I assume you have those right from the beginning of the recording?
It may be that your editing with videoredo is introducing your sync problems. Is the a/v in sync if you play the transport stream directly (such as with powerdvd)?

captain_video
01-16-2008, 07:02 AM
Well, I have all those too! ;) (a couple of tuner cards and HDHomeRun).

But the Tivo S3 beats 'em all for simple convenience and reliability. Glad to pay the $7 bucks a month for it!
I'd agree if you're using your PC for recording but if you have a dedicated HTPC to use as a DVR then the convenience factor is a dead heat, IMHO. I've got two S3 Tivos (one hacked and one virgin) in addition to my six ATSC tuner HTPC and they're all going through my HDMI switcher to my surround syetem and HDTV. I have my Harmony 880 remote set up to control everything, including various apps on the HTPC using EventGhost and a USB-UIRT IR receiver. Operating a program such as PowerDVD or BeyondTV is no different than using the S3 Tivos.

bcc
01-16-2008, 12:32 PM
The new version of s3tots clarifies the recently mentioned warning about PCRs, fixes a problem in auto-detection of PIDs from TY streams, and cleans up visual studio compilation.

bcc
01-16-2008, 12:37 PM
You can analyze the transport stream timestamps with something like vobeditScratch that, I was thinking program stream not transport stream for a moment.
Anyways, I've updated s3tots so that the error reporting distinguishes between a wrapped timer and one that has inexplicably has moved backwards. For a wrapped timer you'll see for example
Warning: Timer wrapped, packet 78145, stream 800
If this is what you see, you can safely ignore it (unless you use a buggy player such as vlc which can't handle the wrap).

Given that you only saw 1 warning, and your recording is long (4 hours) this is the most likely case.

ScanMan
01-19-2008, 08:26 PM
Anyways, I've updated s3tots so that the error reporting distinguishes between a wrapped timer and one that has inexplicably has moved backwards.<snip>
Given that you only saw 1 warning, and your recording is long (4 hours) this is the most likely case.Thanks bcc, next time at least I'll know up front. Turns out it was harmless wrapping; the sync turned out to be a non-issue as well. I edited in VideoRedo, saved a transport stream and fed that to HDTV2DVD (which I know is just a front-end to ffmpeg). Anyway, transcoded to a Dual Layer DVD and although there was some expected quality loss, it still looks pretty good. Thanks again bcc, 's3tots' is a great utility.

ScanMan
01-21-2008, 09:30 AM
Hey bcc, have another one for you. Using the 1.1 version of s3tots, I got the following error message.
H:\TyProjects>s3tots -i AFC_Championship.ty
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is AFC_Championship.ty, dest is AFC_Championship.ts
TY set video,audio pid: 800,801. Audio is AC3
Warning: Timer wrapped, packet 45655, stream 800
idx: 10a94 Len remain: f3d0
Assertion failed: FALSE, file s3tots.c, line 228

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.FWIW, I got the same message using 1.0 version as well. It writes over 90% of the file (which is all I need anyway) and the .ts plays fine. What do you think?

bcc
01-21-2008, 07:35 PM
idx: 10a94 Len remain: f3d0
Assertion failed: FALSE, file s3tots.c, line 228So your recording has some garbaged data at this point. I've never seen this, so I wasn't sure which way to code recovery (1. linear search for next transport stream packet, 2. assume next transport stream packet will be OK after 188 bytes, or 3. assume rest of chunk is garbage).

You could send in the problem chunk and then I'd know which recovery method is best. Otherwise I'll just do the most conservative thing, #3, in the next release. On the other hand, that f3d0 length field suggests that #2 would work better.

I wonder what your tivo does when you try to play across this error point.

ScanMan
01-21-2008, 10:49 PM
So your recording has some garbaged data at this point. I've never seen this, so I wasn't sure which way to code recovery (1. linear search for next transport stream packet, 2. assume next transport stream packet will be OK after 188 bytes, or 3. assume rest of chunk is garbage).

You could send in the problem chunk and then I'd know which recovery method is best. Otherwise I'll just do the most conservative thing, #3, in the next release. On the other hand, that f3d0 length field suggests that #2 would work better.

I wonder what your tivo does when you try to play across this error point.Couple of quick questions:
How do I calculate the location of the bad chunk (in H:M:S) from the error message?
How do I extract the offending chunk?

bcc
01-22-2008, 12:25 AM
How do I calculate the location of the bad chunk (in H:M:S) from the error message?
How do I extract the offending chunk?You would estimate the chunk number based upon the file size of your .ts file. <ts file size in bytes>/131072=# of valid chunks.
Assuming you're working with a .ty file instead of .tmf you can use the -s and -n arguments to s3tots to home in on exactly which chunk is causing a problem.
Then, tychopper or dd to extract the sample.

In anticipation, I've already coded solution #2 mentioned above, if you'd rather just test drive a possible fix.

lrhorer
02-03-2008, 11:22 PM
Hey bcc,

Great program!

I'm having a little problem, though. I want s3tots to automatically translate .ty files which show up in the appropriate directory, so I wrote a couple of bash scripts to handle the processing. The problem is, s3tots complains about the filenames I use, I think almost surely because they have spaces in them. I'm running Debian Etch under Linux kernel 2.6.18-5-amd64. I tried this to replace the spaces with "\ ":



videofilein=$(echo $1 | sed 's/.\///' | sed 's/ /\ /')
videofileout=$(echo $1 | sed 's/.\///' | sed 's/ - .ty//' | sed s'/.ty//' | sed 's/ /\ /')
videofileout=$videofileout.ts

sourcedir=/RAID/Server-Main/Movies/TiVo_HD/
targetdir=/RAID/Recordings/

s3tots -i $sourcedir$videofilein -o $targetdir$videofileout
and also this to enclose the whole string in quotes:



videofilein=$(echo $1 | sed 's/.\///')
videofileout=$(echo $1 | sed 's/.\///' | sed 's/ - .ty//' | sed s'/.ty//')
videofileout=$videofileout.ts\"

sourcedir=\"/RAID/Server-Main/Movies/TiVo_HD/
targetdir=\"/RAID/Recordings/

s3tots -i $sourcedir$videofilein -o $targetdir$videofileout

The command line is passed to the routine by find, so it is a fully qualified string, and I checked that the variables have the right form. How can I get s3tots to react properly to the filenames, all of which have spaces in them? It works fine from the Linux prompt, of course by enclosing the names in quotes.

bcc
02-03-2008, 11:29 PM
It works fine from the Linux prompt, of course by enclosing the names in quotes.I don't understand what you're asking; if it works from the command line what more are you expecting s3tots to do? Works for me:
% s3tots 'a b c.ty'
s3tots: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Version 1.2, Source is a b c.ty, dest is a b c.ts
TY set video,audio pid: 90,91. Audio is AC3
17213468 transport stream bytes
Recording elapsed time: 47 seconds Rate: 2.72 Mbps

lrhorer
02-04-2008, 05:49 PM
It doesn't work from the script, which is what I want to handle the transcoding. I tried replacing " " with "\ " and also placing quotes around the entire string as I demonstrated above, but neither works. I confirmed the script works fine if the filenames have no spaces in them, but every file extracted from the TiVo has spaces in the filename.

Jamie
02-04-2008, 07:27 PM
It doesn't work from the script, which is what I want to handle the transcoding. I tried replacing " " with "\ " and also placing quotes around the entire string as I demonstrated above, but neither works. I confirmed the script works fine if the filenames have no spaces in them, but every file extracted from the TiVo has spaces in the filename.This is a shell issue, not an application issue.

Put quotes around the outside of the variable expansion if there are spaces or other special characters inside the variable value.

For example,

% x="a b c"
% y="d e f"
% s3tots -i "$in" -o "$out"
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 0.6, Source is a b c, dest is x y z
...

lrhorer
02-04-2008, 09:54 PM
Thanks, I just found that out. That's counter-intuitive. I would have thought the quotes force everything inside them to be taken as a literal, rather than forcing the contents of the variables inside the quotes to be taken as a literal. In any case, it's working!

Jamie
02-05-2008, 07:41 PM
Thanks, I just found that out. That's counter-intuitive. I would have thought the quotes force everything inside them to be taken as a literal, rather than forcing the contents of the variables inside the quotes to be taken as a literal. In any case, it's working!Read up on the difference between single and double quotes.

lrhorer
03-10-2008, 01:58 AM
I'd agree if you're using your PC for recording but if you have a dedicated HTPC to use as a DVR then the convenience factor is a dead heat, IMHO. I've got two S3 Tivos (one hacked and one virgin) in addition to my six ATSC tuner HTPC and they're all going through my HDMI switcher to my surround syetem and HDTV.
I disagree quite strongly. First of all, the last thing I would ever want to do is record anything for my theater from an ATSC tuner, with the exception of PBS. The networks had very little other than garbage on them when I was born (50 years ago), and they've gotten rather steadily worse over time.

Secondly, the Tivo has a host of important features which to the best of my knowledge are missing or poorly implemented in every other platform. The three biggies are wishlists, suggestions, and hyperfine combination search criteria. The entire list is quite long.


Operating a program such as PowerDVD or BeyondTV is no different than using the S3 Tivos.
Oh, yes it is. I cannot speak very much to Beyond TV, but PowerDVD certainly does not let me copy between any DVR and the PC from any DVR on the network. It won't show me program information like the actors, director, year, or program synopsis on the screen while selecting videos. I also have yet to find either a motherboard or video card which works well with either of my HD sets. Most won't work at all at 1080i (let alone 1080p), and those which I have tried that do look terrible at 1080i. I have also run into significant performance issues with PowerDVD, Nero Showtime, and every other application I've tried.

Beyond TV looks like it has some interesting features; especially its Beyond TV Link and SmartSkip (if it works well) features, but the fact it runs on Windows pretty much eliminates it at the outset. Not only that, but I am skeptical CableLabs will ever certify Beyond TV for use with CableCards as long as SmartSkip and Beyond TV Link are functional. As for SDV support, I'm pretty sure you can just forget about it entirely.

I'm also not sure what analogs to TiVoWebPlus, TyTool, MFS_FTP, etc. are available for Beyond TV. I do however look forward to the day TiVo enables MPEG-4 support on the S3 class units, which of course is supported by Beyond TV.

bcc
03-11-2008, 02:21 AM
It [powerdvd] won't show me program information like the actors, director, year, or program synopsis on the screen while selecting videos. It'll do all that and more (cover art) in conjunction with media center. Please, most every feature of the tivo can be had in hardware/software from others at this point. Lets face it, tivo has let the competition catch up.

I also have yet to find either a motherboard or video card which works well with either of my HD sets.I recommend the MSI NX8500GT-MTD256EH video card ($70 at newegg). Does mpeg4 avc & vc1 decoding in hardware, so it works well for blu/hd-dvd even with low-end CPUs.
I do however look forward to the day TiVo enables MPEG-4 support on the S3 class units, which of course is supported by Beyond TV.If tivo wakes up and does the development to support mpeg-4 playback, I'm sure they'll want to sell you a new box for their effort. I'll be surprised if they support it on "old" hardware given their track record.

Wait a minute, what's all this doing in an s3tots thread? Moderator?

lgkahn
03-20-2008, 12:00 AM
is a .ts a streameable format I have been looking for a way to stream my .tmf files stored on my pc.. the .asx etc is not streameable as the whole file must be downloaded b4 playback begins.

Jamie
03-20-2008, 12:27 AM
Mpeg transport streams are designed for streaming: link (http://en.wikipedia.org/wiki/MPEG_transport_stream).

lgkahn
03-20-2008, 03:38 PM
ok thanks jamie cool. but just realized this doesn't work for series 2 and looking for some way to stream my series 2 shows.

thanks

lrhorer
03-25-2008, 10:08 AM
I'm getting quite a few errors along with aborted conversions. Since I don't have any other way to try to convert to .ts, I'm stuck. Here are some entries from the logs:

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Ghostbusters - Sat-03-08-08-HDNETMV.ty, dest is /RAID/Recordings/Ghostbusters - Sat-03-08-08-HDNETMV.ts
TY set video,audio pid: d8,d9. Audio is AC3
Warning: Continuity error for stream d8, packet 48324028
Warning: Continuity error for stream d9, packet 48324030
Packet 81768523, audio pid now 21 (vid pid d8)
Packet 81768533, audio pid now 22 (vid pid d8)
Audio is now MPEG
Packet 81768541, audio pid now 21 (vid pid d8)
Packet 81768601, audio pid now 22 (vid pid d8)
Audio is now MPEG
Packet 81768605, audio pid now 21 (vid pid d8)
Packet 81768648, audio pid now 22 (vid pid d8)
Audio is now MPEG
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/The Bourne Supremacy - Sat-03-08-08-TNTHD.ty, dest is /RAID/Recordings/The Bourne Supremacy - Sat-03-08-08-TNTHD.ts
TY set video,audio pid: ad,ae. Audio is AC3
Warning: Continuity error for stream ad, packet 7186308
Warning: Continuity error for stream ae, packet 7186336
Warning: Continuity error for stream ae, packet 17866957
Warning: Discontinuity flag set for stream ae, packet 17866957
Packet 17867109, stream ae Time warped by 121
Warning: Continuity error for stream ae, packet 30977698
Warning: Discontinuity flag set for stream ae, packet 30977698
Packet 30977716, stream ae Time warped by 60
Warning: Continuity error for stream ae, packet 31999756
Warning: Discontinuity flag set for stream ae, packet 31999756
Packet 31999793, stream ae Time warped by 60
Warning: Continuity error for stream ae, packet 42017037
Warning: Discontinuity flag set for stream ae, packet 42017037
Packet 42017052, stream ae Time warped by 61
Warning: Continuity error for stream ae, packet 51224128
Warning: Discontinuity flag set for stream ae, packet 51224128
Packet 51224145, stream ae Time warped by 60
Packet 73718047, audio pid now 21 (vid pid ad)
Packet 73718059, audio pid now 22 (vid pid ad)
Audio is now MPEG
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Shrek - Sun-03-23-08-TNTHD.ty, dest is /RAID/Recordings/Shrek - Sun-03-23-08-TNTHD.ts
TY set video,audio pid: ad,ae. Audio is AC3
Warning: Continuity error for stream ae, packet 11761527
Warning: Discontinuity flag set for stream ae, packet 11761527
Packet 11761538, stream ae Time warped by 31
Unparsable chunk: 27411
s3tots: s3tots.c:844: parse_chunk_rec: Assertion `0' failed.
Exit Code: 134

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Back to the Future Part II - Mon-03-24-08-HBOHD.ty, dest is /RAID/Recordings/Back to the Future Part II - Mon-03-24-08-HBOHD.tsTY set video,audio pid: 209,20a. Audio is AC3
idx: 7c1c Len remain: 1818c
s3tots: s3tots.c:228: ts_sanity: Assertion `0' failed.
Exit Code: 134

tiver
03-25-2008, 11:21 AM
bcc: does ffmpeg also fall into the category of players (like VLC) that cannot handle the timer wrapping ?

I have a relatively benign error output:

[root @ home]#s3tots -i pti.tmf -o /delete/pti.ts
s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is pti.tmf, dest is /delete/pti.ts
TY set video,audio pid: 1001,1002. Audio is AC3
Warning: Timer wrapped, packet 16721498, stream 1001
6111766072 transport stream bytes
Recording elapsed time: 2701 seconds Rate: 17.26 Mbps
[root @ home]#

So, just one error ... but the resulting video, while playable, refuses to allow forward/backward skip, and won't thumbnail properly.

I am using XBMC for OSX (uses ffmpeg), and I have some other .tmf files that decoded (with one or more timer wrap errors) and do not display this strangeness.

So I think osxbmc/ffmpeg can handle the timer wraps ... sometimes. But when it can't, it can't parse through the file to move in skip ahead / reverse and it can't thumbnail.

Comments ?


Edit:


Ok, actually, the clip that successfully works after decoding has these, different, errors:

TY set video,audio pid: 42,43. Audio is AC3
Warning: Discontinuity flag set for stream 42, packet 12181440
Packet 12181440, stream 42 Time warped by 2
Packet 12181448, stream 42 Time warped by 95443
Warning: Discontinuity flag set for stream 42, packet 12185310
3916405096 transport stream bytes
Recording elapsed time: 1801 seconds Rate: 16.59 Mbps

So perhaps ffmpeg can handle "time warps" but not "time wraps" ?

bcc
03-25-2008, 01:54 PM
bcc: does ffmpeg also fall into the category of players (like VLC) that cannot handle the timer wrapping ?I don't use ffmpeg for playback, but I just tried a simple ffmpeg -vcodec copy -acodec copy -i x.ts x.mpg on a .ts with a wrapped timer and ffmpeg handled it ok (and the resulting recording didn't have a warped timer).

So, just one error ... but the resulting video, while playable, refuses to allow forward/backward skip, and won't thumbnail properly.Those sound like limitations of your player, trick play operations are normally possible with mpeg2-ts.
So perhaps ffmpeg can handle "time warps" but not "time wraps" ?For players I've tested that can't handle PCR wrapping, the result is a glitch in video decoding (massive macroblock errors) and some players hang as well until one skips over the wrap point. For your warped timer warnings, your stream had the discontinuity flag set at the same points and so those are probably just normal transitions between programs. I believe I should simply expect the timer to warp when the discontinuity flag is set. You could check your show to answer your own question on that one.

bcc
03-25-2008, 02:12 PM
I'm getting quite a few errors along with aborted conversions. Since I don't have any other way to try to convert to .ts, I'm stuck. Here are some entries from the logs:For the first one, look like you ran into garbage at end of recording after 14GB into your show. What is the size of the actual feature? I bet the resulting .ts has the complete movie.

(You might try my mfs_uberexport, here (http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15) , to see if it addresses the garbage at end of this recording. If nothing else it'll save you some disk space).

Actually looks like the same answer for the rest of your examples except perhaps the last one. Not sure what is wrong with that last one, you'd probably have to upload a sample for me to be sure. I did get rid of that assertion check in version 1.2 however, I'll see if I can post an update before leaving town.

tiver
03-25-2008, 04:59 PM
bcc:

Ok, I made an error - osxbmc does not use ffmpeg for mpeg decoding, it uses libmpeg2.

So when I go to the libmpeg2 people, I should tell them:

"libmpeg2 does not handle PCR wrapping properly"

and that is good information, or can I be more specific ?

bcc
03-25-2008, 05:10 PM
bcc:

Ok, I made an error - osxbmc does not use ffmpeg for mpeg decoding, it uses libmpeg2.

So when I go to the libmpeg2 people, I should tell them:

"libmpeg2 does not handle PCR wrapping properly"
No, I don't think you've shown that. You haven't mentioned whether you're seeing symptoms of PCR wrap playback problems, have you?
You've indicated that you're having trouble with trick play operations, so you could ask the player authors why they don't handle trick play operations on transport streams (ff/rew)?

xine&mplayer use libmpeg2 and don't have such trick play problems.
Also, mplayer doesn't have a problem with PCR wrap whereas xine does. So libmpeg2 isn't even the right place to point the finger for PCR wrap playback issues, as far as I know.

tiver
03-25-2008, 05:47 PM
well, the FF/REW, etc., works fine on other mpegs, as well as the s3tots-converted mpegs that have time warps, but not time wraps.

So yes, there could be some other third problem that is causing the inability to jump around and thumbnail these particular mpegs, but since these "trick play operations" are successful _in general_ in osxbmc, it would seem to be the time-wrapped mpegs.

ALSO, I am curious, where do these problems come from originally ? Why is tivo recording these streams in a way that produces these errors ?

bcc
03-25-2008, 08:41 PM
well, the FF/REW, etc., works fine on other mpegs, as well as the s3tots-converted mpegs that have time warps, but not time wraps.Oh, I wish I realised you were making that distinction before. In that case, it'd seem your player has a timer wrap bug. Just guessing, but I bet it tries to compute the recording duration by looking at the PCR at the end of file and the PCR at the beginning of the file, in order to estimate skip points. If one is careless with their timer math and simply subtracts, it's pretty clear how that leads to nonsense values, and leaves one unable to compute any skip points.
ALSO, I am curious, where do these problems come from originally ? Why is tivo recording these streams in a way that produces these errors ?It is not an error for mpeg2-ts streams to have PCR timers that wrap. In fact the spec specifically allows for this and requires an implementation to be able to handle the situation in order to be compliant. A broadcaster may reset the PCR between shows, or at will, in which case the timer may never wrap. Otherwise the timer is going to wrap every 13+ hours, and in that case your chances of seeing the wrap in a recording are ~10% (guesstimating recording length).

To answer your question, the problem originates in players that are not sufficiently careful with their timer arithmetic. Such players may get along just fine normally with freshly authored/remuxed/transcoded content.

You may want to search for a different player, one that is accustomed to playing content that is taken directly from OTA/DVB/tivo broadcast.

lrhorer
03-26-2008, 04:22 AM
For the first one, look like you ran into garbage at end of recording after 14GB into your show. What is the size of the actual feature? I bet the resulting .ts has the complete movie.
I can't say one way or the other. The source file is long gone. I don't recall for certain whether I re-recorded the show, or the .ts was intact. I think the former.


(You might try my mfs_uberexport, here (http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15) , to see if it addresses the garbage at end of this recording. If nothing else it'll save you some disk space).
Not really, because the source .ty file is deleted as soon as it's processed. Errors beyond the end of the program are no big deal. As long as the .ts is whole, everything else is fine.


Actually looks like the same answer for the rest of your examples
If you mean it ran into garbage (or what it thinks is garbage), that's pretty obvious. If you mean the .ts is whole, not even close. The resulting .ts files only contain a fraction of the program. The .ts for Back to the Future Part II, for example, only contains 2.5 out of 11 Gigs, or about 22 minutes.

I tried re-running s3tots on the source file for Back to the Future Part II, and got precisely the same results. I tried re-running Shrek, and got different results, but still far from a complete recording.


except perhaps the last one. Not sure what is wrong with that last one, you'd probably have to upload a sample for me to be sure. I did get rid of that assertion check in version 1.2 however, I'll see if I can post an update before leaving town.

What's the best way for me to trim out a chunk from the .ty file which should contain the error?

lgkahn
05-20-2008, 01:03 PM
this sounds like a really good solution to the problem of vserver not working with series 3 hd stream.. you could compile this for the tivo itself and use this to read stdin and convert the stream on the fly to a .ts anyone undertaken this... in this way you could have the view links back in tivowebplus for hd streams

thanks

Jamie
05-20-2008, 01:57 PM
Take a look at this (http://www.dealdatabase.com/forum/showthread.php?t=58805).

lgkahn
05-20-2008, 03:54 PM
getting close anyone here see anything wrong with my run line modified the tivoweb latest extract module for .ts files here is the log entry

bash-2.02# tail tivoweb.log
216.177.20.66 - - [20/May/2008 15:51:04] /extract
loading module extract (0)
-(MOD::action_extract)--(mfs_path=/Recording/NowShowingByClassic)
216.177.20.66 - - [20/May/2008 15:51:08] /{{Lord of War}{2005-01-01}{}{0525 AM W
ed Aug 17 2005}{TMCw}}{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw
}}{}{174509}.ts
-(action_.ty sock14 {{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw}
}{{Lord of War}{2005-01-01}{}{0525 AM Wed Aug 17 2005}{TMCw}}{}{174509}.ts} {hea
d_req 0 last_modified 0 noheader 0 ts 1})--(fsid=174509)
s3tots: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Version 1.3, Source is -, dest is (null)
TY set video,audio pid: 21,22. Audio is MPEG
Warning: Discontinuity flag set for stream 22, packet 0
Warning: Discontinuity flag set for stream 21, packet 44
bash-2.02#


looks like it is not finding the output

here is the execute line

if $tmf {
exec bash -c "${mfs_bin_dir}mfs_uberexport -t -x $part_files" <$xmlf >@$chan &
} elseif $ts {
# mfs_uberexport -v -R 174509 | s3tots - > 2.ts
exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid | s3tots -" >@$chan &
} else {
exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid ; cat $xmlf" >@$chan &
}
return 0
}


thanks

got it working will let you know how streaming works needed this for the run line extra -

exec bash -c "${mfs_bin_dir}mfs_uberexport -R $fsid | s3tots - -" >@$chan &

lgkahn
05-20-2008, 08:42 PM
got this working but it doesnt stream.. ie saves b4 starting to play.. assume I need a special mime header for streaming and cannot find out what it should be for a .ts stream any help would be appreciated.

lrhorer
10-12-2008, 07:40 PM
I have a file which throws an assertion failure under both s3tots and tytompg at what I assume is the same spot (it certainly looks like it from the output file). Here is the output from s3tots:

s3tots: Copyright (c) 2004-2007 B.C. <bcc24x7@gmail.com>
Version 1.1, Source is /RAID/Server-Main/Movies/TiVo_HD/Wait Until Dark - Wed-09-24-08-HDNETMV.ty, dest is /RAID/Server-Main/Movies/TiVo_TS/Wait Until Dark - Wed-09-24-08-HDNETMV.ts
TY set video,audio pid: 1ea,1eb. Audio is AC3
Warning: Timer wrapped, packet 42326591, stream 1ea
Packet 70874202, audio pid now 1a3 (vid pid 1ea)
Packet 70874220, audio pid now 1a4 (vid pid 1ea)
Packet 70874221, audio pid now 1a3 (vid pid 1ea)
Packet 70874268, audio pid now 1a4 (vid pid 1ea)
Packet 70874269, audio pid now 1a3 (vid pid 1ea)
Packet 70874296, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
Packet 70874297, audio pid now 1a3 (vid pid 1ea)
Packet 70874305, audio pid now 1a4 (vid pid 1ea)
Packet 70874306, audio pid now 1a3 (vid pid 1ea)
Packet 70874325, audio pid now 1a4 (vid pid 1ea)
Packet 70874326, audio pid now 1a3 (vid pid 1ea)
Packet 70874331, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
s3tots: s3tots.c:336: ts_parse_adaptation: Assertion `0' failed.
Exit Code: 134

And this from tytompg:

RAID-Server:/RAID/Server-Main/Movies/TiVo_TS_Error# ./tytompg -y -3 -s 10 "Wait Until Dark - Wed-09-24-08-HDNETMV.ty"
tytompg: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Multiplexer version 1.1, Demuxer version 1.0
Source is Wait Until Dark - Wed-09-24-08-HDNETMV.ty
TY set video,audio pid: 1ea,1eb. Audio is AC3
AC3 audio at 48kHz, 384 kbps, 1536 bytes/sync frame
Video frame size is 1920x1080 - high definition. Frame rate 29.970030
Video bit rate is 38810400 bits/sec., 38810 Kbit/sec.
TY set video,audio pid: 1ea,1eb. Audio is AC3
Warning: Timer wrapped, packet 35101919, stream 1ea
Packet 63649530, audio pid now 1a3 (vid pid 1ea)
Packet 63649548, audio pid now 1a4 (vid pid 1ea)
Packet 63649549, audio pid now 1a3 (vid pid 1ea)
Packet 63649596, audio pid now 1a4 (vid pid 1ea)
Packet 63649597, audio pid now 1a3 (vid pid 1ea)
Packet 63649624, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
Warning: Tossing ac3 audio frame with bad sync data at end of packet. length 3086. Chunk 102951
Packet 63649625, audio pid now 1a3 (vid pid 1ea)
Packet 63649633, audio pid now 1a4 (vid pid 1ea)
Packet 63649634, audio pid now 1a3 (vid pid 1ea)
Packet 63649653, audio pid now 1a4 (vid pid 1ea)
Packet 63649654, audio pid now 1a3 (vid pid 1ea)
Packet 63649659, audio pid now 1a4 (vid pid 1ea)
Audio is now AC3
tytompg: demux_ty.c:416: ts_parse_adaptation: Assertion `0' failed.
Aborted

I would really like to save this movie, but with the last 3 minutes or so missing, it's pretty much a goner. Does anyone have any suggestions?

pdd
03-09-2009, 10:56 PM
I am having alot of trouble converting the output of s3tots into a mp4 or avi using FFMPEG.

See
http://dealdatabase.com/forum/showthread.php?p=302064#post302064

Has anyone had any success with this on a windows machine?

Does anyone have any clues as to what is going wrong with mine as I am doing a simple conversion.

Is it possible the Australian Tivo's are in a slightly different format?

If anyone is willing to try with a small 1 minute file from my Australian Tivo please pm me.

Apologies for the semi double post but this was probably a more relevent place to ask as opposed to the other thread.

Thanks

PS. tivodecode will not work for me as we have not been provided with MAK keys in Australia at present.

lgkahn
03-09-2009, 11:09 PM
i use tsmuxer or video redo to convert.. they both work well.. txmuxer works for 90% of hd conten for the other 10 videoredo works

pdd
03-09-2009, 11:17 PM
videoredo give the error

'Unable to open: Video Program Stream Not Found'

Already tried that :(

The TS file outputted from s3tots won't play back in VLC of mplayer either.

txmuxer could pull out the video but seemed to freak out at the audio track and ignored it.

Hmmmm...

bcc
03-10-2009, 05:06 AM
The TS file outputted from s3tots won't play back in VLC of mplayer either.What diagnostics do you get from s3tots, vlc, and mplayer? Does video play, audio, or neither? What if you try the s3tots arguments to limit the .ts to just audio or just video, do the individual streams play fine?

pdd
03-10-2009, 06:01 AM
Thanks for the response here are some outputs I got.

Last few lines from s3tots

Packet 909331, audio pid now 200 (vid pid 80)
Packet 909361, audio pid now 28a (vid pid 80)
Packet 909362, audio pid now 200 (vid pid 80)
Packet 909377, audio pid now 28a (vid pid 80)
Audio is now MPEG
Packet 909378, audio pid now 200 (vid pid 80)
Packet 909394, audio pid now 28a (vid pid 80)
Packet 909395, audio pid now 200 (vid pid 80)
Packet 909407, audio pid now 28a (vid pid 80)
Packet 909408, audio pid now 200 (vid pid 80)
Packet 909435, audio pid now 28a (vid pid 80)
Packet 909436, audio pid now 200 (vid pid 80)
Packet 909450, audio pid now 28a (vid pid 80)
Packet 909451, audio pid now 200 (vid pid 80)
171479312 transport stream bytes
Recording elapsed time: 250 seconds Rate: 5.22 Mbps


Error from ffmpeg first line is the command it runs.

"D:\VideoShare\Tivo\Encoders\ffmpeg.exe" -i D:\VideoShare\Tivo\mpegs\16.9\TenNewsAtFive.ts "D:\VideoShare\Tivo\mpegs\16.9\TenNewsAtFive.mp4"
FFmpeg version Sherpya-r11050, Copyright (c) 2000-2007 Fabrice Bellard, et al.
libavutil version: 49.5.0
libavcodec version: 51.48.0
libavformat version: 51.19.0
built on Nov 18 2007 09:00:58, gcc: 4.2.1 [Sherpya]
Input #0, mpegts, from 'D:\VideoShare\Tivo\mpegs\16.9\TenNewsAtFive.ts':
Duration: 00:04:10.5, start: 15766.057411, bitrate: 5475 kb/s
Program 1
Stream #0.0[0x80]: Video: mpeg2video, 90000.00 tb(r)
Stream #0.1[0x200]: Audio: liba52, 32000 Hz, stereo, 448 kb/s
picture size invalid (0x0)
Cannot allocate temp picture, check pix fmt


When playing in VLC I see the progress bar moving across but there is no video or sound and the seconds timer stays at 0:00:00
Codec Details in VLC


Stream 0
Codec:mpgv
Language:
Type: Video
Stream 1
Codec: a52
Language:
Type: Audio


Mplayer gives this error

D:\VideoShare\Tivo\mpegs\16.9>c:\System\Apps\MPlayer\mplayer.exe TenNewsAtFive.t
s
MPlayer Sherpya-SVN-r25803-4.2.2 (C) 2000-2008 MPlayer Team
CPU: Intel(R) Pentium(R) Dual CPU E2200 @ 2.20GHz (Family: 6, Model: 15, Step
ping: 13)
CPUflags: MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled for x86 CPU with extensions: MMX MMX2 SSE SSE2
MPLAYER:
Playing TenNewsAtFive.ts.
DEMUXER: TS file format detected.
DEMUXER: NO VIDEO! AUDIO A52(pid=650) NO SUBS (yet)! PROGRAM N. 1
==========================================================================
AUDIO: Opening audio decoder: [liba52] AC3 decoding with liba52
Using SSE optimized IMDCT transform
a52: CRC check failed!
Using MMX optimized resampler
AUDIO: AUDIO: 44100 Hz, 2 ch, s16le, 192.0 kbit/13.61% (ratio: 24000->176400)
AUDIO: Selected audio codec: [a52] afm: liba52 (AC3-liba52)
==========================================================================
MPLAYER: AO: [dsound] 44100Hz 2ch s16le (2 bytes per sample)
MPLAYER: Video: no video
MPLAYER: Starting playback...
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
a52: CRC check failed!
AUDIO: a52: error at resampling
DEMUX: TS_PARSE: COULDN'T SYNC
MPLAYER:
MPLAYER:

MPlayer interrupted by signal 2 in module: uninit_ao
MPLAYER:
Exiting... (End of file)

D:\VideoShare\Tivo\mpegs\16.9>

Hopefully that might narrow things down???

lgkahn
03-10-2009, 06:59 PM
that doesnt look like an s3tots outputted file to me.. are you sure.. that audio codec is strange i think it is directsound with s3tots does not generate?

pdd
03-10-2009, 07:07 PM
that doesnt look like an s3tots outputted file to me.. are you sure.. that audio codec is strange i think it is directsound with s3tots does not generate?

Definitely is a s3tots outputted file. As expected the new Australian Tivo's are slightly different. I have sent the ty and ts file to some people who are looking into it.

Jamie has had some success running s3tots with the -t option which I thought was active by default from the documentation?? ffmpeg appears to handle it ok.

I just ran it with -t and got a file that ffmpeg can encode video looks fine however I am not able to test the audio (and audio sync) as I am not at the machine where I ran the encode.

bcc
03-10-2009, 09:01 PM
It looks like Ausie TY3 recordings can have 3 separate payload streams instead of just 2 like s3tots was assuming. This was throwing s3tots off and causing it to think the audio stream was constantly changing. It was also picking the wrong stream for the video. I've posted a new version, version 1.3 which should address this.

Running with -t helped because it kept s3tots from mis-declaring which streams were which. However, lots of programs don't know how to discover the streams on their own and depend upon those PAT/PMT tables to tell them which streams are which.

Anyways, I think it's fixed now.

pdd
03-10-2009, 09:29 PM
Anyways, I think it's fixed now.

Thanks bcc it looks like it is working perfectly now.

ffmpeg is no longer dying in a heap. Will test it more thoroughly when I get home from work.

bkdtv
03-10-2009, 10:48 PM
Could someone tell me wheteher the TY files downloaded from the TiVo are the same size as reported by the TiVo on the INFO screen?

Or, if you are still able to access the TiVo's built-in web server, check the size, in bytes, of a recording reported by the TiVo:

https://<your tivo ip>/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying

After you download a TY file, is the byte count identical to that reported above?

pdd
03-11-2009, 06:08 AM
Could someone tell me wheteher the TY files downloaded from the TiVo are the same size as reported by the TiVo on the INFO screen?

Tivo HD says
Disc Usage: Less than 1% (1.00GB)

Note I have a 1Tb disk in as well.


Or, if you are still able to access the TiVo's built-in web server, check the size, in bytes, of a recording reported by the TiVo:

https://<your tivo ip>/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying

We dont have this feature available yet.

TivowebPlus however says it is
1030 MB

MfsFtp says
1,080,033,280 bytes


After you download a TY file, is the byte count identical to that reported above?

After downloading windows says it is
1,080,038,725 b

After processing with s3tots it is
171,479,312 b

I think that covers everything.

Also audio and video have been tested with this new version of s3tots and then converting with ffmpeg and all looks good on the one video I have tried.

Also Jamie asked me in a pm to check is SAP was active as this might be the cause of multiple audio streams
http://www.tivo.com/setupandsupport/technicalsupport/setupmytivodvr/How_Do_I_Enable_SAP_.html
I checked the Australian TivoHD and we do not have this option.

Thanks again everyone for your assistance.

bcc
03-11-2009, 01:59 PM
After downloading windows says it is
1,080,038,725 b

After processing with s3tots it is
171,479,312 bNote that most of that 1GB is pre-allocated garbage that is a waste of time to download or store off-tivo. That explains the size discrepancy vs the output from s3tots. I posted a fix for mfs_uberexport over here:
http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15

which addresses the problem. This change ought to have been incorporated into mfs-utils by now but apparently it hasn't.

pdd
03-11-2009, 06:41 PM
I posted a fix for mfs_uberexport over here:
http://www.dealdatabase.com/forum/showpost.php?p=295133&postcount=15


I tried to install this yesterday but it seemed to cause my most recent version of mfs_ftp to fail to download ty files. I am pretty confident all the patches have been applied to the version I am running. I replaced the file then killed all instances of mfs_ftp then restarted it. (Maybe I should have restarted the machine there could have been other stray things running.) I will retry again this afternoon.

It is just a matter of replacing the mfs_uberexport in the mfs_ftp directory isn't it?

Also bcc in your own time is it possible for you to apply the fixes to s3tots for Aust Tivo to the mips s3tots in the first post of this thread?
http://www.dealdatabase.com/forum/showthread.php?p=295133
No hurry if you don't have to tools for compilation (Like myself). I can use mfs_ftp as is with ty's for now.

Thanks

Jamie
03-11-2009, 07:29 PM
I tried to install this yesterday but it seemed to cause my my most recent version of mfs_ftp. I am pretty confident all the patches have been applied to the version I am running. I replaced the file then killed all instances of mfs_ftp then restarted it. (Maybe I should have restarted the machine there could have been other stray things running.) I will retry again this afternoon.That mfs-uberexport build is for mfs32 only, as far as I know. There's a long delayed project to merge the mfs32 and mfs64 mfs-utils versions, but it's a project I never seem to have time to get to. mfs-utils is all GPL'd and the source for both the mfs32 and the mfs64 versions are in CVS on sourceforge: link (http://sourceforge.net/scm/?type=cvs&group_id=138561). mfs64 is in the branch tagged "mfs-utils64". I'm happy to add other developers if anyone is inclined to pitch in; just send me your sourceforge moniker.

bcc
03-11-2009, 11:12 PM
My mfs-uberexport patch has been ready to go for almost a year and a half. It's been out in the field with no bug reports. Just needs to be applied. I don't have a tivohd to do any more testing with so I cannot test mfs64, and so it doesn't make sense for me to own any mfs64 issues.

Yes, my build is for 32 bit systems. It pre-dates the 64 bit source being on sourceforge.

junk22
04-25-2009, 11:21 AM
Hi,

Having some trouble and hoping that someone can point me in the right direction.

I have an Australian TivoHD, using s3tots, works OK if I try to retrieve the 5 video tours that came with the Tivo, but trying to download a ts file for anything I've recorded I get the following error:

bash-2.02# ./mfs_tsexport 35029
s3tots: Copyright (c) 2004-2008 B.C. <bcc24x7@gmail.com>
Version 1.3, Source is -, dest is -
Unparsable chunk: 0
s3tots: s3tots.c:817: ty3_parse_chunk_rechdr: Assertion `0' failed.
./mfs_tsexport: line 8: 943 Broken pipe $exportprog $args
944 Aborted | $bindir/s3tots - -

I'm using the 64 bit version of mfs_uberexport and v1.3 of s3tots.

Thanks!

bcc
04-25-2009, 12:50 PM
"Unparsable chunk: 0" means you're hitting an EOF situation right at the beginning. So I'd suspect the FSID you're passing in to the tsexport script is invalid (or the script is still picking up the old 32bit version of mfs_uberexport). I assume things fail when you simply try to use mfs_ftp to extract the .ts.

I'd simplify the test by invoking mfs_uberexport directly.

Jamie
04-25-2009, 02:09 PM
Here's a shot in the dark: The tour videos come unencrypted. Are you sure your other recordings are unencrypted? You might check with drmcheck.tcl (http://www.dealdatabase.com/forum/showthread.php?p=288922#post288922).

junk22
04-30-2009, 05:18 AM
Here's a shot in the dark: The tour videos come unencrypted. Are you sure your other recordings are unencrypted? You might check with drmcheck.tcl (http://www.dealdatabase.com/forum/showthread.php?p=288922#post288922).

Bingo, thanks for your help.

tivo4mevo
05-27-2009, 10:29 PM
Assuming that the recordings are unscrambled, any idea whether s3tots would work with an x-tivo-raw-tts file?

i.e., one downloaded in this fashion: http:// w.x.y.z/download/videoName.TiVo?Container=%2FNowPlaying&id=123456&Format=video/x-tivo-raw-tts

bcc
11-15-2009, 04:50 PM
I've updated the source to include OSX portability fixes.
New version is 1.4. No new binaries since there were no functionality changes from 1.3.

philhu
12-14-2009, 12:35 PM
Can someone explain why this, plus all the encryption patches, etc have over using Tivo Desktop and tivodecode?

I have a hacked tivo, but do not want the encryption patches and all that 'breaks'

So, I use the CCI patches, and download most everything using tivo desktop or Pytivo, then run it automagically thru tivodecode to mpeg2

Why is s3tots better?

Jamie
12-14-2009, 01:17 PM
In my opinion, the main advantage is that "native" extraction is much faster (about 3x) than the Qualcomm ciphered and "remuxed as a program stream" extraction provided by TiVoDesktop, pyTivo, etc. Some more background: here (http://dealdatabase.com/forum/showthread.php?t=57757), if you read the whole thread.

bcc
12-14-2009, 08:47 PM
A few other points besides the big performance difference.

With s3tots you also get to avoid some of the other ttg pitfalls: its remux to mpeg2 program stream may error out for some of you content or produce buggy results. With s3tots you have your choice of remuxers, if you even need one at all (hint: you don't for playback on a desktop). Another distinction would be that tivodesktop doesn't run on linux and is closed source.

That said, s3tots is a tool that is not intended for users who need their software marketed to them. :) If you're happy with tivo desktop and it works for all your content, great, I'm not trying to sell you anything.

Also some perspective: s3tots was written before TTG even existed on the s3.

simmonsjeffreya
12-30-2009, 04:43 PM
I just wanted to make it aware to you that ld on OS X 10.6 doesn't support the --hash-style=both option. To fix this I simply edited the Makefile and removed that option. Will this have any ill effects or is it fine to do? (I'm pretty sure I had been using it this way for a month or two, but figured I'd bring it up for others.)

MacBook-Pro:src jeff$ make
cc -c -Wall -O3 -Wuninitialized s3tots.c
cc -c -Wall -O3 -Wuninitialized debug.c
cc -c -Wall -O3 -Wuninitialized bits.c
cc -c -Wall -O3 -Wuninitialized crc.c
cc -c -Wall -O3 -Wuninitialized utils.c
cc -Wall -O3 -Wuninitialized -Wl,--hash-style=both -o s3tots s3tots.o debug.o bits.o crc.o utils.o
ld: unknown option: --hash-style=both
collect2: ld returned 1 exit status
make: *** [s3tots] Error 1

bcc
12-30-2009, 09:32 PM
I just wanted to make it aware to you that ld on OS X 10.6 doesn't support the --hash-style=both option. To fix this I simply edited the Makefile and removed that option.Yes, I'm aware. I fixed this after you last mentioned that you still were having compiling trouble, but didn't cut a new release just for this one trivial item. Make is failing because the OSTYPE environment variable is not exported if you're using bash as your shell script. I had tested the makefile with tcsh so things worked fine for me. As a workaround you can simply type 'export OSTYPE' before running make. Deleting those LDFLAGS values for OSX is an OK workaround too.