Packet dump needed of MRV transfer

[mikesown]

Can someone with MRV(two Series3/HD, not series2 for the moment) please post or email me a packet dump of the first few seconds(including initialization) of an MRV transfer? I aim to develop an MRV emulator for the PC so that transfers will go much faster than TTG. If you want to email me the dump, email 1337mail [atatatat] gmail [dotdotdot] com.

Thanks,
Mike

[moyekj]

Some packet sniffing prompted by you revealed the following:
MRV transfers can be initiated from a PC very similar to TTG transfer but with the addition of video format specification at the end. For example:

http://192.168.1.101/download/Grey's%20Anatomy.TiVo?Container=%2FNowPlaying&id=1169277&Format=video%2Fx-tivo-raw-tts

Note the format specification at the end: &Format=video%2Fx-tivo-raw-tts
(%20=space %2F=/)

While this is all well and good and avoids having the Tivo have to re-mux and re-encrypt and hence speeds up transfers to MRV speeds, the question is what can be done with the resulting encrypted transport stream file on the PC?

[bcc]

Some packet sniffing prompted by you revealed the following:
MRV transfers can be initiated from a PC very similar to TTG transfer but with the addition of video format specification at the end. For example:

http://192.168.1.101/download/Grey's%20Anatomy.TiVo?Container=%2FNowPlaying&id=1169277&Format=video%2Fx-tivo-raw-tts

Note the format specification at the end: &Format=video%2Fx-tivo-raw-tts
(%20=space %2F=/)

While this is all well and good and avoids having the Tivo have to re-mux and re-encrypt and hence speeds up transfers to MRV speeds, the question is what can be done with the resulting encrypted transport stream file on the PC?

Wow, lacking a working MRV setup I had tried to guess the HTML post command that would get x-tivo-raw-tts but hadn't quite manged. See here:
http://dealdatabase.com/forum/showpo...8&postcount=46

Now, I took a look at what is extracted when one modifies the HTML POST as you found, and the resulting file is *not* encrypted! In fact it looks a lot like the raw mpeg2 transport stream content that s3tots processes. If you attempt playback with xine you'll see that it can almost play the video&audio streams as-is.

So the idea here:

http://dealdatabase.com/forum/showpo...5&postcount=58

was on the money...
Guess it's time to see if s3tots can be made to clean up this new file format.

[bcc]

Now, I took a look at what is extracted when one modifies the HTML POST as you found, and the resulting file is *not* encrypted! In fact it looks a lot like the raw mpeg2 transport stream content that s3tots processes. If you attempt playback with xine you'll see that it can almost play the video&audio streams as-is.

Note that I have the no encryption patch in my tivoapp. Maybe someone else could verify whether or not the recordings are scrambled without the tivoapp patch.

[moyekj]

Note that I have the no encryption patch in my tivoapp. Maybe someone else could verify whether or not the recordings are scrambled without the tivoapp patch.

Mine definitely seems encrypted extracted from a non-hacked S3. I have not found any app that can recognize it - Womble, VLC, TSReaderLite, etc. all do not recognize the file. Direct Show Dump doesn't know what to do with it. Tivo Decoder UI does something with it but crashes about 1/4 of the way through and the resulting file it spits out is not anything recognizable either (probably because it expects program stream instead of transport stream).

Did you verify that the modified transfer is indeed about 2x faster than normal TTG transfer? (Not much point pursuing this too much if the speedup is not significant enough).

[bcc]

Mine definitely seems encrypted extracted from a non-hacked S3. I have not found any app that can recognize it - Womble, VLC, TSReaderLite, etc. all do not recognize the file. Direct Show Dump doesn't know what to do with it. Tivo Decoder UI does something with it but crashes about 1/4 of the way through and the resulting file it spits out is not anything recognizable either (probably because it expects program stream instead of transport stream).

vlc doesn't read mine either. You need to test with a player that is really permissive about the mpeg streams (it'll search for the headers when they are not where they should be). Have you tried xine or mplayer?

Did you verify that the modified transfer is indeed about 2x faster than normal TTG transfer? (Not much point pursuing this too much if the speedup is not significant enough).

No I haven't checked. A point would be that with this method, one could bypass use of mfs_ftp/tserver&mfs-utils and extract with stock software on their tivo (without any pesky DRM added to the recording).

[moyekj]

Just tried mplayer - no dice. "Cannot seek backward in linear streams! Seek failed Exiting... (End of file)" I'm very sure I have an encrypted file. I've dealt a lot with mpeg2 transport streams and in it's current form this file is not it.

[bcc]

Ok, so I benchmarked regular TTG vs TTG with x-tivo-raw-tts. With regular TTG:
.tivo file of 620864088 bytes, 405 seconds to extract. That's 11Mbit/sec. Transfer rate was pretty constant during the download as well.

With x-tivo-raw-tts:
ts file of 647653868 bytes, 142 seconds. That's 35Mbit/sec. Ditto the constant transfer rate.

So we finally have some performance results that quantify just the overhead of adding the Qualcomm cipher to the recordings. Wow, that's a big performance hit.

[moyekj]

Ok, so I benchmarked regular TTG vs TTG with x-tivo-raw-tts. With regular TTG:
.tivo file of 620864088 bytes, 405 seconds to extract. That's 11Mbit/sec. Transfer rate was pretty constant during the download as well.

With x-tivo-raw-tts:
ts file of 647653868 bytes, 142 seconds. That's 35Mbit/sec. Ditto the constant transfer rate.

So we finally have some performance results that quantify just the overhead of adding the Qualcomm cipher to the recordings. Wow, that's a big performance hit.

That's very good info. If you tune both tuners of your S3 to channels you don't receive for the duration of the transfer you can speed it up even more to about 44Mbit/sec or so. I would assume you can monitor Tivo resources in use (perfmeter or "top" style monitoring) and can determine that the transfer is still CPU limited?

So looks like very little chance of this benefiting users with unhacked Tivos? Personally I don't have the skills necessary for the PROM hack and wouldn't feel comfortable letting someone else do the hack either. Any other way?

[bcc]

That's very good info. If you tune both tuners of your S3 to channels you don't receive for the duration of the transfer you can speed it up even more to about 44Mbit/sec or so. I would assume you can monitor Tivo resources in use (perfmeter or "top" style monitoring) and can determine that the transfer is still CPU limited?

My benchmark was with an s3 with both tuners tuned to channels I don't receive, and the system was in standby. Net connection was via built-in ethernet, stock MTU, stock kernel, and gig-e switch to a gig-e host.

So looks like very little chance of this benefiting users with unhacked Tivos? Personally I don't have the skills necessary for the PROM hack and wouldn't feel comfortable letting someone else do the hack either. Any other way?

Well I haven't verified that TTG leaves the recording scrambled with x-tivo-raw-tts without a tivoapp change. What I see are headers before every ~128K of payload so you might just be having problems with your players being confused by those headers.

If it is in fact true that extractions remain scrambled, the weak link might be to figure out how to get the equivalent of unscramble.o working on a PC. You can feed the scrambler content via TTCB so it might not require getting a secret out of the crypto chip.

[bcc]

If it is in fact true that extractions remain scrambled,

Ok, I looked & agree they're scrambled when tivoapp isn't patched (except for the header info).

[Jamie]

...
So we finally have some performance results that quantify just the overhead of adding the Qualcomm cipher to the recordings. Wow, that's a big performance hit.

Is it really just the qualcomm cipher? Isn't there a remux involved in a non-raw TTG transfer that is being skipped with raw?

[AlphaWolf]

Just tried mplayer - no dice. "Cannot seek backward in linear streams! Seek failed Exiting... (End of file)" I'm very sure I have an encrypted file. I've dealt a lot with mpeg2 transport streams and in it's current form this file is not it.

Open the downloaded file with a hex editor and look at the first two bytes. If they aren't the same as a normal unencrypted stream, then it is definitely encrypted. I don't know if these newer streams have a different magic number, otherwise I would tell you what it is.

[Jamie]

Open the downloaded file with a hex editor and look at the first two bytes. If they aren't the same as a normal unencrypted stream, then it is definitely encrypted. I don't know if these newer streams have a different magic number, otherwise I would tell you what it is.

The "magic" number is the same (see drmcheck.tcl). It might not be the first two bytes in the data stream, depending on what other metadata is being sent with the raw ty data.

This could still be useful on a hacked tivo. It's a way of extracting without need the vagaries of mfs_ftp. It also could be useful on an unhacked tivo if we could at least send it back to the tivo to play it. Of course, I assume the copy protection bits restrict what you can extract via http, while you have no so restrictions with mfs_ftp.

[bcc]

Is it really just the qualcomm cipher? Isn't there a remux involved in a non-raw TTG transfer that is being skipped with raw?

Oh right, there would be the mpeg2 transport stream to program stream conversion as well. But that should be computationally fairly easy - more like reformatting the streams than re-muxing. The program stream is already muxed at the correct rate and with all the timestamps. For reference, notice how quickly videoredo can perform a ts->ps rewrite.