PDA

View Full Version : TTG and MRV announced for S3 Tivos in November



captain_video
09-08-2007, 08:14 AM
TivoPony has posted the announcement over at the TCF. TTG and MRV are going to be a reality for S3 Tivos coming this November. Here's the link:

http://www.tivocommunity.com/tivo-vb/showthread.php?t=365225

spaceman1013
09-08-2007, 07:48 PM
I wonder if this will work with the Series 3 content once its activated in Nov.

TiVoToGo DRM cracked:
http://arstechnica.com/news.ars/post/20061205-8358.html

BTW- I hope its okay to discuss DRM workarounds here, I tried searching for a forum rules but did not see anything. I found this thread, but it only talks about theft of DTV or Tivo service and nothing about DRM.

http://dealdatabase.com/forum/showthread.php?t=30792&highlight=forum+rules

AlphaWolf
09-08-2007, 11:35 PM
We've been breaking tivo's DRM since 2002 here.

spaceman1013
09-09-2007, 08:51 PM
Well not technically, we are just working around it by disabling the encryption flag on recordings. So we are not messing with DRM, which I believe is against the DMCA act even for fair use. At least thats the way I understood it.

But since it looks like the only S3 or S3 HD hack available is a hacked PROM, then getting shows off a S3 without PROM hacking it means using TTG and then using something like the DRM remover. That is if the CL DRM has not changed in the S3 Tivos.

Curious, will it ever be possible for a kernal hack for S3 HD Tivos? I am assuming yes but that it will take time since the kernal code was only recently in the hands of DDB hackers.

spaceman1013
09-09-2007, 10:13 PM
Also sounds like MPEG4 editing is planned for VideoReDo. That means frame accurate editing of Tivo MPEG4 files with only recoding on a few frames between cuts which means fast final output.

Not even sure if the TyTools developer is experimenting with MPEG4 frame accurate editing yet. Is he?

Here are some excerpts from the VideoReDo forum indicating this:

08-21-2007, 02:12 PM
It's been started. No ETA until we get close to a beta.
Dan Rosen ( VideoReDo )

09-04-2007
Please, no more replies to this thread. We hear you load and clear and have already started on this project. Its going to take a number of months so please be patient. When we have product to test we will post a message over in the beta section.
Dan Rosen ( VideoReDo )

Original Thread over at VideoReDo forums:

H.264 (MPEG-4) editing
http://www.videoredo.net/msgBoard/showthread.php?t=3350&page=6&highlight=mpeg4

AlphaWolf
09-09-2007, 10:28 PM
Well not technically, we are just working around it by disabling the encryption flag on recordings. So we are not messing with DRM, which I believe is against the DMCA act even for fair use. At least thats the way I understood it.

In the old days we used to modify the kernel so as to disable the crypto wrapper so that the shows would be read and written plaintext (this had the side effect of not being able to play back previously recorded shows that were encrypted.) Today we are modifying tivoapp so that it doesn't even try to encrypt the shows as it stores them to the disk (it also doesn't even generate an encryption key.) Two different methods of achieving the same end result. Both are in effect circumventing the DRM system tivo uses.

There isn't a whole lot that tivo can do about this though, namely because the copyrights that would be violated aren't held by them. Tivo would have to get the content producers themselves to sue you, and even then, the content producers would have to prove that you used this to circumvent the copy protection on their works in particular.

It's kind of a catch 22. It would be pretty well impossible to enforce copy controls on DVR's, unlike DVD's which are encrypted and sold by the content producers themselves.

The only people who could *possibly* be able to sue you would be directv if you did this to a combo unit, since they own the video stream, and not necessarily the content contained within that stream.


But since it looks like the only S3 or S3 HD hack available is a hacked PROM, then getting shows off a S3 without PROM hacking it means using TTG and then using something like the DRM remover. That is if the CL DRM has not changed in the S3 Tivos.

Curious, will it ever be possible for a kernal hack for S3 HD Tivos? I am assuming yes but that it will take time since the kernal code was only recently in the hands of DDB hackers.

The kernel hack that was done on the HDTivos and all previous generation S2 hardware was accomplished by means of an exploit in the way that the kernel checked the integrity of the ramdisk contained in its own image file. Basically it made it so that the kernel couldn't integrity check all of the boot scripts.

In order to effectively do this on all hardware later than the HDTivos, we would need to find a similar exploit that can be taken advantage of by modifying any of the non-MFS resources contained on the hard disk that are accessed somewhere in the init stage.

Any exploits other than that can be easily rendered totally useless (or very impractical) by a simple software update, therefore there is no point in even investigating such an exploit, or publishing such an exploit even if you do manage to find one.

spaceman1013
09-10-2007, 12:37 AM
Any exploits other than that can be easily rendered totally useless (or very impractical) by a simple software update, therefore there is no point in even investigating such an exploit, or publishing such an exploit even if you do manage to find one.Are you saying then that a PROM hack could be rendered useless? Also with the PROM hack, would it eventually be possible to get all the previous hacks working on Tivo HD?

AlphaWolf
09-10-2007, 01:23 AM
Are you saying then that a PROM hack could be rendered useless? Also with the PROM hack, would it eventually be possible to get all the previous hacks working on Tivo HD?

Well no, because a prom hack allows you to do anything you want with the kernel, and thus by extension, anything you want to any of the files contained on the hard disk. With a prom modification we can update the software all we want, and then modify it all we want, because we have already taken control over the boot stage, which is the key. Without the prom hack on the other hand, we are extremely limited in what we can do to affect the boot stage of a tivo.

--

On second thought, I am not quite sure I understood your question earlier. By kernel hack, you were referring to the killhdinitrd kernel exploit, correct? And HDTivo was a rather vague description on my part - by that I meant the HR10-250.

spaceman1013
09-10-2007, 01:40 AM
On second thought, I am not quite sure I understood your question earlier. By kernel hack, you were referring to the killhdinitrd kernel exploit, correct? And HDTivo was a rather vague description on my part - by that I meant the HR10-250.Yes, I did mean the killhdinitrd hack. Actually, I was more interested in the Series 3 hacking thats going on right now. So by Tivo HD, I meant the Series 3 ones and not the DTivo HD. But its good to know there is a software hack for the DTivo HR10-250 rather than a PROM only hack - am I understanding that right?

I also don't know what hacks work on the HR10-250. Can you pull off unencrypted shows and run TivoWebPlus and does TyTool work with the ty files?

I know that it looks like we are a long way from getting the common Tivo hacks we are used to in a Series 3. Seems the ty format even changed. And with most of the hackers here focusing on DTivo units, it seems that S3 hacking may take some time. Does disabling encryption on S3 as simply as finding the hack point in the tivoapp file?

But hopefully we will see a Series 3 DTivo in the future now that Liberty media has taken over and Tivo and DTV are now talking on a more friendly basis.

And some news out on the wire seems to indicate this:
http://arstechnica.com/news.ars/post/20070801-updates-show-more-signs-of-new-love-between-tivo-directv.html
http://arstechnica.com/news.ars/post/20070606-tivo-directv-may-get-back-together-comcast-to-launch-tivo-in-august.html
http://seekingalpha.com/article/22199-directv-now-in-liberty-media-s-hands-analysts-play-out-the-scenario
http://www.tvpredictions.com/tivo060507.htm

Which would mean that all those S3 hacks done today could translate into DTivo S3 hacks. So maybe it is not that much of a waste of time for these DTivo hackers here.

AlphaWolf
09-10-2007, 03:00 AM
Yes, I did mean the killhdinitrd hack. Actually, I was more interested in the Series 3 hacking thats going on right now. So by Tivo HD, I meant the Series 3 ones and not the DTivo HD. But its good to know there is a software hack for the DTivo HR10-250 rather than a PROM only hack - am I understanding that right?

Yeah, killhdinitrd was targeted specifically at the HR10-250 in fact.


I also don't know what hacks work on the HR10-250. Can you pull off unencrypted shows and run TivoWebPlus and does TyTool work with the ty files?

Same as any other S2 unit.


I know that it looks like we are a long way from getting the common Tivo hacks we are used to in a Series 3. Seems the ty format even changed. And with most of the hackers here focusing on DTivo units, it seems that S3 hacking may take some time. Does disabling encryption on S3 as simply as finding the hack point in the tivoapp file?

Don't know yet, I am still waiting for enough free time to begin prom modding my tivohd.

spaceman1013
09-10-2007, 03:54 AM
I am still waiting for enough free time to begin prom modding my tivohd.Is that a Series 3 Tivo HD you are talking about?

If so, how do you like it and what cable provider do you use?

AlphaWolf
09-10-2007, 12:46 PM
Is that a Series 3 Tivo HD you are talking about?

If so, how do you like it and what cable provider do you use?

Cox, but I haven't really used it yet.

ronnythunder
09-10-2007, 03:13 PM
in a way, i almost think it's better that the s3/tivohd hacking be prom-mod-only. tivo will never take hacks that require this with more than a grain of salt, because there are a very, very small number of people that would ever do it (much smaller than the number of people that would take advantage of a software-only exploit).

having said that, the gpl v3 is coming, and it takes square aim at tivo and other vendors that seek to use gpl content while locking users out of the hardware.

it'll be interesting to see. if i get a tivohd, i'll do so with the intent to have it prom modded, then the fun can begin. :)

ronny

Narf54321
09-10-2007, 05:30 PM
having said that, the gpl v3 is coming, and it takes square aim at tivo and other vendors that seek to use gpl content while locking users out of the hardware.

"Tivozation" and Vonage-ization of the GPL software, i.e. checking that only specific vendor-supplied binaries will run on purchased hardware, is certainly the reason behind all the GPL3 hubub. But in reality I doubt it will have much effect on Tivo. They're still using v2.4-based kernels (GPL2) and probably have no plans to change. Plus, the tivoapp binary isn't GPL anyway. Besides, I was reading a discussion about possible vendor workarounds/loopholes to GPL3 by running hypervisor type systems to restrict access to parts of the hardware.

I'm guessing the only way we're ever going to get back to software-based exploits is if somebody figures out a valid SHA-1 signature for the Tivo kernel.

AlphaWolf
09-11-2007, 02:20 AM
"Tivozation" and Vonage-ization of the GPL software, i.e. checking that only specific vendor-supplied binaries will run on purchased hardware, is certainly the reason behind all the GPL3 hubub. But in reality I doubt it will have much effect on Tivo. They're still using v2.4-based kernels (GPL2) and probably have no plans to change. Plus, the tivoapp binary isn't GPL anyway. Besides, I was reading a discussion about possible vendor workarounds/loopholes to GPL3 by running hypervisor type systems to restrict access to parts of the hardware.

Either that or they'll do like what linksys did and migrate their software over to vxworks or something similar. (though linksys did this because they wanted a lighter kernel than the linux kernel that would require less hardware to run)

IMO that fat dirty hippie Richard Stalin needs to quit being such a communist bastard because it isn't earning him any favors except among other communists. Even Linus Torvalds doesn't like his ideology.


I'm guessing the only way we're ever going to get back to software-based exploits is if somebody figures out a valid SHA-1 signature for the Tivo kernel.

I wonder how much these two nuggets would help:

http://www.schneier.com/blog/archives/2005/02/sha1_broken.html
http://nsa.unaligned.org/

They claim to be able to find any SHA-1 hash collision within a day.

bcc
09-11-2007, 02:23 PM
I wonder how much these two nuggets would help:

http://www.schneier.com/blog/archives/2005/02/sha1_broken.html
http://nsa.unaligned.org/

They claim to be able to find any SHA-1 hash collision within a day.Since the kernel is signed with ElGamal, it would seem this only could help if you wanted to break the train of trust by attacking an sha1 signed file in the root filesystem. Assuming you succeeded at this, and posted an exploit, all tivo has to do is change the initrd image to use a stronger hash function.

Jamie
09-11-2007, 02:32 PM
Since the kernel is signed with ElGamal, it would seem this only could help if you wanted to break the train of trust by attacking an sha1 signed file in the root filesystem. Assuming you succeeded at this, and posted an exploit, all tivo has to do is change the initrd image to use a stronger hash function.Isn't the thing that is ElGamal signed an SHA-1 hash of the kernel image? I'm just basing this on ADH's post here (http://www.dealdatabase.com/forum/showthread.php?p=192565&highlight=chain+trust#post192565). If so, then in theory if you could produce a modified kernel that had the same SHA-1 hash as the original, you could use the original signature. That's essentially what killhdinitrd does, relying on the fact that parts of the kernel image were not included in the hash with the old PROMs. I don't think the brute force or other cracks on SHA-1 give you what you need, but I'm no crypto expert.

bcc
09-11-2007, 03:37 PM
Isn't the thing that is ElGamal signed an SHA-1 hash of the kernel image? I'm just basing this on ADH's post here (http://www.dealdatabase.com/forum/showthread.php?p=192565&highlight=chain+trust#post192565). If so, then in theory if you could produce a modified kernel that had the same SHA-1 hash as the original, you could use the original signature. That's essentially what killhdinitrd does, relying on the fact that parts of the kernel image were not included in the hash with the old PROMs. I don't think the brute force or other cracks on SHA-1 give you what you need, but I'm no crypto expert.I thought that was old information - that old proms just used an sha hash as the kernel signature. If s3 proms are doing an El Gamal signature of *just* the hash, then that leaves the hash as a weak link. Weak enough? Sounds like one needs to put together FPGA hardware for brute-force attacks to run in a practical amount of time, which is harder than modding the PROM IMO.

Jamie
09-11-2007, 03:54 PM
I thought that was old information - that old proms just used an sha hash as the kernel signature. If s3 proms are doing an El Gamal signature of *just* the hash, then that leaves the hash as a weak link. Weak enough? Sounds like one needs to put together FPGA hardware for brute-force attacks to run in a practical amount of time, which is harder than modding the PROM IMO.I think the info in ADH's post is still current. There's a discussion here (http://en.wikipedia.org/wiki/ElGamal_signature_scheme) of ElGamal and how it is only as secure as the hash function used (SHA-1 in this case).

I'm sure killhdinitrd was harder to develop than doing one prom mod too. Possible motivations for developing a software hack: for fun. To be a her0: once you have a modified kernel that passes the security checks, anyone can use it without need a prom mod, so you save a lot of other people the work of doing a prom mod.

Of course, when/if an exploit is developed, there's a pretty good chance it will be closed in the next hardware release.

tivo4mevo
09-11-2007, 04:17 PM
I think the info in ADH's post is still current. There's a discussion here (http://en.wikipedia.org/wiki/ElGamal_signature_scheme) of ElGamal and how it is only as secure as the hash function used (SHA-1 in this case).A signature scheme is only as strong as its weakest link (the hash in this case), but I believe that the Wang et. al. attacks on full-round SHA-1 are collision attacks and not the preimage (http://www.cryptography.com/cnews/hash.html) attacks needed to create rogue kernels that would verify as valid.

Software-only exploits also lower the barrier to entry, which can develop more interest in the hobby (though it also lets in the riff-raff... you know, those people with unimaginative usernames. ;) )

Jamie
09-11-2007, 04:40 PM
... I believe that the Wang et. al. attacks on full-round SHA-1 are collision attacks and not the preimage (http://www.cryptography.com/cnews/hash.html) attacks needed to create rogue kernels that would verify as valid.This (http://www.heise-security.co.uk/news/77244) more recent work might be more relevant, although I still don't think we are there yet.

Some of this ground was covered back last year: link (http://www.dealdatabase.com/forum/showthread.php?p=262919#post262919).

Guess I should go shopping for a new handle....

tivo4mevo
09-11-2007, 04:56 PM
Still looks to be collision and not preimage from a cursory read (though the new developments allow a block of the message to be well-formatted), but my jest did not come through quite right. No slight intended; it was mean to be self-referential, as handles including "tivo" seem to be a dime a dozen and because I was part of the riffraff ushered in by killhdinitrd. :o

I'm shopping for a new handle... Count Von Count sounds about right.

Jamie
09-11-2007, 05:02 PM
Still looks to be collision and not preimage from a cursory read (though the new developments allow a block of the message to be well-formatted), but my jest did not come through quite right. No slight intended; it was mean to be self-referential, as handles including "tivo" seem to be a dime a dozen and because I was part of the riffraff ushered in by killhdinitrd. :o

I'm shopping for a new handle... Count Von Count sounds about right.Yes, it is a collision attack, not a preimage attack, so perhaps not useful.

AlphaWolf
09-12-2007, 08:06 PM
I thought that was old information - that old proms just used an sha hash as the kernel signature. If s3 proms are doing an El Gamal signature of *just* the hash, then that leaves the hash as a weak link. Weak enough? Sounds like one needs to put together FPGA hardware for brute-force attacks to run in a practical amount of time, which is harder than modding the PROM IMO.

AFAIK this is standard practice in any digital document signing. Hash the document, then sign the hash. Unless my understanding is wrong...I dunno. I read a short guide to cryptography once that is targeted at people who have no prior knowledge of cryptography and it went over the basic concepts in like 5 pages or so.

My thought here was that we could create just enough code to boot off of and redirect to an arbitrary spot where we could put another kernel, then from there add junk data at the end of that to cause a collision with the signed hash. Essentially this would become a partial password match, as opposed to just starting with nothing, and would of course require significantly more memory to pull off, but not much more processing power I would venture to guess.

bcc
09-12-2007, 09:25 PM
AFAIK this is standard practice in any digital document signing. Hash the document, then sign the hash. Unless my understanding is wrong...I dunno. I read a short guide to cryptography once that is targeted at people who have no prior knowledge of cryptography and it went over the basic concepts in like 5 pages or so.
In crypto systems such as kerberos, signing is usually done on tuples that include more than just 1 thing so I wouldn't assume the signature is just over the hash. References I saw just indicated that the prom was doing SHA1{Kernel} or El Gamal{Kernel} not El Gamal{SHA1{Kernel}}. I've been fortunate enough to not need to disassemble the prom myself so I'll just trust that the prom is indeed doing El Gamal{SHA1{Kernel}} on the s3 boxes.

My thought here was that we could create just enough code to boot off of and redirect to an arbitrary spot where we could put another kernel, then from there add junk data at the end of that to cause a collision with the signed hash. Essentially this would become a partial password match, as opposed to just starting with nothing, and would of course require significantly more memory to pull off, but not much more processing power I would venture to guess.Well after I publicly mentioned how the kernel load address could be tampered with, as it was outside the hash, tivo blocked that hole. I've verified that it can no longer be tampered with directly, and it sounds like the state of the art of sha1 cracking is not up to what you're suggesting (the preimage point jamie made).

Suggest you just practice more with chipquik, it's not that hard really; come on in the water's fine:) I got it done with a cheap radio shack soldering iron even. It took me a couple tries on the first one before I realized I had to fully clean off the chipquik alloy and re-tin the pads before putting the socket on but after that the process worked pretty well.

AlphaWolf
09-12-2007, 09:56 PM
Suggest you just practice more with chipquik, it's not that hard really; come on in the water's fine:) I got it done with a cheap radio shack soldering iron even. It took me a couple tries on the first one before I realized I had to fully clean off the chipquik alloy and re-tin the pads before putting the socket on but after that the process worked pretty well.

Yeah I am going to regardless of whether or not a soft attack surfaces. I just haven't had the time to mess with it lately. I actually made a video of my prom removal btw, and that part I did just fine. As soon as I get the time to edit that I'll post it.

It's the getting the socket on there that is giving me hell. Or at least, that is where I lifted the pad on my S1 anyways. I am sure I can repair it, but I am not getting my hopes up as there are other things I possibly damaged in the process. I accidently flicked some chipquik on the CPU, I cleaned it off but some of the pins bent a tad while doing so...no reason to suspect damage, but its still ugly. That and I accidentally trimmed a little of that green stuff off of the top of some of the traces between the pads...they are still plenty conductive though, just hoping that no solder sticks to them upon socket installation.

Yeah I made quite a mess of things. Ironically my very first prom removal went completely without error on the other hand and looks perfectly clean with no scorch marks or scrapes anywhere. This is the one I already posted pics of, and it looks beautiful. It was the second removal that got ugly.

newbie
09-21-2007, 02:48 PM
According to TCF software version 9.1.L5 is being rolled out. It's claimed this software has MRV capability but tivo won't enable this capability until some time in the future.

AlphaWolf
09-21-2007, 05:19 PM
That L5 nomenclature sounds like a beta version.

bcc
09-21-2007, 05:27 PM
That L5 nomenclature sounds like a beta version.I think "L" means limited release (to unsuspecting end users). Ie the typical limited early rollout. I got 9.1L5 and I'm not in the beta.

newbie
09-21-2007, 06:45 PM
It's been posted that the features are:

Enchanced Wishlists
Multiroom viewing for TiVo HD and S3 (needs to be enabled by TiVo*)
Crestron Integration for S3
Improvement in Emergency Alert Handling
Bug fixes

According to a tivo rep this is the "measured part of the rollout" After two weeks,if the call center doesn't find problems it will be rolled out to the rest of the units (over several weeks).

jkozee
10-23-2007, 03:45 PM
Looks like it arrived early! http://www.engadgethd.com/2007/10/23/tivo-series3-and-hd-finally-get-tivotogo-mrv-esata-drive-othe/

jkozee
10-23-2007, 04:42 PM
It would be interesting to see transfer speeds for HD content from TivoHD<->TivoHD for:

1) Integrated NIC with stock drivers
2) Integrated NIC with backport drivers
3) Gigabit USB NIC with backport drivers
4) Gigabit Jumbo Frames USB NIC with backport drivers

Hopefully some combination would allow for faster-than-realtime transfers. I haven't got the hardware to test these scenarios yet, but when I do I'll post the results. If anyone else has the time and hardware to run these tests then please post the results.

Narf54321
10-23-2007, 05:46 PM
Some of those guys are claiming some pretty crazy numbers, one fella is saying around 20Mbps or so.

My own experience with the S3 has thus far never shown more than 4.5mbps with the stock drivers.

Jamie
10-23-2007, 05:55 PM
Some of those guys are claiming some pretty crazy numbers, one fella is saying around 20Mbps or so.

My own experience with the S3 has thus far never shown more than 4.5mbps with the stock drivers.Bits verses bytes? That's the usual confusion with network performance comparisons. mbps normally means "millions of bits per second" in the networking/telecom world. (ref: [link (http://en.wikipedia.org/wiki/Mbps)]) Are you really only seeing 4.5mbps? I haven't timed MRV yet, but with mfs_ftp, I get > 80mbps, with the usual tricks.

jkozee
10-23-2007, 06:57 PM
I haven't timed MRV yet, but with mfs_ftp, I get > 80mbps, with the usual tricks.

Usual tricks as in gigabit/jumbo frames/backport/custom kernel?

Those numbers look great. That's 33.5 GB/hour and 3.7 times what we need assuming 9 GB/hour for HD. Let's hope MRV doesn't doesn't eat up all that overhead.

I know you have better things to do, but if you run any performance tests I'm sure there are many people interested in numbers.

Jamie
10-23-2007, 07:06 PM
Usual tricks as in gigabit/jumbo frames/backport/custom kernel?Yes. Gigabit; jumbo frames;custom kernel.



Those numbers look great. That's 33.5 GB/hour and 3.7 times what we need assuming 9 GB/hour for HD. Let's hope MRV doesn't doesn't eat up all that overhead.We'll see. TTG has always been a dog, but MRV and "go back" haven't been too bad.



I know you have better things to do, but if you run any performance tests I'm sure there are many people interested in numbers.Will do, as time permits. I have an S3 and a TiVoHD. The TiVoHD doesn't have all the hacks yet, but I'm working on it.

Here are some raw netperf numbers:

TiVoHD, 100mbps, custom kernel, bcmemac driver (built in NIC)

bash-2.02# netperf -H 192.168.2.100 -- -S 65536 -s 65536
TCP STREAM TEST to 192.168.2.100
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

131072 131070 131070 10.03 35.62Series3, 1000mbps, jumbo frames. custom kernel, agigausb
bash-2.02# netperf -H 192.168.1.100 -- -S 65536 -s 65536
TCP STREAM TEST to 192.168.1.100
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

131072 131070 131070 10.00 105.49

Roger Dylan
10-24-2007, 06:44 AM
TivoDecode Manager is working well so far on the s3.

EDITED TO ADD: Mpeg2 only, not mpeg4.

Jamie
10-24-2007, 03:48 PM
For what it is worth, S2 <-> S3 MRV transfers appear to use TTG/TTCB protocol. Look at the TransferStatus messages in tvlog, for example. The upshot is that they aren't particularly fast: 10-15Mbps, in my setup. S2<->S2 transfers are faster: I saw ~ 25mbps last night. I haven't had a chance to try S3<->THD yet.

bcc
10-24-2007, 03:55 PM
From what I can tell, TTG/TTCB always convert the recordings to mpeg2 ps before transmitting out the wire. This means an extra conversion step for s3 boxes (which use mpeg2 ts to store recordings natively). I assume s3<->s3 transfers stay "native". Anyone know how to get s3<->PC transfers to use native mode as well?

Jamie
10-24-2007, 04:00 PM
From what I can tell, TTG/TTCB always convert the recordings to mpeg2 ps before transmitting out the wire. This means an extra conversion step for s3 boxes (which use mpeg2 ts to store recordings natively). I assume s3<->s3 transfers stay "native". Anyone know how to get s3<->PC transfers to use native mode as well?I believe s2<->s2 is native. I haven't done s3<->s3 yet. We need a full superpatch port for that :)

bcc
10-24-2007, 04:05 PM
I believe s2<->s2 is native. I haven't done s3<->s3 yet. We need a full superpatch port for that :)I thought s3<->s3 MRV was working fine, which is what I meant by s3<->s3. tivopony's post about native vs non-native performance implied that s3<->s3 would be native (which probably means mpeg2 ts). I don't have 2 activated s3s to find out. It would be nice if s3<->PC could be made native as well (should improve performance).

Here's hoping tivo updates the SDKs to reflect current reality.

Jamie
10-24-2007, 04:54 PM
I thought s3<->s3 MRV was working fine, which is what I meant by s3<->s3.Doesn't work for me with the various patches we like to use around here, e.g. to prevent encryption. It is an advertised feature on an unhacked S3/THD.

I don't think tivo->PC will ever be native, since the encryption is completely different on the PC side. I suppose you could write a PC side program that emulated an S3 MRV client to receive native format, but it wouldn't be much use from an unhacked tivo, since they would be encrypted. On a hacked tivo, you can't get much more native than mfs_uberexport-to-vserver, or mfs_ftp.

lrhorer
10-24-2007, 06:42 PM
Yes. Gigabit

Um, how are you doing Gigabit? Is there a Gig-e hardware hack out for the S3?

lrhorer
10-24-2007, 06:47 PM
I don't think tivo->PC will ever be native, since the encryption is completely different on the PC side. I suppose you could write a PC side program that emulated an S3 MRV client to receive native format, but it wouldn't be much use from an unhacked tivo, since they would be encrypted.
Actually, that woud be great for my purposes. I don't need to be able to watch the video from a PC. The only thing I need TTG for is to offload content onto the server to free up space on the TiVos without losing the recordings. I'll be upgrading my Linux sever (running Galleon) to a 4TB RAID array shortly.

Jamie
10-24-2007, 06:54 PM
Um, how are you doing Gigabit? Is there a Gig-e hardware hack out for the S3?The backport drivers support the S3 platform. The AGIGAUSB dongles are available here (http://www.icyant.com/estore/control/Computer3G/productdetails?id=69066), though they aren't as cheap as they once were.

bcc
10-24-2007, 07:19 PM
I don't think tivo->PC will ever be native, since the encryption is completely different on the PC side. What you're saying here makes no sense to me. The .tivo file encryption (qualcom encryption) should be happening on the tivo side, otherwise the content is not protected on the wire (unless there's another level of encryption I don't know about). And then it's necessarily the same encryption algorithm on the PC to decrypt.
The stream itself, whether it's native transport stream, or non-native program stream, should be a separate issue.

If you look in tivoapp, there's reference to a video/x-tivo-raw-tts content-type. This suggests TTG protocol support for extracting transport streams. It may be as simple as specifying the content-type in the HTML request to get .tivo files with transport stream content.

bcc
10-24-2007, 08:23 PM
Ok, now that I've actually looked at tivodecoder (I'm new to .tivo format), it looks like .tivo file format assumes mpeg2 ps, and so extracting to a PC in mpeg2 ts was probably not implemented.

Jamie
10-24-2007, 09:18 PM
What you're saying here makes no sense to me. The .tivo file encryption (qualcom encryption) should be happening on the tivo side, otherwise the content is not protected on the wire (unless there's another level of encryption I don't know about). And then it's necessarily the same encryption algorithm on the PC to decrypt.
The stream itself, whether it's native transport stream, or non-native program stream, should be a separate issue.

If you look in tivoapp, there's reference to a video/x-tivo-raw-tts content-type. This suggests TTG protocol support for extracting transport streams. It may be as simple as specifying the content-type in the HTML request to get .tivo files with transport stream content.Yes, I believe the re-encryption is done on the tivo side, and that's what makes TTG transfers so slow. Anything that is re-encrypted doesn't sound native to me, but perhaps I didn't understand what you meant by "native". I interpreted it as "as stored on disk on the tivo". I thought the point was to avoid any processing on the tivo side to get the highest transfer rates possible. mfs_uberexport is still probably the best bet for that. It's really doing a straight disk to network copy.

bcc
10-24-2007, 09:26 PM
Yes, I believe the re-encryption is done on the tivo side. Anything that is re-encrypted doesn't sound native to me, but perhaps I didn't understand what you meant by "native".s1/s2 boxes store recordings in mpeg2 ps format. s3 boxes store recordings in mpeg2 ts format. By "native" I mean the mpeg2 format in which recordings are stored on the tivo's disk. I think this is what tivopony means as well.
So, an s2<->s3 transfer involves some stream conversion to switch between formats (non-native transfer). An s3<->s3 or an s2<->s2 transfer can be native. An s3<->pc transfer is apparently always non-native as the tivo server stores the content in a .tivo file which seems to presume mpeg2 ps. I may be wrong - there may be a way to make .tivo files with mpeg2 ts...

non-native transfers involve extra overhead to convert between formats. For the tivos I think that means some non-trivial performance impact. Ie worth avoiding if possible.

Jamie
10-24-2007, 09:36 PM
... By "native" I mean the mpeg2 format in which recordings are stored on the tivo's disk. I think this is what tivopony means as well.Old style MRV actually copies the ty streams, even the encrypted ones, exactly as they are stored on disk with no processing. You can extract them and diff them and they are identical. The only thing that changes is in the metadata: the CSO/Encryption keys, which are rewritten to map to the different crypto chip.

bcc
10-24-2007, 09:49 PM
Old style MRV actually copies the ty streams, even the encrypted ones, exactly as they are stored on disk with no processing. You can extract them and diff them and they are identical. The only thing that changes is in the metadata: the CSO/Encryption keys, which are rewritten to map to the different crypto chip.So how does current MRV work? I've never had MRV 'till yesterday and still don't have 2 sub'd ones, so I haven't been able to see it in practice.

Jamie
10-24-2007, 10:06 PM
So how does current MRV work? I've never had MRV 'till yesterday and still don't have 2 sub'd ones, so I haven't been able to see it in practice.I'm assuming that "native" MRV is "old style MRV" and "non-native" MRV is the TTG/TTCB protocol. But I haven't spent anytime sniffing network packets yet.

jkozee
10-26-2007, 11:26 AM
Here are some raw netperf numbers I have gotten from my TiVoHD. For reference I've included a S2.5 which I believe runs about 3x faster than stock thanks to the backport drivers. Thanks Jamie, ADH, burbinator and anyone else who made that possible.

Series2.5, 1000mbps, jumbo frames, custom kernel, agigausb

tivo:/# netperf -H 192.168.168.4 -- -S 65536 -s 65536
TCP STREAM TEST to 192.168.168.4
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

65536 131070 131070 10.02 25.52

Edit: Moved TiVoHD results to soapbox derby (http://www.dealdatabase.com/forum/showpost.php?p=289774&postcount=269).

Jamie
10-26-2007, 12:15 PM
Thanks for those results. They are consistent with what I am seeing. The TiVoHD doesn't do as well as the Series3, but is still respectable.

It might make sense to move further network performance discussions over to the soapbox derby (http://dealdatabase.com/forum/showthread.php?t=39328) thread.

Jamie
10-30-2007, 09:31 PM
I thought s3<->s3 MRV was working fine, which is what I meant by s3<->s3.Doesn't work for me with the various patches we like to use around here, e.g. to prevent encryption. ...I was wrong on this point. Transfers of unencrypted shows works fine for me now, with just the "nocso" patch equivalent. I think I was just testing before I had all the necessary keyring keys -- I had an HM_MAK_... but not an MRV_... key.

bcc
10-31-2007, 07:25 PM
I was wrong on this point. Transfers of unencrypted shows works fine for me now, with just the "nocso" patch equivalent. I think I was just testing before I had all the necessary keyring keys -- I had an HM_MAK_... but not an MRV_... key.So then, that brings me back to my original question. If MRV on the s3s means "native" mpeg2 ts transfers, then that'd be a big performance win over TTG/TTCB which isn't native for the s3s. If the PC servers could be made to use an MRV-like transfer, they could get the performance win of "native" transfers as well.

Jamie
10-31-2007, 11:45 PM
So then, that brings me back to my original question. If MRV on the s3s means "native" mpeg2 ts transfers, then that'd be a big performance win over TTG/TTCB which isn't native for the s3s. If the PC servers could be made to use an MRV-like transfer, they could get the performance win of "native" transfers as well.There's nothing more native than vserver(tivo) to mfs_uberexport(pc). It's a direct disk to network copy. I get >80mbps in s3=>pc transfers that way. "Native" MRV s3<=>s3 transfers are reported (http://www.tivocommunity.com/tivo-vb/showthread.php?t=371710) on TCF to max out at ~44mbps, and ~21mpbs thd<->thd with stock tivo software. I've personal measured s3<->thd native MRV at ~27mbps sustained, with peaks at ~35mbps, using gige, jumbo frames, custom kernel ,etc. So on a hacked tivo, I'd still recommend hacking extraction tools, if performance is important. On an unhacked tivo, there really isn't the possibility of "native" tivo<->pc transfers, in the sense of no processing on the stream, given the encryption issues previously discussed.

bcc
11-01-2007, 12:44 AM
Lemme start over.
s3->pc on non-hacked tivo. Uses TTG. Non-native. Slow.
s3->s3 on hacked or non hacked tivo. Uses MRV. Native (reportedly). "Fast".

So to make s3->pc faster on non-hacked tivo, change PC servers to use MRV not TTG, or use TTG with mpeg-ts content-type specified.
May be feasible, would be worth seeing what goes out on the wire for s3<->s3 MRV case.

Jamie
11-01-2007, 07:18 AM
....So to make s3->pc faster on non-hacked tivo, change PC servers to use MRV not TTG,You'd get a stream encrypted with tivos on box encryption, which there is no public way to crack off the tivo.
...or use TTG with mpeg-ts content-type specified.You'd still have the tivo doing the expensive decrypt/reencrypt, which is what makes TTG slow. It's also not at all clear that tivo included code to do this on the tivo side. Give it a shot, if you feel like it, but I'm not optimistic that you'll see much of a performance improvement.

bcc
11-01-2007, 02:29 PM
You'd get a stream encrypted with tivos on box encryption, which there is no public way to crack off the tivo.Thanks, that'd make sense. I didn't want to assume up front that they did the encryption in a sound manner (using the tivo's private keys). You had said that old MRV copied the ty streams exactly with no processing.
Oh well nevermind, sorry for the trouble, back to s3tots :)

lrhorer
11-04-2007, 03:41 PM
You'd get a stream encrypted with tivos on box encryption, which there is no public way to crack off the tivo.
That would be perfectly fine with me. As long as the space is freed up on the TiVo and the Tivo can easily see and download the file from the file server, I'm in heaven. I just need to get around the S3 2TB limit on drive space.

mr_zorg
12-11-2007, 02:58 AM
I just got an s3 and set it up, does anyone know if tivoserver works with an s3 and MRV? According to tivo's website, all tivo's using MRV must be using 9.1 software. Does that suggest that the MRV protocol may have changed in a way that is not compatible with tivoserver? Any ideas?

Jamie
12-11-2007, 08:59 AM
I just got an s3 and set it up, does anyone know if tivoserver works with an s3 and MRV? According to tivo's website, all tivo's using MRV must be using 9.1 software. Does that suggest that the MRV protocol may have changed in a way that is not compatible with tivoserver? Any ideas?
no; yes; use pytivo or tivo.net.

mr_zorg
12-11-2007, 02:24 PM
no; yes; use pytivo or tivo.net.
Thank you!