PDA

View Full Version : Live Extraction (making a DSR-6000 into a TCP/IP video source)



dschuetz
01-21-2003, 04:01 PM
For some time now, I've been thinking about building a network-based media system, where multiple video streams are recorded on a central server, and played back via simple (and hopefully, quiet) set-top boxes. A big stumbling block for this is getting lots of video information into the server. Video capture cards are a pain in the neck and require a lot of horesepower under Linux to compress the video.

Has anyone successfully turned a TiVo (especially a DirecTV unit, like the DSR-6000) into, essentially, a tv-tuner streaming to an IP address?

What I'm hoping for is something where I can say, on the TiVo (conceptually -- there'll be much more complex programs involved, obviously):

% echo "channel 202" > /dev/tuner1
% echo "channel 4" > /dev/tuner2
% cat /dev/video1 | netcat mysever 3701
% cat /dev/video2 | netcat myserver 3702

The idea being that the TiVo tuners would have to be easily controllable from an internal user program (not simply an external computer spitting IR codes), and that the resultant streams could be sent as fully-encoded MPEG-2 (with digital 2.0 or 5.1 audio, hopefully) to some process on another box.

I know there's been a lot of work getting pre-recorded streams out of the box, but I haven't seen anyone talk about anything like I'm describing.

The big advantage to something like this is that I could get 4 simultaneous DirecTV channels recorded (I have two DSR-6000s), while to do this with "traditional" methods I'd have to have four individual cheap-o DTV receivers, with their audio and video (analog, no less) piped into video capture cards on 4 separate Linux systems, which (obviously) would be a whole lot more expensive.


Any pointers to discussions, detailed examinations of the DTivo filesystem and device structure, etc., would be greatly appreciated!!

tungsten2k
01-22-2003, 06:14 PM
http://www.dealdatabase.com/forum/showthread.php?s=&threadid=16715&highlight=playitsam

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=9340&highlight=playitsam

and the other way...

http://www.dealdatabase.com/forum/showthread.php?s=&threadid=15265&highlight=playitsam

ps - it's the little button right between the "f.a.q." and "home" ;)

dschuetz
01-23-2003, 09:10 AM
Tungsten -- Thanks for the reply, but I think you misunderstood what I'm looking for. The playitsam program, from what I've been reading, seems to be a way to take a pre-extracted stream, stored on a PC, and view it on TiVo hardware. I'm trying to do the reverse (sort of).

Essentially, I want to replace the TiVo hardware at the set-top level -- with something that hopefully has a flash drive and passive cooling, so it won't be annoying me with fans and hard drive noises. It'll replicate the TiVo functionality (interface and whatnot), along with providing MP3 playback, web surfing, and maybe some game emulation. Something like Freevo or MythTV. All the video / audio data will come from a central media server in the basement, probably with a huge drive (or a moderately huge RAID array).

The crux of the problem is getting video ONTO that media server -- and, if we're talking about Linux tuner cards, you either require a beefy box for each cheap tuner card (to do the compression), or a $200 external MPEG2 compressor with a cheaper cpu, which amounts to the same thing. Plus, you'd need a dedicated DirecTV tuner for each channel you want to simultaneously record.

So, what I'm interested in doing, is using my current DSR6000 hardware to receive the paid-for DTV signal, and somehow get it to the hard drive on the media server. I figure, with luck (and with putting enough of the processing stages on the server) I can get both tuners in each DSR6000 sending realtime streams at once, giving me 4 (or 6, if I buy a third unit) channels of live tv on my server.

All the extraction software and discussion I see focuses on either getting shows onto a TiVo, or on getting pre-recorded shows off. I want to skip the entire recording process (on the TiVo side) and get the raw (or close to it), decrypted, satellite MPEG stream, live, to send realtime to another system.

I've been trying to find detailed, low-level faq's on how the DTivo hardware works (with, for example, a list with explanation of all the /dev devices), but haven't found anything like that yet. I'm *assuming* that there'll be devices to control each tuner, and devices that simply stream raw, decrypted data for use by native TiVo code. Those are what I want to tap into. In the long term, I even forsee killing myworld altogether and using only custom software to control the tuners and serve the streams (which would protect me against TiVo updates that'd clobber the system, but might also preclude me from buying any PPV shows, which I can live with.)

"Extra credit will be provided" if the "raw tuner stream devices" can provide HD streams (which TiVo hardware doesn't handle, but maybe the server and future set-top boxes could).


I've seen at least one other post asking for something similar, but he didn't have any responses, either. :(


So, has anyone conclusively ruled this sort of thing out? Or has nobody even tried going down this path? It could open up a whole new world, linking TiVo hackers with the growing HTPC and network-based PVR hacking worlds...


david.

tungsten2k
01-23-2003, 02:59 PM
Originally posted by dschuetz
Tungsten -- Thanks for the reply, but I think you misunderstood what I'm looking for.
nope, perfectly clear.

[junk deleted...]
Essentially, I want to replace the TiVo hardware at the set-top level -- [...] All the video / audio data will come from a central media server...

The crux of the problem is getting video ONTO that media server --
All the extraction software and discussion I see focuses on either getting shows onto a TiVo, or on getting pre-recorded shows off. I want to skip the entire recording process (on the TiVo side) and get the raw (or close to it), decrypted, satellite MPEG stream, live, to send realtime to another system.


dude, if you're confortable reverse-engineering the entire mfs file system api, have at it. but recording shows and extracting them simultaneously is doable right now and is simple. are you concerned that it'll require the meager amount of space on the DTivo to store the recordings before they are deleted ? you should be able to extract the streams automagically to your media server right after recording has started once you work out the timing issues (hence the Playback info i pointed you to) and have your media server issue a delete to the dtivo once the end of the show show is out. sure it may sound kludgey, but it is pretty much all there right now... and solid.

the driving factor of your project is the user-experience, and with a short delay, the end-user experience is exactly the same as what you're talking about. so what is the problem ? what you want to do is like unveiling the entire mfs iceburg compared to the stream demuxing efforts that are going on right now. not to say it is impossible but unless some astounding info breaks, it is probably a ways off.

ya gotta think outside the box ;)

dschuetz
01-23-2003, 04:55 PM
> ya gotta think outside the box

I thought I was! :)

I'd thought of the kluge of manually scheduling something and then grabbing the resultant file, after a short delay, but I wasn't sure that you could read from a still-open fsid. Plus, I'm not sure how this would work for casual channel surfing -- if you had to "schedule" a recording for each channel, as you passed into it. (or maybe Playback can deal with that). And, finally, with two "recording" processes going, as well as whatever other TiVo processes run, I'm not convinced that I could run two full-resolution extraction streams and feed them out the network card.

Or, maybe to avoid the problem of "scheduling a recording" to get a live feed as you surf into a channel, is the fsid for the live buffer known? Can that be read real-time? (again, I'm ignoring for now whether there's enough power to extract both of those and send them over the net). (of course, doing even one would be helpful to the cause).


what you want to do is like unveiling the entire mfs iceburg compared to the stream demuxing efforts that are going on right now. not to say it is impossible but unless some astounding info breaks, it is probably a ways off.Okay, now this is at least getting closer to really answering the question. It's sounding like there isn't a device, per se, that simply spits out MPEG data. I'm gathering that the native software is controlling the tuner(s) and receiving the streams through underlying hardware calls that we haven't deciphered. Is that a fair assessment?


Thanks for the help so far, but please, anyone else, speak up! I'm still not convinced that there's no way to get closer to what I'm looking for, even if it isn't as simple as I might have (over-optimistically, I knew) hoped. In the meantime, I'm gonna look more closely at Playback and Playitsam...

(another part of my question, was are there any very detailed technical references for the TiVo system, giving real information about the apps, file system, stream format, etc.? Or is it all sort of "self documented" on an ad-hoc basis within the source code of all the extraction and webserver sorts of apps?) (the traditional Hacking FAQ sorts of resources don't go into this kind of detail....)


Thanks again!


david.

old7
01-23-2003, 05:14 PM
Just out of curiosity is this related to this (http://ask.slashdot.org/article.pl?sid=03/01/22/2041211&mode=thread) story on Slashdot?

Old7

old7
01-23-2003, 05:30 PM
Originally posted by dschuetz
> And, finally, with two "recording" processes going, as well as whatever other TiVo processes run, I'm not convinced that I could run two full-resolution extraction streams and feed them out the network card.

DirecTivos have the horsepower to deal with two recording processes and two extraction streams. You won't want to watch anything from that box out any other connections. There is enough horsepower now to record two streams, extract to net and view one from s-video. Shouldn't be too much different.

Setup one of the extraction streams to use a different port and there should be plenty of bandwidth on a 100 base network. Put in multiple nics in the server to reduce network traffic. One nic per DirecTivo feeding two streams. If the server has SCSI based RAID there should be plenty of bandwith in the server to stream in 8-12 streams (4-6 DirecTivos) with room left over to stream several streams back out. Getting the data in the server looks pretty straight forward.

Ok, so what are you really trying to accomplish?

Old7

dschuetz
01-24-2003, 09:16 AM
DirecTivos have the horsepower to deal with two recording processes and two extraction streams. You won't want to watch anything from that box out any other connections.It's no problem that I won't be able to use the DTivo for actual watching, they'll be in the server rack in the basement. It's good to hear that they'll have horsepower, now I just need to figure out whether I can stream the "live buffer" w/out significant delay -- because I'm really not sure I'll be able to start a recording "on demand" as someone surfs to a channel...


Setup one of the extraction streams to use a different port and there should be plenty of bandwidth on a 100 base network. Put in multiple nics in the server to reduce network traffic. One nic per DirecTivo feeding two streams. If the server has SCSI based RAID there should be plenty of bandwith in the server to stream in 8-12 streams (4-6 DirecTivos) with room left over to stream several streams back out. Server and network issues are real, but separate. I'd figured at the least on having a 100 mbps switch (I've got a nice Cisco that a friend has loaned me), with a gigabit uplink to the server. That ought to keep traffic issues pretty well covered, for inbound as well as outbound (server to set-top) traffic. Disk issues are another problem, and will probably entail going to SCSI RAID, as you suggest, in the long term, but I can cross that bridge after I've proven to my boss (that is, my wife) that the thing'll work at all. :)


> Ok, so what are you really trying to accomplish?

I'd thought that was pretty clear. I want to have, basically, much more functional set-top boxes, with features like:

* web browsing
* music playing (preferably with a flourescent display so I can select MP3s w/out turning on the TV)
* gaming (arcade emulation, maybe some lower-intensity PC games)
* regular PVR features
* HDTV PVR features (as long as I can get an HDTV signal into the server, a set-top can display it. With enough power, of course).
* DVD caching (whenever I watch a DVD, I sorely miss the TiVo's way of navigating around in the film -- the "instant replay" and the multi-speed reverse...)
* Full networkability -- so I can start a show downstairs, pause it, turn out the lights, and finish it upstairs in bed.
* less noise -- the hard drive noise bugs the heck out of me. Doesn't quite keep me up at night, but sometimes it's really annoying.

(I think you'll agree that doing this with off-the-shelf TiVo hardware isn't going to work....) So, to me, the best approach seems to be slim set-top boxes, a big server, and video sources in the basement feeding the server. ReplayTV (and TiVo, with some of the enhanced features discussed for Series 2 units) seem to be taking a different approach, distributing the load across multiple machines around the house, each being simultaneously a client and server. But that still fails to address some of my requirements (plus, I'd have to replace my current units, which are less than 2 years old, and that just bugs me.)

You'd asked if this had anything to do with a recent slashdot story, and it's "no," I've been ruminating on this for a couple of years now. I've only now got the time to really start investigating it (spent most of last year getting adjusted to a new house). There has been a lot of talk on slashdot lately about homebrew PVRs, but that's just sort of an interesting coincidence.


Right now, specifically, I'm trying to ascertain whether I'm WAY out in left field, or if there's actually a possibility this might work. I can't start actually hacking anytime soon, as I don't have a TurboNet card (and they seem to be out of stock! anyone know what's up with that?), and I don't want to disturb my two current units, so I have to find and buy a new one (I've no illusions about the length of downtime this will entail. :) )

What would be really good is if I could find other people interested in pursuing this as a larger goal, but so far, I've seen precious few people really pursuing this approach. Maybe it's because it's a crazy idea, or maybe I'm just one of the first to really see the possibilities. I don't know.