PDA

View Full Version : MRV Protocol?



DocTauri
02-23-2005, 09:50 PM
All, Iíve kind of been sitting on my butt, hoping someone would write some sort of Linux based MRV server. I havenít seen one yet, so decided to try it myself. Iíd like to share what Iíve learned so far here, and hope that others who may be more familiar with this can shed some light or point me in the right direction.

The following posts come from running tcpdumpís as well as netstat Ėln to determine which ports are being used, and grabbing packet data. According to netstat Ėln, ports 2190/udp, 2190/tcp, 2191/tcp, and 2300/tcp are all listening (we also see iptables punching holes during boot for these ports). I have seen all of these ports being used, except 2190/tcp. I havenít fully stressed the box to see if I can figure it out yet though.

I welcome any help to reverse the MRV protocol.

Thanks!
Doc

DocTauri
02-23-2005, 09:51 PM
Port 2190/udp is how the Tivo or HMO server advertises itself to the rest of the network. A single packet is sent to the broadcast address of the subnet. From what Iíve seen, they appear to do this about once per minute, but if they stop, it may take up to 15 minutes to fall out of another Tivoís list.

Below are two advertisement packets, one from a Tivo, the other from a JavaHMO server.

Advertisement packet from a Tivo

18:46:45.949381 192.168.1.19.2190 > 192.168.1.255.2190: udp 163 (DF)
0x0000 4500 00bf 0000 4000 4011 b5cb c0a8 0113 E.....@.@.......
0x0010 c0a8 01ff 088e 088e 00ab f915 7469 766f ............tivo
0x0020 636f 6e6e 6563 743d 310a 7377 7665 7273 connect=1.swvers
0x0030 696f 6e3d 342e 302e 3162 2d30 322d 322d ion=4.0.1b-02-2-
0x0040 3234 300a 6d65 7468 6f64 3d62 726f 6164 240.method=broad
0x0050 6361 7374 0a69 6465 6e74 6974 793d 4630 cast.identity=F0
0x0060 4630 4630 4630 4630 4630 4630 460a 6d61 F0F0F0F0F0F0F.ma
0x0070 6368 696e 653d 5469 766f 4445 560a 706c chine=TivoDEV.pl
0x0080 6174 666f 726d 3d74 6364 2f53 6572 6965 atform=tcd/Serie
0x0090 7332 0a73 6572 7669 6365 733d 5469 566f s2.services=TiVo
0x00a0 2d53 6572 7665 5463 6456 6964 656f 2d31 -ServeTcdVideo-1
0x00b0 3a32 3139 312f 7476 6275 735f 7633 0a :2191/tvbus_v3.


Advertisement packet from a JavaHMO server

18:47:17.214067 192.168.1.32.zephyr-hm > 192.168.1.255.2190:
0x0000 4500 00b9 418f 0000 8011 7435 c0a8 0120 E...A.....t5....
0x0010 c0a8 01ff 0838 088e 00a5 11de 5469 566f .....8......TiVo
0x0020 436f 6e6e 6563 743d 310a 4d61 6368 696e Connect=1.Machin
0x0030 653d 4454 2d5a 4143 480a 4964 656e 7469 e=DT-ZACH.Identi
0x0040 7479 3d7b 3439 3033 3043 4233 2d46 3843 ty={49030CB3-F8C
0x0050 322d 3439 3544 2d39 3738 342d 3132 4144 2-495D-9784-12AD
0x0060 3133 3435 3943 4531 7d0a 4d65 7468 6f64 13459CE1}.Method
0x0070 3d42 726f 6164 6361 7374 0a50 6c61 7466 =Broadcast.Platf
0x0080 6f72 6d3d 7063 2f57 696e 4e54 3a35 2e31 orm=pc/WinNT:5.1
0x0090 2e32 3630 300a 5365 7276 6963 6573 3d54 .2600.Services=T
0x00a0 6956 6f4d 6564 6961 5365 7276 6572 3a38 iVoMediaServer:8
0x00b0 3038 312f 6874 7470 0a 081/http.


The fields appear to be as follows:

tivoconnect=1
Not sure what this is, probably just advertising that it speaks v1 of the Tivo Connect protocol.

swversion=4.0.1b-02-2-240
Obviously the version of OS on the Tivo

method=broadcast
Not sure the purpose of this, as Iíve not seen any other ďmethodsĒ announced.

identity=<number>
For a Tivo, this is the Tivo DVR Service number of the box.
For a HMO server, this is

machine=<name>
For a Tivo packet, this is the name that will appear in the Now Playing list.
For a HMO server, this is
platform=<value>
This is the type of hardware the advertiser is running on.
A Tivo advertises: tcd/Series2
My HMO server advertises: pc/WinNT:5.1.2600


services=<value>
This is the type of services running on this box, and the port they run on.
A Tivo advertises: TiVo-ServeTcdVideo-1:2191/tvbus_v3
A HMO server advertises: TiVoMediaServer:8081/http.

Of note is the fact that [1] all fields donít exist in both packets, so obviously not everything is required; [2] Field order doesnít seem to matter (notice that ďmethodĒ comes before ďmachineĒ in the Tivo packet, but after in the HMO server packet); and [3] In a Tivo packet, the fields are lower case, in a HMO server packet, theyíre mixed case.

DocTauri
02-23-2005, 09:51 PM
Space holder until we find out what this port does

DocTauri
02-23-2005, 09:53 PM
It appears that MRV listings are provided via this port. Remember that during the advertisement phase, we notify that we are listening on 2191, I assume that we could change this as required, modify the packet data accordingly, and still get listings to work.

Hereís a capture of a listing from my test box, it has only 1 show on it.

In this example, .11 is the box Iím watching TV on, pulling the listing from .19.

Notice in the 6th packet (20:32:55.148455), the requesting Tivo, after establishing the 3 way handshake (and not sure what the other two packets did), passes an 0x01, then itís Tivo Service #. Iím assuming the service # is passed as part of the protection that normally would only allow them to talk if they were on the same account. Iím assuming the 0x01 is possibly the command code for ďlistĒ.

After ackíing the packet, our serving Tivo then responds with an 0x2b, then itís own service number. Iím assuming 0x2b signifies a response to a previous list (also, for the 0x01 above and the 0x2b here, Iím counting 54 bytes into the packet to bypass the Ethernet and tcp headers).

This is where things get murky for me. After ackíing the response, the requesting Tivo responds with a 0x2b, then a series of 0x00ís, then an 0x06 followed by ďSiBusConnectionĒ end padded with 0x00ís.

Now, we push data back and forth a few more times until the serving Tivo sends itís show list (with description). Iím not sure whatís going on after that either, but they appear to be repetitive, perhaps some form of keep-aliveís?

DocTauri
02-23-2005, 09:54 PM
This appears to be the port that MRV movies are actually transferred on. I havenít yet captured anything here, trying to get my arms around 2191/tcp first.

DocTauri
02-23-2005, 09:57 PM
In case we need it.

Thanks again for any help.

Doc

mrblack51
02-24-2005, 04:04 AM
FYI: the broadcast packets and such were well documented in tivo's HMO documentation. Thanks for looking into this though, nice to see some interest.

check out
www.tivo.com/developer/
go to the tivo desktop resources

TiVoConnectDiscovery.pdf
HmoMusicPhotosSpecv1.1.pdf

edit2: you should check out http://alt.org/forum/index.php?t=msg&th=118&start=0&rid=135

JJBliss
02-24-2005, 09:31 AM
Also consider looking into sniffing the tvbus on the tivo itself to provide a clue as to what exactly is happening on a native MRV transfer.

The latest public release of the routerplus kernel module and the associated tools allow for a lot of investigation that would have ordinarily been obscured.

DocTauri
02-24-2005, 10:31 AM
Exactly the kind of info I was looking for. Thanks for the pointers guys!

Doc

wsewell
03-03-2005, 08:05 PM
I recently threw together a quick Ruby script to allow forwarding of the MRV functionality from one Tivo to another through the internet (VERY SLOWLY of course). If anyone is interested I can post it. I've been working to improve it so that it can collect all the NowShowing data from the local Tivos and compile them into one big NowShowing list (for sharing with a friend or using as a single 'SuperTivo'... this should be a short step away from a MRV server).

In doing this I have figured out some stuff (but not nearly everything):

2190 (as the guys mentioned) is just to allow Tivos to discover each other. If you are adept at Perl/Ruby programming, you can throw together what looks like a Tivo in a couple minutes by forging one of these 'beacon' packets.

2191 actually provides for messaging; basically everything but the beacon and the actual transfer of shows. In other words: Asking for NowShowing List, Asking for show details (ie when you click on a show and are about to 'record' it from the other Tivo), and starting the transfer of the show. I'm currently stuck in the middle of trying to decode the NowShowing data, at least to the point that it can be parsed (and could use some help). I'll post some data in a next post. It appears that the 'details' data for a show is nearly identical to the data for that show in the whole, big NowShowing list that in the beginning. Not sure if they are identical, but VERY close to each other. Lot's of redundancy here. I"ve found one (and maybe two?) unique IDs for the shows, which is the number used for asking for a transfer to start. During all the rounds of messaging (ie every time the client server makes a new connect() and you see a SYN packet) the Tivos trade their Tivo IDs (or at the very least,the client sends his ID).

2300 is the port that the actual transfer of the shows happens on. On sniffing, there are several 'small' packtes (100-200 bytes or so) that must be setup of some kind, followed by LOTS of larger packtest. On a quick inspection by my friend, the larger packets appear to simply be the same data as one would get for an MFS_FTP download, which I guess makes sense (both my Tivos are hacked and descrambled). After many failed intERnet transfers, I realized that the unique ID of the client (sent during messaging) must be known to the Tivo from a recent beacon or the serving Tivo refuses to start the transfer. This seems obvious, but at first, I was using a fake Tivo ID for the tunnel locally, and letting th packes from the remote tivo (including it's ID) just pass through. Essentially I had to broadcast a beacon on the remote Tivo's behalf.

More to come....

wsewell
03-03-2005, 09:04 PM
Ok. So now I'm knee deep in trying to figure the NowShowing list stuff out. I'm attaching a short script I wrote in Ruby that pretends to be a server and provides a nowshowing list with 1 show, and you can click on the show to see it's details. Basically it just presents the whole packets from my network captures. I've been using it to try and figure out what some of the fields in the data are. If you use it BE CARFUL becaus changing some fields can and WILL CRASH YOUR TIVO. And by crash I mean reboot, I've not had any data loss (yet). I'm also attaching a compilation of the 'details' captures from several different shows with several different settings (ie folders on/off, alphabetical listing on/off).

Some quick notes. There are several timestamps. Basically, the are just 64bit extensions of unix timestamps. Not 64bit time, but 32bit timestamps (seconds since the epoch) TIMES 1000000000 (I think that's the right number, anyway it's a power of ten multiplyer NOT a power of 16). In particular there are timestamps for when the actual recording started (2 seconds before the show starts) when the actual recording stops (on time) the Displayed start time and length (lenght represnted as a rather small timestamp) and a timestamp for when it's OK to delete the show (ie the "Keep until at least...." value).

I've found a couple of Unique show IDs (one of which is used during show transfer). I think I've found the 'sorted alphabetically' flag.

The field I've labelled 'Number from last listing' appears to be the index number of the show you are looking at the details for, as it appeared in order in the last listing view. So if you have folders turned off and select the first show, this value is 0, if you select the second show, this value is 1. With folders on, I beleive the initial now showing data is just sent to include the first show from each folder, on clicking on a folder, the server tivo then sends another now showing list of just that folder. Thus this field looks like it's the index from the LAST view (either whole nowshowing or inside a folder).

The two things I'm having the most trouble with are:

1) RIght in the middle of my data, one of the shows diverges and it's record ends up being longer (before the title string) of the other shows. I don't know what this field is or why it's longer. Though I think I've found the preceeding size field, so this may not be an issue.

2) It appears that there is some large record length indicator somewhere, because I can't do anything simple like changing the length of the show title (and change the size field for the title of course) without the Tivo crashing.

Any help looking over the data, playing with fakeserver, or posting more captures would be very helpful. Specifically try capturing discrete events like getting the NowShowing list with folders on/off, alphabetical on/off, folder listing, and details showing. (For the case of folder listing and details showing it would be nice to have at least some minimal listing information about the previous view - ie write down the list of shows and send it along with the capture). I'll rummage through my stuff and post any extra captures I have. With this kind of effort: the more data and the more different kinds of data, the better.

DocTauri
03-04-2005, 09:36 AM
Great work wsewell, I'll try to look into (and absorb more) what you've done this weekend.

WRT: listing w/folders on/off, I have no factual data to base this on, but I'd bet that, as this is a per box setting, the data comes over the same regardless of the host setting, then it's up to the client (receiving) tivo to display it appropriately.

Doc

alldeadhomiez
03-04-2005, 11:17 AM
Ok. So now I'm knee deep in trying to figure the NowShowing list stuff out. I'm attaching a short script I wrote in Ruby that pretends to be a server and provides a nowshowing list with 1 show, and you can click on the show to see it's details. Basically it just presents the whole packets from my network captures. I've been using it to try and figure out what some of the fields in the data are. If you use it BE CARFUL becaus changing some fields can and WILL CRASH YOUR TIVO.

You should be able to use the tvidl facilities built into the 7.1 tivosh to parse/decode the tvbus messages, if you put them into the same format as rpsniff (http://www.pvrhax0r.com/forum/showthread.php?threadid=51) (rpsniff -t "") does. This is probably an easier (reboot-free) way to validate your new packets. Unfortunately, this facility is not available on other versions.

The ID for a recording is the fsid of the Recording object, padded from the right to 64 bits with 0xff. ex: 0x00001234ffffffff

Somewhat OT:

Not only are there tvbus messages for querying the now playing list; there are also messages that allow you to queue and dequeue MRV transfers. These can definitely be injected locally, and perhaps from the netwok as well. Example:


# msg: sess=12 idx=80000089 prot=10628 len=19 flags=00010000 csum=83f5f5 dst=12
# 000: 12 34 00 03 00 01 06 26 00 21 f0 25 00 00 00 09 .4.....&.!.%....
# 010: 00 15 82 c8 ff ff ff ff 01 00 00 00 00 00 00 00 ................
puts "serial 4:"
try { puts [tvidl serialtoxml "\x12\x34\x00\x03\x00\x01\x06\x26\x00\x21\xf0\x25\x00\x00\x00\x09\x00\x15\x82\xc8\xff\xff\xff\xff\x01"] } catch errCode { puts "ERROR" }

prot 0x10624/msg 0x1061f is TvPvrRecordingQueueCreateMsg, which is sent to a local myworld thread to initiate an MRV transfer from another box. The information needed to start the transfer is the other unit's TCD_ID and the fsid of the Recording object.

prot 0x10631/msg 0x10629 is TvPvrRecordingQueueAllMsg, which sends back a list of the queued MRV recordings on the local machine.

DocTauri
03-04-2005, 01:54 PM
ADH, is there any reference anywhere describing TvBus and the tvidl facilities in more detail? Is the tvidl facility some form of interapplication communication? I'd like to understand this process better.

Another project I'd like to do is to be able to programmatically tell my Tivo to MRV a show from another unit, but, if I understand you correctly, this facility is only availble on 7.x and up, so whatever [tvidl] method I discover for doing this in 7.x wouldn't work on my 4.0.1b boxes?

Thanks,
Doc

wsewell
03-04-2005, 06:13 PM
Thanks for the info ADH.

After looking through the info at the alt.org link posted above, I realized that all the NowShowing data sent during MRV messaging is just data described by the IDLs. Specifically, /tvlib/idl/0x14105.tvbin (and all its subtypes). So now it's just a matter of drudging through the IDL files to understand the format and writing code to parse it (no more trial and error). Thanks to drnull's IDL Parser, this shouldn't be a huge task (figure I'll drop all the optional data the first run through).

DocTauri: Did you read the info at alt.org linked to above? It has a pretty good dscription of IDL - after you wrap your brain around the 'self-describing' data descriptions :)

alldeadhomiez
03-04-2005, 06:29 PM
So now it's just a matter of drudging through the IDL files to understand the format and writing code to parse it

Right - as I mentioned, software 7.1 has this built into tivosh, so it should not be too hard to clone the functionality. What might be trickier is finding a way to efficiently represent and traverse these data structures in C.

In general, it seems that the "protocol number" on the message header references both the recipient process and the state machine used to handle the messages. The first protocol number after 12340003 in the message body references the actual message format and should be one of the entries in the state table.

DocTauri - the local and remote tvbus features are present in 4.0+. Note, however, that idl files (and thus message formats) sometimes change between versions. If you look at the tivovbi sources I posted, you'll see that there is different handling for 4.x and 5.x remote messages, since a serial number field was added for 5.1+.

jbuehl
03-05-2005, 08:26 PM
It's good to see someone is looking into figuring out the MRV protocol. I'm interested in building a Linux based MRV server.


Some quick notes. There are several timestamps. Basically, the are just 64bit extensions of unix timestamps. Not 64bit time, but 32bit timestamps (seconds since the epoch) TIMES 1000000000 (I think that's the right number, anyway it's a power of ten multiplyer NOT a power of 16). In particular there are timestamps for when the actual recording started (2 seconds before the show starts) when the actual recording stops (on time) the Displayed start time and length (lenght represnted as a rather small timestamp) and a timestamp for when it's OK to delete the show (ie the "Keep until at least...." value).

The mfs objects store dates as a 32 bit value which is unix time divided by 86400 (the number of seconds in a day). Times are 32 bit values that are the number of seconds since midnight of that day. In general I would guess that looking at the binary representation of data in the mfs objects might be helpful in figuring out the formats of the MRV messages.

wsewell
03-07-2005, 10:42 PM
Yeah, I'm working on a Linux server but I have only gotten as far as starting to decode the NowShowing information and dump it into a data structure. The actual transfer of show data may prove more difficult to understand/implement.

I don't know about the mfs format for time (like I said I haven't gotten there yet), but the field name for all the timestamps in the NowShowing data according to the IDL entry is 'nanoseconds'. Which matches with what I figured out (and posted) earlier, before I realized the IDLs spelled everything out so clearly. I think I had the right number of zeros :)

wsewell
03-09-2005, 06:25 AM
Ok. I've got the first bits of the MRV server come together. The program can broadcast it's own fake Tivo ID (999999999999999) then sit and wait for any other Tivos on the network to speak up. When one does, it asks for that Tivo's Now Showing list, prints it in a pretty format, and then exits. I'm actually storing all the (relevant) info, but only displaying the show title and whether it's a folder or not. Just as a Tivo would do. The next step is to produce a Now Showing list for other Tivo's to see.

In the meantime it would be useful if others could test my program out on their network. Volunteers only please :) The message exchange is fairly regular, and I tried to put in error catching where I felt like it was necessary, but there is always the chance your Tivo could catch on fire and burn your house down. So use at your own risk. I'm including two i686 binaries, built on Debian unstable. They were compiled with GTKMM_LIBS b/c I'm working on an interface too (interface isn't actually shown though). One binary is the standard executable, the other (obivously) has DEBUG stuff in it. Try running the non-debug exec first, if it works correctly let me know :) If it does anything other than print the Now Showing list, please run the DEBUG exec and dump it to a file. A packet capture at the same time would be helpfull as well :)

I'm also including the source if you'd like to take a look, or feel more comfortable building it yourself. You'll need the development libraries to do a build. All the interesting stuff happens in NowShowingEntry.cc and NowShowingCollector.cc. The DEBUG defines are at the top of Mynet.cc and NowShowingCollector.cc (Off by default). I'm open to any suggestions, but please don't judge my code to harshly at this point.

Let me know if you have any problems/questions/concerns.

mikertx
03-09-2005, 10:25 AM
OK, now that the fire department is gone I can report my results. ;)

As you said I had to install the libgtkmm libs on my system. After
that it ran just fine. I have three DTiVo's running version 4.01 and
it showed each of them. It only produced the "Now Showing" list
for one of them and then exited. This is probably expected behaviour
at this point.

Keep up the good work!
--
Mike

mikertx
03-09-2005, 04:01 PM
So, seems like your next step is to generate a "Now Showing" list from the server
that will show up on the TiVo's. I would propose that the data for this list come
from the "showing.xml" file in tmf files. If your servers uses Riley's format then
all-the-better IMHO. A .tmf file is just a tar ball whose files look like this,

showing.xml
part00.ty
part01.ty
...

This way we could populate a server using mfs_ftp and play back using
your app. Of course you might like to go both ways with your app but at
least it would leave the options open.
--
Mike

wsewell
03-09-2005, 04:25 PM
Yes this is indeed. There is actually a loop there that will eventually poll the Tivos regularly for any change in NowShowing information. But for debugging purposes, it just picks the first one and dumps the info on it. Thanks for the help miker!


I have three DTiVo's running version 4.01 and
it showed each of them. It only produced the "Now Showing" list
for one of them and then exited. This is probably expected behaviour
at this point.

wsewell
03-09-2005, 04:31 PM
Sounds like a great idea, thanks for the input. Upon looking at a showing.xml file it doesn't seem like it would be too difficult to parse.


I would propose that the data for this list come
from the "showing.xml" file in tmf files.
...
This way we could populate a server using mfs_ftp and play back using
your app. Of course you might like to go both ways with your app but at
least it would leave the options open.

I do plan to 'go both ways' :) but I don't see any reason to reinvent the wheel here (I'll need a format for storing the relevant show information along with the video data anyway). I'll check to see if all the 'required' data is included in the xml file, even if it's not, there's a good chance it could be spoofed, depending on the field.

AhoyMatey
03-09-2005, 09:13 PM
There may be easier (and better) ways of pulling out the xml info from a tmf file, but I wrote a utility in cpp (using VisualStudio) a while back to create an xml file from a tmf file. I've converted in to c, and it can be compiled for use with Cygwin and for the TiVo. main() basically just parses a given directory (or the pwd if no arg is given) for tmf files to convert. If an xml file exists for a tmf file, the creating of a new xml file is skipped. The work is really done in extract_xml(), which you could modify to your needs. Once you have the xml structure in memory, it would be easy to parse.

(Had to really google around to find functions to make life easier for main(). Probably could be done a lot better. Really missed perl string handling. :rolleyes: )

SR712
03-09-2005, 09:36 PM
I think WinRAR will separate the .tys and the .xml for you, too, on your windows box.

AhoyMatey
03-09-2005, 09:52 PM
I think WinRAR will separate the .tys and the .xml for you, too, on your windows box.Yeah, when I was trying to figure all of this out, I'd rename a .tmf file to a .tar file so that I could load it with winzip. showing.xml is always the first file, followed by the partXX.ty files (as described on alt.org). I designed the code to be able to find a showing.xml in any file position though.

The nice thing about using the function is that it allows you to automatically create xml files from a directory containing a big number of tmf files without any manual intervention. For this application though, the parser can pull the xml file out of the tmf straight into memory for parsing...

jbuehl
03-09-2005, 11:09 PM
I have 2 Tivos with MRV enabled and tivotunnel recognized both Tivos but only listed the Now Showing list of one of them. So far so good :)

I like the idea of using .tmf files on the server as they are self-describing and portable. It would be nice to be able to organize them in directories on the server that relate to the groups that are in Now Showing. Tivo uses the Series to define groups which is in the xml of a .tmf file. So the server could either advertise groups based on the directories that files are in, or it could advertise them based on the xml it found in the files. Maybe this could be a configuration option. The nice thing about directories is that you can invent your own groups (requires extra work, but it's do-able)

Just thinking out loud at this point...

DocTauri
03-14-2005, 03:58 PM
A .tmf file is really just a tarball, so a "tar -xvf <filename>.tmf showing.xml" will extract just the xml portion.

Doc

AhoyMatey
03-15-2005, 02:56 PM
A .tmf file is really just a tarball, so a "tar -xvf <filename>.tmf showing.xml" will extract just the xml portion.

DocMy code was orginally for the PC, so I didn't have the luxury of a tar executable at that time (didn't even know about Cygwin :rolleyes: ). The PC server will have to parse the xml to send the info as MRV info. It will probably also have to parse the parts to send a selected show to the TiVo. The code can be easily adapted for that. Whichever language the server is utlimately written in...

lgkahn
04-01-2005, 09:55 AM
how is this project coming.. as many know due to my pc tivo server software using menus to push/pull shows from to the tivo ... I have lots of movies on my pc media server about 1 terabyte now..

so I believe a better way to do this would be to have the pc server pretend to be a tivo so you can use hmo also..

I am willing to work on this with others and try to get it working..

I only have one of my tivos running 4.0.1b currently
the rest are running 3.1.1e... havent' upgraded yet..

wsewell
04-04-2005, 04:21 PM
Coming along, but slowly. Spent all my free time last week (and a day off work) on a home project. So far the server can see other Tivos, grab their NowShowing Lists, and produce it's own list (from that compiled data). Will be working on the actual transfer of videos off the PC next, and ASAP. As soon as I can get the code organized a bit, I'll put it up somewhere for others to help if they like. Prolly on sourceforge. I'll try to keep everyone aprised on my progress until then.

woodie
04-19-2005, 01:36 PM
My WinXP cygwin-x machine was missing gtkmm.
I'll set this up on a debian box... and see what happens.
Is there more current code than the files from 03-09-2005?

When transferring from the PC to the TiVo, will the server be expected
to convert various formats (like DivX AVIs) to .ty format? As long as the
conversion process takes less time than the transfer, it seems reasonable.

wsewell
04-20-2005, 04:30 PM
My WinXP cygwin-x machine was missing gtkmm.
I'll set this up on a debian box... and see what happens.
Is there more current code than the files from 03-09-2005?


There is. If you can wait a few days (prolly this sunday or so) I'll be releasing all the source I have so far. Neither codebase does anything very useful yet though, so don't get too excited :) The newer source does not need gtkmm <holding my breath>so you should be able to build it under cygwin</holding my breath>


When transferring from the PC to the TiVo, will the server be expected to convert various formats (like DivX AVIs) to .ty format? As long as the conversion process takes less time than the transfer, it seems reasonable.

Ideally, yes. Initially, I'm just going to set it up so that you can easily offload shows from your Tivo for later viewing pleasure, or use .ty files from another Tivo if a friend lends you an episode of something you missed. Other file formats are definately on my wishlist too, so I plan to get to that.

woodie
04-22-2005, 02:45 PM
After creating some lib symlinks (~2.4 -> ~2.0) the compiled app conplained
that the libraries were the wrong size... or something like that. The debain sarge
box I was using has new glib, gtk, etc. I look forward to getting your new code. :-)

Besides the WinXP and debian/sarge boxes, I'm hoping to enhance a SA1/Turbonet TiVo
so that its 'Now Showing' list will be browsable/transferrable from my SA2 using MRV.
The first step could be a MRV-server / TivoWebPlus client proxy, running on a PC.

woodie
04-22-2005, 05:47 PM
Seems like 'offload shows from your Tivo for later viewing pleasure' is a solid goal.
I use multiple TiVos to record mutliple shows that would otherwise conflict, and I'd
like to be able to do that from my SA1/Turbonet unit as well... mostly becasue I can
remotely schedule a recording even when the unit is in another room.

I imagine the 'same household' requirement is holding up this feature:
http://tivohme.sourceforge.net/?page=faq
Q: Can I download video to the DVR?
A: These features will be included in a future release.

NutKase
04-22-2005, 10:04 PM
I'm hoping to enhance a SA1/Turbonet TiVo
so that its 'Now Showing' list will be browsable/transferrable from my SA2 using MRV.

That sounds interesting. Let me know if I can test or help in some way.

Just got into S1 hacking so I'm behind right now...but...


NutKase

wsewell
04-25-2005, 05:07 AM
OK. Here we go. This version is a bit slimmer than the last (no stupid gui libraries needed). As I said before, it still doesn't do anything uber-useful. Basically just grabs the now showing lists from all your Tivos, and pretends to have those shows.

I wanted to have more done before I release any code, but I've got a final exam (not studied yet), final project (not started yet) and full-time work for the next week and a half. And a first baby on the way in a couple weeks - hopefully after my final & project :) So I may be 'busy' for a while and I wanted to at least give any ambitious people out there a start. I hope to be working on it more soon, but it's hard to say.

So here it is. I GPLed it, so feel free to tear it apart, and do whatever you want, but please pass down the GPLed rights (as per the license). If anyone wants to make minor changes/bugfixes to share, just PM me (or make your own tarball). I'd like to sourceforge it at some point too.

As always, your Tivo may crash and lose the newest episode of your wife's favorite show before she gets a chance to watch it. So use at your own risk. Worst thing that ever happened to me was a Tivo reboot and that was during development.

Do post your results good, bad, or otherwise. As well as any comments or suggestions.

Oh... and I'm working on Debian/unstable so your mileage may vary. I wrote a lot of the initial code on my Mac (OSX) so it should work there as well, but I haven't tried it recently. Good luck :)

EDIT:
BTW, I included a README file that tries to explain how it works, and what needs to be done to get it fully functional. Post any questions and I'll try to reply, PM me if I don't reply for a bit.

woodie
04-25-2005, 05:51 AM
From versions 5.2.2-01-2-275 and 7.1a-02-2-240, I get this...

Programs cannot be transferred from the TivoServer DVR
to the <My Tivo> DVR because the TivoServer
DVR is not enabled for transfers.

. . .

wsewell
04-25-2005, 06:28 AM
From versions 5.2.2-01-2-275 and 7.1a-02-2-240, I get this...

Programs cannot be transferred from the TivoServer DVR
to the <My Tivo> DVR because the TivoServer
DVR is not enabled for transfers.

. . .

I think maybe you need to apply superpatch-4all and/or run set_mrv_name.tcl, otherwise your Tivos check to make sure that the tivoserver is on the same billing account... which obviously it is not.

Getting this working with completely unhacked Tivos should be on my TODO list in the README file (probably lowest priority considering all the work yet to be done). If you'd like to work on this, I can send you some packet captures of the transactions with a hacked Tivo.

I know there are others on the forum that understand HMO better than myself, and could comment on the feasability of this.


If you are hacked, sorry for the spew. If you could, please load up ethereal to sniff packets when you try to connect, and post the capture file. I'll try to take a look if I get a chance, or maybe someone else who gets it to work (and has more time) can help out.

lgkahn
05-11-2005, 09:16 PM
started testing... had to add the following lines to Mynet.cc to get it to compile on fedora core 3

#include <errno.h>

extern int errno;

alldeadhomiez
05-11-2005, 10:11 PM
started testing... had to add the following lines to Mynet.cc to get it to compile on fedora core 3

#include <errno.h>

extern int errno;

IIRC you should only use the first line (the #include).

THardie
05-15-2005, 12:48 PM
OK. Here we go. This version is a bit slimmer than the last (no stupid gui libraries needed). As I said before, it still doesn't do anything uber-useful. Basically just grabs the now showing lists from all your Tivos, and pretends to have those shows.Can you post the latest source code then? What I was thinking is I'll take this code, and finish it up under windows to do what you had originally intended (tivo tunnel), so I can link my tivo's with my friends over the internet...

Do you plan on doing that? If not, I'll be more than happy to do some work on it (and of couse, keep it GPL'd)

Terry.

SpoonsJTD
05-24-2005, 12:43 AM
Have you seen this pdf (http://a423.g.akamai.net/7/423/1788/3e68318a7ea60a/www.tivo.com/developer/i/TiVoConnectDiscovery.pdf) for the TiVo Connect Automatic Discovery Protocol Specification?

Not sure it's the right version, it doesn't talk about MRV, and it probably just validates what you reverse engineered already, but it's a good reference. The discussion of the high frequency vs. low frequency mode is interesting. Other specs here (http://www.tivo.com/developer/download_resources.asp).

wsewell
05-24-2005, 03:58 PM
I have, actually. And you're right, it's basically what had already been figured out by several ppl with net sniffing. I did not actually implement the "high frequency" mode thing. The Tivoserver's beacon period is constant; I was too lazy to add support for detecting when a _new_ Tivo is added. Feel free to add this feature and send me a patch :)


Have you seen this pdf (http://a423.g.akamai.net/7/423/1788/3e68318a7ea60a/www.tivo.com/developer/i/TiVoConnectDiscovery.pdf) for the TiVo Connect Automatic Discovery Protocol Specification?

Not sure it's the right version, it doesn't talk about MRV, and it probably just validates what you reverse engineered already, but it's a good reference. The discussion of the high frequency vs. low frequency mode is interesting. Other specs here (http://www.tivo.com/developer/download_resources.asp).


PS - I actually got some preliminary work done on the real guts of the show transfer stuff. I'll post any interesting updates.

EvilJack
05-29-2005, 07:10 AM
I just found this thread... this is what I've been looking for. ;)

I seem to have a slight issue with this running on Linux FC3.
My Two 6.2 DTivos have been patched/hacked to support the
HMO/MRV stuff. I can see each tivo in each other's now
showing list and I can stream shows between them, so I think
that this stuff is set up correctly.

When I run this tivoserver app, I see my two Tivos, but the
now showing list is sort of blank... I see:
========================================================
===========> Listing for tvroom <============
*
*
*
*
*
*
*
*

========================================================
===========> Listing for office <============
*
*
*
*
*
*
*
*

========================================================

I took a peek at the source and it seems to say that I should
see show info on each '*' line.... but .... you see what I see.

Anyone have any ideas. I know a bit of C, a lot of php ( and a
ton of REXX ) but I've never done much with c++ other than
a 'Hello world!' type program.

jack

EvilJack
05-29-2005, 07:25 AM
Now I'm getting:
========================================================
===========> Listing for tvroom <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'tvroom'
===========> Listing for office <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'office'
===========> Listing for tvroom <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'tvroom'
===========> Listing for office <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'office'
===========> Listing for tvroom <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'tvroom'
===========> Listing for office <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'office'
===========> Listing for tvroom <============
ERROR ReadU32: 0
WARNING: error receiving from Tivo 'tvroom'

... that doesn't look right...



... here is the 'debug' output when I get the above error...

===========> Listing for tvroom <============
*NET: Socket()
*NET: Connect( 3232235887, 2191 )
*NET: Write( 0x120x340x00x10x390x390x390x390x390x390x390x390x390x390x390x390x390x390x390x00x00x00x00x40x00x00x00x 00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x0 )
*NET: Read( 4 )
*NET: Error reading from stream: 0
*NET: Errno: 0
ERROR ReadU32: 0
*NET: Read( 20 )
*NET: Error reading from stream: 0
*NET: Errno: 0
WARNING: error receiving from Tivo 'tvroom'
===========> Listing for office <============
*NET: Socket()
*NET: Connect( 3232235888, 2191 )
*NET: Write( 0x120x340x00x10x390x390x390x390x390x390x390x390x390x390x390x390x390x390x390x00x00x00x00x40x00x00x00x 00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x0 )
*NET: Read( 4 )
*NET: Error reading from stream: 0
*NET: Errno: 0
ERROR ReadU32: 0
*NET: Read( 20 )
*NET: Error reading from stream: 0
*NET: Errno: 0
WARNING: error receiving from Tivo 'office'
===========> Listing for tvroom <============
*NET: Socket()
*NET: Connect( 3232235887, 2191 )
*NET: Write( 0x120x340x00x10x390x390x390x390x390x390x390x390x390x390x390x390x390x390x390x00x00x00x00x40x00x00x00x 00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x0 )
*NET: Read( 4 )
*NET: Error reading from stream: 0
*NET: Errno: 0
ERROR ReadU32: 0
*NET: Read( 20 )
*NET: Error reading from stream: 0
*NET: Errno: 0
WARNING: error receiving from Tivo 'tvroom'




jack

EvilJack
05-29-2005, 07:55 AM
Here is a bit of 'debug' output... I selected 1 ( maybe 2 )
'shows ( * )' for my output. I 'see' the show description
in the debug info... just don't see them on the normal
output.....



*
*NET: Read( 4 ) 0x0 0x0 0x1 0x4 |....|
*NET: Read( 4 ) 0x0 0x0 0x0 0x0 |....|
*NET: ReadSkip( 16 )
*NET: Read( 4 ) 0x12 0x34 0x0 0x3 |.4..|
*NET: Read( 4 ) 0x0 0x1 0x41 0x5 |..A.|
*NET: Read( 4 ) 0x1 0xab 0x96 0x1d |....|
*NET: Read( 4 ) 0x0 0x0 0x2 0x59 |...Y|
*NET: Read( 240 ) 0x0 0x0 0x0 0x7 0x1 0x0 0x0 0x0 0x36 0x0 0x0 0x0 0x32 0x12 0x34 0x0 0x3 0x0 0x1 0x41 0x4 0x1 0xe7 0xe8 0x2e 0x0 0x0 0x0 0x22 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0xd 0x5e 0x38 0xff 0xff 0xff 0xff 0x1 0x0 0x6 0x78 0x89 0xff 0xff 0xff 0xff 0x0 0x0 0x0 0x7 0x0 0x0 0x0 0xf 0x1 0x0 0x0 0x0 0x62 0x0 0xd 0x5e 0x38 0xff 0xff 0xff 0xff 0x0 0x0 0x0 0x4 0x1 0x0 0x0 0x0 0x2 0x0 0x0 0x0 0x8 0xf 0x81 0x40 0xf4 0x2a 0xde 0x40 0x0 0x0 0x0 0x0 0x8 0xf 0x80 0xa3 0xca 0x91 0xa 0xac 0x0 0x0 0x0 0x0 0x8 0xf 0x80 0xa7 0x11 0x74 0x93 0xaa 0x0 0x0 0x1 0x0 0x0 0x0 0x2 0x0 0x1 0x0 0x0 0x0 0x1c 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x14 0x0 0x0 0x0 0x10 0x12 0x34 0x0 0x3 0x0 0x1 0x2 0x15 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x2 0x1 0x0 0x0 0x0 0x25 0x0 0xd 0x5e 0x38 0x0 0x0 0x0 0xb 0x0 0x0 0x0 0x8 0xf 0x80 0xa3 0xcb 0x8 0x40 0x40 0x0 0x0 0x0 0x0 0x8 0x0 0x0 0x3 0x46 0x30 0xb8 0xa0 0x0 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x1 0x6e 0x0 0x0 0x0 0x6 0x41 0x6c 0x69 0x61 0x73 0x0 0x1 0x0 0x0 0x0 0x11 0x42 0x65 0x66 0x6f 0x72 0x65 0x20 0x74 0x68 0x65 0x20 0x46 |........6...2.4....A........"...........^8.......x.................b..^8...................@.*.@.....................t........... .................4.......................%..^8.............@@........F0............n....Alias......B efore the F|
*NET: Read( 4 ) 0x0 0x0 0x1 0x4 |....|
*NET: Read( 4 ) 0x0 0x0 0x0 0x0 |....|
*NET: ReadSkip( 16 )
*NET: Read( 256 ) 0x6c 0x6f 0x6f 0x64 0x0 0x1 0x1 0x0 0x0 0x0 0x51 0x4f 0x6e 0x6c 0x79 0x20 0x53 0x79 0x64 0x6e 0x65 0x79 0x2c 0x20 0x4a 0x61 0x63 0x6b 0x2c 0x20 0x49 0x72 0x69 0x6e 0x61 0x2c 0x20 0x56 0x61 0x75 0x67 0x68 0x6e 0x20 0x61 0x6e 0x64 0x20 0x4e 0x61 0x64 0x69 0x61 0x20 0x63 0x61 0x6e 0x20 0x73 0x61 0x76 0x65 0x20 0x74 0x68 0x65 0x20 0x77 0x6f 0x72 0x6c 0x64 0x20 0x66 0x72 0x6f 0x6d 0x20 0x45 0x6c 0x65 0x6e 0x61 0x27 0x73 0x20 0x70 0x6c 0x61 0x6e 0x2e 0x0 0x0 0x0 0x0 0x6 0x0 0x0 0x1 0x70 0x0 0x0 0x1 0x77 0x0 0x0 0x0 0x69 0x0 0x0 0x0 0x7c 0x0 0x0 0x0 0x8c 0x0 0x0 0x1 0x81 0x0 0x0 0x0 0xb 0x0 0x0 0x0 0x10 0x47 0x61 0x72 0x6e 0x65 0x72 0x7c 0x4a 0x65 0x6e 0x6e 0x69 0x66 0x65 0x72 0x0 0x0 0x0 0x0 0xe 0x47 0x61 0x72 0x62 0x65 0x72 0x7c 0x56 0x69 0x63 0x74 0x6f 0x72 0x0 0x0 0x0 0x0 0xb 0x52 0x69 0x66 0x6b 0x69 0x6e 0x7c 0x52 0x6f 0x6e 0x0 0x0 0x0 0x0 0xf 0x56 0x61 0x72 0x74 0x61 0x6e 0x7c 0x4d 0x69 0x63 0x68 0x61 0x65 0x6c 0x0 0x0 0x0 0x0 0xc 0x4c 0x75 0x6d 0x62 0x6c 0x79 0x7c 0x43 0x61 0x72 0x6c 0x0 0x0 0x0 0x0 0xe 0x57 0x65 0x69 0x73 0x6d 0x61 0x6e 0x7c 0x4b 0x65 0x76 0x69 0x6e 0x0 0x0 0x0 0x0 0xe 0x47 0x72 0x75 0x6e 0x62 0x65 0x72 0x67 0x7c 0x47 0x72 0x65 0x67 0x0 0x0 0x0 0x0 0xc 0x4d 0x61 0x65 0x73 |lood......QOnly Sydney, Jack, Irina, Vaughn and Nadia can save the world from Elena's plan.........p...w...i...|................Garner|Jennifer.....Garber|Victor.....Rifkin|Ron.....Varta n|Michael.....Lumbly|Carl.....Weisman|Kevin.....Grunberg|Greg.....Maes|
*NET: Read( 4 ) 0x0 0x0 0x1 0x4 |....|
*NET: Read( 4 ) 0x0 0x0 0x0 0x0 |....|
*NET: ReadSkip( 16 )
*NET: Read( 112 ) 0x74 0x72 0x6f 0x7c 0x4d 0x69 0x61 0x0 0x0 0x0 0x0 0x18 0x42 0x65 0x6e 0x65 0x64 0x69 0x63 0x74 0x7c 0x52 0x6f 0x62 0x65 0x72 0x74 0x20 0x50 0x61 0x74 0x72 0x69 0x63 0x6b 0x0 0x0 0x0 0x0 0xc 0x42 0x72 0x61 0x67 0x61 0x7c 0x53 0x6f 0x6e 0x69 0x61 0x0 0x0 0x0 0x0 0xa 0x4f 0x6c 0x69 0x6e 0x7c 0x4c 0x65 0x6e 0x61 0x0 0x0 0x0 0x1 0x0 0x0 0x32 0x80 0x0 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x2 0x1 0x0 0x0 0x0 0xd 0x0 0x0 0x0 0x6 0x41 0x6c 0x69 0x61 0x73 0x0 0x1 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 |tro|Mia.....Benedict|Robert Patrick.....Braga|Sonia.....Olin|Lena......2....................Alias...............|
*
*NET: Read( 4 ) 0x0 0x0 0x1 0x4 |....|
*NET: Read( 4 ) 0x0 0x0 0x0 0x0 |....|
*NET: ReadSkip( 16 )
*NET: Read( 4 ) 0x12 0x34 0x0 0x3 |.4..|
*NET: Read( 4 ) 0x0 0x1 0x10 0x9 |....|
*NET: Read( 4 ) 0x1 0x4b 0x3d 0xa3 |.K=.|
*NET: Read( 4 ) 0x0 0x0 0x0 0xa |....|
*NET: Read( 16 ) 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x8 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 |................|
*NET: WriteU32( 260 )
*NET: WriteU32( 0 )
*NET: WriteU32( 82179 )
*NET: WriteU32( 33 )
*NET: WriteU32( 33685504 )
*NET: WriteU32( 24819797 )
*NET: WriteU32( 305397763 )
*NET: WriteU32( 69642 )
*NET: WriteU32( 23982125 )
*NET: WriteU32( 17 )
*NET: WriteBool8()
*NET: WriteU32( 415 )
*NET: WriteU32( 8 )
*NET: WriteU32( 8 )
*NET: WriteU32( 8 )
*NET: Write( 0x760x1f0x3c0x7f0x760x1f0x3c )
*NET: Read( 4 ) 0x0 0x0 0x1 0x4 |....|
*NET: Read( 4 ) 0x0 0x0 0x0 0x0 |....|
*NET: ReadSkip( 16 )
*NET: Read( 4 ) 0x12 0x34 0x0 0x3 |.4..|
*NET: Read( 4 ) 0x0 0x1 0x10 0x9 |....|
*NET: Read( 4 ) 0x1 0x4b 0x3d 0xa3 |.K=.|
*NET: Read( 4 ) 0x0 0x0 0x0 0xa |....|
*NET: Read( 16 ) 0x0 0x0 0x1 0x9f 0x0 0x0 0x0 0x0 0x1 0x0 0x0 0x0 0x0 0x0 0x0 0x0 |................|
*NET: WriteU32( 260 )
*NET: WriteU32( 0 )
*NET: WriteU32( 82179 )
*NET: WriteU32( 33 )
*NET: WriteU32( 33685504 )
*NET: WriteU32( 24819797 )
*NET: WriteU32( 305397763 )
*NET: WriteU32( 69642 )
*NET: WriteU32( 23982125 )
*NET: WriteU32( 17 )
*NET: WriteBool8()
*NET: WriteU32( 415 )
*NET: WriteU32( 8 )
*NET: WriteU32( 8 )
*NET: WriteU32( 8 )
*NET: Write( 0x760x1f0x3c0x7f0x760x1f0x3c )

========================================================

EvilJack
05-29-2005, 08:23 PM
I recently threw together a quick Ruby script to allow forwarding of the MRV functionality from one Tivo to another through the internet (VERY SLOWLY of course). If anyone is interested I can post it.

I'd like to see it. I can't figure out the c++ code posted
father down. Maybe your code will show some light on it.

jack

RavenStL
05-30-2005, 04:45 PM
I have been toying with this also. Couple questions:

Is the Data (command to ask for list) on the TCP packet start with 1234 0001 ? The first page Doc Tauri said it started with 0001 but I see the C code sending the 1234.

I am toying with Visual Basic and ethereal packet sniffer and see a few differences. My packets are 2 bytes larger in header size, no biggee I think. Just a 0000 at the end. But still 54 bytes as doc said, but some packets are only 52 and thats why I ask about the 1234. Is that part of theheader or actual data to send to the Tivo?

Last question, I have a 15 digit Service number. I see the 3810000FFFFFFFF int he packet. so is the service number 381-0000-FFFF-FFFF? or is it some sort of format I cannot see?

would I use the hexadecimal value to represent my service number (to spoof) or the ascii value of the service number? Mine is like 151-0000-A000-3DFA or whatever.

I am guessing I use the ascii representation of my service number digits?

RavenStL
05-30-2005, 05:15 PM
Got it working. 1234 appears needed and Ascii values work great.

So Doctauri, are your headers only 52 bytes?

darrin75
05-30-2005, 05:16 PM
Does anyone have a windows binary build to play with?

EvilJack
05-30-2005, 05:32 PM
Has anyone but the author of tivoserver and tivotunnel gotten
them to run on linux and work with a DTV Tivo? My 6.2 DTV Tivo
doesn't seem to work well with this server code. Is it just me,
or is it the code in general?

I've spent a few hours looking / testing the code and get
get a handle on what's wrong.

jack

RavenStL
05-30-2005, 06:58 PM
Made some progress, I may be behind others, but I like to post what I find.

Apparently when you get the remote Service Number response from the serving Tivo, position # 0x0033 is 2b (using DocTauri's posted dump). That value may change as it does with my DirecTivo HDVR2. I store that value and use it on the next packets as seen on DocTauris. 0x0033 is always 2b. BUT on my HDVR2's, if I rerun the 2b will change to something else. Apparently incrementing.

So what types and models are otehrs working on this for?

I am playing with HDVR2's running 4.01.b-02-2-240.

wsewell
06-02-2005, 06:12 PM
Just for the record I've been testing on two different boxes. My main development box is a hacked Series 2, running 4.0.1b-02-2-240, with no Tivo service. I also regularly use my DTivo which is hacked with a standalone software version of 4.0.1-01-2-240 (to enable MRV et al.), with service.

The fact that you originally got lots of '*'s is good (you're right these should have show information after them). It could just be a matter of me assuming the title is always in the same spot. There are at least a couple "title" fields: show title, series title. I can't tell much from just what you posted, it would be easier for me to debug if you could dump the output to a file ( like this "./tivoserver > log.txt"). Just hit CTRL+C after a few minutes. If you are familiar with ethereal (or another network traffic sniffer) a dump would be really helpful too. I can work with the debug output, just a bit slower.

As for the error you were getting in your second post. I'm not sure why that's occuring. It looks like the serving tivo is closing the network connection because 0 (zero) bytes are returned as being read. Did this happen a lot? Did it happen after the tivoserver had been running a while (and some phantom show info shown) or as the first output on a later run?


Has anyone but the author of tivoserver and tivotunnel gotten
them to run on linux and work with a DTV Tivo? My 6.2 DTV Tivo
doesn't seem to work well with this server code. Is it just me,
or is it the code in general?

I've spent a few hours looking / testing the code and get
get a handle on what's wrong.

jack

EvilJack
06-02-2005, 06:31 PM
The fact that you originally got lots of '*'s is good (you're right these should have show information after them). It could just be a matter of me assuming the title is always in the same spot. There are at least a couple "title" fields: show title, series title. I can't tell much from just what you posted, it would be easier for me to debug if you could dump the output to a file ( like this "./tivoserver > log.txt"). Just hit CTRL+C after a few minutes.
As for the error you were getting in your second post. I'm not sure why that's occuring. It looks like the serving tivo is closing the network connection because 0 (zero) bytes are returned as being read. Did this happen a lot? Did it happen after the tivoserver had been running a while (and some phantom show info shown) or as the first output on a later run?

that's kind of the issue... that WAS the output from something
like: ./tivoserver > log.txt. It also showed me two lines
saying it found my two tivos, but that was it... that WAS the
output.....

after about 3 minutes... it started with the 'connection'
issue.

I'll post ( or tell you where I put it ) some more detailed
debug output that I came up with. Not sure if it will help or
not.

I know a bit about tcpdump but I'll have to look and see
what options give all of the data and not just the first
x bytes. I'll see if I can gather that info later tonight.

jack

wsewell
06-02-2005, 07:05 PM
Yeah, I need the whole debug output unfortunately. From the very beginning down to at least where the second show info shows up (or more). Just gzip up your log file and post it (or PM me and you can email me w/ it directly if you don't wanna post the whole file).


that's kind of the issue... that WAS the output from something
like: ./tivoserver > log.txt. It also showed me two lines
saying it found my two tivos, but that was it... that WAS the
output.....


That's why I asked. Looks like there's a bug earlier on that misreading something, and eventually leading to the Tivos getting out of sync and the serving Tivo closes the connection. Just a theory thogh :) Let's get the first bit worked out first and see if this problem persists.



after about 3 minutes... it started with the 'connection'
issue.


Thanks for helping debug :)

wsewell
06-03-2005, 02:06 AM
Okay EvilJack,
It looks like your serving Tivo is sending the now showing data in a slightly different format than what tivoserver can handle (which is based on what I've groked form the IDL files on my own Tivos). Despite the fact that the IDL version numbers seem to be the same as my own (ie the 0x0003 in 0x12340003) that you may see in the debug logs.

For those that care, it looks to be an extra optional field (5 bytes, first one set to 0x1) just after bitstreamFormat in TvDbRecordingInfo (0x10222) which is in TvDbNowShowingBucketItem (0x14105). Of course it could be an extra field after where my TvDbRecordingInfo field ends and before where my TvDbShowingInfo ends in TvDbNowShowingBucketItem. But either way, there are an extra 5 bytes.

EvilJack: I need to take a look at what your Tivos think is correct. I assume you can telnet and ftp to/from your Tivo? If so, first telnet to your Tivo, cd to /tvlib/idl, and make sure there are a lot of .tvbin files there, especially check for 0x14105.tvbin and 0x10222.tvbin (guess you could do this in ftp too). Then fire up ftp and download those two files and send them to me. If you're curious, you can look at them with a hex editor, or using the parser that drnull wrote:

http://alt.org/forum/index.php?t=msg&th=118&start=0&rid=135

It would be nice to have all of the .tvbin files in there if there does turn out to be a version discrepancy. But for now those two should do. If you're uncomfortable sending me random files from your Tivo, I can send you mine and you can diff them :)

-- Edit --

Actually, it looks like the rest of the NowShowingBucket is the same. So you can try to patch my IDL definition in your tivoserver source in file TvStructs.hh (these edits are for the TvDbRecordingInfo if the line numbers are wrong):

Change line 176
from:
&recordQuality, &bitstreamFormat );
to:
&recordQuality, &bitstreamFormat, &extraThing );

And add at line at line 194 like so:

OPTIONAL<LONG> extraThing;

Then recompile, and run and let me know how it goes.

drnull
06-03-2005, 11:08 AM
Okay EvilJack,
It looks like your serving Tivo is sending the now showing data in a slightly different format than what tivoserver can handle (which is based on what I've groked form the IDL files on my own Tivos). Despite the fact that the IDL version numbers seem to be the same as my own (ie the 0x0003 in 0x12340003) that you may see in the debug logs.

Yeah, tragically, it looks like tivo went and changed the formats. I guess if 4.0 tivos can talk to 6.2 tivos and share shows between themselves, there is other Black Magic going on here. Possibly the source tivo sends his idl over so the receiving tivo knows how to interpret the bitstream.

Anyways, I'll send ya the 6.2 .tvbin's if you're interested, but here's what I've seen that changed:



Changes between 4.0 & 6.2

Transition 0x14103: TvDbNowShowingBucket
0x11008: TvBusQueryPropertiesEvent
Added OPTIONAL<LONG> iSpec at the end

0x2f2f: TvDbPresentationBehavior
Added a few enumerated elements
LINE_IN=0x3,
REPEAT_MANUAL=0xd,
SINGLE_MANUAL=0xc,
AUTO_RECORD_SHOWCASE=0xf,
AUTO_RECORD_VIDEO_STORE=0x10

0x2f52: TvDbTunerBehavior (new)
new enumerated type
CACHE_HISTORY=0xa,
DATA_BLIZZARD=0x9,
DATA_DTV_DFB_MANDATORY=0x12,
DATA_DTV_DFB_OPTIONAL_HIGH=0x14,
DATA_DTV_DFB_OPTIONAL_LOW=0x13,
DATA_DTV_SW_UPDATE_MANDATORY=0xf,
DATA_DTV_SW_UPDATE_OPTIONAL=0x10,
DATA_PLAN_Y=0x5,
DELAYED_START=0x11,
EPG=0xc,
FUZZY=0x7,
INDEFINITE_LENGTH=0xb,
LOW_SINGLE_EXPLICIT=0xe,
NOT_APPLICABLE=0x8,
PAY_PER_VIEW=0x3,
SEASON_PASS=0x2,
SINGLE_EXPLICIT=0x1,
TIME=0xd,
TIVO_CLIPS=0x6,
WATCH_ONLY=0x4

0x10222: TvDbRecordingInfo
Added OPTIONAL<TvDbTunerBehavior> tunerBehavior at the end

0x14110: TvDbThemeInfo
Added OPTIONAL<UNSIGNED_LONG_LONG> id at the end
Added OPTIONAL<STRING> strShortName at the end

0x14108: TvDbCollectionInfo
Added OPTIONAL<TvDbPresentationBehavior> enumSspPresentationBehavior at the end

A few of those have the unhappy effect of bungling the now showing retrieval.

Also, almost all of the files are different at the binary level, even if they have the same resulting structure. The examples I've seen of this are files where the first checksum (offset 0x0a) is different, but the rest of the file is identical. Very odd. I still am not sure how that checksum is generated, and it boggles me why they might decide to change it. Possibly it's based on the software version? Example 0x10002, the TvBusStruct class:

4.0 1234 0003 0001 0002 0086 da1f 0000 007d .4........ŕ....}
6.2 1234 0003 0001 0002 00b1 2bdd 0000 007d .4........+›...}

Hope it helps.

wsewell
06-03-2005, 12:32 PM
Wow. Thanks drnull. Actually, EvilJack already tgzed his up and PMed them to me. Thanks tho.


Yeah, tragically, it looks like tivo went and changed the formats. I guess if 4.0 tivos can talk to 6.2 tivos and share shows between themselves, there is other Black Magic going on here. Possibly the source tivo sends his idl over so the receiving tivo knows how to interpret the bitstream.

Anyways, I'll send ya the 6.2 .tvbin's if you're interested, but here's what I've seen that changed:


Interesting... that is odd. Maybe they aren't even using the checksum, but that doesn't explain why they would bother to change it. Maybe you're right and they're using the major version number or something. Thanks for the input, glad it's not just my code that is b0rked.



Also, almost all of the files are different at the binary level, even if they have the same resulting structure. The examples I've seen of this are files where the first checksum (offset 0x0a) is different, but the rest of the file is identical. Very odd. I still am not sure how that checksum is generated, and it boggles me why they might decide to change it. Possibly it's based on the software version?

-- EDIT --

OK. So I patched one of the source files for those of you running version 6.2 out there. Copy it over your TvStructs.hh file and do a 'make clean' then 'make'. Still need to figure out a way to determine which version is in use at runtime. But this should do for now; so at least let you 6.2ers can help me debug :) I verified that it compiles, but I don't have a 6.2 box, so I have no way of knowing whether it runs correctly.

wsewell
06-04-2005, 06:23 PM
Yeah, tragically, it looks like tivo went and changed the formats. I guess if 4.0 tivos can talk to 6.2 tivos and share shows between themselves, there is other Black Magic going on here.

DrNull: Do you know for sure that 4.0 Tivos and 6.2 Tivos can share shows between each other? Tried to 'tunnel' my 4.0 with a 6.2 user with no success. I could see his Tivo in my list, but he could not see mine. When I tried to actually look at the showing list on his Tivo, mine failed. This would be consitent with a 6.2 box being aware of a 4.0 box and ignoring it; and a 4.0 box thinking the 6.2 box was OK until it realized that the nowshowing bucket data was malformed. Of course it could have just been a problem with the tunnel. Thoughts?

weston

DocTauri
06-04-2005, 08:50 PM
I can confirm that 4.0 and 6.2 can share shows. I have a SAS2 running 4.0.1b that acts as a server to my 4x S2 DirecTivos running 6.2, it's all working as expected, just as it did when all units were at 4.0.1b.

Doc

drnull
06-04-2005, 09:01 PM
DrNull: Do you know for sure that 4.0 Tivos and 6.2 Tivos can share shows between each other? Tried to 'tunnel' my 4.0 with a 6.2 user with no success. I could see his Tivo in my list, but he could not see mine. When I tried to actually look at the showing list on his Tivo, mine failed. This would be consitent with a 6.2 box being aware of a 4.0 box and ignoring it; and a 4.0 box thinking the 6.2 box was OK until it realized that the nowshowing bucket data was malformed. Of course it could have just been a problem with the tunnel. Thoughts?

That's why I said "if". :) I'm pretty sure I remembered somebody saying that they had successfully transferred between superpatched 4.0's and 6.2's. Searching...

http://www.dealdatabase.com/forum/showpost.php?p=222240&postcount=10
http://www.dealdatabase.com/forum/showpost.php?p=223984&postcount=4

It appears that some have had success with this. Possibly? I guess we'll have to wait for someone to answer definitively.

Edit: So there ya go. Thanks, Doc

wsewell
06-04-2005, 09:01 PM
I can confirm that 4.0 and 6.2 can share shows. I have a SAS2 running 4.0.1b that acts as a server to my 4x S2 DirecTivos running 6.2, it's all working as expected, just as it did when all units were at 4.0.1b.

Doc

Thanks Doc.

garyw90
06-07-2005, 11:55 PM
Thanks wsewell for your work with helping to understand this protocol.

I've been trying to implement it on .NET, and my Tivo is running 7.1b. It seems like the Tivo is shutting down the connection right ater sending its own ID. I can connect to it, send the ID from the PC, and then get the response. Before I can send the next part of the handshake, the Tivo closes the connection (not by resetting it, but with a FIN).

Could you tell me if you have seen this sort of behavior in your own development? Is there something about the 7.1b protocol that has changed? Does anybody else have 7.1b with this working?

garyw90
06-08-2005, 12:05 AM
No, I do not have this converted to Windows, but I am certainly working on it.

wsewell
06-08-2005, 05:02 AM
I think I did have a similar problem but it ended up being my networking code. I'm not sure if the 7.1b protocol has changed, but if the Tivo is sending a FIN _right_ after it sends its own ID (and before your next packet even gets sent to it) then it seems like it's a network problem, not a protocol problem.

Oh wait, one thing to be sure and check is that you are sending the Beacon _before_ you try to make a connection, or the serving Tivo won't talk to you (and it's possible that it would only go so far as to send its own ID back and then close the connection). So set up another thread or process or program to send a well crafted Beacon to announce the existence of your "fake" tivo. I just did a network dump of the Beacon from my real Tivo to make sure the format was correct, and changed the ID to all 9s. And, btw, the ID you send in the Beacon has to be the same as the one you send when connecting or similar problems occur.


Thanks wsewell for your work with helping to understand this protocol.

I've been trying to implement it on .NET, and my Tivo is running 7.1b. It seems like the Tivo is shutting down the connection right ater sending its own ID. I can connect to it, send the ID from the PC, and then get the response. Before I can send the next part of the handshake, the Tivo closes the connection (not by resetting it, but with a FIN).

Could you tell me if you have seen this sort of behavior in your own development? Is there something about the 7.1b protocol that has changed? Does anybody else have 7.1b with this working?

drnull
06-08-2005, 08:52 AM
Oh wait, one thing to be sure and check is that you are sending the Beacon _before_ you try to make a connection, or the serving Tivo won't talk to you (and it's possible that it would only go so far as to send its own ID back and then close the connection).

Actually, I don't think that's entirely true. In 4.0, the tivo would happily send you the now playing list, and let you get as far requesting a show. At that point, if you hadn't sent a beacon out (and thus weren't in the /State/KnownHost) then the tivo would return a TRANSIENT_NET_ERROR (TvNetworkVideoError, type 0x16baf) to you.

On the other hand, I can confirm that if you send a beacon with one id, and then connect with another id, the tivo clams up. I guess it doesn't like schizophrenic clients.

garyw90
06-08-2005, 10:28 AM
wsewell,
Thanks for your reply. Yep, I've been checking those things and still cannot get it to cooperate. I start one thread that is the listening beacon, then another the broadcast beacon. When I get a beacon notification I fire an event that tries to connect to the Tivo to get the shows.

I agree that it sounds like possibly a network protocol problem, but I'm not seeing where it is. I'm attaching an Ethereal dump of the packets. If you have some traces of a working system, I'd be happy to look through them.

One more dumb question: do I have to do anything to enable MRV on 7.1b? The HMO option is turned on.

Jamie
06-08-2005, 12:01 PM
One more dumb question: do I have to do anything to enable MRV on 7.1b? The HMO option is turned on.You are superpatched aren't you? Without the superpatch, I don't think it can work since the sessions are encrypted and your PC won't be in the tivo group cert.

garyw90
06-08-2005, 12:22 PM
No, there are no patches or hacks on the box at all. This may be unrealistic, but I my goal is to work with an unmodified Tivo. The only thing I have done to the box is replaced the drive and install the PTVUpgrade InstantCake.

Jamie
06-08-2005, 12:45 PM
No, there are no patches or hacks on the box at all. This may be unrealistic, but I my goal is to work with an unmodified Tivo. The only thing I have done to the box is replaced the drive and install the PTVUpgrade InstantCake.Won't work. An unpatched tivo will only MRV with another TiVo on the same account, enabled at www.tivo.com. Also, MRV sessions on unpatched tivos are encrypted, so unless you know the encryption algorithms and keys, you won't be able to communicate beyond the initial handshake.

wsewell
06-08-2005, 07:49 PM
Yeah, everything looks fine to me... right up until the Tivo sends the FIN :) Here's a capture file I took a while back of a browse and the first part of a transfer. It's between my 4.0 standalone (hacked) and my 4.0 directivo (also hacked). Don't know if you've given up after hearing about the encryption. If you still are just interested in a .NET version, you could superpatch your box, and get the MRV server working that way. Then later you could restore the original drive image, and work on the encryption stuff. That's effectively what I'm doing, though I don't know how loft a goal it is to ever get this thing working with an unhacked Tivo.

Oh.. and the file I'm sending is an ethereal .cap file, you should be able to just load it up (makes for a bit easier browsing than a text file). IF you have any probs, I can text-ify it :)



wsewell,
Thanks for your reply. Yep, I've been checking those things and still cannot get it to cooperate. I start one thread that is the listening beacon, then another the broadcast beacon. When I get a beacon notification I fire an event that tries to connect to the Tivo to get the shows.

I agree that it sounds like possibly a network protocol problem, but I'm not seeing where it is. I'm attaching an Ethereal dump of the packets. If you have some traces of a working system, I'd be happy to look through them.

One more dumb question: do I have to do anything to enable MRV on 7.1b? The HMO option is turned on.

AlphaWolf
06-09-2005, 12:39 AM
No, there are no patches or hacks on the box at all. This may be unrealistic

Talk about pandoras box...nay, I think pandoras box looks better in comparison.

wsewell
07-06-2005, 05:05 AM
Ok boys and girls. I have _something_ to share. I finally got the tivoserver working to some extent. It currently only supports TMF files, so crank up mfs_ftp and download some vids from your Tivo to test with. The Now Showing list should work correctly. Video transfer works, but not completely. I'm only releasing the source at this stage because I know of several people waiting to mess around with this stuff. (Also I could use some help debugging; esp. from those ppl running 6.2).

There are some issues with the duration/timestamps (which applies to the green status bar at the bottom, and whether the Tivo stops playing when it gets to the end of the show). These issues with playback are to be expected and it's possible that the Tivo could reboot when it gets to the end of the show, so BE CAREFUL. Also multi-part videos may or may not work.

Please report any issues (other than the ones I've noted above), and let me know if you have any suggestions/bugfixes. As always please use at your own risk, this is very much development software and I could possibly erase all your shows and crash your Tivo :) Please be sure to read the README before you try it out.

mateom199
07-07-2005, 01:10 AM
Im running 6.2, and I'm attempting to compile now. I'll report back my results.
Can't wait for tivo->pc mrv, sounds great.

nams
07-12-2005, 01:37 PM
Worked fine for me on Debian with 6.2

If we can get this to work with TY files created with mplex it would be perfect.

I tried uploading the mplex created TY files to my SD Directivo running 6.2 and HD Directivo running 3.1.5.e download the TMF to my tivoserver but it did not work.

wsewell
07-13-2005, 05:33 PM
Yeah, can't wait to get some more file types working. That's interesting that your experiment didn't work. Did the show not show up in the Now Showing list or was it a problem with the actual video transfer? If it was just a Now Showing list problem, the it's probably a simpl(er) fix. You could help me out by "tar -xvf blah.tmf" the tmf file and sending me the showing.xml file. Be sure to delete the .part files afterwards.


Worked fine for me on Debian with 6.2

If we can get this to work with TY files created with mplex it would be perfect.

I tried uploading the mplex created TY files to my SD Directivo running 6.2 and HD Directivo running 3.1.5.e download the TMF to my tivoserver but it did not work.

wsewell
07-13-2005, 05:42 PM
Also, I forgot to mention that you need to have libtar and libxml2 installed before you compile tivoserver. You need the development libraries for these too (so that's libtar-dev and libxml2-dev under Debian variants). So if you tried to compile and gave up b/c of errors, do this before giving up for good. Sorry I don't have a configure script working yet. If someone has the knowledge/time to do this, it would be super-helpful.

Also, I don't know if anyone (anyone at all!) is a Mac user on this forum. But I do about 50% of my work on my Powerbook (when away from home). So tivoserver should work fine under OSX. The libraries mentioned above are available for OSX either with fink or as source (I think I had to download the libtar source tarball and build/install it the old fashioned way).

darrin75
07-13-2005, 06:34 PM
Here's the error i get under cygwin



Owner@Bobby ~
$ cd /usr/src/tivoserver

Owner@Bobby /usr/src/tivoserver
$ make
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib
Mynet.cc
Mynet.cc:29: error: invalid function declaration
make: *** [Mynet.o] Error 1

Owner@Bobby /usr/src/tivoserver
$

RavenStL
07-13-2005, 09:20 PM
I currently dont have the room for linux and am trying the cygwin emulator.

Here is my error from G++ on cygwin on an XP pro machine




g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Streamable.o -L/usr
/lib Streamable.cc
Streamable.hh: In member function `void TivoType<TYPE>:rint() [with TYPE = int
32_t]':
Streamable.hh:601: instantiated from here
Streamable.hh:143: warning: unsigned int format, uint32_t arg (arg 3)
Streamable.hh: In member function `void TivoType<TYPE>:rint() [with TYPE = cha
r]':
Streamable.cc:155: instantiated from here
Streamable.hh:143: warning: unsigned int format, uint32_t arg (arg 3)
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o BeaconManager.o -L/
usr/lib BeaconManager.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib
Mynet.cc
Mynet.cc:29: error: invalid function declaration
make: *** [Mynet.o] Error 1


ALso, i got the xml2 libraries, but I cant see the libtar library as a components for cygwin.

Jamie
07-13-2005, 10:11 PM
I currently dont have the room for linux and am trying the cygwin emulator.

Here is my error from G++ on cygwin on an XP pro machine
...

ALso, i got the xml2 libraries, but I cant see the libtar library as a components for cygwin.You can fix the invalid function declaration by including <endian.h>. That may not be portable (I don't think *BSD has it; don't know about OSX). You could include it at the point where the ERROR: is now, so it is only included if the byte order macros aren't defined elsewhere.

The signedness warnings can probably be ignored for now and cleaned up later.

libtar isn't a package included in the standard cygwin install from cygwin.com. I believe I saw it in the gnome-cygwin packages somewhere -- haven't looked that hard yet.

BAHitman
07-13-2005, 10:41 PM
I have no idea how to fix this?

I would appreciate any help that could be offered!

Thanks

Jamie
07-13-2005, 11:15 PM
If you don't already have a basic reading knowledge of C/C++, porting this to another platform will probably be over your head. Maybe you can figure out some things by reading the error messages carefully.


Mynet.cc:80: `close' undeclared (first use this function)Probably need to include <unistd.h>


TMF.hh:8:20: libtar.h: No such file or directoryThis indicates you probalby don't have libtar installed.

etc.

It compiled for me on Fedora Core 3 with a minor Makefile change.

BAHitman
07-13-2005, 11:29 PM
yeah, I looked all over google for help with some of some of those errors.

I did install the libtar, and the libxml2. I just got tired of looking around and spinning my wheels.

Where would be a good source of documentation that would tell me to include the unistd.h because of a 'close' undeclared error. some place that lists what functions are in what header file?

I'm not a programmer, but do understand some of it... normally, when I compile something I don't have these kinds of problems...

Thanks again for your help!

Jamie
07-13-2005, 11:42 PM
Where would be a good source of documentation that would tell me to include the unistd.h because of a 'close' undeclared error. some place that lists what functions are in what header file?RTFM :)

"man close" to a bash prompt on your RH system. Generally man pages for functions will list their include dependencies.

wsewell
07-14-2005, 12:02 AM
yeah, I looked all over google for help with some of some of those errors.

First place I start when something won't work too :)



Where would be a good source of documentation that would tell me to include the unistd.h because of a 'close' undeclared error. some place that lists what functions are in what header file?

The problem is (I think) that some versions of g++ include some of the standard headers by default; so the compile works for me and others. This is really my fault, 'cause I should have been including the headers.



I'm not a programmer, but do understand some of it... normally, when I compile something I don't have these kinds of problems...


This is to be expected at this point (that it does not just compile). There's no configure script and until I posted _no_one_ had compiled it but me on my two computers. Just think if I didn't have to go out of town every weekend and only tested in on one system :)

Thanks for all your help testing, and thanks to Jamie for helping me debug peoples' problems.


EDIT: Did you install the libtar development libraries too? If so, see if you can find libtar.h (you can try "locate libtar.h"). It may be in some non-standard place.

wsewell
07-14-2005, 12:17 AM
Oh, and I didn't mention. You need unistd.h for sleep(), so put the #include <unistd> at the top of BeaconManager.cc too :)

TivoWare
07-14-2005, 02:17 AM
I'm trying to get it to compile under FreeBSD and I got past the sleep, close, and printf. Now it does not understand "struct timeval tv"
# gmake
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib Mynet.cc
Mynet.cc: In method `int Mynet::SetSocketTimeout(int)':
Mynet.cc:88: aggregate `struct timeval tv' has incomplete type and cannot be initialized
Mynet.cc:93: confused by earlier errors, bailing out
gmake: *** [Mynet.o] Error 1
Time for bed, will play with it tommorow.

Jamie
07-14-2005, 02:29 AM
Now it does not understand "struct timeval tv"
This is from the linux 'select' man page. Select uses the timeval structure:

/* According to POSIX 1003.1-2001 */
#include <sys/select.h>

/* According to earlier standards */
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

TivoWare
07-14-2005, 11:11 AM
More FreeBSD errors:

gmake
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib Mynet.cc
Mynet.cc: In method `class string Mynet::Read(unsigned int)':
Mynet.cc:239: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
Mynet.cc: In method `int Mynet::ReadString(string &, unsigned int)':
Mynet.cc:345: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
Mynet.cc: In method `int Mynet::ReadString(string &)':
Mynet.cc:381: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
gmake: *** [Mynet.o] Error 1I fixed this by changing s.clear to s = ""

Now I have much bigger problems :(
gmake
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector.o -L/usr/lib NowShowingCollector.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o TivoManager.o -L/usr/lib TivoManager.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o TMF.o -L/usr/lib TMF.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o ShowData.o -L/usr/lib ShowData.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o ShowDatabase.o -L/usr/lib ShowDatabase.cc
ShowDatabase.cc: In method `size_t ShowDatabase::findShowByRecordingId(long long int)':
ShowDatabase.cc:36: warning: comparison between signed and unsigned
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o Server.o -L/usr/lib Server.cc
/usr/include/g++/stl_relops.h: In function `bool operator !=<LONG>(const LONG &, const LONG &)':
Server.cc:111: instantiated from here
/usr/include/g++/stl_relops.h:38: no match for `const TivoType<int> & == const TivoType<int> &'
/usr/include/g++/stl_relops.h:38: candidates are: operator ==(int32_t, int32_t) <builtin>
/usr/include/g++/stl_tree.h:188: bool operator ==(const _Rb_tree_base_iterator &, const _Rb_tree_base_iterator &)
/usr/include/g++/stl_relops.h:38: warning: control reaches end of non-void function `operator !=<LONG>(const LONG &, const LONG &)'
gmake: *** [Server.o] Error 1

wsewell
07-14-2005, 03:22 PM
More FreeBSD errors:

gmake
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib Mynet.cc
Mynet.cc: In method `class string Mynet::Read(unsigned int)':
Mynet.cc:239: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
Mynet.cc: In method `int Mynet::ReadString(string &, unsigned int)':
Mynet.cc:345: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
Mynet.cc: In method `int Mynet::ReadString(string &)':
Mynet.cc:381: no matching function for call to `basic_string<char,string_char_traits<char>,__default_alloc_template<false,0> >::clear ()'
gmake: *** [Mynet.o] Error 1I fixed this by changing s.clear to s = ""


That's interesting. Guess the STL string on freeBSD doesn't have clear()?



Now I have much bigger problems :(
gmake
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector.o -L/usr/lib NowShowingCollector.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o TivoManager.o -L/usr/lib TivoManager.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o TMF.o -L/usr/lib TMF.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o ShowData.o -L/usr/lib ShowData.cc
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o ShowDatabase.o -L/usr/lib ShowDatabase.cc
ShowDatabase.cc: In method `size_t ShowDatabase::findShowByRecordingId(long long int)':
ShowDatabase.cc:36: warning: comparison between signed and unsigned
g++ -Wall -I/usr/local/include/libxml2 `xml2-config --cflags` -c -o Server.o -L/usr/lib Server.cc
/usr/include/g++/stl_relops.h: In function `bool operator !=<LONG>(const LONG &, const LONG &)':
Server.cc:111: instantiated from here
/usr/include/g++/stl_relops.h:38: no match for `const TivoType<int> & == const TivoType<int> &'
/usr/include/g++/stl_relops.h:38: candidates are: operator ==(int32_t, int32_t) <builtin>
/usr/include/g++/stl_tree.h:188: bool operator ==(const _Rb_tree_base_iterator &, const _Rb_tree_base_iterator &)
/usr/include/g++/stl_relops.h:38: warning: control reaches end of non-void function `operator !=<LONG>(const LONG &, const LONG &)'
gmake: *** [Server.o] Error 1


Yeah, I was kinda' afraid this would happen. I was originally going to write all the operator overloads for these types (the ones in Streamable.hh) but I decided it was easier to do the thing you are not supposed to do and have a casting operator, that allows the type (like TivoType<int>) to be cast directly to it's single subordinate (ie int for TivoType<int>). This can lead to all sorts of ambiguities, but it was simpler than writing all those operators for every type. Looks like for your case we'll have to though. If you look in Streamable.hh (line 135) there's a commented out == operator overload. Just uncomment that line and you should be able to get past at least this error. Sorry 'bout that.

On a related note, I think I'll make a sourceforge project for tivoserver at this point. First of all it would be nice to have CVS access so I can move between my two machines more easily. But mostly I'd like a more formal place to log bugs. Being able to fix minor bugs and release by CVS will be helpful too.

wsewell
07-14-2005, 03:23 PM
RavenSTL: Did you get a chance to try Jamie's fix for your problem? I didn't want you to think I'd forgot about you since asked you to post your question in the forum.

TivoWare
07-14-2005, 04:40 PM
Yeah, I was kinda' afraid this would happen. I was originally going to write all the operator overloads for these types (the ones in Streamable.hh) but I decided it was easier to do the thing you are not supposed to do and have a casting operator, that allows the type (like TivoType<int>) to be cast directly to it's single subordinate (ie int for TivoType<int>). This can lead to all sorts of ambiguities, but it was simpler than writing all those operators for every type. Looks like for your case we'll have to though. If you look in Streamable.hh (line 135) there's a commented out == operator overload. Just uncomment that line and you should be able to get past at least this error. Sorry 'bout that.

On a related note, I think I'll make a sourceforge project for tivoserver at this point. First of all it would be nice to have CVS access so I can move between my two machines more easily. But mostly I'd like a more formal place to log bugs. Being able to fix minor bugs and release by CVS will be helpful too.Well.. that got me something different :P


gmake
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector.o -L/usr/lib NowShowingCollector.cc
Streamable.hh: In method `bool TivoType<int>::operator ==(const int32_t &)':
NowShowingCollector.cc:184: instantiated from here
Streamable.hh:135: request for member `value' in `rhs', which is of non-aggregate type `int'
Streamable.hh:135: warning: control reaches end of non-void function `TivoType<int>::operator ==(const int32_t &)'
gmake: *** [NowShowingCollector.o] Error 1

What I have changed for FreeBSD 4.8-RELEASE so far:
(I guess I need to upgrade to 5.x at some point)

BeaconManager.hh
Added: #include <unistd.h>

Mynet.hh
Added: #include <unistd.h>
Added: #include <stdio.h>
Added: #include <sys/time.h>

Mynet.cc
Changed in three places: s.clear() to s = ""

Streamable.hh
Uncommented: bool operator==( const TYPE &rhs ) { return rhs.value == value; }

wsewell
07-14-2005, 06:54 PM
Well.. that got me something different :P

That's what I was afraid of :( Well, two things actually. The line should read:


bool operator==( const TivoType<TYPE> &rhs ) { return rhs.value == value; }

But this will probably cause some ambiguity problems so you'll have to comment out line 137 (the TYPE() operator). Then you'll run into the problem of some missing operators (and possibly some other problems). I'll look at fixing this the 'right' way this evening. Thanks for your help so far.

BAHitman
07-14-2005, 11:02 PM
First place I start when something won't work too :)

I know what u mean...




Thanks for all your help testing, and thanks to Jamie for helping me debug peoples' problems.

Absolutely!


EDIT: Did you install the libtar development libraries too? If so, see if you can find libtar.h (you can try "locate libtar.h"). It may be in some non-standard place.

Yeah, but the .h files weren't in the /usr/include directory... they're there now, and I'm down to this:




$ make -k
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Streamable.o -L/usr/lib Streamable.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o BeaconManager.o -L/usr/lib BeaconManager.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib Mynet.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector.o -L/usr/lib NowShowingCollector.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o TivoManager.o -L/usr/lib TivoManager.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o TMF.o -L/usr/lib TMF.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o ShowData.o -L/usr/lib ShowData.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o ShowDatabase.o -L/usr/lib ShowDatabase.cc
ShowDatabase.cc: In member function `size_t
ShowDatabase::findShowByRecordingId(long long int)':
ShowDatabase.cc:36: warning: comparison between signed and unsigned integer
expressions
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Server.o -L/usr/lib Server.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Utils.o -L/usr/lib Utils.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -o tivoserver -L/usr/lib -lpthread `xml2-config --libs` -ltar tivoserver.cc Streamable.o BeaconManager.o Mynet.o NowShowingCollector.o TivoManager.o TMF.o ShowData.o ShowDatabase.o Server.o Utils.o
/usr/bin/ld: cannot find -ltar
collect2: ld returned 1 exit status
make: *** [tivoserver] Error 1
make: Target `all' not remade because of errors.


The signed/unsigned integer problem is just a silly warning (although that could be a big problem), and it doesn't stop the compiler, but the can't find -ltar.
all I found with google is a site called http://www.cantfindongoogle.com --no help there :)

Thanks!

wsewell
07-15-2005, 04:24 AM
The signed/unsigned integer problem is just a silly warning (although that could be a big problem), and it doesn't stop the compiler

Yeah, I need to fix that. It should only be a problem if there are a _lot_ of shows (ie more than the largest positive signed int32) 'cause it's on a comparison of the showID and I ID the shows with an incrementing counter.


but the can't find -ltar.
all I found with google is a site called http://www.cantfindongoogle.com --no help there :)

Thanks!

Interesting site :) Sound like (from your previous header file problem) that libtar is getting installed into a weird place. See if you can find libtar.so (or even libtar.a). It should be in something like /usr/lib. Once you've found it, try adding


-L/directory/where/libtar/is

to the LIBRARIES line in the Makefile. Hope this works for you.

BAHitman
07-15-2005, 07:19 AM
I fould libtar.so. it's in /usr/lib/gnome-vfs-2.0/

I added the -L/usr/lib/gnome-vfs-2.0/ to the LIBRARIES Line.

a new problem arises. I think it may be a problem with the libtar.so file... but I could be wrong...




g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -o tivoserver -L/usr/lib -lpthread `xml2-config --libs` -ltar -L/usr/lib/gnome-vfs-2.0/modules/ tivoserver.cc Streamable.o BeaconManager.o Mynet.o NowShowingCollector.o TivoManager.o TMF.o ShowData.o ShowDatabase.o Server.o Utils.o
TMF.o(.text+0x169): In function `TMF::loadShowingData()':
: undefined reference to `tar_open'
TMF.o(.text+0x1d1): In function `TMF::loadShowingData()':
: undefined reference to `th_read'
TMF.o(.text+0x1e5): In function `TMF::loadShowingData()':
: undefined reference to `oct_to_int'
TMF.o(.text+0x256): In function `TMF::loadShowingData()':
: undefined reference to `tar_fd'
TMF.o(.text+0x510): In function `TMF::loadShowingData()':
: undefined reference to `tar_close'
TMF.o(.text+0x1e17): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_open'
TMF.o(.text+0x1ea8): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `th_read'
TMF.o(.text+0x1ed4): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `th_get_pathname'
TMF.o(.text+0x1f99): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_skip_regfile'
TMF.o(.text+0x2191): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_fd'
TMF.o(.text+0x262e): In function `TMF::closePartFile(PartFile&)':
: undefined reference to `tar_close'
collect2: ld returned 1 exit status
make: *** [tivoserver] Error 1

wsewell
07-15-2005, 07:51 AM
I fould libtar.so. it's in /usr/lib/gnome-vfs-2.0/

I added the -L/usr/lib/gnome-vfs-2.0/ to the LIBRARIES Line.

a new problem arises. I think it may be a problem with the libtar.so file... but I could be wrong..

You're right, that's the wrong libtar.so. All the libraries in that directory have something to do with gnome/nautilus handling file formats. See if you can find libtar.a. BTW - what method did you use to install libtar (RPM, DEB, source tarball, etc.) and how did you find that one in gnome-vfs-2.0?

nams
07-15-2005, 10:27 AM
Yeah, can't wait to get some more file types working. That's interesting that your experiment didn't work. Did the show not show up in the Now Showing list or was it a problem with the actual video transfer? If it was just a Now Showing list problem, the it's probably a simpl(er) fix. You could help me out by "tar -xvf blah.tmf" the tmf file and sending me the showing.xml file. Be sure to delete the .part files afterwards.

I was able to get this working. After I transfer the TY file(s) I needed to stop mfs_ftp clear the cache then restart mfs_ftp. This causes the xml file to be regenerated with enough info for tivoserver to use. The only error I get with these files now is


WARNING: Could not get Series Information

But this is just a warning and the files are added to the now showing list anyway.

If only I can convert the files directly to TMF format from mpg with the necessary xml data I would be all set.

wsewell
07-15-2005, 04:55 PM
If only I can convert the files directly to TMF format from mpg with the necessary xml data I would be all set.

Glad you got it working :) There is no huge difference between TY files and TMF files (someone please correct me if I'm wrong here). TY files are just the .part files of the video concatenated together. TMF files are actually tarballs of the .part files plus a showing.xml file which contains all the interesting info about a show (title, episode, etc.). So you can probably pretty easily write a converter for TY to TMF that would prompt you for the show information. In fact someone on the forum may have already written such a device. That being said it won't be too hard for me to just add this functionality to tivoserver. It's just not a high priority for me right now 'cause I'm trying to get everyone to the point where they can run (and test) it. You may wanna check out this thread where I found a short quote from AlphaWolf talking about conversion:

http://64.21.8.90/forum/showthread.php?t=21915

I think the thread is dead, but maybe you'll have better luck finding someone who was written a tool to do this. Thanks a lot for testing it out, glad it's at least doing something for you :)

wsewell
07-17-2005, 07:03 AM
I hacked up a few fixes for the problems people have been having compiling. Added some headers includes. Fixed the implicit type cast / missing operator overaloads problem. Fixed up the byteorder (endian) code. Actually borrowed the endian code from the SDL source, so it should work a lot better than mine did. Tested on Debian unstable and OSX 10.3.9.

New Bug: Showing info (ie title, episode title, etc.) for transferred show is incorrect (on client Tivo). Tivoserver is sending the wrong info just before the transfer.

As usual, this is still experimental, so please be careful and don't use it if you are worried about your Tivo crashing.

TivoWare
07-17-2005, 04:28 PM
Great news! I got it compile under FreeBSD 4.x with no changes. (After I updated my libtar port).

Bad news:

I forgot to copy TvStructs-6.2.hh to TvStructs.hh, after I did this I found that you didn't apply your changes to this file. I'm let you know what I find out.

(Edited after I reread the README and saw I didn't use the TvStructs-6.2.hh)

wsewell
07-17-2005, 05:25 PM
Great news! I got it compile under FreeBSD 4.x with no changes. (After I updated my libtar port).

Awesome.



Bad news:

I forgot to copy TvStructs-6.2.hh to TvStructs.hh, after I did this I found that you didn't apply your changes to this file. I'm let you know what I find out.

(Edited after I reread the README and saw I didn't use the TvStructs-6.2.hh)

Righto. Keeping up with 2 files is stupid, so I merged them (and attached it to this post along with a new Enums.hh). You'll just need to edit TvStructs.hh and uncomment the line:

//#define TIVO_VERSION_62

BAHitman
07-17-2005, 09:19 PM
The libtar version I'm using is 1.2.11. I got the sources, did a ./configure, then make, then make install. i then did an updatedb (so I could find stuff...) I have only the 1 libtar.so file for gnome (I don't even use, nor did I install gnome...) I found 2 libtar.a files. one in /usr/local/lib the other in /root/libtar-1.2.11/lib/libtar.a. I tried using the -L/root/libtar-1.2.11/lib and -L/usr/local/lib in the makefile, but neither work...

Do I need a different version of the libtar library?

also, I did upgrade to the latest version, and I only get the errors relating to libtar...


g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Streamable.o -L/usr/lib Streamable.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o BeaconManager.o -L/usr/lib BeaconManager.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Mynet.o -L/usr/lib Mynet.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector.o -L/usr/lib NowShowingCollector.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o TivoManager.o -L/usr/lib TivoManager.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o TMF.o -L/usr/lib TMF.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o ShowData.o -L/usr/lib ShowData.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o ShowDatabase.o -L/usr/lib ShowDatabase.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Server.o -L/usr/lib Server.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o Utils.o -L/usr/lib Utils.cc
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -o tivoserver -L/usr/lib -lpthread `xml2-config --libs` -ltar -L/root/libtar-1.2.11/lib/libtar.a tivoserver.cc Streamable.o BeaconManager.o Mynet.o NowShowingCollector.o TivoManager.o TMF.o ShowData.o ShowDatabase.o Server.o Utils.o
TMF.o(.text+0x169): In function `TMF::loadShowingData()':
: undefined reference to `tar_open'
TMF.o(.text+0x1d1): In function `TMF::loadShowingData()':
: undefined reference to `th_read'
TMF.o(.text+0x1e5): In function `TMF::loadShowingData()':
: undefined reference to `oct_to_int'
TMF.o(.text+0x256): In function `TMF::loadShowingData()':
: undefined reference to `tar_fd'
TMF.o(.text+0x510): In function `TMF::loadShowingData()':
: undefined reference to `tar_close'
TMF.o(.text+0x1e17): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_open'
TMF.o(.text+0x1ea8): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `th_read'
TMF.o(.text+0x1ed4): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `th_get_pathname'
TMF.o(.text+0x1f99): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_skip_regfile'
TMF.o(.text+0x2191): In function `TMF::loadPartHeader(unsigned, PartFile*)':
: undefined reference to `tar_fd'
TMF.o(.text+0x262e): In function `TMF::closePartFile(PartFile&)':
: undefined reference to `tar_close'
collect2: ld returned 1 exit status
make: *** [tivoserver] Error 1


Thanks again, and I'm anxious to get this working

wsewell
07-18-2005, 04:22 AM
The libtar version I'm using is 1.2.11. I got the sources, did a ./configure, then make, then make install. i then did an updatedb (so I could find stuff...) I have only the 1 libtar.so file for gnome (I don't even use, nor did I install gnome...) I found 2 libtar.a files. one in /usr/local/lib the other in /root/libtar-1.2.11/lib/libtar.a. I tried using the -L/root/libtar-1.2.11/lib and -L/usr/local/lib in the makefile, but neither work...

Do I need a different version of the libtar library?

Nope, that's the version I have on my Mac and it works fine there.


also, I did upgrade to the latest version, and I only get the errors relating to libtar...

Cool... well except for the libtar stuff

Not sure what's wrong here, if anyone has any advice it would be helpful. I do have a couple things you can try. First run nm on libtar.a and dump it to a file (ie "nm /usr/local/lib/libtar.a > somefile.txt"). Then look through the file to make sure that it lists tar_open and the other undefined references.

If it checks out. Then try an alternate compile method. First do a make in the tivoserver directory to make sure all the object files are built. Then do:



g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o tivoserver.o -L/usr/lib tivoserver.cc
g++ -o tivoserver *.o -lxml2 -lpthread /usr/local/lib/libtar.a


The first command should return no errors. The second error may or may not give you the same errors as before. Let me know what happens and maybe we can make some progress on what's wrong. Also, it might be helpful to know what OS/distribution (RH9 I think I recall) and also what version of g++ (just do "g++ -v").

darrin75
07-18-2005, 04:46 AM
Owner@Bobby ~
$ cd /usr/src/tivoserver

Owner@Bobby /usr/src/tivoserver
$ make
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector
.o -L/usr/lib NowShowingCollector.cc
In file included from ShowDatabase.hh:6,
from NowShowingCollector.hh:3,
from NowShowingCollector.cc:3:
TMF.hh:8:20: libtar.h: No such file or directory
In file included from ShowDatabase.hh:6,
from NowShowingCollector.hh:3,
from NowShowingCollector.cc:3:
TMF.hh:33: error: ISO C++ forbids declaration of `TAR' with no type
TMF.hh:33: error: expected `;' before '*' token
NowShowingCollector.cc: In static member function `static bool NowShowingCollect
or::ReadHeader(Mynet&, uint32_t, uint32_t)':
NowShowingCollector.cc:259: warning: unsigned int format, uint32_t arg (arg 2)
NowShowingCollector.cc:265: warning: unsigned int format, uint32_t arg (arg 2)
Streamable.hh: In member function `void TivoType<TYPE>:: print() [with TYPE = int
32_t]':
Streamable.hh:610: instantiated from here
Streamable.hh:152: warning: unsigned int format, uint32_t arg (arg 3)
make: *** [NowShowingCollector.o] Error 1

wsewell
07-18-2005, 05:00 AM
Looks like you need to install libtar. If it's not available from cygwin, then you'll have to download it and build it first:

http://www.feep.net/libtar/

If you already did this, then see if you can find libtar.h and libtar.a and report the locations of them to the thread.

Don't forget to install libxml2 too (if you haven't already).



Owner@Bobby ~
$ cd /usr/src/tivoserver

Owner@Bobby /usr/src/tivoserver
$ make
g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o NowShowingCollector
.o -L/usr/lib NowShowingCollector.cc
In file included from ShowDatabase.hh:6,
from NowShowingCollector.hh:3,
from NowShowingCollector.cc:3:
TMF.hh:8:20: libtar.h: No such file or directory
In file included from ShowDatabase.hh:6,
from NowShowingCollector.hh:3,
from NowShowingCollector.cc:3:
TMF.hh:33: error: ISO C++ forbids declaration of `TAR' with no type
TMF.hh:33: error: expected `;' before '*' token
NowShowingCollector.cc: In static member function `static bool NowShowingCollect
or::ReadHeader(Mynet&, uint32_t, uint32_t)':
NowShowingCollector.cc:259: warning: unsigned int format, uint32_t arg (arg 2)
NowShowingCollector.cc:265: warning: unsigned int format, uint32_t arg (arg 2)
Streamable.hh: In member function `void TivoType<TYPE>:: print() [with TYPE = int
32_t]':
Streamable.hh:610: instantiated from here
Streamable.hh:152: warning: unsigned int format, uint32_t arg (arg 3)
make: *** [NowShowingCollector.o] Error 1

darrin75
07-18-2005, 05:22 AM
Owner@Bobby /usr/src/libtar-1.2.11
$ make all
for n in lib libtar doc; do \
( cd $n && make all ) || exit 1; \
done
make[1]: Entering directory `/usr/src/libtar-1.2.11/lib'
gcc -g -O2 -I. -I.. -I. -I../compat -I../listhash -c -o append.o append.c
In file included from ./internal.h:16,
from append.c:13:
./libtar.h:18:17: tar.h: No such file or directory
append.c: In function `tar_append_file':
append.c:124: error: `LNKTYPE' undeclared (first use in this function)
append.c:124: error: (Each undeclared identifier is reported only once
append.c:124: error: for each function it appears in.)
append.c:144: error: `SYMTYPE' undeclared (first use in this function)
append.c:179: error: `REGTYPE' undeclared (first use in this function)
append.c:179: error: `AREGTYPE' undeclared (first use in this function)
append.c:179: error: `CONTTYPE' undeclared (first use in this function)
make[1]: *** [append.o] Error 1
make[1]: Leaving directory `/usr/src/libtar-1.2.11/lib'
make: *** [all] Error 1

Owner@Bobby /usr/src/libtar-1.2.11
$




I get this to when i try to build the libtar???

wsewell
07-18-2005, 05:42 AM
In file included from ./internal.h:16,
from append.c:13:
./libtar.h:18:17: tar.h: No such file or directory


I get this to when i try to build the libtar???

Looks like a problem with tar.h missing in cygwin:

https://lists.feep.net:8080/pipermail/libtar/2002-September/000092.html

Post is kinda old though. Might see if you can upgrade cygwin, don't know if that will help or not. I'll try to dig some more info up.

weston

BAHitman
07-18-2005, 10:03 PM
Nope, that's the version I have on my Mac and it works fine there.



Cool... well except for the libtar stuff

Not sure what's wrong here, if anyone has any advice it would be helpful. I do have a couple things you can try. First run nm on libtar.a and dump it to a file (ie "nm /usr/local/lib/libtar.a > somefile.txt"). Then look through the file to make sure that it lists tar_open and the other undefined references.

If it checks out. Then try an alternate compile method. First do a make in the tivoserver directory to make sure all the object files are built. Then do:



g++ -Wall -I/sw/include/libxml2 `xml2-config --cflags` -c -o tivoserver.o -L/usr/lib tivoserver.cc
g++ -o tivoserver *.o -lxml2 -lpthread /usr/local/lib/libtar.a


The first command should return no errors. The second error may or may not give you the same errors as before. Let me know what happens and maybe we can make some progress on what's wrong. Also, it might be helpful to know what OS/distribution (RH9 I think I recall) and also what version of g++ (just do "g++ -v").

well... I have good news... it compiled... I now have an executable!!!

Thanks!

of course, I deleted all of my TMF files last week... ooh well, time to break out mfs_ftp and wget :)

I'll let you know how well it works!

wsewell
07-19-2005, 02:17 AM
Sweet! Could you relay what you did so others can get it working on RH9 (or I can make changes/fixes as necessary). Glad you got it working.

weston


well... I have good news... it compiled... I now have an executable!!!

Thanks!

of course, I deleted all of my TMF files last week... ooh well, time to break out mfs_ftp and wget :)

I'll let you know how well it works!

TivoWare
07-19-2005, 09:38 AM
I'm running 6.2 on my Tivos and compiled under FreeBSD with no problems. I am getting a core dump when it should start streaming the show.


** Loading Local Shows From '/home/samba/tmf/' **
THUMBS: 268501376
- Dilbert : The Gift
THUMBS: 268501376
- Dilbert : The Takeover
THUMBS: 268501376
- Dilbert : Elbonian Trip
THUMBS: 268501376
- Dilbert : The Little People
THUMBS: 268501376
- Dilbert : Holiday
THUMBS: 268501376
- Dilbert : The Infomercial
Adding Tivo: Main (1510000xxxxxxxx)
0x00000000
0x00014103
0x00014104
SELECTED TITLE:
-------------=> SHOWLIST <=---------------
( 0 ) Dilbert ( Elbonian Trip )
--------------------------------------------
Selected Show = 0
--------------------------------------------
************************ enumView = TOP
************************ iAbsoluteOffset = 0
0x00014103
0x0001100a
<-------------- 0: Dilbert
0x00000000
0x00014103
0x00014104
SELECTED TITLE: Dilbert
-------------=> SHOWLIST <=---------------
( 0 ) Dilbert ( Elbonian Trip )
( 1 ) Dilbert ( The Takeover )
( 2 ) Dilbert ( Holiday )
( 3 ) Dilbert ( The Infomercial )
( 4 ) Dilbert ( The Gift )
( 5 ) Dilbert ( The Little People )
--------------------------------------------
Selected Show = 0
--------------------------------------------
************************ enumView = BOTTOM
************************ iAbsoluteOffset = 0
0x00014103
0x0001100a
<-------------- 0: Dilbert
<-------------- 1: Dilbert
<-------------- 2: Dilbert
<-------------- 3: Dilbert
<-------------- 4: Dilbert
<-------------- 5: Dilbert
0x00000000
0x00014103
0x00014104
SELECTED TITLE: Dilbert
-------------=> SHOWLIST <=---------------
( 0 ) Dilbert ( Elbonian Trip )
( 1 ) Dilbert ( The Takeover )
( 2 ) Dilbert ( Holiday )
( 3 ) Dilbert ( The Infomercial )
( 4 ) Dilbert ( The Gift )
( 5 ) Dilbert ( The Little People )
--------------------------------------------
Selected Show = 1
--------------------------------------------
************************ enumView = BOTTOM
************************ iAbsoluteOffset = 0
0x00014103
0x0001100a
<-------------- 0: Dilbert
0x00000000
0x00010661
0x0001065e
Found Show By ID: 1
<<<<<<<<<<<< Dilbert
0x00000000
0x00000000
0x00016bb4
0x00016bad
--------> Random Magic: '1804289383'
Segmentation fault (core dumped)

AlphaWolf
07-19-2005, 11:29 AM
I think its well time that this thread see a split into the support forums.

wsewell
07-20-2005, 05:05 AM
I think its well time that this thread see a split into the support forums.

Done:
http://www.dealdatabase.com/forum/showthread.php?t=44485



TivoWare: I'll get to your problem as soon as I can. When you get a chance, turn on debugging in the Mynet.hh file, recompile, dump the output to a log file, and post the log in the new thread. If you need help with uncommenting or dumping to a file let me know :)

wsewell
08-31-2005, 06:43 AM
Ok boys and girls. Here's a not-so-short list of features that we'd like to get implemented for tivoserver. I'm posting it here to ask for volunteers. If you'd like to help out, feel free to speak up.

1) streaming AVI support - this is our next-biggest priority, and is being actively worked on, but if you have some experience here le'me know.

2) client-side, ability to pull shows - Part of this (the menuing) is written, but needs to be rewritten a bit. The remainder (the good stuff... the video transfer) is not implemented at all yet.

3) GUI for client side - speaks for itself. Definately something cross platform like wxWindows.

4) Filling in more info for TYs, such as duration and start/stop times
5) improving error handling (including not aborting when an xfer is stopped)
6) Fixing network code to allow for a more graceful shutdown (SIGINT handler is implemented, but doesn't do much).
7) putting in a logging mechanism
8) improve support for 4.x/6.2 (doesn't work for both a 4.x and 6.2 at the same time)
9) native xml support (ie no more libxml)
10) considering a better ("data-driven") model for mapping the XML info to the Tivo client.
11) unique FSIDs - we were thinking about using the timestamp in some way. Thus implicitly changing ID if the file changes for some reason.
12) caching show info across runs of tivoserver
13) do something useful about interrupted transfers (can just "kill" them for now, but would be better to resume them. the latter depends on unique FSIDs).
14) directory watching - watch for new/removed shows in the active video directories.

As I said, AVI support is being worked on. And I'll probably start working on client support since I'm pretty familiar with the code and the MRV interface. But please feel free to volunteer for anything, there's a lot of features we want to get into this thing.

PS - sorry for the cross post.

britrock
02-24-2006, 07:06 PM
I've been reading as much as I can about tivoserver, including this thread, and the one thing that confuses me is why there is no mention of the 'GoBack' feature.

Wouldn't it be a lot easier to use that than to require a hacked tivo? I know this has been done by a few other projects (like galleon), but to be honest this project sounds a lot more like what I'm looking for.

I'm a pretty good coder, and would be glad to help with this if some one can point me in the right direction.

Jamie
02-24-2006, 07:25 PM
I've been reading as much as I can about tivoserver, including this thread, and the one thing that confuses me is why there is no mention of the 'GoBack' feature.

Wouldn't it be a lot easier to use that than to require a hacked tivo? I know this has been done by a few other projects (like galleon), but to be honest this project sounds a lot more like what I'm looking for.

I'm a pretty good coder, and would be glad to help with this if some one can point me in the right direction.The main audience for tivoserver is the dtivo users. dtivo users must hack anyway to enable MRV at all.

That said, the Galleon GoBack feature, according to the author (http://galleon.tv/component/option,com_jd-wp/Itemid,62/p,13/), uses the HMO, not HME protocol. An interesting question is whether that protocol can be used to serve up shows to a DTivo with 6.2 and the superpatch. I'm guessing the answer is no, but I don't know for sure.

britrock
02-24-2006, 07:36 PM
The main audience for tivoserver is the dtivo users. dtivo users must hack anyway to enable MRV at all.

That said, the Galleon GoBack feature, according to the author (http://galleon.tv/component/option,com_jd-wp/Itemid,62/p,13/), uses the HMO, not HME protocol. An interesting question is whether that protocol can be used to serve up shows to a DTivo with 6.2 and the superpatch. I'm guessing the answer is no, but I don't know for sure.

Interesting. Thats too bad. Tivoserver was sounding really good, but I really don't want to hack my tivo. Thanks.

Jamie
02-24-2006, 08:10 PM
Interesting. Thats too bad. Tivoserver was sounding really good, but I really don't want to hack my tivo. Thanks.If you can code, there's nothing stopping you from looking into using the HMO protocol for serving shows with tivoserver. The source to both Galleon and tivoserver is out there. If the protocol works for 6.2 dtivos w superpatch as well as SA tivos, it would have some advantages: Works on unhacked SA tivos. Doesn't require remuxing to ty. It's a documented and supported protocol.If it only works on SA tivos running software 7.x and above, it's not going to be useful to the largest population of tivoserver users (dtivo owners). Perhaps it would be better to enhance Galleon to do what you want in that case.

dishman
02-25-2006, 09:21 AM
I've been reading as much as I can about tivoserver, including this thread, and the one thing that confuses me is why there is no mention of the 'GoBack' feature.

Wouldn't it be a lot easier to use that than to require a hacked tivo? I know this has been done by a few other projects (like galleon), but to be honest this project sounds a lot more like what I'm looking for.

I'm a pretty good coder, and would be glad to help with this if some one can point me in the right direction.

What are you trying to do? If you want to push videos from your computer to your unhacked SA TiVo you can do that with the TiVo Desktop.

wsewell
03-25-2006, 01:16 AM
I've been reading as much as I can about tivoserver, including this thread, and the one thing that confuses me is why there is no mention of the 'GoBack' feature.

It doesn't use GoBack. When I started TivoServer GoBack didn't exist yet, and to be honest I didn't know about it until recently (mostly 'cause I'm always in Tivoserver land). And as Jamie said, most of the ppl on the forum are DTivo users (myself included for the time being). I'm working on an HME server piece but haven't gotten as far as I would have liked yet (only shows a show listing right now) because I'm squeezed for time recently. If you'd like to help out on that effort specifically, I'll probably be making a (CVS) branch pretty soon so I have somewhere to check in to and let others check out and muck w/ the code w/out without breaking the mainline.

woracan
09-02-2009, 11:31 AM
edit2: you should check out http://alt.org/forum/index.php?t=msg&th=118&start=0&rid=135

The link is no longer good. Does anyone have another location for the writeup on idl files and the idl parser program?

tivo4mevo
09-04-2009, 04:39 PM
I believe that the parsing of IDL files was obsoleted by tivo building a parser directly into tivosh (the tvidl command appeared with software version 6.3x/7.x).

Here's a recent post discussing the use of tvidl to parse some idl stored in MFS: link

(http://www.dealdatabase.com/forum/showthread.php?t=60508)Taking a random idl: /tvlib/idl/0x29a63.tvbin One can use tivosh to parse it as follows


tvidl serialtoxml \x12\x34\x00\x03\x00\x01\x00\x09\x00\xC9\xC5\x5D\x00\x00\x00\x92\x00\x02\x9A\x63\x00\x00\x00\x20\x54 \x76\x44\x62\x52\x65\x6D\x6F\x74\x65\x54\x6F\x4C\x6F\x63\x61\x6C\x52\x65\x63\x6F\x72\x64\x69\x6E\x67 \x51\x75\x65\x72\x79\x00\x00\x64\xF6\x45\x00\x00\x00\x03\x00\x00\x00\x22\x00\x00\x00\x06\x53\x74\x61 \x72\x74\x00\x00\x00\x00\x01\x00\x00\x00\x10\x00\x00\x00\x02\x00\x02\x9A\x61\x00\x00\x00\x01\x00\x00 \x00\x00\x00\x00\x00\x27\x00\x00\x00\x0B\x50\x72\x6F\x63\x65\x73\x73\x69\x6E\x67\x00\x00\x00\x00\x01 \x00\x00\x00\x10\x00\x00\x00\x01\x00\x02\x9A\x62\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x0D\x00 \x00\x00\x05\x44\x6F\x6E\x65\x00\x00\x00\x00\x00

<TvBusEnvelope xs:type="TvBusProtocol"><id>170595</id><name>TvDbRemoteToLocalRecordingQuery</name><checksum>6616645</checksum><vState><element><name>Start</name><vTransition><element><direction value="2">RECEIVE</direction><eventId>170593</eventId><gotoState>1</gotoState><bits value="0"/></element></vTransition></element><element><name>Processing</name><vTransition><element><direction value="1">SEND</direction><eventId>170594</eventId><gotoState>2</gotoState><bits value="0"/></element></vTransition></element><element><name>Done</name><vTransition/></element></vState></TvBusEnvelope>

The routerplus kernel module replaces the stock tvbus router, and allows one to sniff tvbus/idl messages, and can format the output as tcl so that they can be easily fed to tivosh for parsing (in the manner described above).