Page 2 of 11 FirstFirst 1234 ... LastLast
Results 16 to 30 of 151

Thread: ExtractStream and PlayStream failing on every sector in a DTivo

  1. #16
    Join Date
    Nov 2001
    Posts
    17
    There is hardware support in the IDE controller (built into the Mediaswitch ASIC) for on-the-fly encryption. It is present in all machines with MediaSwitch version 2 or later.

    Older standalone models apparently have MediaSwitch version1 (I don't know, I don't have one) which also apparently did not support encryption. I have been told that the new AT&T standalone has a later revision of the Mediaswitch and hence is presumably also encrypted.

    Although the encryption has been in the hardware since the DirecTivo was built, it was not enabled until Tivo release 2.5. This was done in a backwards compatible way, so that new recordings post-2.5 are encrypted, but old pre-2.5 recordings are plaintext.

    So if you're wondering why extractstream fails on 2.5 DirectTivo recordings, there is a good reason, having nothing to do with bad FSIDs (-:

  2. #17
    Join Date
    Dec 2001
    Posts
    37
    So simply turn the encryption off again. No use trying to decode the encryption on the disk, simply make it write the recordings unencrypted by reverting to previous versions of the tivo software, or simply turning it off.

    As for the file format, you're saying that the streams coming out of the mpeg decoder chip are already Tivo formated PES style streams. Good. Since I don't have a Tivo in front of me, I can't verify that sort of stuff. I knew that the broadcast format was certainly different..

  3. #18
    Join Date
    Dec 2001
    Posts
    17
    I can't believe that encryption caused the original problem reported by Eric. It just doesn't make sense given the data:

    Attempting to locate tyStream with fsid 119309...
    Sector FSID passed (1d20d) failed to match the FSID value in the sector (0).
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 91 23 1e bc .............#..
    a9 dd 19 85 00 00 00 00 00 00 00 00 00 00 00 00 ................
    This error is not due to any form of encryption. By looking at get_tyStream_sectors() in the PlayStream source code, where this error is reported, you can tell that what's expected is for the above buffer to begin with 0x0001d20d (the fsid).
    sector = *((unsigned long *)(buffer));
    if ((fsid) != (sector)) {
        fprintf(stderr,"Sector FSID passed (%lx) failed to match the FSID value in the sector (%lx).\n", fsid, sector);
    ...
    }
    The fact that the magic signature (0x91231ebc) is in the correct position tells me that this is a valid sector so the "variable sector size causes misalignment" theory is not correct.

    This is a valid sector, just not the right one. Personally, I think the "missing fs mounts" or at least the "FSID calculation is incorrect" theories are on the right track.

    -Ray

  4. #19
    Join Date
    Nov 2001
    Posts
    117

    Thumbs up

    Hey PPCHacker,

    I think that your theroy about the on-the-fly encryption happening on the DTV-Tivo's make some sence. If I'm reading all the posts from this and other threads it looks as those the people having success with extractstream might be those with the stand alone units. But on the other hand I'll need to do more testing but I did reload my drive with the factory 2.0.1 image and extractstream didn't work. I'll give this one more try. I'll also verify that I'm running the factory 1.10 prom also. I'll drop a note to this thread if I come up with a working solution, this should at least help up focus on where the probem is, ie prom, 2.5 software or else where.
    Did I do that...

  5. #20
    Join Date
    Dec 2001
    Posts
    18
    ExtractStream did not work on any version of the DirecTivo. Ever. Same problems now as in previous versions. I highly doubt, again, that encryption has any part here, but rather differences in format of the stream on disk.

  6. #21
    Join Date
    Nov 2001
    Posts
    117
    Well I can difinatly vouge that extractstream 2 doesn't work with 2.0.1 on a Phillips DS6000R. Here is some detailed infor I gleaned from the system in my test. Ho hummm so id PPChack correct ? do we need to disable the encryption ? is the a promhack ? or Do we need to figure out the new way there pushing the bits arround on the DT-Tivo's ?

    bash-2.02# ./showlist.tcl
    <?xml version="1.0" ?>
    <ShowList>
    <Show>
    <Program>Lara Croft: Tomb Raider</Program>
    <Day>Wed</Day>
    <Date>1/2</Date>
    <Station>PPV</Station>
    <Episode></Episode>
    <fsid>15285/14</fsid>
    <Parts>
    <Part>15284</Part>
    <Size>60</Size>
    </Parts>
    <TotalSize>60</TotalSize>
    </Show>
    <Show>
    <Program>America's Sweethearts</Program>
    <Day>Wed</Day>
    <Date>1/2</Date>
    <Station>PPV</Station>
    <Episode></Episode>
    <fsid>13509/15</fsid>
    <Parts>
    <Part>13508</Part>
    <Size>15</Size>
    </Parts>
    <TotalSize>15</TotalSize>
    </Show>
    <Show>
    <Program>DIRECTV PPV Previews</Program>
    <Day>Wed</Day>
    <Date>1/2</Date>
    <Station>DTV</Station>
    <Episode></Episode>
    <fsid>12520/14</fsid>
    <Parts>
    <Part>12519</Part>
    <Size>62</Size>
    </Parts>
    <TotalSize>62</TotalSize>
    </Show>
    </ShowList>

    ./e2 12519
    Attempting to locate tyStream with fsid 12519...
    Sector FSID passed (30e7) failed to match the FSID value in the sector (0).
    Sector FSID passed (30e7) failed to match the FSID value in the sector (0).
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 91 23 1e bc .............#..
    a9 dd 19 85 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    bash-2.02# ./e2 13508
    Attempting to locate tyStream with fsid 13508...
    Sector FSID passed (34c4) failed to match the FSID value in the sector (0).
    Sector FSID passed (34c4) failed to match the FSID value in the sector (0).
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 91 23 1e bc .............#..
    a9 dd 19 85 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

    bash-2.02# ./e2 15284
    Attempting to locate tyStream with fsid 15284...
    Sector FSID passed (3bb4) failed to match the FSID value in the sector (0).
    Sector FSID passed (3bb4) failed to match the FSID value in the sector (0).
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 91 23 1e bc .............#..
    a9 dd 19 85 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

    uname -a
    Linux (none) 2.1.24-TiVo.1 #6 Sat Sep 2 15:01:15 PDT 2000 ppc unknown
    bash-2.02# grep -i monitor /var/log/messages
    Jan 3 08:40:22 (none) Stats: ^MTiVoProm Monitor version 1.100
    Did I do that...

  7. #22
    Join Date
    Dec 2001
    Posts
    17

    fsid calculation problem

    Good news. I spent a few hours digging into the output from sector_scan (ExtractStream -d) and a spreadsheet prog and found that with a small change to the formula used to calculate fsids, I can correctly calculate every tyStream sector. I don't have a cross compile setup yet so I can't actually test this tonight, but if someone else wants to give a shot, please do so...

    Here's the original fsid calculation from ExtractStream:
    Code:
    nsector = (((loff_t)(((fsid * 0x20db2) & 0x3ffff) + 0x462)) << 9);
    for the following fsids, we get these sector values:
    Code:
    ebc2	26643456
    0b0f	49258496
    e9ab	72912896
    ea1f	12705792
    e0bf	133718016
    e0c5	10270720
    ea3f	70148096
    619f	99606528
    24df	99934208
    now if we change the formula to:
    Code:
    nsector = (((loff_t)(((fsid * 0x20db2) & 0xfffff) + 0x462)) << 9);
    changing the mask from 0x3ffff to 0xfffff we get the following sector values:
    Code:
    ebc2	26643456
    0b0f	183476224
    e9ab	207130624
    ea1f	146923520
    e0bf	133718016
    e0c5	10270720
    ea3f	204365824
    619f	233824256
    24df	234151936
    And most importantly the new sector values match what the sector_scan output dumps exactly!
    Code:
    26643456:0x0000ebc2	yes	  yes
    183476224:0x00000b0f	yes	  yes
    207130624:0x0000e9ab	yes	  yes
    146923520:0x0000ea1f	yes	  yes
    133718016:0x0000e0bf	yes	  yes
    10270720:0x0000e0c5	yes	  yes
    204365824:0x0000ea3f	yes	  yes
    233824256:0x0000619f	yes	  no
    234151936:0x000024df	yes	  no
    This is why ExtractStream fails on FSIDs only some of the time, but more so on larger drives! I have a single drive 120GB SAT-T60 and 0xfffff works for me. You're mileage may vary... The mask may be dependant on partition size, or it may have always been 0xfffff but since most FSIDs are found on smaller systems, the upper two bits weren't missed.

    If someone with a cross compiler setup could plug in this new formula and recompile ExtractStream for me I'd really appreciate it. I'm sure this doesn't solve all of the problems with ExtractStream on DirecTivo, but it's a start!

    -Ray
    Last edited by rwaldin; 01-03-2002 at 08:44 AM.

  8. #23
    Join Date
    Jul 2001
    Posts
    63

    bump

    bump bump want to see somthing come out of this.

  9. #24
    Join Date
    Dec 2001
    Posts
    18
    Here's the compiled ExtractStream 0.3a1 with change proposed by rwaldin. I don't have my TivoNet adapter in right now so I can't properly test.

    Enjoy! Hope it works

    [edit] attachment now below...
    Last edited by bobwondernut; 01-04-2002 at 11:50 PM.

  10. #25
    Join Date
    Dec 2001
    Location
    Greenbelt, MD
    Posts
    6
    Originally posted by bobwondernut
    Here's the compiled ExtractStream 0.3a1 with change proposed by rwaldin. I don't have my TivoNet adapter in right now so I can't properly test.
    Unfortunately, it doesn't work any better for me.

    I have a 2.5Xtreme DTivo with a single 120 gig Maxtor drive and a TiVoNet card installed. When I run your version of ExtractStream, I get:

    bash-2.02# ./ExtractStream-0.3a1 -s 39341
    Attempting to locate tyStream with fsid 39341...
    ...tyStream located, sector map follows:
    Start Length
    0d308800 00040000
    0d348800 00020000
    0d368800 00010000
    Block: d308800 Count: 262144 Total blocks 3
    Playing block 0d308800/dev/hda10 has 0x00100000 sectors
    /dev/hda11 has 0x01645800 sectors
    Failed to open /dev/hdb2
    bash-2.02#

    Why is it trying to use /dev/hdb2 ? I don't have a second drive at all.

    Prosaic

  11. #26
    Join Date
    Dec 2001
    Posts
    18
    the partition maps in extractstream are preset by default to hda10,11 and hdbn,n (i forget which exactly). likely we just have to determine which partition maps actually hold the mfs data and compile extractstream again with this data.

    upon further investigation, i'm an *****. hda10 is still the application partition, and hda11 is the media partition. i'll start poking in the code and see what's broken now.
    Last edited by bobwondernut; 01-04-2002 at 09:24 PM.

  12. #27
    Join Date
    Dec 2001
    Posts
    17
    Many thanks to bobwondernut for compiling this, but I'm not getting the results I expected. I get many "sector fails signature check" error messages which indicate that the sectors are not even valid (they don't contain the magic signature) let alone the right sectors. Did you start with the ExtractStream-0.2 or is there really a 0.3 version somewhere? And can you double check to make sure you only changed the 0x3ffff mask to 0xffff (that's five "f"s) and not to 0xffff or something? Also, it would really help to have the following debug output right after the fsid-to-nsector calculation:
    Code:
      fprintf(stderr,"Attempting to locate tyStream with fsid %d...\n", fsid);
      nsector = ( ( (loff_t)(((fsid * 0x20db2) & 0xfffff) + 0x462) ) << 9 );
      fprintf(stderr,"tyStream with fsid %d appears to be at sector %d...\n", fsid, nsector);
    That way we can compare the results with what is dumped by ExtractStream -d.

    Again thanks for compiling this. I guess I need to get my cross compile system going and do some more extensive and interactive debugging... I'm still convinced that the correct mask is 0xfffff.

    -Ray

  13. #28
    Join Date
    Dec 2001
    Posts
    17
    Why is it trying to use /dev/hdb2 ? I don't have a second drive at all.
    Prosaic:

    This is hard coded into ExtractStream. It believes there are only (up to) four MFS Media partitions (hda10, hda11, hdb2, hdb3). I'm pretty sure that on upgraded systems the list of MFS media partitions is different. On my single A drive 120 GB system, I think ?I have six MFS Media partitions: hda10, hda11, hda12, hda13, hda14, and hda15.

    This is something else that we'll need to change in ExtractStream before it works flawlessly on upgraded systems.

    -Ray

  14. #29
    Join Date
    Dec 2001
    Posts
    18
    The version posted before is simply 0.2 with the mod, i labeled it as 0.3 in case some poor schmuck comes along looking for 0.2 and ends up with something that doesn't work.

    here's the build with your debug request added. I'd be happy to let you onto the host i have which has the crosscompile tools if you like. you'll need ssh. pm me here.

    i'm also attaching the source so you can verify i made the change exactly as you asked.

  15. #30
    Join Date
    Nov 2001
    Posts
    117
    Has any one had sucess with this new compile ?
    Last edited by tivomon; 01-05-2002 at 02:57 AM.
    Did I do that...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •