Page 3 of 11 FirstFirst 12345 ... LastLast
Results 31 to 45 of 151

Thread: ExtractStream and PlayStream failing on every sector in a DTivo

  1. #31
    Join Date
    Nov 2001
    Posts
    117
    I'm working on bulding a gcc based cross_compiler so I can assist. I'm following the faq from penguinppc dot org on cross_compiling. I'm in the process of building a glibc and I get the following error when running make after configure. any ideas ?

    make[1]: Entering directory `/tmp/glibc-2.2.3/csu'
    gcc ../sysdeps/unix/sysv/linux/init-first.c -c -O2 -Wall -Winline -Wstrict-prototypes -Wwrite-strings -g -mnew-mnemonics -Wa,-mppc -mpowerpc -fkeep-inline-functions -I../include -I. -I.. -I../libio -I../sysdeps/powerpc/elf -I../linuxthreads/sysdeps/unix/sysv/linux -I../linuxthreads/sysdeps/pthread -I../linuxthreads/sysdeps/unix/sysv -I../linuxthreads/sysdeps/unix -I../linuxthreads/sysdeps/powerpc -I../sysdeps/unix/sysv/linux/powerpc -I../sysdeps/unix/sysv/linux -I../sysdeps/gnu -I../sysdeps/unix/common -I../sysdeps/unix/mman -I../sysdeps/unix/inet -I../sysdeps/unix/sysv -I../sysdeps/unix/powerpc -I../sysdeps/unix -I../sysdeps/posix -I../sysdeps/powerpc/fpu -I../sysdeps/powerpc -I../sysdeps/wordsize-32 -I../sysdeps/ieee754/flt-32 -I../sysdeps/ieee754/dbl-64 -I../sysdeps/powerpc/soft-fp -I../sysdeps/ieee754 -I../sysdeps/generic/elf -I../sysdeps/generic -nostdinc -isystem /usr/lib/gcc-lib/i386-redhat-linux/2.96/include -isystem /usr/local/powerpc-linux/include -D_LIBC_REENTRANT -include ../include/libc-symbols.h -DHAVE_INITFINI -o init-first.o
    In file included from ../string/endian.h:37,
    from ../include/endian.h:1,
    from ../sysdeps/generic/bits/waitstatus.h:63,
    from ../stdlib/stdlib.h:43,
    from ../include/stdlib.h:7,
    from ../sysdeps/unix/sysv/linux/init-first.c:21:
    ../sysdeps/powerpc/bits/endian.h:34:4: warning: #warning Cannot determine current byte order, assuming big-endian.
    cc1: Invalid option `new-mnemonics'
    cc1: Invalid option `powerpc'
    make[1]: *** [init-first.o] Error 1
    make[1]: Leaving directory `/tmp/glibc-2.2.3/csu'
    make: *** [csu/subdir_lib] Error 2
    Did I do that...

  2. #32
    Join Date
    Nov 2001
    Posts
    117
    Also I've looked at the source code posted a few messages back. Thanks for the compile etc. but... it doesn't work on my Phillips DSR6000. I looked and the source for ExtractStream is @ 0.2 + the sectore formula update. I'm posting 0.3 that I got from CVS. maybe someone could add the formula and recompile for us.

    Also here is the output that I get from running the posted compile. Looks like the formula may not work for my hardware. I'm running a 60gb Western Digital. I've rename ExtractStream to es3 my easer to type. Note that sector id might be suspect since it's 0 for fsid 20071 and 429496729 for fsid 20062.

    es3 20071 20062
    Attempting to locate tyStream with fsid 20071...
    tyStream with fsid 20071 appears to be at sector 0...
    Sector fails signature check.
    Sector fails signature check.
    33 38 62 3a 33 3a 31 3a 30 00 09 88 00 00 3b d4 38b:3:1:0.....;.
    18 08 41 30 30 30 65 65 33 63 31 3a 33 3a 31 3a ..A000ee3c1:3:1:
    30 00 09 88 00 00 49 3c 18 08 41 30 30 30 65 65 0.....I<..A000ee
    33 63 32 3a 33 3a 2d 31 3a 31 00 88 00 00 49 44 3c2:3:-1:1....ID
    18 08 41 30 30 30 65 65 33 63 33 3a 33 3a 2d 31 ..A000ee3c3:3:-1
    3a 31 00 88 00 00 29 b8 18 08 41 30 30 30 65 65 :1....)...A000ee
    33 63 34 3a 33 3a 2d 31 3a 31 00 88 00 00 1d 05 3c4:3:-1:1......
    18 08 41 30 30 30 65 65 33 63 35 3a 33 3a 2d 31 ..A000ee3c5:3:-1
    3a 31 00 88 00 00 29 c4 18 08 41 30 30 30 65 65 :1....)...A000ee
    33 63 36 3a 33 3a 2d 31 3a 31 00 88 00 00 2c 95 3c6:3:-1:1....,.
    18 08 41 30 30 30 65 65 34 32 65 3a 33 3a 32 3a ..A000ee42e:3:2:
    30 00 09 88 00 00 45 b2 18 08 41 30 30 30 65 65 0.....E...A000ee
    34 33 65 3a 33 3a 31 3a 30 00 09 88 00 00 40 15 43e:3:1:0.....@.
    18 08 41 30 30 30 65 65 35 30 66 3a 36 3a 35 3a ..A000ee50f:6:5:
    30 00 09 88 00 00 45 b4 18 08 41 30 30 30 65 65 0.....E...A000ee
    35 31 30 3a 36 3a 34 3a 30 00 09 88 00 00 40 16 510:6:4:0.....@.
    18 08 41 30 30 30 65 65 35 33 31 3a 36 3a 35 3a ..A000ee531:6:5:
    30 00 09 88 00 00 28 5a 18 08 41 30 30 30 65 65 0.....(Z..A000ee
    35 34 39 3a 33 3a 2d 31 3a 31 00 88 00 00 65 dc 549:3:-1:1....e.
    18 08 41 30 30 30 65 65 35 65 36 3a 33 3a 31 3a ..A000ee5e6:3:1:
    30 00 09 88 00 00 22 6a 18 08 41 30 30 30 65 65 0....."j..A000ee
    36 64 35 3a 33 3a 31 3a 30 00 09 88 00 00 22 35 6d5:3:1:0....."5
    18 08 41 30 30 30 65 65 36 66 33 3a 33 3a 31 3a ..A000ee6f3:3:1:
    30 00 09 88 00 00 22 6b 18 08 41 30 30 30 65 65 0....."k..A000ee
    37 36 34 3a 33 3a 31 3a 30 00 09 88 00 00 5d c8 764:3:1:0.....].
    18 08 41 30 30 30 65 65 37 38 31 3a 33 3a 2d 31 ..A000ee781:3:-1
    3a 31 00 88 00 00 47 08 18 08 41 30 30 30 65 65 :1....G...A000ee
    37 61 38 3a 33 3a 31 3a 30 00 09 88 00 00 3d a8 7a8:3:1:0.....=.
    18 08 41 30 30 30 65 65 37 61 65 3a 33 3a 2d 31 ..A000ee7ae:3:-1
    3a 31 00 88 00 00 26 77 18 08 41 30 30 30 65 65 :1....&w..A000ee
    37 63 34 3a 33 3a 32 3a 30 00 09 88 00 00 26 78 7c4:3:2:0.....&x
    18 08 41 30 30 30 65 65 37 63 35 3a 33 3a 32 3a ..A000ee7c5:3:2:

    Attempting to locate tyStream with fsid 20062...
    tyStream with fsid 20062 appears to be at sector 429496729...
    Sector fails signature check.
    Sector fails signature check.
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    Did I do that...

  3. #33
    Join Date
    Nov 2001
    Posts
    117
    This is most likely obious to those who have been pooring thru the ExtractStream code, but I saw two places where the sector formula is used.

    if you just grep for 3fff you'll see what I'm talking about. I'm not sure if this matter at this point.

    Also I've found a very similar thread at pvrupgrade dot com. Looks like they were also reviing the fsid/sector formula topic. There were unsure about the function 0x462. They thought this could be some sort of ofset at the headder of the disk. Thay also were unsure about the affect programs like tivomadd would have on this number and the overall placment/layout of the data. Could Tivo have a smarter routine ? I think the answer is in Tridge's head, I've read several places that Tivo asked him not to release a MFS driver. I think the understanding at that level would surely give us the answers we need here. So is it liquor or women that we need to bribe Tridge ?
    Did I do that...

  4. #34
    Join Date
    Dec 2001
    Posts
    18
    neither. i'm seeing some good progress from rwaldin. i'm sure he'll fill everyone in when he's got something a little more concrete. this topic has more hearsay and the worst signal/noise ratio of almost any i've seen here -- a little patience is required is all.

  5. #35
    Join Date
    Dec 2001
    Location
    Lafayette, California
    Posts
    47

    Beer anyone?

    Wow - thanks everyone! It's great to get this sort of current info and thinking from a number of people who really understand this area.

    I wish I could help - I finally got Linux installed on my PC, and now am looking for the cross-compiler files. I guess I should start reading the PlayStream and ExtractStream sources so that I have a better understanding of the issues in there.

    But in the meantime, I'll get the chips&dip and cold beer. Looks like we are going to be here a while

    Eric

  6. #36
    Join Date
    Dec 2001
    Posts
    17

    Good news/Bad news

    Good news, bad news.

    Ok, first the good news. My modified ExtractStream is able to extract raw tyStreams for every single show on my DirecTivo. The bad news is that the streams are not understood by any of the programs that are supposed to split them into seperate a/v files. I'm hoping someone with some more knowledge in this area can take a look at the resulting data and figure out what to do next.

    Three changes have been made to ExtractStream:

    1) I've modified the formula as described earlier (0x3ffff mask is now 0xfffff) plus changes necessary to detect FSIDs that refer to secondary MFS App partitions (as is the case with my 120GB single A drive config). Details can be found in the new mapFSID function.

    2) I've added MFS partition detection, so that ExtractStream doesn't assume a fixed sequence of devices (it used to assume /dev/hda10 /dev/hda11 /dev/hdb2 /dev/hdb3). All you need to do is... nothing. Your system should have the MFS_DEVICE environment variable set to the first MFS device (usually /dev/hda10) and this partition contains info on other MFS partitions. See the source code for the initDeviceList function for details.

    3) In case you can't extract raw tyStreams, I've add a debugging switch to "walk" the MFS App partitions checking the new formulas against the actual data found. This function should produce the debugging output necessary to figure out what went wrong in the case that ExtractStream can't find FSID sectors. Just run ExtractStream -w to walk the MFS App partitions. Running this causes my Tivo to pause until it completes, which is at least a few minutes.

    This is only an experimental release to be used to extract raw tyStreams from DirecTivo systems. If you have an SA Tivo and are willing to try this version, please let me know the results. I don't recommend it as I haven't tried this myself (don't own one), but I am curious whether it works or not.

    To extract raw tyStreams, use the -s switch, like this:

    ExtractStream -s <fsid> [...] | nc -n -w 3 192.168.0.1 1234

    as described in the ExtractStream document at:

    http://pvrhack.sonnik.com/tivo/extract-stream.htm

    Many, many thanks go to bobwondernut for his invaluable assistance. Couldn't have done this without him!

    Enjoy...

    -Ray
    Last edited by rwaldin; 01-07-2002 at 05:24 AM.

  7. #37
    Join Date
    Dec 2001
    Posts
    17
    whoops, forgot the attachment

  8. #38
    Join Date
    Dec 2001
    Location
    Lafayette, California
    Posts
    47

    Looks promising (pass the dip please)

    Thanks, rwaldin and bobwondernut.

    I tried the new ExtractStreamDx1 on the same DTivo and same FSID as my original post, results are below.

    This is a Hughes DTivo with only the original 40GB drive. The stream I am attempting to extract is 3 minutes long.

    - Does everything look OK? Or does "flushing buffers" mean that it gave up?
    - On a DTivo with 2 drives, where do I find the list of partitions? Is there a text file like /etc/fstab, or is this info buried in the database, or is it hidden in a partition table?


    Thanks
    Eric


    ExtractStreamDx1 -w 119309
    found 4 MFS partitions: /dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13
    /dev/hda10 has 1048576 sectors
    /dev/hda10 sector 1048576 (61 passed, 0 failed, 1047332 skipped)
    /dev/hda12 has 1048576 sectors
    /dev/hda12 sector 839680 (62 passed, 0 failed, 839555 skipped)

    ExtractStreamDx1 -s 119309 > out
    found 4 MFS partitions: /dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13
    Attempting to locate tyStream with fsid 119309...
    tyStream with fsid 119309 appears to be on device /dev/hda12 at sector 23860736.
    ..
    /dev/hda12 has 1048576 sectors
    ...tyStream located, sector map follows:
    Start Length
    01880000 00020000
    018a0000 00010000
    018b0000 00008000
    018b8000 00004000
    018bc000 00002000
    Block: 1880000 Count: 131072 Total blocks 5
    Playing block 01880000/dev/hda10 has 0x00100000 sectors
    /dev/hda11 has 0x01ea8c00 sectors
    Block: 18a0000 Count: 65536 Total blocks 5
    Block: 18b0000 Count: 32768 Total blocks 5
    Block: 18b8000 Count: 16384 Total blocks 5
    Block: 18bc000 Count: 8192 Total blocks 5
    Flushing buffers8be000
    bash-2.02# ls -l out
    -rw-r--r-- 1 0 0 113483776 Jan 7 08:26 out
    bash-2.02#
    (113MB for 3 minutes of video)

  9. #39
    Join Date
    Dec 2001
    Posts
    17
    - Does everything look OK? Or does "flushing buffers" mean that it gave up?
    Looks ok to me. Flushing buffers does not mean it gave up, just that it's flushing output buffers before exiting to make sure everything gets written out completly.

    - On a DTivo with 2 drives, where do I find the list of partitions? Is there a text file like /etc/fstab, or is this info buried in the database, or is it hidden in a partition table?
    I think the entire list is stored in the first MFS App partition, at offset 0x24. There you'll find a space separated, zero terminated string listing all MFS partitions. Mine reads "/dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13 /dev/hda14 /dev/hda15". Yours was printed out for you...

    found 4 MFS partitions: /dev/hda10 /dev/hda11 /dev/hda12 /dev/hda13
    I assume that two drive systems are "Married" by adding the necessary entries ("... /dev/hdb2 /dev/hdb3") to the list.

    I seems that there can only be at most two MFS App partitions, the first and third in the list.

    -Ray

  10. #40
    Join Date
    Nov 2001
    Posts
    117
    cheers waldin and bobwondernut,

    It looks better I think the bad fsid's are behind us. I'm encloseing some diag info that I go from this run. Hey anyone which versions of DirectTivo harware and software are people using ?
    Did I do that...

  11. #41
    Join Date
    Dec 2001
    Posts
    17
    Thanks Tivomon,

    Your output looks good. I'm hoping someone finds out how to use the extracted data and makes this all worth while. Meanwhile, I suspect the FSID mapping will be useful to the SA Tivo folks regardless...

    -Ray

  12. #42
    Join Date
    Jul 2001
    Posts
    63

    safe to start dumping?

    So I got the video off. Yes I cant watch it but is it safe to start dumping other stuff as well? So when a fix does come out then I can just run it on the files I extracted.

  13. #43
    Join Date
    Dec 2001
    Posts
    17
    I wish I had a better answer for you, but I don't know. Until someone figures out how to decode/split/whatever these streams, I wouldn't do anything too drastic!

    -Ray

  14. #44
    Join Date
    Jan 2002
    Location
    Medford, MA
    Posts
    23

    Arrow

    rwaldin - I've been having some trouble with a SA TiVo doing video extraction, documented over here - http://www.dealdatabase.com/forum/sh...0465#post20465
    I saw some posts of yours on the ExtractStream group and you seemed like you might be interested in debugging extraction on SAs. Unfortunately, your ExtractStreamDx1 doesn't work *at all* on my SA:

    bash-2.02# AUDIO_OUT=extract/gothic2.m2a VIDEO_OUT=extract/gothic2.m2v ./ExtractStreamDx1 267378 268817 268887
    found 4 MFS partitions: /dev/hda10 /dev/hda11 /dev/hdb2 /dev/hdb3
    Attempting to locate tyStream with fsid 267378...
    tyStream with fsid 267378 appears to be on device /dev/hda10 at sector 134958080...
    /dev/hda10 has 1048576 sectors
    Sector fails signature check.
    Sector fails signature check.
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    ...
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

    Attempting to locate tyStream with fsid 268817...
    tyStream with fsid 268817 appears to be on device /dev/hdb2 at sector 234464768...
    /dev/hdb2 has 8192 sectors
    Failed to lseek/read from disk.
    Attempting to locate tyStream with fsid 268887...
    tyStream with fsid 268887 appears to be on device /dev/hdb2 at sector 225902080...
    Failed to lseek/read from disk.
    Flushing buffers
    bash-2.02#
    However, I very much appreciate the efforts you've made and if it's of any interest to you, I placed the output of ExtractStreamDx1 -w up at:
    http://www.chiappa.net/~chris/dxlog.zip
    Thanks a bunch...

  15. #45
    Join Date
    Dec 2001
    Posts
    18
    Anyone out there with a 2.0x dtivo, please try the dx1, and see if you can process an extracted stream. Now's the time to find out if our pals at tivo have enabled encryption in 2.5, or if we're just dealing with a unique stream format here from what we're used to.

    Let us know you're gonna do it, or I guess eventually i'll just fire up a 2.0 image in a couple of days and try it out myself.

    thx

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •