Page 7 of 99 FirstFirst ... 567891757 ... LastLast
Results 91 to 105 of 1476

Thread: Mfs_Ftp: extract, archive, restore & transfer recordings

  1. #91
    Join Date
    Mar 2002
    Posts
    1,339
    the header chunk of each fsid tells you how many chunks of video it contains in bytes 12-15

    from the mfstools 2 readme

    -r scale
    This option allows you to reduce the amount of RAM TiVo uses by
    increasing the block size for the media storage created with the -x
    option. The acceptable values are 0 to 4, corresponding to 1, 2, 4,
    8 and 16 megabytes. The larger the value, the less RAM will be used
    (and the faster menus will respond) but a (small) amount of storage
    will be wasted by some recordings. At the extreme small end (-r 0)
    some PVRs with a large amount of storage may not be able to perform
    some tasks, such as self repair. (A.K.A. Green Screen) The default
    is 2 (or 4 megabytes). Any TiVo created partitions are created with
    a scale of 0 (or 1 megabyte). It is best to leave this option alone.


    if you have a mfstools restore on a large drive or adjusted the mimimum mfs allocation size with the -r option then sometimes the fsid is actually larger than the size required for the chunks it contains. the space between the last chunk and the end of the fsid is the junk that falls through the cracks. it's just whatever happened to be there when the fsid was allocated. 98% of the time it only happens at the end of the last fsid.

    since tcl is required for tivo db calls and runs on every flavor of os/hardware I tinker with I write tivo utils as tcl scripts. (it's also easy to link in platform specific c modules as functions for speed) if needed they can be "compiled" into an executable with a runtime lib like freewrap as I did with tmf2ty & tmf2mpg. porting is unnecesary for the mac since X includes tcl. the import filter algo is in the ty2fsid proc in mfs_ftp that starts around line 500 and in tmf2ty but disabled by default


    --
    Riley
    Last edited by rc3105; 07-10-2003 at 01:56 AM.

  2. #92
    Join Date
    Nov 2002
    Posts
    221
    Originally posted by rc3105
    RE: AW's teaser. if you extract encrypted recordings as tmf they CAN be restored to the same unit. (slightly easier than archiving mfstools backups that include recordings)
    By "same unit", do you mean the same harddrive as well, or could you d/l and reload scrambled tmf on a fresh HD install? Can you do it with TY+?
    Scott

    SAT-T60 Ver 3.5

  3. #93
    Join Date
    Mar 2002
    Posts
    1,339
    AW has a bee in his bonnet and noodged me 'n mbm to update our insert routines... (if only I were that young and energetic again!)

    so far scrambled recs seem to be tied to the crypto chip. they're working on 3.1 and I'm focusing on 2.x. I'll be moving a cyrpto between boards this evening to pin down whether it's tied to the chip or the drive image. my $$$'s on the chip

    scrambled ty+ should insert ok, but tmf is better for archiving scrambles as the "junk between the cracks" is preserved and that may be a factor in restoring scrambles


    --
    Riley
    Last edited by rc3105; 07-10-2003 at 11:48 AM.

  4. #94
    Join Date
    Jan 2002
    Location
    Sonoran Desert
    Posts
    2,823
    Originally posted by rc3105

    so far scrambled recs seem to be tied to the crypto chip. they're working on 3.1 and I'm focusing on 2.x. I'll be moving a cyrpto between boards this evening to pin down whether it's tied to the chip or the drive image. my $$$'s on the chip
    Are you still trying to get oprah to play?

    BTW, when you say you are moving the chip, are you actualy talking about physicaly moving the chip? or cloning the serial number?
    Before PMing me: Iím not your personal tech support. If you have a question, ask in public so I don't have to repeat if somebody else asks. If you want images or slices, use emule. I will ignore all support PMs.

    Sponsor a vegetarian! I have taken the pledge, how about you?

  5. #95
    Join Date
    Mar 2002
    Posts
    1,339

    scrambling & speed issues

    simply cloning the sernum doesn't work so I'll be physically desoldering the !@#$@#$ surface mount crypto chip. gotta replace the tuners / volt reg & who knows what else on those boards anyway. should be fun in the root canal sense of the word
    Originally posted by needo
    How exactly does this work? I have a scrambled stream on my TiVo 4.0. and tried to download the .ty (Same error as before.) I then downloaded the .tmf without any problems, but it still choked on the tmf2ty conversion. And I ended up with an invalid .ty.

    What am I missing?
    scrambled recs can't be converted to mpeg on the pc (yet). they can be backed up, restored and played back on the EXACT same tivo, or they can be decrypted into regular ty in the tivo, then extracted & converted to mpg. mfs_stream can't read scrambles so to archive w/o decrypting you have to use mfs_tarstream to extract as tmf
    speed issues - the biggie
    here I thought it was something complex & it was just the !@#$% ideturbo module, figures. thanks mbm!

    to check your tivo boot parms issue this command from bash

    bootpage -q /dev/hda

    you should get roughly this

    IP address: 192.168.0.2
    Primary boot partition: 3
    Alternate boot partition: 6
    Hostname: unnamed
    Net boot kernel name: linux.px
    Boot parameters: root=/dev/hda4 runideturbo=false
    MAC address: bd:ca:43:47:75:90
    RF channel: 0
    Standby: 109
    1

    if the "Boot paramters:" line DOES NOT include "runideturbo=false" then you need to add it

    note whatever your root parms are. "root=/dev/hda4" or "root=/dev/hda7" and update them like this

    bootpage -P "your_current_root_parms runideturbo=false" /dev/hda

    reboot, try an insert. should be MUCH faster
    Last edited by rc3105; 12-10-2003 at 04:34 AM.

  6. #96
    Join Date
    Jul 2003
    Posts
    3

    insert speed

    Oh yeah - this is nice!

    Here are some transfer examples once the runideturbo=false parameter is added:

    TIVO -> TIVO: 416 KBs
    PC -> TIVO: 409 KBs

    Seems like if I let it queue up about 5-10 mins, I can watch the show "real time" as it streams from the source.

    Congrats rc3105 you've got a winner!

    P.S. This is using 2 SVR-2000's - which are Series 1 standalones.

  7. #97
    Join Date
    Dec 2001
    Posts
    173

    Re: insert speed

    Originally posted by tivowatcher
    PC -> TIVO: 409 KBs

    Seems like if I let it queue up about 5-10 mins, I can watch the show "real time" as it streams from the source.
    Even at that speed you really should be able to start watching after a minute or so (though my experience is all on DTivos).

  8. #98
    Join Date
    Feb 2003
    Posts
    17
    WOW, that is sooo much faster I am now having the problem of inserts running so fast they are affecting normal use

    I have extract running around 1,500k while recording at best quality and inserts around 1,000k again while recording at best however this severely impedes on normal use I'll have to get experimenting on throttles.

    LOVING IT

  9. #99
    Join Date
    Nov 2002
    Posts
    221
    Originally posted by mini__me
    WOW, that is sooo much faster I am now having the problem of inserts running so fast they are affecting normal use

    I have extract running around 1,500k while recording at best quality and inserts around 1,000k again while recording at best however this severely impedes on normal use I'll have to get experimenting on throttles.

    LOVING IT
    1000KB Wheew! glad to hear it. I saw that 400KB insert speed mentioned earlier, so I didn't want to get my hopes up.

    Already did the bootpage cmd (from work), just can't wait to get home to test upload speeds.

    As always thanks Riley (and of course mbm)
    Scott

    SAT-T60 Ver 3.5

  10. #100
    Join Date
    Jul 2003
    Posts
    769
    I'm getting about 550-650KBS uploads and am more than satisfied with the Tivo's performace. Recordings and menus all look fine so far. That's a good enough speed for me. It seems like high quality recordings go in at 'real time' now. That is, 1 hour of video uploads in about an hour.

  11. #101
    Join Date
    Nov 2002
    Posts
    221
    Originally posted by osetivo
    1000KB Wheew! glad to hear it. I saw that 400KB insert speed mentioned earlier, so I didn't want to get my hopes up.

    Already did the bootpage cmd (from work), just can't wait to get home to test upload speeds.

    As always thanks Riley (and of course mbm)

    Much better, With both tuners on empty channels I'm now getting ~650KB/s up from 250. Geesh 1,000 KB/s, I'm still jealous.
    Scott

    SAT-T60 Ver 3.5

  12. #102
    Join Date
    Jul 2001
    Posts
    48
    The ideturbo module is intended to put the drives in "streaming media" mode; a few tests suggest that drive access actually slows down while ideturbo is loaded, adding runideturo=false to the boot params will prevent the ideturbo module from loading:
    Code:
    bash-2.02# bootpage -p /dev/hda
    root=/dev/hda4
    bash-2.02# bootpage -P "root=/dev/hda4 runideturbo=false" /dev/hda
    Updated boot page on /dev/hda
    Playing around with hdparm can lead to further performance gains.

    Also, the version of mfs_import distributed with mfs_ftp is rather inefficient; a newer version is avaiable here: http://alt.org/forum/index.php?t=msg&th=55&start=20
    http://tivo.samba.org/download/mbm
    E4pFXEMBEEMXXv2L0TlAFOYC3/2HtWFvYiL3md0h2cxuU1BFugTKBBaOi1GH/7265DTD4a57
    7fg1JOK8+3nCiZvRjl11Bit4LuaXA4KjPh0OHCyFIpSP2VJkb5pkY2M5HPlBN0/UawyQBhSM
    CVnB02kbxifsgVYcYfEiTG2qfIdFXmstrEhW9gpe+5OxEYid979qu1Esg2YHNA7W8tSTd1t9
    88LYW46AhE01Uts8pa4TgZazxlo/FkMAS3i/Oqtm7Rf8C6QzXmbDgbN+fP+Fcu53FOtZXNXX
    ClRoZSB0cnV0aCBhYm91dCBhIG1hbiBsaWVzIGluIHdoYXQgaGUgaGlkZXMgLU1hbHJhdXgK

  13. #103
    Join Date
    Mar 2002
    Posts
    1,339
    mfs_ftp is a basic ftp implmentation. some os's and proxy's can mount it as a remote filesystem, other's can't. try & few & see what works for you

    setpri is there to prevent inserts from eating so many cpu cycles that menu's get sluggish. get it from alt.org, put it in /var/mfs_ftp/ & make it executable or don't worry about it. it's there to slow things down. the log just lets you know whether it was found/used

    mbm's optomized mfs_import is a bit faster BUT it has to be renamed to mfs_stdinsert (or mfs_ftp edited appropriatly) and it only works with mfs_ftp versions > 1.2.8f it doesn't affect transfer speeds so much as lowering the cpu load for a given transfer rate (which lets you insert faster w/o impacting system performance)

    if you add runideturbo=false to your boot parms you have to reboot the tivo before that takes effect. check to see if ideturbo is loaded with "cat /proc/modules" if it's not listed you're good

    hdparms can make a big difference, this is safe for stock quantums "hdparm -c1 -d1 -m8 -S0 /dev/hda /dev/hdb" try it from bash before adding to rc.sysinit or rc.remote-login

    increasing info(ithrottle) on line 21 slows inserts down. values can be 0 to 10

    increasing info(insert_priority) accelerates inserts if setpri is available. values are 1 (lowest) to 99 (highest) 5 works nicely here

    ithrottle & insert_priority will interact differently depending on your hd / network / running-hacks / sw-version / viewing-behavior / karma. the worst case scenario is a dtivo/tivonet/Q30 recording 2 shows & rewinding a third while inserting. even under those conditions I'm getting 500k/sec+ here. a standby dtivo/turbonet should run 1,200k/sec+ easily

    don't try to watch scrambled recordings untill import is complete. depending on your noscramble module/settings it could cause a reboot

    test your network / hdparms / insert speed & tweak your settings with the tivo in standby. once you find the peak rate then try a worst case scenario & re-adjust. dtivo style partition layout can make a big difference on slow drives like the stock quantums (see previous posts re: mfstools restore with the -p option)
    Last edited by rc3105; 09-30-2003 at 01:33 AM.

  14. #104
    Join Date
    Jul 2003
    Location
    NYC
    Posts
    339

    Re: some performance notes from an xplusz'd 2.5.2 dtivo running 1.2.8d

    Originally posted by rc3105

    check your hdparms with hdparm -i /dev/hda and pay attention to MaxMultiSect. if you have dual drives you also need to do this for /dev/hdb--
    Riley
    I have a quantum and my maxmultisect=16, should I go with 8 or 16?

    Do I have to reboot?

    Thanks

    Keep up the good work, thanks for working on this!

    /dev/hda:
    Model=QUANTUM FIREBALLlct15 30, FwRev=A01.0F00, SerialNo=314015418411
    Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
    RawCHS=16383/16/63, TrkSize=32256, SectSize=21298, ECCbytes=4
    BuffType=3(DualPortCache), BuffSize=418kB, MaxMultSect=16, MultSect=off
    DblWordIO=no, maxPIO=2(fast), DMA=yes, maxDMA=2(fast)
    CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=58633344
    tDMA={min:120,rec:120}, DMA modes: mword0 mword1 *mword2
    IORDY=on/off, tPIO={min:120,w/IORDY:120}, PIO modes: mode3 mode4


    bash-2.02# hdparm -m8 /dev/hda

    /dev/hda:
    setting multcount to 8
    multcount = 8 (on)
    bash-2.02# hdparm -i /dev/hda

    /dev/hda:
    Model=QUANTUM FIREBALLlct15 30, FwRev=A01.0F00, SerialNo=314015418411
    Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
    RawCHS=16383/16/63, TrkSize=32256, SectSize=21298, ECCbytes=4
    BuffType=3(DualPortCache), BuffSize=418kB, MaxMultSect=16, MultSect=8
    DblWordIO=no, maxPIO=2(fast), DMA=yes, maxDMA=2(fast)
    CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=58633344
    tDMA={min:120,rec:120}, DMA modes: mword0 mword1 *mword2
    IORDY=on/off, tPIO={min:120,w/IORDY:120}, PIO modes: mode3 mode4

  15. #105
    Join Date
    Mar 2002
    Posts
    1,339
    AVD:

    if the drive MaxMultiCount is 16 then use that. the point was not to set multicount to more than the reported MaxMultiCount for the drive. the hdparm settings are lost when you power-down or reboot the tivo. once you figure out what options you're going to use add them to a startup script somewhere
    Last edited by rc3105; 12-10-2003 at 04:40 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •