Page 89 of 99 FirstFirst ... 39798788899091 ... LastLast
Results 1,321 to 1,335 of 1476

Thread: Mfs_Ftp: extract, archive, restore & transfer recordings

  1. #1321
    Join Date
    Aug 2004
    Posts
    102

    mfs_import freeze

    Quote Originally Posted by Jamie View Post
    Where did "Using new MFS_Import options..." in the log come from? That's not in any version of the mfs_ftp.tcl source I can find, and I believe those changes are causing problems. You don't want the verbose option with mfs_import unless you redirect output to a file. Otherwise the stderr output from mfs_import may confuse mfs_ftp.
    The message was added to the mfs_ftp file so that I do not forget that I modifed the code to turn on the verbose output, and redirect the output to a file.

    Here's a current mfs_import output (file attached).

    I'm suspecting a tcp problem as mfs_import apparently is still trying to get data, and the client had timed out (timeout set to 180 seconds).

    Should I try a different set of usb drivers? Any suggestions?

    Again, an unmodified version of mfs-Ftp seems to freeze in the exact same way. The older data_in method never fails !
    Attached Files Attached Files

  2. #1322
    Join Date
    Aug 2004
    Posts
    4,085
    Quote Originally Posted by tas3086 View Post
    The message was added to the mfs_ftp file so that I do not forget that I modifed the code to turn on the verbose output, and redirect the output to a file.
    ...
    How was this tmf file generated? Is it a capture from mfs_ftp, or generated with other tools?

    Do a "tar tv your_file.tmf" on your PC (e.g. in cygwin). Is the third part exactly 536870912 bytes long? What does "ls -l your_file.tmf" show for the length? If the file length is shorter than the combined lengths in the tar header, you'll get this kind of behavior where mfs_import is trying to read more data from the socket than what the client is sending. I think, though, that it will see the socket connect closed when the client finishes, and exit with an error rather than just hang forever.

    It is possible that unreliable network drivers are an issue here. I've lost track of what hw/sw you are running, but I'm pretty sure that that the stock 3.1.5 usb drivers have been reported to be unreliable.

    If you think the problem is loaded related, you can play around with the priority and rate control options in mfs_ftp. These are mfs_ftp options, but in the end they get passed down to the -p and -r options of mfs_import:
    Code:
            -p <priority>       Priority  0: ts; 1-99 RT FIFO
    ...
            -r <ms>             rate control (throttle)
                                  -'ve  : no delay (default)
                                  0     : sched_yield() between chunks
                                  +'ve  : # of ms to delay between chunks
    Higher priority and/or a delay between chuncks might help work around the problem, if it is flakey usb drivers under load.
    Last edited by Jamie; 08-23-2007 at 02:03 PM.

  3. #1323
    Join Date
    Aug 2004
    Posts
    102
    Quote Originally Posted by Jamie View Post
    How was this tmf file generated? Is it a capture from mfs_ftp, or generated with other tools?

    Do a "tar tv your_file.tmf" on your PC (e.g. in cygwin). Is the third part exactly 536870912 bytes long? What does "ls -l your_file.tmf" show for the length? If the file length is shorter than the combined lengths in the tar header, you'll get this kind of behavior where mfs_import is trying to read more data from the socket than what the client is sending. I think, though, that it will see the socket connect closed when the client finishes, and exit with an error rather than just hang forever.
    File has 8 parts. First 7 are 536870912 part 8 is:503316480
    total file length 4,685,873kb long enough to get past part 3.

    Originally created with mfs_ftp from 3.x system.
    h10-250 6.3d system with .20 kernel and stock usb drivers.
    Should I try the backport drivers?

    Upload speeds mfs_import vs old data_in seem to be the same slow 600KB.
    have tried upping the priority to 80. seems to help but still freezes here and there. Old data-In method is fine. Adding additional rate delay just makes things a lot slower.

    Maybe I will do an IP trace and see exactly what is happening when it does freeze.

    Thanks for all your help.. More later.

    Just noticed, I'm running with rate 0. I'll try the rate -1 and see what happens

    Update: throttle rate of -1 seems to have solved the problem. mfs_ftp Upload completed successfully. Will try one more time.
    Last edited by tas3086; 08-25-2007 at 01:21 AM. Reason: Update

  4. #1324
    Join Date
    Dec 2003
    Location
    SF Bay Area
    Posts
    15
    I had mfs_ftp working fine, but today I get the error shown below when trying to read the contents of any of the folders (e.g. "ty+"). This is on a HDVR2 running 6.2 (patches applied).

    Code:
    03:11:06:PM - readlinefromsocket: "sock22"
    03:11:06:PM - echo to verify: "CWD /ty+"
    03:11:06:PM - parseline:
    "CWD /ty+"
    03:11:06:PM - 250 Directory change successful.
    03:11:06:PM - readlinefromsocket: "sock22"
    03:11:06:PM - echo to verify: "PWD"
    03:11:06:PM - parseline:
    "PWD"
    03:11:06:PM - PWD:
    03:11:06:PM - 257 "/ty+" is current directory.
    03:11:06:PM - readlinefromsocket: "sock22"
    03:11:06:PM - echo to verify: "TYPE A"
    03:11:06:PM - parseline:
    "TYPE A"
    03:11:06:PM - 200 Type set to I
    03:11:06:PM - readlinefromsocket: "sock22"
    03:11:06:PM - echo to verify: "PORT 192,168,1,10,4,58"
    03:11:06:PM - parseline:
    "PORT 192,168,1,10,4,58"
    03:11:06:PM - PORT 192,168,1,10,4,58
    03:11:06:PM - data channel "sock23" to 192.168.1.10:1082
    03:11:06:PM - 200 PORT command successful.
    03:11:06:PM - readlinefromsocket: "sock22"
    03:11:06:PM - echo to verify: "LIST -aL"
    03:11:06:PM - parseline:
    "LIST -aL"
    03:11:06:PM - NLIST: "LIST -aL" ("" == LIST command)
    03:11:06:PM - 150 Opening ASCII mode data connection for file list.
    03:11:06:PM - list_type is "LIST"
    03:11:06:PM - update_rec_fsids: forced 1
    bgerror invoked with error
    
     " can't scan path (0x70009)
     "
    
    errorInfo:
    can't scan path (0x70009)
    
        while executing
    "mfs scan $dirName -start $prefix -count 50"
        ("uplevel" body line 1)
        invoked from within
    "uplevel $body"
        invoked from within
    "transaction {uplevel $body}"
        (procedure "RetryTransaction" line 5)
        invoked from within
    "RetryTransaction {
            if { [catch {mfs scan $dirName -start $prefix -count 50} batch] } {
                global errorCode errorInfo
                if {..."
        (procedure "ForeachMfsFile" line 9)
        invoked from within
    "ForeachMfsFile fsid name type "$info(mfspath)" "" {
      RetryTransaction { set rec [db $db openid $fsid] ; set State [dbobj $rec get State] }
      if { [ls..."
        (procedure "update_rec_fsids" line 5)
        invoked from within
    "update_rec_fsids 1"
        (procedure "NLST" line 10)
        invoked from within
    "NLST   $line   $fsock "
        ("LIST" arm line 1)
        invoked from within
    "switch $cmd {
      USER  { USER  $fsock  $line }
      PASS  { PASS  $fsock  $line }
      CWD   { CWD   $args   $fsock }
      CDUP  { CDUP  $fsock }
      DELE  { DELE  $fsock $line..."
        (procedure "parseline" line 6)
        invoked from within
    "parseline $line $sock "
        (procedure "readlinefromsocket" line 10)
        invoked from within
    "readlinefromsocket sock22"
    
    re-initializing mfs_ftp
    
    close the current ftp connection and simply open another
    
    "core dump" :p
    
    info(version): 1.2.9p-patchlevel-20070717b
    info(tswv): 6.2-01-2-151
    info(dbl): 5
    info(ithrottle): 2
    info(insert_priority): 10
    info(multithreaded): 0
    info(saveuntil): suggestion
    info(name_detail): 5
    info(bjuggle): 0
    info(active): 0
    info(ac_interval): 1800
    info(gatewayip): 127.0.0.1
    info(gatewayport): 3105
    
    
    03:11:07:PM - abortcheck: "bgerror"
    03:11:07:PM - reset_control_socket:
    catch close lastsock val ""
    03:11:07:PM - setup_control_socket:
    03:11:07:PM - "192,168,1,26:3105" ready for connections
    Update: I found this patch: http://www.dealdatabase.com/forum/sh...d.php?p=245152

    But now I get the following fatal error:

    Code:
    Dumping mempool to /tmp/BlockFailure.6357
    Tmk Assertion Failure:
        DumpArenaAndBlocksAndDie, line 1472 ()
    Tmk Fatal Error: Thread tivosh <6357> strayed!
    Paste the following into a shell to get a backtrace...
    
    bt -t /tvbin/tivoapp <<END_OF_BT
      read 0x2aaa8000 /lib/ld.so.1
      read 0x2ab04000 /lib/libutil.so.1
      read 0x2ab48000 /lib/libdl.so.2
      read 0x2ab8c000 /lib/libpthread.so.0
      read 0x2abe8000 /lib/libm.so.6
      read 0x2acb0000 /lib/libc.so.6
      read 0x2ae4c000 /lib/libnss_files.so.2
      read 0x2ae9c000 /lib/libnss_dns.so.2
      read 0x2aee4000 /lib/libresolv.so.2
      0x013b3e3c 0x013b3c90 0x013b3f84 0x00f58234 0x00f9312c 0x00fc579c 0x00fc5234
      0x00ff4184 0x00ff5c40 0x00fb9144 0x00f8eb18 0x00fecc98 0x00fb7a28 0x00f8eb18
      0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644
      0x00fb7a28 0x00f8eb18 0x00f9fd3c 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28
      0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00f8e770 0x00f91280 0x00fd37d0
      0x00fd30a0 0x0100127c 0x00fe31c8 0x00fe3714 0x00fb672c 0x00f8da78 0x00fb7a28
      0x00f8eb18 0x00fd99ec 0x00fdd23c 0x00f58150 0x00612c48 0x00403090 0x2acc13fc
    END_OF_BT
    
    Tmk Fatal Error: Thread tivosh <6357>: assertion failure
    
    To view the blocks, run:
       $TIVO_ROOT/devbin/poolview.tcl <app-with-symbols> /tmp/BlockFailure.6357
    
    In the UI that comes up, find your block by address (see above)
    This will help you identify the type and ownership of the blocks.
    Last edited by twostop; 08-25-2007 at 06:55 PM.

  5. #1325
    Join Date
    Aug 2004
    Posts
    102
    Quote Originally Posted by twostop View Post
    I had mfs_ftp working fine, but today I get the error shown below when trying to read the contents of any of the folders (e.g. "ty+"). This is on a HDVR2 running 6.2 (patches applied).
    I've had some problems with smart ftp. Try filezilla at http://filezilla.sourceforge.net/ and see if that helps. Be sure that you are going to port 3105.

  6. #1326
    Join Date
    Aug 2004
    Posts
    102

    Nice Mfs_ftp Install Doc

    I found a reference to some nice mfs_ftp installation documentation here: http://www.dealdatabase.com/forum/sh...5&postcount=13

    Please note that I believe that the following needs to be added to the settings file. I did not see it documented.:

    set info(seriescreate) 1 ;# Allow Series to be created? 0 = disabled, 1/others = enabled.

    and I seem to have good success using:

    set info(ithrottle) -1

  7. #1327
    Join Date
    Jan 2005
    Posts
    41
    Have a question regarding insertion on a 6.3D HR10. I am pulling off the shows from a 6.2 SD D*Tivo and insterting into the HR10 ( Using tmf format). The entire process works great. But folders on the HR10 won't group the shows anymore. It's like the keys to the show are lost and the Tivo doesn't know they are related. Any suggestions ?
    (2) HR10-250 500 GB upgraded to 6.4a
    (2) HR20s
    JimW

  8. #1328
    Join Date
    May 2007
    Posts
    449
    followed all the instructions from http://www.dvrpedia.com/MFS_FTP and believe I have this ready to run. This set of instructions confuses me, can someone please clarify?

    Shutting Down MFS_FTP

    To shutdown MFS_FTP, FTP to the TiVo (port 3105) and download the "shutdown" file.
    Or, you can telnet into port 3105 and type in exit
    If you kill a tivosh script that accesses the mfs db it forces a reboot. mfs_ftp has a ping_pong "heartbeat" that shuts down the program if /var/mfs_ftp/abort.txt exists. The default interval is 15 mins. Abortcheck is also called whenever a new control channel is opened. So, create a file named /var/mfs_ftp/abort.txt, wait 15 minutes, mfs_ftp will exit, then delete the file abort.txt.

    Perhaps this will be more clear when I crank it up and perhaps not.

    Does mfs_ftp shut down after 15 minutes of inactivity?
    Will my Tivo reboot each time it shuts down?
    Can I just leave it running?
    What if the Tivo reboots, does that hurt anything if I don't have abort.txt?

    Can I get the *****t version???

  9. #1329
    Join Date
    Jun 2006
    Location
    Dougal County
    Posts
    1,015
    Quote Originally Posted by Barefooter View Post
    folders on the HR10 won't group the shows anymore. It's like the keys to the show are lost and the Tivo doesn't know they are related. Any suggestions ?
    http://dealdatabase.com/forum/showthread.php?t=51734

    Quote Originally Posted by Soapm
    Does mfs_ftp shut down after 15 minutes of inactivity?
    Will my Tivo reboot each time it shuts down?
    Can I just leave it running?
    What if the Tivo reboots, does that hurt anything if I don't have abort.txt?
    Mfs_ftp doesn't shut down every 15 minutes. If you start it up, it'll run until you kill it. Creating abort.txt is just one way of shutting it down, doesn't matter if that's ever there or not. I usually just download shutdown.txt.

  10. #1330
    Join Date
    Jan 2004
    Location
    Chebeague Island, Maine
    Posts
    812
    hackman automates that process and makes life a little easier.
    David R. Hill............... TivoWeb Hack Manager
    Chebeague Island........From the land of champions:
    Casco Bay..................The Boston Red Sox
    Maine........................The New England Patriots
    58.5 gig HDR-112 since October, 2000
    240 gig, 306 hours since January, 2004 (same machine)
    120 gig DSR6000 DirecTiVo

    "There are 10 kinds of people -- those who count in binary and those who don't."

  11. #1331
    Join Date
    May 2007
    Posts
    449
    Quote Originally Posted by PortlandPaw View Post
    hackman automates that process and makes life a little easier.

    I was hoping to hear that. Hackman is my friend. So will it appear automatically in Hackman when I start it up or should I delete the config file and let it look again?

  12. #1332
    Join Date
    May 2007
    Posts
    449
    I have mfs_ftp working. However, I can't get it to start automatically when the Tivo starts because the drive has to be RW to start it. I also can turn the drive back to RO while it is running. Is this normal behavior? Should I just put the mount RW command in the author file?

  13. #1333
    Join Date
    Aug 2004
    Posts
    4,085
    Quote Originally Posted by Soapm View Post
    I have mfs_ftp working. However, I can't get it to start automatically when the Tivo starts because the drive has to be RW to start it. I also can turn the drive back to RO while it is running. Is this normal behavior? Should I just put the mount RW command in the author file?
    Put the cache directory, and the port3105.log file somewhere on /var, and symbolicly link them to the root file system directory where you have mfs_ftp installed.

    Either that, or just install it on var and be prepared to refresh it if var should get cleared.

  14. #1334
    Join Date
    May 2007
    Posts
    449
    Quote Originally Posted by Jamie View Post
    Put the cache directory, and the port3105.log file somewhere on /var, and symbolicly link them to the root file system directory where you have mfs_ftp installed.

    Either that, or just install it on var and be prepared to refresh it if var should get cleared.
    How do I undo an existing symlink? Where do they store?

    I had created a link from the root to /var/mfs_ftp so I think I should undo that one first.

  15. #1335
    Join Date
    Jun 2006
    Location
    Dougal County
    Posts
    1,015
    A symlink is stored where ever you made it. Delete it in the usually manner. If you have your mfs_ftp directory in root, then you could do this:
    Code:
    mkdir -p /var/mfs_ftp/cache
    touch /var/mfs_ftp/port.3105.log
    ln -sf /var/mfs_ftp/cache /mfs_ftp/cache
    ln -sf /var/mfs_ftp/port.3105.log /mfs_ftp/port.3105.log

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •