Page 1 of 20 12311 ... LastLast
Results 1 to 15 of 286

Thread: MFS Soapbox Derby: aka what is your download speed?

  1. #1
    Join Date
    Jul 2004
    Location
    California
    Posts
    298

    MFS Soapbox Derby: aka what is your download speed?

    Well, with the help of several people here on DD, in particular Jamie, I have been able to increase my MFS download speeds from 1400 to now over 3300+KB/sec. My best so far is 3349. The key to getting this has been:
    • 2.4.27 backport drivers (here)

    • Unified MFS software (here )

    • usbcore.o loading tweak: (insmod /lib/modules/usbcore.o log2_irq_thresh=4)

    • Installed vwait patch for 4.01b systems (either the tcl patches or the Tivoapp patch located here)


    HDVR2, running 2.4.18 stock kernel and 4.0.1b-02 software. FA-120 USB adapter.

    I allowed my Tivo to settle down after reboot, which is key to getting correct download speed test. (used TOP to wait until no process was taking more than about 8%, which often takes an hour after reboot). I also tuned both tuners to 'blank' channels so that there would be no buffer recording activity (yes, that is not completely real world, but provides a good way to compare apples to apples).

    I have not tried using the recompiled kernel with CONFIG_FILTER and CONFIG_NETFILTER turned off, but that is next. Word is that can give another 10-20%. That might get me above the magic 3500 mark!

    How about we start a little competition to see who can get the best repeatable download speeds, and post what tweaks you made to get them
    Last edited by blueman2; 11-22-2004 at 12:09 AM.

  2. #2
    Join Date
    Aug 2004
    Posts
    4,087
    Here's a new one to beat:

    Code:
    tivo# netperf -H 192.168.2.100 -C -c -- -S 65536 -s 65536
    TCP STREAM TEST to 192.168.2.100
    Recv   Send    Send                          Utilization       Service Demand
    Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
    Size   Size    Size     Time     Throughput  local    remote   local   remote
    bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB
    
    131072 131072 131072    10.00        94.59   98.30    2.72     85.136  4.719
    Code:
    fedora % lftp 192.168.2.107 -p 3105
    ftp 192.168.2.107:~> get '{Late Show With David Letterman}{2004-08-03}{}{04.35AM Wed Sep 19, 2001}{KRQE}.tmf'
    1342184448 bytes transferred in 181 seconds (7.09M/s)
    This is with a $15 usb2 Ali M5632 cable (link). It uses the usbnet kernel module. I bumped the mtu up to 15000. The rest was as blueman2 described, but with the kernel from here.

    I've only been able to get this cable to work linux-to-linux. The windows driver uses a different framing protocol. I think it's fixable, with some work, since the 2.6.9 usbnet.c supports the RNDIS framing. In the meantime, I use a linux box as a tivo-to-ethernet bridge.

    At this point the cpu is clearly the bottleneck.

    The usb host-to-host cables aren't really that practical unless you happen to have a linux PC in close proximity to your tivo.
    Last edited by Jamie; 11-16-2004 at 01:36 AM.

  3. #3
    Join Date
    Jul 2004
    Posts
    89
    Quote Originally Posted by Jamie
    Here's a new one to beat:


    This is with a $15 usb2 Ali M5632 cable (link). It uses the usbnet kernel module. I bumped the mtu up to 15000. The rest was as blueman2 described, but with the kernel from here.

    I've only been able to get this cable to work linux-to-linux. The windows driver uses a different framing protocol. I think it's fixable, with some work, since the 2.6.9 usbnet.c supports the RNDIS framing. In the meantime, I use a linux box as a tivo-to-ethernet bridge.

    At this point the cpu is clearly the bottleneck.

    The usb host-to-host cables aren't really that practical unless you happen to have a linux PC in close proximity to your tivo.
    Is it possible to link two tivos together using this cable? MRV would really be almost real time instead of the current ****ing real time. Each tivo would have it's own ethernet interface and see the USB cable as an Out of Bound interface for MRV backups.
    Magnus Unus

  4. #4
    Join Date
    Aug 2004
    Posts
    4,087
    Quote Originally Posted by my0gr81
    Is it possible to link two tivos together using this cable? MRV would really be almost real time instead of the current ****ing real time. Each tivo would have it's own ethernet interface and see the USB cable as an Out of Bound interface for MRV backups.
    I haven't tried it, but yes it should work. You might have to play around to get the routing just right, so that tivo-to-tivo transfers would use the higher speed links. This does make a lot of sense if you have several tivo's in close proximity to each other, so I take back my "impractical" remark about these cables.
    Last edited by Jamie; 11-16-2004 at 02:48 PM.

  5. #5
    Join Date
    Jul 2004
    Location
    California
    Posts
    298
    Quote Originally Posted by Jamie
    Here's a new one to beat:
    Code:
    fedora % lftp 192.168.2.107 -p 3105
    ftp 192.168.2.107:~> get '{Late Show With David Letterman}{2004-08-03}{}{04.35AM Wed Sep 19, 2001}{KRQE}.tmf'
    1342184448 bytes transferred in 181 seconds (7.09M/s)
    Good heavens! Over 7MB/s! I am curious, what is your speed now with regular networking (like an FA-120) using your tweaked setup? You must be getting near 4.0MB/s.

    I think the max data rate for DirecTV for HDTV is 19 Mbps, or 2.4MB/s. Typical for HBO is 14Mbps or so (under 2.0MB/s). So even with overhead and momentary CPU slowdowns due to other tasks, we are now operating at the range of real time transfer for HDTV, right?

  6. #6
    Join Date
    Aug 2004
    Posts
    4,087
    Quote Originally Posted by blueman2
    Good heavens! Over 7MB/s! I am curious, what is your speed now with regular networking (like an FA-120) using your tweaked setup? You must be getting near 4.0MB/s.
    The best I've seen is ~3.5MB/sec, so not that much different than yours. I've seen ~ 60Mbps in netperf sending tests. I'm still having issues getting good receiving performance, e.g. for MRV transfers. My MRV tivo-to-tivo transfers still run at about 2MB/s, though I haven't tried tivo-to-tivo through the usb host-to-host cables yet.

  7. #7
    Join Date
    Jul 2004
    Location
    Boston
    Posts
    141
    is there a similar thread for 3.1.1? im getting around 1.2meg/sec and would love to get the kind of rates you guys are seeing. FA-120 here as well.

  8. #8
    Join Date
    Aug 2004
    Posts
    4,087
    Quote Originally Posted by robr
    is there a similar thread for 3.1.1? im getting around 1.2meg/sec and would love to get the kind of rates you guys are seeing. FA-120 here as well.
    None of this is specific to 4.X/5.X, as far as I know.

    The 2.4.4 version of the 2.4.27 usb2 backport drivers ought to work with 3.1.1. Grab the latest from post #4 in this thread.

    The unified MFS software should work with all tivo software versions.

    The vwait problem doesn't apply to 3.1.1.

  9. #9
    Join Date
    Jul 2004
    Location
    California
    Posts
    298

    4MB/s!!!

    OK, this is a BIT of a cheat, but I can now get 4MB/s download via ethernet on my HDVR2 with 4.01b-02. Used all the same tweaks outlined in my first post, but finally found a way to temporarily stop myworld on my Tivo using the TvLauncher -StopAll command. SwitcherStart -k was the command for S1 units and it took a bit of trial and error to find the command for S2 units.

    Attached is my BPFTP screen capture. Not the 7MB/s Jamie got with his USB networked Tivo, but pretty good for standard USB2 Ethernet download from Tivo to PC.

    Of course, stopping myworld stops all Tivo functionality, so you only want to do this if you need to download a bunch of shows overnight or something. With myworld stopped, Tivo will not record or even play.
    Last edited by blueman2; 11-18-2004 at 04:50 PM.

  10. #10
    Join Date
    Feb 2004
    Location
    Chicago
    Posts
    877
    Quote Originally Posted by blueman2
    • usbcore.o loading tweak: (insmod /lib/modules/usbcore.o log2_irq_thresh=4)
    Okay, time for my stupid question of the day. Coming from the 3.x world, I'm used to always having to load my drivers manually, via the rc.sysinit.author file. Since 4.x has this capability already built in, this is no longer "technically" necessary.

    However, to achieve the tweaking as described above, one would have to manually load the driver(s). Thus, my question. With 4.x, when/where are the drivers loaded automatically? I'm assuming tivoapp handles this, and therefore, to load the drivers with "tweaked" settings, one would have to "insmod" them prior to tivoapp loading them on its own, i.e. load them early on in the rc.sysinit process before tivoapp gets launched.

    Lastly, if this is the case, is tivoapp smart enough to know that the drivers are already loaded, and therefore, not attempt to load them again? I'm concerned about the possibility of "double loading" drivers.
    1 HR10-250, upgraded to 570 SD hours, hacked, 6.3b.
    1 HDVR2, upgraded to 206 hours, hacked, 6.2.
    1 HDVR2, upgraded to 168 hours, hacked, 6.2.
    tyExtract - Automated batch extraction utility
    YacMon - YAC Server log monitor for new call(s) notification via email/text message

  11. #11
    Join Date
    Jul 2004
    Location
    California
    Posts
    298
    Well, you sure as heck know a lot about 4.0 world!! You are exactly right that in 4.x, Tivoapp loads some (not all) of the network drivers. Normally, this is not an issue. For the backport drivers, the best method is to symlink the driver names that Tivoapp uses into the new drivers you want Tivoapp to use. That way you avoid Tivoapp loading the wrong driver version by mistake.

    And, yes, since Tivoapp loads the drivers, the "insmod /lib/modules/usbcore.o log2_irq_thresh=4" line must come before Tivoapp. What I did was to create a file called test.conf in the root directory (not etc/tc.d). ANy file with this name in the root gets executed before rc.sysinit. It is there that I have my network drivers start. Here is my test.conf:


    echo +++++++++++++ Starting test.conf +++++++++++++++++

    /sbin/insmod /lib/modules/usbcore.o log2_irq_thresh=4
    sleep 2
    /sbin/insmod /lib/modules/ehci-hcd.o
    sleep 2
    /sbin/insmod /lib/modules/usb-ohci.o
    sleep 2
    /sbin/insmod /lib/modules/usbnet.o

    echo ++++++++++++++ Finished test.conf ++++++++++++++++++
    I then load the rest of the drivers (for USB hard drive support, for example) in the rc.sysinit.author file.

    And, no, you do not have to worry about a driver being loaded twice, so long as it is exactly the same driver. It will simply refuse to load if already loaded; no harm done. The only concern is if you left your old drivers in place, and you have two competing drivers (such as ax8817x.o and usbnet.o) getting loaded, which will hose your system.

  12. #12
    Join Date
    Feb 2004
    Location
    Chicago
    Posts
    877
    Perfect. I was hoping this logic made sense. Just wanted to confirm what others were doing.

    I'm already using /test.conf for monte'ing into my custom kernel (test.conf calls check.sh), so, on the second pass through, I'll insmod the drivers at that time, i.e:

    Code:
    if [ "$chainloaded" != "1" ] ; then
        echo "Running monte."
        /sbin/insmod -f /chainload/kmonte.o
        /chainload/monte /chainload/vmlinux.px root=$root console=2,115200 dsscon=true runideturbo=false upgradesoftware=false chainloaded=1
    else
        echo "Chainloaded environment in place, skipping monte."
        #Insmod drivers here
        insmod /lib/modules/usbcore.o log2_irq_thresh=4
        ...etc
        ...etc
    fi
    Just gotta get those speeds you guys are referencing! This is the only thing left to do, i.e. I'm using the new backport drivers, patched tivoapp for vwait, and am using the unified MFS files posted by Jamie with a patched mfs_ftp.tcl file.

    John
    Last edited by JohnSorTivo; 11-18-2004 at 07:36 PM.
    1 HR10-250, upgraded to 570 SD hours, hacked, 6.3b.
    1 HDVR2, upgraded to 206 hours, hacked, 6.2.
    1 HDVR2, upgraded to 168 hours, hacked, 6.2.
    tyExtract - Automated batch extraction utility
    YacMon - YAC Server log monitor for new call(s) notification via email/text message

  13. #13
    Join Date
    Sep 2001
    Location
    West of Bermuda
    Posts
    1,020
    ok, so is 3.5 pretty much the most anyone is getting over ethernet? i seem to be topping out at about 3.15 with this setup:

    hdvr2 or dsr7k, both on 3.1.1c
    config_filter and _netfilter disabled in the kernel
    backport drivers
    unified mfs_*
    log2_irq_thresh=4
    tytool
    usb200m

    the other possible biggie is that my setup requires me to run two 10/100 switches in series (one at the site of the tivos, the other at the pc, although the two are only about 8 feet apart and the cable runs are short). the switches are the cheap linksys, but they do show 100mb and fdx for everything. i've thought about temporarily trying a direct ethernet xover cable between the tivo and the pc to see if that's a factor.

    i've also tried log2_irq_thresh=3 which made transfer speeds marginally slower, and 5 which made them very marginally faster, but i'm not sure i'm comfortable with keeping it at 5.

    comments?

    ronny

  14. #14
    Join Date
    Jun 2004
    Posts
    147
    Quote Originally Posted by ronnythunder
    ok, so is 3.5 pretty much the most anyone is getting over ethernet? i seem to be topping out at about 3.15 with this setup:
    I have peaked at 3.6Mb/s a few times but normally run consistantly around 3Mb/s. The thing is, i have not used the unified_MFS patch yet. Just the backport drivers and the thresh hold adjustment. I am hoping to get the MFS patch loaded this evening. Cant wait to see how that helps..

  15. #15
    Join Date
    Aug 2004
    Posts
    4,087
    Quote Originally Posted by ronnythunder
    i've also tried log2_irq_thresh=3 which made transfer speeds marginally slower, and 5 which made them very marginally faster, but i'm not sure i'm comfortable with keeping it at 5.
    In my experience, larger log2_irq_thresh values tends to help for tx and hurt for rx. I've also added some options to an unreleased usbnet.c that allows you to control the rx and tx queue sizes and the mii mode (full/half duplex, etc). For reasons I don't fully understand, sending seems to like slow interrupts and long queues, while receiving likes the opposite. I also get better receive speeds in half duplex rather than full. Running the released usbnet.o with log2_irq_thresh=4 seems to be a happy medium with reasonable send and receive performance.

    I noticed that a netperf test to localhost can only get 60Mb/sec on a TiVo S2SA, while on my i386 box, it's around 3Gb/sec. That gives you an idea of how much slower the tcp/ip stack runs on the TiVo processor.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •