Results 1 to 13 of 13

Thread: tivosh CPU usage in v4+

  1. #1
    Join Date
    Nov 2003
    Posts
    291

    tivosh CPU usage in v4+

    I've been doing some investigation into the CPU usage, and have come to some conclusions:

    1) This problem affects all event-driven tivosh scripts... if it calls vwait, it's gonna spin

    2) The CPU usage is not just an artifact, and unless some tricky manipulation is done with process priorities, it will have an affect, especially on other tivosh scripts

    Has anyone found a way to write an event-driven script that doesn't spin? (other than running a wait/update loop)


    Note: this investigation was done on a 5.x machine... if 4.x behaves differently, please post the info here.

  2. #2
    Join Date
    Jun 2001
    Posts
    3,108
    Quote Originally Posted by BTUx9
    I've been doing some investigation into the CPU usage, and have come to some conclusions:

    1) This problem affects all event-driven tivosh scripts... if it calls vwait, it's gonna spin

    2) The CPU usage is not just an artifact, and unless some tricky manipulation is done with process priorities, it will have an affect, especially on other tivosh scripts

    Has anyone found a way to write an event-driven script that doesn't spin? (other than running a wait/update loop)


    Note: this investigation was done on a 5.x machine... if 4.x behaves differently, please post the info here.
    5.x machine? as in a dvd unit, or non-dvd unit running 5.x software.
    Step one: search button!
    Silly Wabbit, guides are for kids

  3. #3
    Join Date
    Nov 2003
    Posts
    291
    Quote Originally Posted by mrblack51
    5.x machine? as in a dvd unit, or non-dvd unit running 5.x software.
    DVD unit. It's not mine... someone I helped graciously gave me telnet access. It's a Toshiba, I think. BOY it has a lot of processes running.

  4. #4
    Join Date
    Aug 2003
    Posts
    1,285
    Quote Originally Posted by BTUx9
    I've been doing some investigation into the CPU usage, and have come to some conclusions:
    Check the TivoWebPlus thread. It has a good explaination of what is really happening with regards to tivosh and CPU usage.

    S.

  5. #5
    Join Date
    Nov 2003
    Posts
    291
    Quote Originally Posted by Sleeper
    Check the TivoWebPlus thread. It has a good explaination of what is really happening with regards to tivosh and CPU usage.
    I read falcontx's post, and he's got some wrong assumptions (unless v4.x is VERY different than 5.x)

    1) having a process spinning would not increase CPU temp.

    2) the script isn't running at idle priority when spinning, it's at whatever priority the rest of the script runs at

    3) top's cpu usage reporting is, by its very nature, twitchy... the script really IS grabbing the cpu whenever it's allowed to, at its priority, but being at a low priority, other background tasks make that % fluctuate

    4) the spinning DOES affect other processes, just not the native tivo ones, because they run at much higher priority. It especially affects other tivosh scripts, bash, telnet, etc. When 2 spinning tasks are running at the same priority, it affects the system more, because it forces constant context switches. The reason this isn't apparent to more people is that the biggest time-waster in most scripts is mfs access, which probably runs at the tivo utilities' higher priority (not 100% sure about that, tho)

    The test for this is trivial. Just write a proc with busy-work in it, that takes between 100 and 1000 ms, repeatedly time it and report the results, then, in another bash window, run a script that has a simple vwait in it. it immediately doubles the timings. If you check with top, it shows that the waiting task is using basically the same CPU as the actively running timing task. Not Good. To prove that it's not at idle priority, run the timing task using nice... the slowdown is extreme. If this same test is run on 3.x, the waiting task has almost NO affect on the timing task (after the initial load).

    I've included a pair of scripts, for those of you who may want to check it out. They will abort when they detect a file named abort in the current directory.

    If you run these on a 4.x system, please post the results, I want to make sure this isn't JUST a 5.x issue.

    ALSO, I put in the abort check because, even tho they don't access mfs, using ^C on these scripts panicked the tivo (5.x). Can anyone confirm whether that's true for 4.x, i.e. does killing a non-mfs script in 4.x cause a reboot?
    Attached Files Attached Files

  6. #6
    Join Date
    Jan 2002
    Posts
    1,778
    This is also a 4.x issue. It appears that the vwait code has been reworked somewhat in 4.0+. Checking the tivoapp disassembly against the tcl source code shows some unusual stuff that I don't quite follow.

    Normally, vwait will call select() with a nonzero, non-NULL delay timeval - 1ms IIRC. On 4.0+ the delay is zero (0 seconds + 0 us), which returns immediately. As a result, the process is probably wasting even more cycles than we know about because it is spinning in both user mode and kernel mode, and constantly switching contexts. This also means that the "timingtest" script does not accurately model the spinning behavior we see with vwait, since it is a simple userland busy-wait loop.

    There are modifications that can be made which change the delay back to 1ms, but until the ramifications of that change are understood I'd rather not post it.

  7. #7
    Join Date
    Nov 2003
    Posts
    291
    Quote Originally Posted by alldeadhomiez
    This also means that the "timingtest" script does not accurately model the spinning behavior we see with vwait, since it is a simple userland busy-wait loop.
    Actually, the timingtest script is intended to show the affect on a non-event-driven script, which is why I kept away from kernel-space. And, as I said earlier, I'm even more worried about 2 or more scripts "fighting" because of the constant context switches.

    Are any of the tivo background tasks tcl-based, and if so, do you know what mechanism they are using instead of vwait (because, obviously, they are not spinning).

    Also, any idea if this is a bug, or an example of crippleware (tivo vs. hackers)?

    PS: thank you for coming forward to back up my findings. All the responses I'd seen to this issue previously had been on the order of "it's an artifact... it doesn't really affect the tivo"
    Last edited by BTUx9; 04-16-2004 at 07:04 PM.

  8. #8
    Join Date
    Jan 2002
    Posts
    1,778
    Quote Originally Posted by BTUx9
    Actually, the timingtest script is intended to show the affect on a non-event-driven script, which is why I kept away from kernel-space.
    Ah, okay.

    Are any of the tivo background tasks tcl-based, and if so, do you know what mechanism they are using instead of vwait (because, obviously, they are not spinning).
    I don't believe so. I think they prefer to avoid resident tcl scripts, as they are resource hogs. At least that was what they claimed in the comments in the "stub" tclient script that shipped with 3.1.0 on the Series 1.

    Also, any idea if this is a bug, or an example of crippleware (tivo vs. hackers)?
    My guess would be indifference.

    All the responses I'd seen to this issue previously had been on the order of "it's an artifact... it doesn't really affect the tivo"
    I think most of the TiVo software runs with fifo/rr "real time" priorities so it might not be affected too badly, but this problem annoys me because it tends to slow down the other low-priority stuff I run on the box.

  9. #9
    Join Date
    Nov 2003
    Posts
    1,754
    Quote Originally Posted by alldeadhomiez
    This is also a 4.x issue. It appears that the vwait code has been reworked somewhat in 4.0+. Checking the tivoapp disassembly against the tcl source code shows some unusual stuff that I don't quite follow.

    Normally, vwait will call select() with a nonzero, non-NULL delay timeval - 1ms IIRC. On 4.0+ the delay is zero (0 seconds + 0 us), which returns immediately. As a result, the process is probably wasting even more cycles than we know about because it is spinning in both user mode and kernel mode, and constantly switching contexts. This also means that the "timingtest" script does not accurately model the spinning behavior we see with vwait, since it is a simple userland busy-wait loop.

    There are modifications that can be made which change the delay back to 1ms, but until the ramifications of that change are understood I'd rather not post it.
    Could a different call be written to do what vwait is supposed to and then have the scripts modified to avoid vwait.
    Malfunct

    HDVR2 - 120hours - Extraction enabled
    SD-DVR40 - Unhacked (for now)

  10. #10
    Join Date
    Nov 2003
    Posts
    291
    Quote Originally Posted by malfunct
    Could a different call be written to do what vwait is supposed to and then have the scripts modified to avoid vwait.
    Well, as a stopgap measure, "vwait x" could be replaced by the following code, in almost all scripts, with no other modifications. The delay could be adjusted for responsiveness vs. CPU usage, and with some fairly minor changes to the original script, it could be made more efficient.
    Code:
    set y $x
    while {$x == $y} {
      update
      after 400 }
    For some scripts, where vwait is designed never to return (event-driven routines call exit), this could be written as while 1 {

  11. #11
    Join Date
    Nov 2003
    Location
    San Jose, CA
    Posts
    18
    wow, chaging vwait reload in TWP to the update after 400 code drastically reduces TWP's CPU usage. TWP cpu is now ~5% vs the previous 66%+. I'm guessing less CPU usage will mean less heat given off.

  12. #12
    Join Date
    Nov 2003
    Posts
    1,754
    Quote Originally Posted by edy
    wow, chaging vwait reload in TWP to the update after 400 code drastically reduces TWP's CPU usage. TWP cpu is now ~5% vs the previous 66%+. I'm guessing less CPU usage will mean less heat given off.
    Could the people maintaining TWP please implement this fix in an upcoming version now that we know the causes of the CPU usage, the disadvantages to the use, and more to the point how to eliminate it.
    Malfunct

    HDVR2 - 120hours - Extraction enabled
    SD-DVR40 - Unhacked (for now)

  13. #13
    Join Date
    Nov 2003
    Posts
    291
    Quote Originally Posted by edy
    wow, chaging vwait reload in TWP to the update after 400 code drastically reduces TWP's CPU usage. TWP cpu is now ~5% vs the previous 66%+. I'm guessing less CPU usage will mean less heat given off.
    Not a change in heat, but the software should run happier (both hacks and tivo).

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •