Tweaking nosdd.tcl to prevent audio skips

[OvitivO]

I want to thank AW for his fine contributions to the Tivo community. nosdd is most definitely a great tool to play with on my extra Tivo.

One thing I have noticed on the T60, however, is that the nosdd script will cause the audio output to go out for about a half-second. I've looked through the tcl code for both nosdd and the reference function from mfslib.tcl, but I really can't determine a more efficient method by which to check the tyDb records.

Nevertheless, I am not a tcl guru, but I have thought of a few things:

Would it help any if there was a slight pause while checking each record? Or, is the opening of the database causing the most overhead?

I'll keep looking at the code, but if anyone has any additional recommendations, please feel free to share.


Thanks!

[AlphaWolf]

I considered doing this, but I don't get skips so I won't be able tell weather or not its effective. As I said earlier, tivoweb is able to parse the todo list, and I don't think this causes freezing for anybody, but I can't tell for certain. If it doesn't cause you any freezing when you look at the todo list in tivoweb, let me know, and I will look at what they are doing differently.

As a temporary solution, see if adding 'after 250' before the last bracket helps any. That will create a 250ms delay between each loop, but it will make the script take much longer to complete (and possibly not cancel the SDD in time).

[OvitivO]

By commenting out everything except the top of the script, I still get the brief drop in A/V.

e.g.:



#!/tvbin/tivosh

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

proc strim {str} { return [string trim $str "\{\} "] }

set db [dbopen]



(I'll focus on this area for now.)

Still testing different things -- let me know if you think of anything else.

Thanks!

[OvitivO]

OK, I've narrowed it down to "set db [dbopen]"

This makes sense, but now I'm at a loss to debug/improve this process.


:confused:

[AlphaWolf]

This means that you have a very filled up database. Almost any script you run will freeze when you first start it, unless you clean out your tivo first.

Since cleaning out your tivo regularly isn't how a non hacked tivo should have to function, going about it that way is a bad idea, so we have two options; find an alternative way of opening the database, or have nosdd loop infinitely. I don't like the latter, but I am not enough of a TCL guru to know how to work around the prior.

MrBlack51 and I were playing around with tivotitle to fix the freezing that it gets, he said he found a way to get a handle on the database without it freezing, but I never got back to him on that.

Meanwhile I suppose I will look into it.

EDIT: try this just for a test, run tivosh, then just type 'dbopen'. If your audio doesn't freeze, then that means we have to slow things down at the start.

[mrblack51]

here is the edited TivoTitle that AW and I worked on. If I am really paying attention, I can hear a micro-skip in the audio when the dbopen occurs, but its a whole lot better than it was with the version I was using. If it helps anyone, great.

As per the readme, version number has been incremented and comments added.

[OvitivO]

So far, this modification appears to be working, and although there is a very, very slight hiccup in audio, it seems to be much better.

I'd like to get some confirmation about this still obtaining the end result of removing SDDs. Thanks!


#!/tvbin/tivosh
# SDD unscheduler by AlphaWolf_HK, mrblack51

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

proc strim {str} { return [string trim $str "\{\} "] }

# initializing db now, due to A/V glitch issue

set dbPoolSize [expr 100 * 1024]

exec sleep 1
puts stdout "set dbopen"
set db [dbopen $dbPoolSize]
puts stdout "done with set dbopen"

# check tyDbs

ForeachMfsFile fsid name type /Recording/Active "" {
if { $type == "tyDb" } {
RetryTransaction {
set rec [db $db openid $fsid]
set showing [dbobj $rec get Showing]
set program [dbobj $showing get Program]
set title [strim [dbobj $program get Title]]
set state [dbobj $rec get State]

if {$state == 6 && $title == "TiVo Service Data"} {
puts "Found $title at FSID: $fsid and unscheduling it."
dbobj $rec set State 2
}
}
}
}

[AlphaWolf]

That would work, though you don't need all of that...try this (in fact this might work better):


#!/tvbin/tivosh

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

setpri fifo 1
after 1000
set db [dbopen 102400]

ForeachMfsFile fsid name type /Recording/Active "" {
if { $type == "tyDb" } {
RetryTransaction {
set rec [db $db openid $fsid]
set showing [dbobj $rec get Showing]
set program [dbobj $showing get Program]
set title [dbobj $program get Title]

if {$title == "{TiVo Service Data}"} {
dbobj $rec set State 2
}
}
}
}


Let me know how it works so I can make a more formal release in the other thread.

EDIT: If it still skips, try lowering the number after dbopen til it stops.

[OvitivO]

That definitely appears to be helping w/ the A/V skips.

Just wondering, however. Was a pool size of 102400 arbitrarily picked?

Thanks again guys -- I may be able to run this a lot more often to catch the smaller files. (I did notice a few SUCCESS entries in svclog that were about 2 minutes apart a few days ago.)

:D

[AlphaWolf]

Er...hold everything for a sec....Embeem pointed out to me that nice does nothing for a tivo, and setpri should eliminate all skip problems, period. I know that most people here are using 25xtreme...so that brings a question, does 25xtreme include setpri? Reason I ask is because I have been using 'setpri fifo 1' for all of my hacks, and none of them have ever caused any freezing, at all.

Try this, at bash type: 'setpri' (no quotes)

If you see "command not found", then you DON'T have it, or haven't installed it properly, hence why you still see skipping all the time, even if you use 'setpri fifo 1' in your rc.sysinit.

Download setpri from the below link, put it in /bin or somewhere in your path entry, then on the line before you call cron in your rc.sysinit, add this line:

setpri fifo 1

Try using the origional nosdd script that way. It shouldn't freeze at all.

http://tivo.samba.org/download/mbm/bin/

[almontebarnes]

Originally posted by AlphaWolf
...so that brings a question, does 25xtreme include setpri?


No, it does not. At least it is not in any of the directories in the standard 25Xtreme path

[OvitivO]

Yup, setpri really helps out. I ran this nosdd script every minute for about an hour and didn't notice any A/V issues.

Thanks again to everyone.

The final script I ended up using:


#!/tvbin/tivosh

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

setpri fifo 1
after 1000
set db [dbopen 102400]

ForeachMfsFile fsid name type /Recording/Active "" {
if { $type == "tyDb" } {
RetryTransaction {
set rec [db $db openid $fsid]
set showing [dbobj $rec get Showing]
set program [dbobj $showing get Program]
set title [dbobj $program get Title]
set state [dbobj $rec get State]

if {$title == "{TiVo Service Data}"} {
puts "Found $title at FSID: $fsid and unscheduling it."
dbobj $rec set State 2
}
}
}
}


(I decided to keep the unscheduling details for logging purposes.)

fwiw, my test platform is a T60

[OvitivO]

For those that are interested, here's my nosdd log. I run the script every 2 minutes, and i thought this would be of interest to those that were not sure how often the feed attempts to send the sdd through the stream.

(I'll set up item-specific timestamps soon, if this is hard to read.)

[...]
Fri Aug 30 06:00:00 UTC 2002
Found {TiVo Service Data} at FSID: 841194 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Fri Aug 30 07:00:00 UTC 2002
Found {TiVo Service Data} at FSID: 850349 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 850373 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 08:00:00 UTC 2002
retrying after errTmActiveLockConflict ...
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 850384 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
retrying after errTmActiveLockConflict ...
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 850393 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 09:00:00 UTC 2002
Found {TiVo Service Data} at FSID: 850394 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
retrying after errFsLockConflict ...
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Found {TiVo Service Data} at FSID: 850405 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 10:00:00 UTC 2002
Found {TiVo Service Data} at FSID: 850422 and unscheduling it.
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
retrying after errTmActiveLockConflict ...
retrying after errTmActiveLockConflict ...
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 11:00:01 UTC 2002
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 12:00:00 UTC 2002
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
retrying after errTmActiveLockConflict ...
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
retrying after errTmActiveLockConflict ...
Fri Aug 30 13:00:01 UTC 2002
Found {TiVo Service Data} at FSID: 845008 and unscheduling it.
Found {TiVo Service Data} at FSID: 847853 and unscheduling it.
Found {TiVo Service Data} at FSID: 850374 and unscheduling it.
Fri Aug 30 14:00:00 UTC 2002
[...]

errFsLockConflict and errTmActiveLockConflict also threw up a few flags (or at least piqued my interest).

[genericuser]

Does your post indicate that out of 30 runs per hour only the run at the top of the hour (XX:00) is doing any work?

Or am I missing something?

[OvitivO]

The times are only hourly place markers. To alleviate the potential for confusion, I modified the script to give a more accurate description of the time at which each fsid was unscheduled.


#!/tvbin/tivosh

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

setpri fifo 1
after 1000
set time [clock format [clock sec] -format %Y-%M-%H:%M:%S]
set db [dbopen 102400]

ForeachMfsFile fsid name type /Recording/Active "" {
if { $type == "tyDb" } {
RetryTransaction {
set rec [db $db openid $fsid]
set showing [dbobj $rec get Showing]
set program [dbobj $showing get Program]
set title [dbobj $program get Title]
set state [dbobj $rec get State]

if {$title == "{TiVo Service Data}"} {
puts "$time :: Found $title at FSID: $fsid and unscheduling it."
dbobj $rec set State 2
}
}
}
}


You all may find [clock] unnecessary, but it gives me a little bit more detail about the sdds.

[AlphaWolf]

If you want timestamps in your logging, you might try it like this:


#!/tvbin/tivosh
# SDD unscheduler by AlphaWolf_HK

EnableTransactionHoldoff true
tvsource $tcl_library/tv/mfslib.tcl

after 250
set db [dbopen 10240]

ForeachMfsFile fsid name type /Recording/Active "" {
if { $type == "tyDb" } {
RetryTransaction {
set rec [db $db openid $fsid]
set showing [dbobj $rec get Showing]
set program [dbobj $showing get Program]
set title [dbobj $program get Title]

if {$title == "{TiVo Service Data}"} {
puts "Found $title in FSID: $fsid on [exec date] and unscheduling it."
dbobj $rec set State 2
}
}
}
}


Read the post I made to the "tips of the trade" sticky thread to find out what you should do with setpri.

EDIT: ah n/m, thought you were doing a puts before the loop, my bad.

[jfkjfk]

Where is the log, or how do I enable logging of the nosdd?

I'm no linux guru..
I do have nosdd running via cron..
seems to be working I haven't taken a service download since the 23rd... that's when I got it setup. :)


thanks..

jfkjfk

[AlphaWolf]

You will have to instruct sh to log its output. You can do that by making this be the command line:

/hack/nosdd.tcl >> /var/log/nosdd.log

Also its been pointed out that nosdd might be fully effective if you have it run every minute (crontab entry * * * * * ), though I haven't yet confirmed this, but it sounds logical as I notice the "Finnished" status SDDs are at odd minutes. (like 4:14).

[jfkjfk]

/hack/nosdd.tcl >> /var/log/nosdd.log
Ok I should have known that one..
Changed my cron line..
So you guys are suggesting running every minute?
a crontab line like

0-59 * * * * /hacknosdd.tcl >> /hack/cron.log &

or actaully would

* * * * * /hacknosdd.tcl >> /hack/cron.log &

be the same thing?

thanks..

jfkjfk

[genericuser]

* * * * * Will run it every minute.

It looks like your command line is a bit weird. Did you mean /hack/nosdd.tcl??

Also, I would call the log filesomething more meaningful like:
/hack/nosdd.log

Lastly... No need for the & at the end of the line.

Here is the entry I would use for you:

* * * * * /hack/nosdd.tcl >> /hack/nosdd.log
00 00 * * * rm /hack/nosdd.log

The second line keeps only 24 hours of log file before deleting it. This way your log file does not grow without bound.

[jfkjfk]

hey Genericuser,
thanks for the reply...

Did you mean /hack/nosdd.tcl??
yep typo..
/hack/nosdd.log
already changed that.. but good idea..

also realized that in order for logs to be writable after remounting the root ro.. the logs must be in the /var directory... so I've moved them there..

Lastly... No need for the & at the end of the line.
I thought it was unnecessary.. but I'm glad that you've confirmed this...

00 00 * * * rm /hack/nosdd.log

VERY GOOD idea.. I had actually thought about something like this.. but have not done so.. but I WILL...

thanks again...

jfkjfk