Results 1 to 9 of 9

Thread: NpkChannelDefinition

  1. #1
    Join Date
    Nov 2003
    Posts
    36

    NpkChannelDefinition

    (This sort of rambles and goes nowhere for a while.)

    (Urgh... I have to specify fonts by name? I chose "Monaco,Lucida Console,monospace" for fixed-pitch text.)

    This saga started a couple of months ago, when I decided to do a capacity upgrade on one of my two TCD240040 boxes to make room for lots of festive stuff.

    Since it had been a while, I used Upgrade Configuration No. 3 from Hinsdale's How-To. The new drive worked fine with the exception that the background video on the (red) Now Playing screen had a few frames of the (green) TiVo Central background video once in every loop. I tried copying the drive again, but the problem persisted. I'd been unable to get a straight dd copy to work following mfsadd, so I made do with the glitchy background video.

    The two TCD240040 boxes had been Pending Restart from 9.1ish (perhaps 9.2ish) to various new versions, ending up with 9.3.2a, for a few months. Several reasons (among them the nightly reboots, version envy, hope that the background video might be reinstalled, and YouTube (silly me—when TiVo sent me a message about YouTube on TiVo on my Series2 box, I thought that meant YouTube on TiVo would be available on my Series2 box)) motivated me to finally take the upgrade.

    When there was finally nothing I'd want to record due to be on for a couple of days, I performed the upgrade on the other pre-9.3.2a box, "Basement S2 SA". installSw.itcl, neutered kernel, network drivers, iptables, /etc/fstab (for my /hack partition), tvapppatch, /etc/rc.d/rc.sysinit.author, cross fingers, uncross fingers, check bootpage, recross fingers, and reboot.

    Everything was fine on this first box, so I performed the OS upgrade on the box with the glitchy background video on the Now Playing List, "Spare". Again, everything was fine.

    But when I did a MRV transfer of a TTCB programme, I found that the actors, description, and most of everything was no longer being transferred with the programme. I could read them on the Program (Basement S2 SA) screen, but only the Recording/@Showing/@Date, Recording/@Showing/@Time, and Recording/@Showing/@Program/@Title were transferred, along with a link to the corresponding Series record (presumably via the Recording/@Showing/@Program/@Series/@TmsId field). [The @ signs indicate field names, as opposed to MFS paths; I am abusing XPath notation here.]

    Further experimentation showed that adding a value, even a bogus one, to the Recording/@Showing/@Program/@TmsId field caused the Recording/@Showing/@Program/@Description field to be transferred. (This is also the behaviour I see with a programme actually recorded by TiVo on Basement S2 SA and transferred to Spare.) (This also reminds me of the Recording/@Showing/@Program/@IsEpisode field controlling whether or not the year of the Recording/@Showing/@Program/@OriginalAirDate is shown on the Program Details screen.)

    Anyway, I remembered the difficulty I had trying to MRV TTCBed programmes until I removed the Recording/@AuxiliaryData field, saw the Recording/@NpkChannelDefinition field, noted that I'd meant to decode it every time I'd seen it for years now, and decided to have a look at it.

    NpkChannelDefinition

    The first thing I thought was that I wanted a hex/ASCII dump of the field value. Perl to the rescue:

    Code:
    $ echo '184 50344978 -1602223616 -1164204796 -1476395008\
     256 1836321280 825309811 3356983 167772161 282 66048 196608\
     65588 1459945472 5919824 1 1128418830 1717977376 1634298985\
     25972 1 65544 20 302055424 196660 21797890 12888343 3473408\
     302120960 196660 21666818 1509363 17432576 58851328 -1476132864\
     3412480 -1677590525 -421658295 203 263 65792 1 257 66304 -4172542 16777215'\
     | perl -ne 'print join " ", map {unpack("H*", pack("N", $_))} split, "\n";'
    000000b8 03003412 a0800200 ba9ba504 a8000000 00000100 6d740a00 31313a73
    00333937 0a000001 0000011a 00010200 00030000 00010034 57050000 005a5450
    00000001 43424e0e 66664120 61696c69 00006574 00000001 00010008 00000014
    12010000 00030034 014c9c02 00c4a917 00350000 12020000 00030034 014a9c02
    001707f3 010a0000 03820000 a8040000 00341200 9c020003 e6de0149 000000cb
    00000107 00010100 00000001 00000101 00010300 ffc05502 00ffffff 00000000
    Some of that looks promising: 333937 is "397" in ASCII; 5a5450 is "ZTP"; 43424e is "CBN"; 666641 is "ffA", 61696c69 is "aili", and 6575 is "et". So flip the endianness and add character output:

    PHP Code:
    #!/usr/bin/perl -w

    use strict;

    sub safe_chr {
        
    my $val shift;
        return 
    chr $val if $val >= 0x20 and $val <= 0x7f;
        return 
    ".";}

    while (<>) {
        
    my @vals split;
        foreach 
    my $val (@vals) {
            
    $val substr sprintf("%08x"$val), -8;  # or unpack "H*", pack("l<", $val)
            
    $val =~ s/(..)(..)(..)(..)/$4.$3.$2.$1.
                
    " (" safe_chr(hex $4) . safe_chr(hex $3)
                     . 
    safe_chr(hex $2) . safe_chr(hex $1)
                     . 
    ")\n"/e;
        }
        print 
    join "", @vals"\n";

    and

    Code:
    $ echo '184 50344978 -1602223616 -1164204796 -1476395008\
     256 1836321280 825309811 3356983 167772161 282 66048 196608\
     65588 1459945472 5919824 1 1128418830 1717977376 1634298985\
     25972 1 65544 20 302055424 196660 21797890 12888343 3473408\
     302120960 196660 21666818 1509363 17432576 58851328 -1476132864\
     3412480 -1677590525 -421658295 203 263 65792 1 257 66304 -4172542 16777215'\
     | ./npk/decode_list_of_signed_int32s
    b8000000 (....)
    12340003 (.4..)
    000280a0 (....)
    04a59bba (....)
    000000a8 (....)
    00010000 (....)
    000a746d (..tm)
    733a3131 (s:11)
    37393300 (793.)
    0100000a (....)
    1a010000 (....)
    00020100 (....)
    00000300 (....)
    34000100 (4...)
    00000557 (...W)
    50545a00 (PTZ.)
    01000000 (....)
    0e4e4243 (.NBC)
    20416666 ( Aff)
    696c6961 (ilia)
    74650000 (te..)
    01000000 (....)
    08000100 (....)
    14000000 (....)
    00000112 (....)
    34000300 (4...)
    029c4c01 (..L.)
    17a9c400 (....)
    00003500 (..5.)
    00000212 (....)
    34000300 (4...)
    029c4a01 (..J.)
    f3071700 (....)
    00000a01 (....)
    00008203 (....)
    000004a8 (....)
    00123400 (..4.)
    0300029c (....)
    4901dee6 (I...)
    cb000000 (....)
    07010000 (....)
    00010100 (....)
    01000000 (....)
    01010000 (....)
    00030100 (....)
    0255c0ff (.U..)
    ffffff00 (....)
    Now we're cooking!

    Some of you will be ahead of me here, but I went in and worked it out by hand. The basic unit of information is what I call a can_has: either a 0x00 or { a 0x01 followed by data }. Sometimes that data starts with an int32_t giving the length of what follows; sometimes it's a fixed size.

    A brief sidetrack into the meaning of Block vs. Syndicated vs. Local vs. Network programming took me to (WO/2003/007596) ELECTRONIC PROGRAM GUIDE FOR PROCESSING CONTENT-RELATED INFORMATION CONFIGURED USING A REFERENCE INFORMATION MODEL.

    Once I'd worked most of it out (with some extra data from URL:http://dealdatabase.com/forum/sh...t=50893&page=8 confusing me somewhat), I thought I'd see if TiVo had left any documentation for me.
    Code:
    bash-2.02# grep -i npkchanneldefinition\
     /tvlib/* /tvlib/*/* /tvlib/*/*/* /tvlib/*/*/*/*\
     /tvlib/*/*/*/*/* /tvlib/*/*/*/*/*/*
    screwed up my terminal [fix: echo -e '\x0f'] but gave some hits in /tvlib/idl/, somewhere I had theretofore ignored.

    [To be continued...]

  2. #2
    Join Date
    Nov 2003
    Posts
    36
    Google for "idl 12340003" turned up http://dealdatabase.com/forum/showth...t=41852&page=2, and ADH's first message—
    Quote Originally Posted by alldeadhomiez
    Right - as I mentioned, software 7.1 has this built into tivosh
    —on that page got me to look in /tvlib/, where I found /tvlib/tcl/tv/TvIdl.tcl.

    So there's a much easier way to decode these things:

    Code:
    % tvidl dbtotext 0 {184 50344978 -1602223616 -1164204796
     -1476395008 256 1836321280 825309811 3356983 167772161 282
     66048 196608 65588 1459945472 5919824 1 1128418830 1717977376
     1634298985 25972 1 65544 20 302055424 196660 21797890 12888343
     3473408 302120960 196660 21666818 1509363 17432576 58851328
     -1476132864 3412480 -1677590525 -421658295 203 263 65792 1
     257 66304 -4172542 16777215}
    TvNpkChannelDefinition {
        idChannel: << not present >>
        idGuideSource: "tms:11793" 
        channelBits: 2586 (0x00000a1a) (*unknown*)
        eSourceType: 2 (0x00000002) (CABLE)
        channelNumber: 
            TvBusChannelNumberV2 {
                major: 52 (0x0034) 
                minor: << not present >>
            } TvBusChannelNumberV2
        strCallSign: "WPTZ" 
        strLongName: "NBC Affiliate" 
        strDescription: << not present >>
        logo: 
            TvNpkChannelLogo {
                iLogo: 65556 (0x00010014)
                iBackupLogo: 0 (0x00000000)
            } TvNpkChannelLogo
        staleInfo: << not present >>
        networkInfo: TvBusAnyStruct
            {
                TvNpkChannelDefinitionNetworks {
                    vNetworks: [ TvBusAnyStruct
                        {
                            TvNpkChannelDefinitionExternalAnalog {
                                idTuningSignalSource: 142949396513960 (0x00008203000004a8) {33283/1192}
                                idOriginatingSignalSource: << not present >>
                            } TvNpkChannelDefinitionExternalAnalog
                        }
    TvBusAnyStruct
                        {
                            TvNpkChannelDefinitionTMS {
                                eServiceTier: 1 (0x00000001) (BASIC)
                                fLocalOnly: false 
                            } TvNpkChannelDefinitionTMS
                        }
    ] 
                } TvNpkChannelDefinitionNetworks
            }
    
        eGuideProvider: 1 (0x00000001) (TMS)
        eConnector: 3 (0x00000003) (COMPOSITE)
        idStation: 657237370470399 (0x000255c0ffffffff) {153024/-1}
        homeChannelNetworkIndex: << not present >>
    } TvNpkChannelDefinition
    And for good measure:

    Code:
    % tvidl struct TvNpkChannelDefinition -info
    idChannel {optional {unsigned long long}}
     idGuideSource {optional string}
     channelBits {optional enum TvNpkChannelBitField}
     eSourceType {optional enum TvNpkChannelSignalSourceType}
     channelNumber {optional struct TvBusChannelNumberV2}
     strCallSign {optional string}
     strLongName {optional string}
     strDescription {optional string}
     logo {optional struct TvNpkChannelLogo}
     staleInfo {optional anystruct}
     networkInfo {optional anystruct}
     eGuideProvider {optional enum TvNpkChannelGuideProvider}
     eConnector {optional enum TvNpkInputConnectorType}
     idStation {optional {unsigned long long}}
     homeChannelNetworkIndex {optional long}
    and

    Code:
    % tvidl enum TvNpkChannelGuideProvider -info
    UNKNOWN 0 TMS 1 DIRECTV 2 GEMSTAR 3
    So now I can get channel numbers and callsigns in my inserted programmes, as well as logos if they're installed. As a bonus, the NpkChannelDefinition fields I've constructed remove the "Video: Unknown Quality" line; I believe this to be because I set the DIGITAL flag (0x0800) in the TvNpkChannelDefinition/@channelBits field, and so I believe this flag to indicate that the channel is recorded directly as a digital bitstream, rather than from an analogue source.

    My guess for the need for an NpkChannelDefinition field in a SeasonPass record is for quick access to the TmsId field, Station record, and SignalSource record.

    As for Recording/@AuxiliaryData:

    Code:
    % tvidl dbtotext 0 {35 50344978 -761593344
     -933553663 318767104 251658240 1702060354 1953391981 540168992 16723}
    TvDbRecordingNetworkTransferAuxData {
        friendlyServiceName: "Basement S2 SA" 
    } TvDbRecordingNetworkTransferAuxData
    Of course, I've also seen dumpobj tell me -prettystructs prints marshalled structures in an unmarshalled
    form (-ps)
    many a time. And what does /tvlib/tcl/tv/dumpobj.tcl have to say about TiVo's marshalling protocol?

    Code:
    proc FMarshalledStruct { type value } {
        # Marshalled structs are represented as int arrays with some special
        # characteristics.  So first of all the basic type must be int.
        if { ($type == "int") } {
            # Now we look at the first element in the integer array.  It is the
            # length, in bytes, of the entire array (minus 5, who knows why?!?),
            # if it's a marshalled struct -- ACTUALLY, this turns out not to be
            # true, so we don't check for that.
            set countVal [lindex $value 0]
            set lenVal [llength $value]
            # And also, the first value is a special magic number which identifies
            # a marshalled struct
            if { ([lindex $value 1] == 50344978) } {
                 return 1
            }    
        }
    
        return 0
    }
    (The first element appears to be the length, in bytes, of the entire array, minus 4 for the length value, minus the number of padding bytes needed to get the whole thing up to a multiple of 4 bytes. So it should be length_in_bytes - {4..7}.)

    Hmmm... maybe they took some fields out of the idl files that describe parts of the MRV metadata transfers. And maybe put some dependencies in, like Recording/@Showing/@Program/@TmsId required to get the Recording/@Showing/@Program/@Description transferred.

    To routerplus?

    (And lookee what I just found here:
    Code:
    % tvidl struct TvDbSeasonPass -info
    objectName string
     dayOfWeekLocal {sequence long}
     duration {optional long}
     endTimePadding {optional long}
     keepTime {optional long}
     maxRecordings {optional long}
     npkChannelDefinition {sequence struct TvNpkChannelDefinition}
     presentationFeature {sequence enum TvDbPresentationFeature}
     priority {optional long}
     recordingQuality {optional enum TvDbRecordQuality}
     series {optional struct TvDbSeries}
     showStatus {optional enum TvDbShowStatus}
     startTimeLocal {optional long}
     startTimePadding {optional long}
     station {optional struct TvDbStation}
     theme {optional struct TvDbTheme}
     type {optional enum TvDbSeasonPassType}
     auxiliaryData {optional struct TvBusMarshalledStruct}
     logoIndex {optional long}
     uiType {optional long}
     fUseShowingEndPadding {optional boolean}
    So they've at least thought about having Season Passes that spread across multiple channels?)

  3. #3
    Join Date
    Nov 2004
    Posts
    420
    Great post! This works for a many other values stored in MFS.

    Code:
    % tvidl dbtotext 0 {53 50344978 2073756160 -483896316 620756992 0 16777216 419430400 65536 285736960 640985883 5796085 16777216 16777472 0}
    TvCopyProtectionInfo {
        vSegment: [ ]
        firstViewingStart: << not present >>
        baseStartTime: << not present >>
        recordingDuration: << not present >>
        fairUseRules:
            TvPvrFairUseRules {
                deleteAfterStr: << not present >>
                deleteAfter: << not present >>
                userAccessible:
                    TvBusTime {
                        nanoseconds: 1232767163496231000 (0x111bab3426f57058) {287025972/653619288}
                    } TvBusTime
                keepMinimally: << not present >>
                viewTimer: << not present >>
                lockedUntilStr: << not present >>
                lockedUntil: << not present >>
                fCanMrv: false
                fCanTtg: false
                fCanBurn: false
            } TvPvrFairUseRules
    } TvCopyProtectionInfo

  4. #4
    Join Date
    Jan 2009
    Posts
    13
    Quote Originally Posted by dah31 View Post
    To routerplus?
    I'd definitely recommend it. Routerplus will let you snoop on all kinds of activity. rpsniff.c is easily hacked up to aid in piping messages thru tivosh for easy parsing/finding info of specific targets. The messages picked up from routerplus can also be used to easily correlate with info from the /tvlib/idl/*.tvbin files.
    Code:
    Ex : 
    protocol : 0x10357
    name (words[1]) : 0x1034f
    <TvBusEnvelope xs:type="TvPvrMenuItemRequest"><vSpec><element><area value="22">TIVO_CENTRAL_6</area></element></vSpec></TvBusEnvelope>
    
    % strings /tvlib/idl/0x1034f.tvbin
    TvPvrMenuItemRequest
    vSpec
    Been using it alot lately to try to correlate info in the tivo's tcl scripts and router messages with code in tivoapp. Makes things much easier to explore.

    I see that the NpkChannelDefinition field caused other problems, like using TivoWebPlus for backing up season passes. With a little elbow grease it shouldn't be too hard to integrate this stuff with the TivoWebPlus module to fix that issue also.
    Last edited by alferd.packer; 01-31-2009 at 02:33 PM.

  5. #5
    Join Date
    Mar 2005
    Posts
    235
    Quote Originally Posted by alferd.packer View Post
    I see that the NpkChannelDefinition field caused other problems, like using TivoWebPlus for backing up season passes. With a little elbow grease it shouldn't be too hard to integrate this stuff with the TivoWebPlus module to fix that issue also.
    IIRC, even if you create a valid NpkChannelDefinition the SP still does not populate the todo list. I seem to remember that I had to edit the created SP in the TiVo UI and save it again before the todo list would actually populate.

  6. #6
    Join Date
    May 2005
    Posts
    913
    Quote Originally Posted by jkozee View Post
    IIRC, even if you create a valid NpkChannelDefinition the SP still does not populate the todo list. I seem to remember that I had to edit the created SP in the TiVo UI and save it again before the todo list would actually populate.
    This happens in versions without NpkChannelDefinition, too (6.2 and its ilk). I assume there's some sort of "SP has been changed" message that doesn't get sent. The odd thing is that the SP sometimes DOES work, with no further action required.

    FWIW, it's not necessary to edit each SP... if I add SPs from TWP, I then adjust the priority of SPs in the tivo UI (usually moving a show from above the new SPs to the end of the list), and that gets them all recognized.
    Former Tivo Hacker (retired)

  7. #7
    Join Date
    Mar 2005
    Posts
    235
    Quote Originally Posted by BTUxNine View Post
    This happens in versions without NpkChannelDefinition, too (6.2 and its ilk). I assume there's some sort of "SP has been changed" message that doesn't get sent. The odd thing is that the SP sometimes DOES work, with no further action required.

    FWIW, it's not necessary to edit each SP... if I add SPs from TWP, I then adjust the priority of SPs in the tivo UI (usually moving a show from above the new SPs to the end of the list), and that gets them all recognized.
    Good to know, I'll try the priority trick. Also, I created a SP using a self created NCD and rebooted the system. The todo list wasn't populated, but the show did record. I'll play around later and see if I can confirm this. It looks like a lot of the info is optional, so hopefully we can just add the basics and get it working.

    Also, I watched router traffic and didn't see anything useful, but may the priority change trick will show something.

  8. #8
    Join Date
    Mar 2005
    Posts
    235
    BTW, here's how I created my NCD:
    Code:
    tvidl serialtodb [tvidl xmltoserial {
    <TvBusEnvelope xs:type="TvNpkChannelDefinition">
      <idGuideSource>tms:11793</idGuideSource>
      <channelBits value="2586"/>
      <eSourceType value="2">CABLE</eSourceType>
      <channelNumber>
        <major>52</major>
      </channelNumber>
      <strCallSign>WPTZ</strCallSign>
      <strLongName>NBC Affiliate</strLongName>
      <logo>
        <iLogo>65556</iLogo>
        <iBackupLogo>0</iBackupLogo>
      </logo>
      <networkInfo xs:type="TvNpkChannelDefinitionNetworks">
        <vNetworks>
          <element xs:type="TvNpkChannelDefinitionExternalAnalog">
            <idTuningSignalSource>14757267935521146024</idTuningSignalSource>
          </element>
          <element xs:type="TvNpkChannelDefinitionTMS">
            <eServiceTier value="1">BASIC</eServiceTier>
            <fLocalOnly>false</fLocalOnly>
          </element>
        </vNetworks>
      </networkInfo>
      <eGuideProvider value="1">TMS</eGuideProvider>
      <eConnector value="3">COMPOSITE</eConnector>
      <idStation>657237370470399</idStation>
    </TvBusEnvelope>
    }]

  9. #9
    Join Date
    Mar 2005
    Posts
    235
    It appears that the todo list will populate, it is just not immediate, when creating season passes. I have posted a patch in the TWP thread that will create a NpkChannelDefinition object if one cannot be located. After the SP is added you can force an update as noted above.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •