Series2 OSD

[alldeadhomiez]

Originally Posted by erhan

FWIW, here is an elseed binary compiled with a "elseed starting.." message to eliminate that issue. It also has a couple of additions of my own, for example formatting the log file better and additional debugging. Nothing too different from the original.

Please post a diff - this is a development forum. Thanks.

[erhan]

Apologies. The attached diff is against dah31's port using his libosd.

[dah31]

Originally Posted by dgi

I am currently extending tivovbi to handle the full closed-captioning protocol. I would be willing to work with someone that has a Series2 to broaden the source so that it could run on those machines as well. Right now the biggest holdup is the lack of a complete Latin-1 font instead of the MS-DOS codepage 437. I would welcome help with at least five things that comprise a wishlist for my current unreleased version:

Spookily enough, porting tivovbi to Series2 was the carrot that originally motivated me to grab the axe and split open /tvbin/osdwriter ....

The next biggest holdup for Series2 will be figuring out how to get at the caption data (though I've got one idea towards that).

Originally Posted by dgi

1) to complete my partial Latin-1 font that I produced by rearranging the existing MS-DOS codepage 437 font

I've spent altogether too much time looking for something suitable. In that size, there aren't a lot of bitmap fonts apart from `novelty' type ones. I found a 16x26 font for X, but it only does 7-bit ASCII. There's also an 8x13 that comes with X, but that's not a great solution either.

Then there's the wider issue of a font that actually looks nice on screen. I personally rather like the one used in EEG decoder chips (their own decoders and some television sets), apart from the (lack of) descenders.

FreeType support will widen the choices somewhat. :-D

Originally Posted by dgi

2) to make flashing text produce some kind of visual feedback in OSD mode

This has never been a high priority for me. I've only actually seen flashing used once, thankfully (in an episode of Laverne and Shirley).

As for implementation, I reckon something along the lines of rendering the text into two separate buffers (`flash on' and `flash off', as Mr. Miyagi might say) and use some sort of timer to memcpy(3) them into the buffer. That'd probably work okay for pop-on mode, but roll-up would be a particular pain, particularly once smooth scrolling is implemented (see 47 CFR 119(f)(1)(iii)).

Originally Posted by dgi

3) to make sure the Text Restart command is handled properly (I have not encountered this command in a captioning stream)
4) to fix the OSD bug that occurs when a preamble code specifies a different row in roll-up mode

I noticed this one a couple of weeks ago on a programme on a PBS channel and recorded some of it for that very reason. :-B

Originally Posted by dgi

5) to handle the Delete to End of Row command

That one seems straightforward enough.

You might like to look at my hacks on tivovbi at <URL:http://www.dah31.org/tivo/tivovbi-1.04.tar.bz2>.

--
dah31

[dah31]

Originally Posted by erhan

I think it's a bug.

No, it's a feature. ;-)

Originally Posted by erhan

I've played with elseed and compiled several variations and it seems like the OSD is not initialized until the first time it is actually used. So, on startup it just displays some garbled stuff. As soon as a call comes in, the screen clears.

I also tried a ClearOSD() call as soon as elseed starts and that also seemed to do the trick.

EDIT: Forgot to mention, this happens only when you start yac/elseed the first time after a reboot. Once the OSD is "cleared", subsequent restarts are fine.

Erhan

Here's what I think happened: I took the ClearOSD() (or memset(tivbuffer, 0, 720*480*4)) call out of SetupTextOSD() to get a better idea of the zordering in example1, but I neglected to test the other programs between recompiling and releasing. Sorry.

(I actually ran that elseed on my mother's DVR39, and she called me to ask why Dick van Dyke was looking through frosted glass...! I added the ClearOSD() call and kept fiddling with FreeType.)

--
dah31

[dgi]

Originally Posted by dah31

Spookily enough, porting tivovbi to Series2 was the carrot that originally motivated me to grab the axe and split open /tvbin/osdwriter ....

The next biggest holdup for Series2 will be figuring out how to get at the caption data (though I've got one idea towards that).

IMHO, getting mbm to release his source to his kernel modules tvbi-test.o and the like would take care of the bulk of that.

Originally Posted by dah31

I've spent altogether too much time looking for something suitable. In that size, there aren't a lot of bitmap fonts apart from `novelty' type ones. I found a 16x26 font for X, but it only does 7-bit ASCII. There's also an 8x13 that comes with X, but that's not a great solution either.

Then there's the wider issue of a font that actually looks nice on screen. I personally rather like the one used in EEG decoder chips (their own decoders and some television sets), apart from the (lack of) descenders.

FreeType support will widen the choices somewhat. :-D

Have you actually tried scaling the 8x13 font? It might be sufficient, as long as it uses the ISO Latin-1 codepage.

Originally Posted by dah31

This has never been a high priority for me. I've only actually seen flashing used once, thankfully (in an episode of Laverne and Shirley).

As for implementation, I reckon something along the lines of rendering the text into two separate buffers (`flash on' and `flash off', as Mr. Miyagi might say) and use some sort of timer to memcpy(3) them into the buffer. That'd probably work okay for pop-on mode, but roll-up would be a particular pain, particularly once smooth scrolling is implemented (see 47 CFR 119(f)(1)(iii)).

If you would like to add that to my as-yet unreleased version, please do!

Originally Posted by dah31

I noticed this one a couple of weeks ago on a programme on a PBS channel and recorded some of it for that very reason. :-B

Which one? The Text Restart command, or the OSD bug? Any ideas on how to fix the OSD bug?

Originally Posted by dah31

That one seems straightforward enough.

You might like to look at my hacks on tivovbi at http://www.dah31.org/tivo/tivovbi-1.04.tar.bz2.

--
dah31

I tried to download that, but your webserver won't let me because of a lack of read permissions. I would love to make a diff to 1.03 and incorporate your changes into my as-yet unreleased version.

[alldeadhomiez]

Originally Posted by dgi

IMHO, getting mbm to release his source to his kernel modules tvbi-test.o and the like would take care of the bulk of that.

That will not help in this case. mbm's modules locate and export the caption1Tail and caption1Buffer symbols in the CS22 driver, pxmpegdecode.o. Since we have no pxmpegdecode kernel module (or CS22 chip, for that matter) on a Series 2, we will need to find the equivalent functionality in brcmdrv-rb.o. Then we can find a way to snag the buffer contents out of kernel memory (as tivovbi does), or intercept the syscalls that transmit caption data to the Broadcom driver.

I can think of three simple approaches:

1) Figure out which ioctl is used to set caption data on a Series 1, locate the appropriate section of code in tivoapp, then find the corresponding function on a Series 2 tivoapp.

2) Log all ioctls coming from myworld, try to dereference anything that looks like it might be a pointer, and see if any caption data turns up.

3) See if the S1 and the S2 happen to use the same ioctl.

What would be slick is to write a kernel module that sniffs the caption data coming from userland, and then puts it in our own buffer (called, of course, caption1Buffer and caption1Tail). If done properly, we probably won't even need special S2 support in tivovbi for straight logging.

[dah31]

Originally Posted by dah31

This is libosd 0.3, a library containing on-screen display (OSD)
routines for Series1 and Series2 TiVo units, tested on a Philips
DSR7000 running 4.0.

The API is intended to be more or less backwards compatible with the
one(s) used in tivovbi, elseed, yac, and newtext2osd, with
enhancements for slanted and coloured text. (I have successfully
ported elseed, yac, and newtext2osd to Series2 with this library.)
See the example for usage.

The Series2 code came from disassembling /tvbin/osdwriter from 4.0.
Some of the variable and function names refer to addresses in
osdwriter (which I have also reconstructed source for).


Notes:

We now have TrueType font support, thanks to FreeType <URL:http://www.freetype.org/>

All the OSD globals have been encapsulated into a structure (struct osd_info) (except got_own_surface (missed that one)). That should be a big step towards having multiple surfaces in one program. Likewise with struct ft_info for FreeType.

Also: newtext2osd's find_colors() is in osd.c (access to S1 colours by name), as is get_colour() (convert colours <0x100 to S2 equivalents).

setupBrcmDisplay() and SetupTextOSD() have a new argument, dont_clear_buffer. Set it if you don't want the display buffer cleared; 0 will clear it.

TodoExercises for the reader:

o test Series1 support. I'm pretty sure it doesn't work just
now....

o fix bugs

o colour name to value mapping, something like X's rgb.txt?

o changeable fonts

Done.

o certainly, a font encoded for (say) Latin-1, rather than MS-DOS
codepage 437.

Done.

o FreeType would probably work well for those last two.

Yes, it does.

o Fix spelling inconsistencies (functions, variables, and types should use `colour'; #define `color' equivalents)

o Put got_own_surface in the osd_info structure.

Please note that the attached files are not ZIP archives.

I've included a copy of the FreeType library (put it in a directory in your LD_LIBRARY_PATH), but you'll need to grab the sources to compile anything.

Oh, and there's an environment variable FONT_PATH, which you set to a colon-separated list of directories to search for fonts. openfont_ex() (and openfont()) searches the exact path given, then the FONT_PATH, then the current directory.

--
dah31

[dgi]

certainly, a font encoded for (say) Latin-1, rather than MS-DOS
codepage 437.



Done.

Ooooo, lemme see!

EDIT: Oh, it uses a freetype lib. So much for adapting tivovbi to use a different font.

[dgi]

dah31: I have incorporated your 1.04 changes to tivovbi for the most part. Here is a preliminary snapshot of my version with your changes. Hopefully we can start extending it together from here. It is NOT a zip file.
EDIT: I have removed the snapshot as it is no longer current.

PS-Also, I got mbm's final cvs snapshot before he abandoned it. It's broken, but it's really rewritten, and doesn't use tvbi-test.o anymore. E-mail me for details.

[dah31]

Originally Posted by dgi

Have you actually tried scaling the 8x13 font? It might be sufficient, as long as it uses the ISO Latin-1 codepage.

Here's a screenshot I put together with netpbm.

Originally Posted by dgi

Which one? The Text Restart command, or the OSD bug? Any ideas on how to fix the OSD bug?

The OSD roll-up bug. I haven't looked at it yet, but I don't see it being too difficult (famous last words!).

--
dah31

[dgi]

That screenshot looks good.

Integrating your OSD changes (skewed italics) seems to have introduced another bug. Whenever the captions go all the way to the right edge (column 32), the last character gets only half-drawn. If it's italic, it's hardly visible.

[pcl]

I just tried out the libosd 0.3 and the patched elseed 0.6 binary, and ended up with a rebooting Tivo, and seemingly no log output.

I put the contents of libosd-0.3/lib into /var/hack/lib, put /var/hack/lib in my LD_LIBRARY_PATH, and executed './elseed -c conf/elseed.conf'. Things loaded as expected, and then I dialed my number. This caused the system to reboot.

I attempted to capture stdout and stderr, but no output was generated.

I had previously installed the vanilla elseed 0.6.0 and configured it to broadcast call info to my Mac. That worked, but no display on the TV. When running the patched elseed, my Mac did not receive any call notification.

I briefly tried to do some debugging (first, by compiling a helloworld binary), but was foiled by not having <tivo/osd/osd.h> available on my cross-compiling machine. At that point, I threw up my hands and typed up this post.

I'm quite new to Tivo compiling. I do have a cross-compiler up and running on a Linux box, and have written one little hello world program that seems to run, but have not been indoctrinated in the ways of the Tivo just yet. Any help (including appropriate, and hopefully content-rich, flames) would be much appreciated. I'm looking to get elseed working (or rather, to get caller id info on my TV), so help to that end is what I'm really looking for, but obviously, this is a developer forum, so info about what headers etc. I need for cross-compiling with the new OSD stuff or any other relevant info is probably what I'm looking for.

System info: Freshly-hacked S2 standalone running 4.0.1b with a single 160-gig drive, with no LBA hack.

-Patrick

[erhan]

Originally Posted by pcl

I briefly tried to do some debugging (first, by compiling a helloworld binary), but was foiled by not having <tivo/osd/osd.h> available on my cross-compiling machine. At that point, I threw up my hands and typed up this post.

You need to "make" libosd first and "make install' to get the include files and the library to the right place. Don't forget to modify the Makefile to point to your cross compiler directory for both libosd and elseed Makefiles.

[dah31]

Originally Posted by pcl

I just tried out the libosd 0.3 and the patched elseed 0.6 binary, and ended up with a rebooting Tivo, and seemingly no log output.

I finally managed to borrow a portable telephone and figure this one out. The problem was that I changed the parameters for osdit() and didn't prototype it:

[elseed.h:]
void osdit ();

[elseed.c:]
int main(int argc, char *argv[]) {
  [...]
  osdit();
  [...]
}

[...]

void osdit(struct osd_info *osd, struct ft_info *ft) {
  [...]
}

Moral: use prototypes!

--
dah31

[alldeadhomiez]

Here is a fake "pxmpegdecode" module for the Series 2 platform which emulates the circular CC/XDS buffers from the Series 1 CS22 driver. Source and a MIPS 2.4.18/4.0 binary module are included. CC tests with tivovbi 1.03 in text mode were successful; XDS might work but is untested. There's a chance it will work on the Series 1 as well if you rename the stock pxmpegdecode.o and adjust the startup scripts accordingly.

Building tivovbi for MIPS required #include's for time.h and errno.h. dgi's tivovbi sources from post #24 worked after I took the additional step of turning off parity checking.

Due to the growing number of schemes in which freely available TiVo hacks are being sold for profit, this code and any derivative work is for personal use only and any redistribution is prohibited. Exceptions: you may post compiled binaries in this thread if you include any source changes, you may post patches in this thread, and the license will revert to GPL on December 31st, 2005.

Edit: I have pulled the original attachment and posted version 20040601a. There were bugs in my kernel module which caused the parity issue alluded to earlier. dgi's version now runs correctly, unmodified, with the fixed module.

Edit: New version - 20040602 fixes XDS handling.

Edit: New version - 20040602a cleans up the intercept.

Edit 2005/02/02: s2vbi and tivovbi development has moved to this thread