PDA

View Full Version : building kernel for s2?



ronnythunder
09-07-2003, 11:52 PM
ok, i've searched here, tivocommunity.com and alt.org, and i'll apologize in advance if i missed it, but...

has anyone successfully built a kernel from tivo's sources for series 2 dtivo (hdvr2/dsr7k)? here's what i've done so far:

1. build (well, installed) cross compiler from tivoutils.sf.net
2. verified cross compiler works by building some progs and testing on hdvr2
3. downloaded all of the tarballs in the 3.1/3.2 section of tivo.com/linux
4. untarred them into a common dir

when i go to the linux-2.4 dir and do "make", i get this:
ismdefs:12: /include/make/commondefs: No such file or directory
Makefile:341: arch//Makefile: No such file or directory
make: *** No rule to make target `arch//Makefile'. Stop.i'm obviously missing some stuff. any ideas? i did see mrblack51's thread here (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=10453), but it doesn't seem to be relevant to the 3.1/3.2 stuff.

thanx and sorry if i missed something; if i need to rtfm, please let me know where and i will

ronny

ronnythunder
09-08-2003, 06:31 PM
ok, i'm getting a little farther but no dice. i build the cross compiler on another machine using the script on tivoutils.sf.net instead of the prebuilt tarball. again, i can build simple programs. i figured out that i need to put at least this on my make line when trying to build the kernel:
make ARCH=mips CROSS_DIR=/usr/local/mips-tivoi can then get the config step to go using:
yes "" | make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config. however, when i to do a make dep (with the variables set of course), i get this:
[root@brick linux-2.4]# make ARCH=mips CROSS_DIR=/usr/local/mips-tivo dep
Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -o scripts/mkdep scripts/mkdep.c
make: Wall: Command not found
make: [scripts/mkdep] Error 127 (ignored)
if [ ! -f /usr/installs/mipstemp/linux-2.4/include/asm-mips/offset.h ]; then \
touch /usr/installs/mipstemp/linux-2.4/include/asm-mips/offset.h; \
fi;
make[1]: Entering directory `/usr/installs/mipstemp/linux-2.4/arch/mips/boot'
make[1]: Nothing to be done for `dep'.
make[1]: Leaving directory `/usr/installs/mipstemp/linux-2.4/arch/mips/boot'
scripts/mkdep -- init/*.c > .depend
/bin/sh: line 1: scripts/mkdep: No such file or directory
make: *** [dep-files] Error 127any ideas? i'm sure it's something stupid, but it's sure not obvious.

thanx,

ronny

ronnythunder
09-08-2003, 06:52 PM
i feel like i'm having a conversation with myself :) ok, i may be on to something. i finally did this:
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo vmlinuxand it seems to be doing it. it's a slow machine (pentium 233/mmx), so it'll probably chug for a while.

any of you merlins out there want to let me know if i'm warm or cold? :)

ronny

ronnythunder
09-08-2003, 10:49 PM
ok, i think we have success! the build completed without error, and produced something that looks like a kernel. of course, it's a tivo 4.0 kernel (if that makes a difference, probably does) and i need a 3.1, so i just laid down the 3.1 kernel source and am building it.

i've got a new hdvr2 coming on thursday (yay ffdvr) and i've got a 160gb drive... hmmm.. mips-lba48 anyone?

ronny

ronnythunder
09-09-2003, 02:02 AM
edit: the kernel is known to have some problems, so i'm withdrawing it for now.

ok, is anyone out there feeling lucky? i won't be able to test this until thursday at the earliest, but i've built a candidate kernel for hdvr2/dsr7000 that has mark lord's lba48 patches referenced here (http://www.tivocommunity.com/tivo-vb/showthread.php?postid=710705#post710705). i've enclosed it; in theory, all one needs to do is dd it in place of an existing 3.1 kernel (i'm assuming a monte system), run killinitrd just to make sure there's not one active, and it should run.

of course, to try the lba48 features, you'd have to boot mfstools with something that supports lba48 (e.g. knoppix) and restore and expand an image onto a big drive and redo all of the monte stuff.

if anyone is brave enough to try it before me, please let me know if it works.

thanks

ronny

mrblack51
09-09-2003, 03:31 AM
why not post a diff or patch file which can be applied to the stock tivo kernel stuff so others can apply it to their source, as well as being able to use a pre-compiled version?

Or, did you just apply his patch as is?

ronnythunder
09-09-2003, 10:01 AM
edit: the patch is now known to be incomplete, so i'm withdrawing it for now.

ask and ye shall receive. this is comparing before to after from the 2.4.4 kernel sources (tivo 3.1/3.2); i haven't tried the 2.4.18 (tivo 4.0) stuff yet as i don't have a tivo with that version.

should be easy to apply to an existing tree with the patch command, although, i just did it by hand on my system.

ronny

ronnythunder
09-09-2003, 04:46 PM
edit: sorry folks, a field report shows issues with this version as well; bacl to the drawing board.

ok, here's the updated patch and kernel in one file. note that i built this on my pc, and it has *not* been tested in a tivo yet. my plan is still to test in on thursday if possible.

note also that, while i could have taken out the scrambling here and saved the kmem necessity, i didn't do so this time around; i wanted to keep the changes to a minimum.

enjoy

ronny

Dogbert5
09-09-2003, 04:51 PM
Nice job Ronny. I'm glad to see you got it to compile. I will probably try this out on my Tivo some time this week.

ronnythunder
09-09-2003, 05:15 PM
dogbert5, you're a brave and honarable person! :)

in case i didn't stress it enough in my other post, though:

i made the mods and built the kernel in good faith
i have *not* tested the kernel
if it breaks your tivo, as they say, you get to keep both pieces

having said that, the changes are actually fairly simple to apply to a 2.4.4 kernel. but, the actual lba48 calls don't get made unless the os is requesting lba's above the lba28 limit. so, it's conceivable that it could work for a while, but not when it gets into the high area.

i think my testing methodology is going to be something like this: build the system with the 160gb drive and this kernel, and make sure it boots. then, make sure the basic tivo functionality works, and maybe a test show to make sure.

then, i'll probably set up a bunch of 24 hour manual recordings to go two at a time until it's full, which, in theory, would take as long as three days (!); by then, we'll know for sure!

ronny

mrblack51
10-02-2003, 12:04 PM
alrighty, so I tried what you mentioned ronny, but it didn't seem to work straight away. I used some of the previous info I had, and this is what seemed to work on my RH9 box (note: i installed everything from the RH9 install):

1. grab from http://tivoutils.sourceforge.net - usr.local.mips-tivo.tar.bz2. bunzip2 it and then untar it while you are sitting in the root dir of your system. should create /usr/local/mips-tivo

2. grab all the 3.1-3.2 stuff from tivo.com/linux, tar xzvf them all into a directory. for example, lets use /home/mrblack51/tivosrc

3. -NEW- grab the TIVO_2_0.tgz from the 2.0.3 section of tivo.com/linux. this file has all the commondefs stuff and whatnot. tar xzvf it while in the same dir as you were for step 2 (/home/mrblack51/tivosrc)

4. execute the following:


cd linux-2.4
export ROOT=/home/mrblack51/tivosrc/cmd/root
export TV_FEATURE_STRONG_CRYPTO=0
yes "" | make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo dep
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo vmlinux.px modules modules_install INSTALL_MOD_PATH=/tmp/tivo


alternatively, if you want to use menuconfig instead of the config, you can do the following instead of the "yes ..." line:



make ARCH=mips CROSS_DIR=/usr/local/mips-tivo NATIVE_CC=/usr/bin/gcc menuconfig


that produced a vmlinux and didn't have any crazy errors when I tried it. as always (esp. with cross compiler stuff), YMMV.

edit: modified a few of the commands based on a david bought post (http://www.dealdatabase.com/forum/showthread.php?s=&threadid=25823&highlight=compil%2A+kernel). added menuconfig note. also added TV_FEATURE_STRONG_CRYPTO environment variable. see my post below about edits to the scripts dir

chrsturbo
10-11-2003, 02:27 PM
Well after starring at Todd's patches and Mlords stuff from tivocommunity I think I have this working. Rember LBA48 is ONLY for the stout as your MFS WILL be totally corrupted should it even start the switcher without a LBA48 kern. So here's a patch to tivos kernel and a binary for LBA48 no scramble and no initrd. Anyone brave enough:)
-Chris

chrsturbo
10-11-2003, 02:29 PM
Here's the kernel.
-Chris

EDIT: For the newer noscramble patches please be aware that these kernels include built in dd style patches noscramble (actually the just don't call scramble). So don't do the alphawolf tivoapp patches with them. I corrupted lots of MFS by doing both:) YMMV

chrsturbo
10-12-2003, 02:33 PM
A little advice for the intrepid S2 LBA48er. Remember your Monte part (the romfs partition) CAN'T be past LBA28 (>137Gig) case the U5 kernel can't see it. I used the bootstrap partition from the hacked pair (hda2 for me). Also mine didn't work with 600gig and a 512meg swap till I actually dd the image of the swap off from a little one that would boot and I'd run mkswap -v1 on. Seems mfsfix REALLY hates huge mfs parts and no swap. Just FYI.
-Chris

mrblack51
11-06-2003, 03:41 AM
also, in case anyone needs it:

Quote:



perl script fixes in scripts/
=============================
For each of the following three files in the scripts directory:

elf2ppceval
elfextract
makeppceval

After the first "open" statement, add the line:

binmode ifile;

And save the changes.


thats from gneuf from AVS

chrsturbo
02-20-2004, 06:45 PM
Just to keep this freedom train rolling. Here's a 2.4.18 (tivo 4.0) patch for lba48. Just about everything in the originals is already in 4.0 (bigswap etc).

chrsturbo
02-20-2004, 06:48 PM
And the kernel.... Sorry I had to use bzip2 on it to get it to fit the 1 meg. Most linuxen have it.

EDIT: I think because these kernels have the scrambler commented out they are NOT compatible with the Alphawolf tivoapp patch. please be aware that running both has caused MFS corruption for me.

EDIT2: Ok I think I found the issue this kernel has the fix patch et al will be out when I can confirm it. ANYONE running this kernel is advised to upgrade.

EDIT3: The problems above were not the Alphawolf patches but my own lba48 being tricky. This binary and the one since EDIT2 are fine. As is running the noscramble tivoapp patch.

chrsturbo
02-27-2004, 11:14 PM
And the kernel.... Sorry I had to use bzip2 on it to get it to fit the 1 meg. Most linuxen have it.
UPDATE: Looks like there may be an issue with this kernel. I think my cross compiling environment may not be "fresh" enough for 4.0.
PROCEED AT YOUR OWN RISK. I have toasted my MFS several times:(

chrsturbo
03-05-2004, 10:49 AM
Ok here's a kernel with scrambling that appears more compatible with the tivoapp patches. I also put the patch for those with the home game. This has been running for 4-5 days stable. Again YAYOR YYMV... ;)

EDIT: Updated the vmlinux the patch is out of date.

kjmcdonald
03-05-2004, 07:54 PM
So the last kernel has scrambling turned on?
but the one beofre it doesn't?

Are both stable? or is the one before last know to have problems?

Were the problems with th scrambling or the LBA48 patch?

-Kyle

tytyty
03-06-2004, 07:54 PM
chrsturbo

Awesome tool here I took your suggestion and put my romfs in 5 and other than dd'ing other images to properlocations and reinstalling bootpage this works like a charm. 142 hrs on a 160gig will let it fill and see if there is any corruption etc..

RKone
03-08-2004, 08:54 PM
I've been trying to compile a 3.1 kernel with Wireless extensions. I've followed mrblack51's guide, but I'm running into a problem.

I get a /usr/local/mips-tivo/bin/gcc 1: Syntax error "(" unexpected error when I try to make vmlinux. Actually, if I try to run /usr/local/mips-tivo/bin/gcc without any arguments, I get the same error message. Since I've had 2 drives fail in the past 2 weeks, my linux box is down - I'm running this in cygwin.
I even tried to re-download usr.local.mips-tivo from sourceforge, I have the same problem.

I've also tried the cygwin cross compiler from this thread (http://www.dealdatabase.com/forum/showthread.php?t=27280&page=1&pp=15&highlight=cross+compiler+cygwin), it starts, but then fails when assembling files in the kernel folder - giving a list of asm instructions used but not supported.

Anyone else using Cygwin get a kernel compiled?

chrsturbo
03-09-2004, 03:07 AM
So the last kernel has scrambling turned on?
but the one beofre it doesn't?

Are both stable? or is the one before last know to have problems?

Were the problems with th scrambling or the LBA48 patch?

-Kyle
Yes the last kernel is for 4.0 LBA48 with scrambling. Both are stable however combining a kernel with scrambling off with the tivoapp patches by AlphaWolf results in MFS corruption as far as I can tell. The problem are with the scrambling and the modified tivoapp. The lba48 has run for 7 days now with no problems.

EDIT: As stated above this is the raving of a decieved madman. Alpha's patches are fine with any of these kernels and all the binaries are good to go. The patch needs updateing but I'll put that in a later post to keep this confusion to a minimum

kjmcdonald
03-09-2004, 03:29 PM
Does the no-scramble kernel work without the 'tivoapp' patch? does it scramble or not? Will tivoapp
still play back recordings?

I admit I haven't really been looking for scrambling info in all the reading I've done lately, but if I recall correctly, I thought that on 4.0 the kmempatch that edits the live kernel required the tivoapp patch also?

I also though I saw something about permanently
applying the kmem patch to the kernel image on the disk?

Will either of these no-scrambling methods also cause
problems with your new kernel?

How was your no-scrambling hack achieved?

-Kyle

David Bought
03-09-2004, 05:21 PM
Does the no-scramble kernel work without the 'tivoapp' patch? does it scramble or not? Will tivoapp
still play back recordings?

Stupid NEWBIE question, get the hell out of the S2 Hacking Forum and especially out of this thread.

This is where the grown-ups play. Go back to where you belong.

fearlessrogue
03-10-2004, 12:32 PM
Forgive the question, but I am lazy so I thought I would ask rather than looking through the patch. Do the patches above have the FPU register fix in it?

The patch is show here:
http://www.pvrhax0r.com/forum/showthread.php?threadid=30

RKone
03-11-2004, 02:45 AM
Ok. I've compiled my kernel, installed it, but now my tivo won't startup, and bash spits out this:

(...)
monte: Short read on image (got 0x3fffe0 of 0x2cc27fe0 expected bytes)
monte Input/output error

I'm fairly sure this is a result of a "couldn't allocate" error message I was getting when doing the vmlinux.px compile stage - I got around it by short-circuiting two files, scripts/elfextract and scripts/makeppceval. They both have comments near the beginning saying "put the file in buffer $ibuf". The script would die on this line:

read (ifile,$ibuf,$ifilesize) == $ifilesize or die("couldnt allocate\n");

I commented out the "or die" part in both these files, and the make finished the kernel files without complaining...

edit: nevermind, I found out this problem was a bug with perl. Why does it seem like linux is always trying to work against me? /edit

Just to make sure I'm not doing anything dumb:

Once I have vmlinux.px, I simply dd if=vmlinux.px of=/dev/hda6, correct?
(assume I have the right path to vmlinux.px and the proper hard drive and partition for my target)

RKone
03-11-2004, 04:32 AM
I simply dd if=vmlinux.px of=/dev/hda6, correct?

To answer my own question, no. After compiling the kernel, you need to apply the killinitrd patch for your kernel revision, downloadable from tivoutils (http://tivoutils.sourceforge.net/). Then dd away...

David Bought
03-11-2004, 11:19 AM
After compiling the kernel, you need to apply the killinitrd patch for your kernel revision, downloadable from tivoutils (http://tivoutils.sourceforge.net/).

Wrong, you should not apply a binary patch when you are building from source.

RKone
03-11-2004, 11:51 AM
Wrong, you should not apply a binary patch when you are building from source.

I thought about that after posting, obviously it would be best to modify the source directly. Do you know what to modify to fix the "no RAMdisk" error? I don't, but killinitrd did fix my kernel. If it wasn't so late last night, I probably wouldn't have even tried it. OTOH, the 3.1 killinitrd program obviously checks to make sure the proper bits are in the proper spot - it works with multiple kernels. So I'm guessing that it'll work fine in my case (and it has been since before I made that last post).

David Bought
03-11-2004, 01:31 PM
I thought about that after posting, obviously it would be best to modify the source directly. Do you know what to modify to fix the "no RAMdisk" error?

Start with CONFIG_BLK_DEV_INITRD.

cdn
03-13-2004, 01:41 PM
Here's the kernel.
-Chris

EDIT: For the newer noscramble patches please be aware that these kernels include built in dd style patches noscramble (actually the just don't call scramble). So don't do the alphawolf tivoapp patches with them. I corrupted lots of MFS by doing both:) YMMV

Curious to know if anyone is using chrsturbo's 3.1.x lba48 kernel on a phillips dsr704. My box is monte'd using sleeper's ISO, running 3.1.1b. I tried replacing my kernel with this one, and my machine booted to the point where I got the "Almost there, a few more second..." screen, and it stayed on that screen for somewhere between 5-10 seconds before rebooting. I had to dd the old kernel back onto the partition so the machine could boot again.

I did a bootparam -p and got back hda7 plus the monte env vars, etc. so I dd'd this kernel into hda3. (I saved what was there, so recovering was a piece of cake).

Thanks
-cdn

chrsturbo
03-13-2004, 02:00 PM
Curious to know if anyone is using chrsturbo's 3.1.x lba48 kernel on a phillips dsr704. My box is monte'd using sleeper's ISO, running 3.1.1b. I tried replacing my kernel with this one, and my machine booted to the point where I got the "Almost there, a few more second..." screen, and it stayed on that screen for somewhere between 5-10 seconds before rebooting. I had to dd the old kernel back onto the partition so the machine could boot again.

I did a bootparam -p and got back hda7 plus the monte env vars, etc. so I dd'd this kernel into hda3. (I saved what was there, so recovering was a piece of cake).

Thanks
-cdn
Hmm sounds like it isn't happy:) Is there any way you can put a console on it and watch the output? That'd be very helpful.
-Chris

chrsturbo
03-13-2004, 02:05 PM
Forgive the question, but I am lazy so I thought I would ask rather than looking through the patch. Do the patches above have the FPU register fix in it?

The patch is show here:
http://www.pvrhax0r.com/forum/showthread.php?threadid=30

No these kernels DO NOT have the fpu fix. I am adding it in. I am also still having problems with the mfs going corrupt after several days and can't seem to find why so again this is bleeding edge so be prepared to bleed. For those interested the error I see after a while is:

Filesystem assert: 0 at fsnodetable.C line 808 in TwStatus FsNodeTable::GetPage(long unsigned int, FsBuffer*&)

I have applied the fpu patches and am continuing testing but I am not confident thats the issue. Going back to the patches:(
-Chris

cdn
03-14-2004, 04:27 AM
Hmm sounds like it isn't happy:) Is there any way you can put a console on it and watch the output? That'd be very helpful.
-Chris

I couldn't find my serial cable, so I made a new one. I figured it out. Here is what I saw:


SIOCSIFNETMASK: No such device
SIOCADDRT: Network is unreachable
/etc/rc.d/rc.sysinit.author: cd: /usr/mfs_ftp: No such file or directory
/etc/rc.d/rc.sysinit.author: /usr/mfs_ftp/mfs_ftp.tcl: No such file or directory
Unable to handle kernel paging request at virtual address 02fc682c, epc == 800b23c4, ra == 80
0b2364
Oops in fault.c:do_page_fault, line 172:
$0 : 00000000 80120000 02fc6820 00000001
$4 : 7fff73e0 00000001 802e0e4c 802e0d8c
$8 : 802e0d80 801466f0 04b08c41 82ce1420
$12: 00000001 82699d60 00000001 82699d60
$16: 00000008 00000001 834e1000 802e0d80
$20: 7fff73e0 801466f0 80146718 00000000
$24: 00000000 00db63e8
$28: 82698000 82699c28 801466f0 800b2364
epc : 800b23c4
Status: 9001d403
Cause : 00000008
Process myworld (pid: 128, stackpage=82698000)
Stack: b000001c 00000003 834e1000 8002df5c 82699dc0 82699c58 00000000 00000000
00000000 00000000 00000000 00000000 00000001 00000000 00000001 00000000
00000000 00000000 00000000 00000000 00000001 00000000 00000002 00000000
00000000 7fff73e0 00000008 00000000 00000001 00000000 00000003 00000000
00000000 0005fb7e 00000000 00000000 00000001 00000000 00000004 00000000
00000000 ...
Trace: 800b23c4 800b28c0 800aad48 8005219c 8005a8ec 8001cb04 00db63fc
Code: 8fc200e4 02802021 01621021 <8c43000c> ae5e001c ae43000c ae400024 8fa20150 0220282
1
Kernel panic: Die called

Rebooting in 1 seconds..


At first I wondered if I had forgot to initialize my swap partition, as this was a newly created drive with 300meg swap, and in fact I had forgot, but the tivo should be able to start up fine w/o it. I went back and created it but that wasn't the problem. Seeing that mfs_ftp was being called made me remember that I was running sleeper's scripts, and it calls kmem to install the noscramble patch, which is sure to scramble kernel memory on this kernel. I commented the kmem line out of rc.sysinit.author and I'm up and running!

Thanks chrsturbo.

-cdn

chrsturbo
03-14-2004, 11:51 AM
At first I wondered if I had forgot to initialize my swap partition, as this was a newly created drive with 300meg swap, and in fact I had forgot, but the tivo should be able to start up fine w/o it. I went back and created it but that wasn't the problem. Seeing that mfs_ftp was being called made me remember that I was running sleeper's scripts, and it calls kmem to install the noscramble patch, which is sure to scramble kernel memory on this kernel. I commented the kmem line out of rc.sysinit.author and I'm up and running!

Thanks chrsturbo.

-cdn
Well it looks like its getting past the initial drive stuff so I don't think its anything obviously stupid done be me (leaving only my more regular unobvious stupidity). I do know that appswitcher (or what ever incarnation of tivosh it is) doesn't like a huge mfs without swap. A trick I have been using is to make the swap under a small mfs and then dd that off. I have been standardizing on 512meg:) Did you see it mount the swap? The message should look like:
Activating swap partitions
Adding Swap: 524276k swap-space (priority -1)

where that number is your size swap. Sorry to zone in on the swap but its all I can see right now.
-Chris

chrsturbo
03-14-2004, 11:53 AM
BTW whats in your rc.sysinit.author after mfs_ftp? No kmems in there are there?:)
-Chris

cdn
03-15-2004, 11:17 AM
BTW whats in your rc.sysinit.author after mfs_ftp? No kmems in there are there?
-Chris


Yeah, that's what the problem was... I had put that at the very very end of my post above, I think you missed it :-)



... I was running sleeper's scripts, and it calls kmem to install the noscramble patch, which is sure to scramble kernel memory on this kernel. I commented the kmem line out of rc.sysinit.author and I'm up and running!


So I've been up and running for 2 days now. Yay!

Thanks again.

-cdn

chrsturbo
03-15-2004, 11:43 AM
Yeah, that's what the problem was... I had put that at the very very end of my post above, I think you missed it :-)



So I've been up and running for 2 days now. Yay!

Thanks again.

-cdn
I think I may have found my issue but it is going to take several days to record enough stuff to tell. You want to ride the edge with me and swap the kernel or wait? I feel it fair to warn everyone else those last kernels have issues. As soon as I have it tested as fixed I'll replace them.
-Chris

AlphaWolf
03-15-2004, 01:32 PM
3. -NEW- grab the TIVO_2_0.tgz from the 2.0.3 section of tivo.com/linux. this file has all the commondefs stuff and whatnot. tar xzvf it while in the same dir as you were for step 2 (/home/mrblack51/tivosrc)


Hmm...this file doesn't seem to be there anymore :confused: anybody know of an alternate location?

FYI, I am trying to build a 5.1 kernel with all fixes, plus the fixes outlined here (http://www.dealdatabase.com/forum/showthread.php?t=31267&page=2&pp=15) for support for RID tivos.

RKone
03-15-2004, 03:24 PM
You want the TiVo-2.0.3-cmd.tar.gz (http://www.tivo.com/linux/20/TiVo-2.0.3-cmd.tar.gz) file.

AlphaWolf
03-15-2004, 05:20 PM
hoo-ah hoo-ah, heres a 5.1 kernel that should be compatible with RID units. I haven't tested it yet though, so it might not work.

EDIT: nope, doesn't work, attachment removed.

AlphaWolf
03-16-2004, 01:10 AM
Ok earlier I was having a problem that was caused by stupidity, and now that I fixed it, I have gotten some useful debug console output from the serial port:



Output enabled
Kernel signed by 'Kernel release key'
Hashing kernel... done
Checking signature... done.
Incompatible
CPU revision is: 00005430
FPU revision is: 00005410
Primary instruction cache 32kb, linesize 32 bytes.
Primary data cache 32kb, linesize 32 bytes.
Linux version 2.4.20 (root@mediaserv) (gcc version 3.0) #3 Mon Mar 15 20:28:19 M
ST 2004
Determined physical RAM map:
memory: 02000000 @ 00000000 (usable)
On node 0 totalpages: 8192
zone(0): 8192 pages.
zone(1): 0 pages.
zone(2): 0 pages.
Kernel command line: root=/dev/hda4 dsscon=true console=2,115200 upgradesoftware
=false
Monotonic time calibrated: 81.00 counts per usec
Calibrating delay loop... 161.79 BogoMIPS
Contiguous region 1: 1048576 bytes @ address 0x81780000
Contiguous region 2: 524288 bytes @ address 0x81880000
Contiguous region 8: 7340032 bytes @ address 0x81900000
Contiguous region of 8912896 bytes total reserved at 0x81780000.
Memory: 22156k/32768k available (1222k kernel code, 10612k reserved, 81k data, 6
4k init, 0k highmem)
Dentry cache hash table entries: 4096 (order: 3, 32768 bytes)
Inode cache hash table entries: 2048 (order: 2, 16384 bytes)
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
Buffer-cache hash table entries: 1024 (order: 0, 4096 bytes)
Page-cache hash table entries: 8192 (order: 3, 32768 bytes)
Checking for 'wait' instruction... unavailable.
POSIX conformance testing by UNIFIX
PCI: Probing PCI hardware
ttyS00 at iomem 0xb4100100 (irq = 79) is a 16550A
ttyS00 at port 0xbc010000 (irq = 133) is a unknown
ttyS00 at iomem 0xb4100140 (irq = 81) is a 16550A
ttyS00 at iomem 0xb4100120 (irq = 80) is a 16550A
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
Starting kswapd
Serial driver version 5.05c (2001-07-08) with MANY_PORTS SHARE_IRQ SERIAL_PCI en
abled
ttyS00 at 0xb4100100 (irq = 79) is a 16550A
ttyS01 at 0xbc010000 (irq = 133) is a unknown
ttyS02 at 0xb4100140 (irq = 81) is a 16550A
ttyS03 at 0xb4100120 (irq = 80) is a 16550A
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
hda: Maxtor 2F040L0, ATA DISK drive
ide0 at 0x400-0x407,0x438 on irq 87
hda: 80293248 sectors (41110 MB) w/2048KiB Cache, CHS=4998/255/63
Partition check:
hda: [mac] hda1 hda2 hda3 hda4 hda5 hda6 hda7 hda8 hda9 hda10 hda11 hda12 hda13
hda14
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
PPP generic driver version 2.4.2
PPP Deflate Compression module registered
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 2048 bind 4096)
ip_conntrack version 2.1 (256 buckets, 2048 max) - 152 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
Kernel panic: No bootable RAMdisk
<0>Rebooting in 1 seconds..


Note that the first few lines of text only show up if I press the enter key just as the 31u5 kernel is booting. I've never heard anything about a RAMdisk before on a tivo...I am glad to see that the kernel at least tries to boot, which means my cross compiler is working fine. I don't know enough about the kernel to be able to find out anything more about this. Does anybody have any pointers?

Sleeper
03-16-2004, 01:32 AM
I don't know about the Tivo specifically, but on a intel box there are 2 way the kernel can boot.

a) kernel + initrd (compressed fs) in ram. linuxrc sets up necessary modules and mounts new root fs & pivots root
b) kernel + no initrd, "root=" is specified in boot loader.

I'm guessing Tivo uses "a". So your kernel image probaply needs a initrd that contains a "linuxrc" that mounts root and calls init (or rc.sysinit).

AlphaWolf
03-16-2004, 01:52 AM
Well, I got a hint in IRC from emuroot, and found that you have to edit the init/do_mounts.c:



#if defined(CONFIG_BLK_DEV_INITRD) && \
defined(CONFIG_TIVO) && !defined(CONFIG_TIVO_DEVEL)
panic("No bootable RAMdisk");


change the "panic" to "printk"

BTW: I also made a similar change to init/main.c earlier before trying any of this:



#endif
panic("No init found. Try passing init= option to kernel.");
}


change panic to printk like above. Both changes are necessary.

It seems that the rc.sysinit is now running, which means the kernel might be good at this point? Don't know, but its doing more than it was before:



<snip where last kernel left off>
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 64k freed
Starting rc.sysinit
Running boot Stage A_PreKickstart scripts
bash: no job control in this shell
Scanning for configuration files
TmkLogger: <132>Jan 2 00:00:03 AttachSharedMemoryFile[9]: Can't find key HpkSha
redMemoryPool on attach

Tmk Assertion Failure:
HPK_Result TivoSerialCryptoResetOn(HPK_CryptoInstance*), line 137 ()
TmkLogger: <129>Jan 2 00:00:03 TmkAssertionFailure[9]: (HPK_Result TivoSerialC
ryptoResetOn(IHPK_CryptoInstannce*), line 137 (v))
imk Fatal Eroror: Thread crypkto <9> strayed!
TmkLogger: <129n>Jan 2 00:00:03g crypto[9]: Tmk Fatal Error: Thrsead crypto <
9> sttrayed!
0x004a27e7c 0x00423e80r 0x00421d64 0x00t4131fc 0x0041fb7uc 0x00412fe8 0x0p0412
d20
TmkLog ger: <129>Jan 2s 00:00:03 cryptoc[9]: 0x00427e7rc 0x00423e80 0x0i0421d
64 0x004131pfc 0x0041fb7c 0xt00412fe8 0x00412sd20
0x2ac794 5c
TmkLogger: f<129>Jan 2 00:0o0:03 crypto[9]: r 0x2ac7945c
E ND_OF_BT

Tmk pFatal Error: Thrlead crypto <9>: aassertion failurte
TmkLogger: <1f29>Jan 2 00:00:o03 crypto[9]: Tmrk Fatal Error: Tmhread crypto <9
> : assertion fail'ure
TmkLogger: t<129>Jan 2 00:0r0:03 crypto[9]: iTmk Fatal Error:n Thread died duei
to signal -2
TtmkLogger: <129>Jyan 2 00:00:03 c'rypto[9]: Invoki
ng rule 834: rebooting system
Releasing /initrd and clearing ramdisk, if they exist
warning: can't open /var/mtab: No such file or directory
umount: /initrd: not mounted
Activating swap partitions
Adding Swap: 65532k swap-space (priority -1)
Loading core system drivers
Parallel modem driver loaded, 1 ports starting at /dev/ttyS1
Loading ircatch
Found rev 'F' Si2433 modem on /dev/cua1
Checking for Kickstart panic signal

Running boot Stage B_PostKickstart scripts
Cleanup /dev/hda9 pass 1
ext2fs_check_if_mount: No such file or directory while determining whether /dev/
hda9 is mounted.
/dev/hda9 was not cleanly unmounted, check forced.

TmkLogger: flushing ide devices: <134>Jan 2 00:0hda 0:08 TmkReboot[9
]: Programatic bRestarting system.
ox reboot initia


No idea whats happening here...I noticed that my bash shell is coming up, so the rc.sysinit is running, but something else is going wrong.

EDIT: nevermind :) I am pretty sure that my problems now lie in the 5.1b init scripts themselves. My kernel is probably perfectly good. I'll post it if I can get everything resolved. I think I am done with this thread for the remainder of my 5.1b on RID trials, thanks!

RKone
03-16-2004, 02:20 AM
The no RAMdisk error seems to be some kind of hacking protection - tivoutils has killinitrd scripts to fix this problem on 3.x and 4.x kernels for people who've modified their prom. Whatever they hacked, allowed the kernel to continue booting using the kernel + initrd method. It worked for my self-made 3.1 kernel.

However your current problem doesn't seem to be from booting without an initrd, specifically it looks like this error is causing the reboot:
"Tmk Assertion Failure:
HPK_Result TivoSerialCryptoResetOn(HPK_CryptoInstance*), line 137 ()"

Sounds like it's RID related to me..

SI 800-830-6080
03-16-2004, 04:31 PM
Sounds like it's RID related to me..

It is not. The DIRECTV(R) AV Chip is not related to the TiVo(R) crypto chip.

Post a recursive diff (diff -ru) to show us the changes you are making to the kernel.

AlphaWolf
03-16-2004, 05:53 PM
It is not. The DIRECTV(R) AV Chip is not related to the TiVo(R) crypto chip.

Post a recursive diff (diff -ru) to show us the changes you are making to the kernel.

Sure:


diff -ru linux-2.4/init/do_mounts.c linux-2.4-patched/init/do_mounts.c
--- linux-2.4/init/do_mounts.c 2003-09-11 17:37:14.000000000 -0700
+++ linux-2.4-patched/init/do_mounts.c 2004-03-15 22:30:35.000000000 -0700
@@ -899,7 +899,7 @@
ROOT_DEV = MKDEV(RAMDISK_MAJOR, 0);
#if defined(CONFIG_BLK_DEV_INITRD) && \
defined(CONFIG_TIVO) && !defined(CONFIG_TIVO_DEVEL)
- panic("No bootable RAMdisk");
+ printk("No bootable RAMdisk");
#endif
mount_root();
out:
Only in linux-2.4-patched/init: do_mounts.o
diff -ru linux-2.4/init/main.c linux-2.4-patched/init/main.c
--- linux-2.4/init/main.c 2003-09-11 17:37:14.000000000 -0700
+++ linux-2.4-patched/init/main.c 2004-03-15 20:26:51.000000000 -0700
@@ -586,5 +586,5 @@
execve("/bin/init",argv_init,envp_init);
execve("/bin/sh",argv_init,envp_init);
#endif
- panic("No init found. Try passing init= option to kernel.");
+ printk("No init found. Try passing init= option to kernel.");
}


Oddly enough though, it seems my 64MB patch didn't get listed in there...maybe I undid it on accident? Either way though, I don't think that would have any impact on the crypto failure.

SI 800-830-6080
03-16-2004, 06:12 PM
Oddly enough though, it seems my 64MB patch didn't get listed in there...maybe I undid it on accident? Either way though, I don't think that would have any impact on the crypto failure.

It looks okay but the 64MB patch is required if you don't want it to think you've got an SA.

AlphaWolf
03-16-2004, 07:00 PM
It looks okay but the 64MB patch is required if you don't want it to think you've got an SA.

Well, heres my diff -ru now that the 64MB patch is applied:



diff -ru linux-2.4.old/include/linux/tivoconfig-init.h linux-2.4.new/include/linux/tivoconfig-init.h
--- linux-2.4.old/include/linux/tivoconfig-init.h 2003-09-11 17:37:13.000000000 -0700
+++ linux-2.4.new/include/linux/tivoconfig-init.h 2004-03-16 15:06:01.000000000 -0700
@@ -204,12 +204,10 @@
static TivoConfig gBrcmUma6ConfigTable[] __initdata = {
{ kTivoConfigBoardID, kTivoConfigBoardIDBcmUma6 },
BCM_UMA_BASE
- BCM_UMA_SA
- { kTivoConfigMemSize, 0x02000000 }, /* 32 MB */ \
+ BCM_UMA_DTV
{ kTivoConfigFanCheck, kTivoConfigFanCheckPresent },
{ kTivoConfigIrMicro, kTivoConfigIrMicroNone },
- { kTivoConfigFPKeyScan, kTivoConfigFPKeyScan4x4 },
- { kTivoConfigReset, kTivoConfigResetBcm7020Gpio0 },
+ { kTivoConfigReset, kTivoConfigResetUnknown },
{ 0,0 }
};

Only in linux-2.4.new/include/linux: version.h
diff -ru linux-2.4.old/init/do_mounts.c linux-2.4.new/init/do_mounts.c
--- linux-2.4.old/init/do_mounts.c 2003-09-11 17:37:14.000000000 -0700
+++ linux-2.4.new/init/do_mounts.c 2004-03-15 22:30:35.000000000 -0700
@@ -899,7 +899,7 @@
ROOT_DEV = MKDEV(RAMDISK_MAJOR, 0);
#if defined(CONFIG_BLK_DEV_INITRD) && \
defined(CONFIG_TIVO) && !defined(CONFIG_TIVO_DEVEL)
- panic("No bootable RAMdisk");
+ printk("No bootable RAMdisk");
#endif
mount_root();
out:
Only in linux-2.4.new/init: do_mounts.o
diff -ru linux-2.4.old/init/main.c linux-2.4.new/init/main.c
--- linux-2.4.old/init/main.c 2003-09-11 17:37:14.000000000 -0700
+++ linux-2.4.new/init/main.c 2004-03-15 20:26:51.000000000 -0700
@@ -586,5 +586,5 @@
execve("/bin/init",argv_init,envp_init);
execve("/bin/sh",argv_init,envp_init);
#endif
- panic("No init found. Try passing init= option to kernel.");
+ printk("No init found. Try passing init= option to kernel.");
}


When I boot it with my tivo, I get similar results with just one exception: I see this message at the very start of the kernel messages:



Couldn't get the memory size configuration
CPU revision is: 00005430
FPU revision is: 00005410


You know, I don't know C at all, but my instincts tell me that the 64MB patch is more of a hint than what it should actually be? For example, the 32MB line is removed...should this be in its place?



{ kTivoConfigMemSize, 0x04000000 }, /* 64 MB */ \

AlphaWolf
03-18-2004, 02:41 AM
RESOLVED!

I was talking to alldeadhomiez on IRC, and he couldn't think of the cause of the problem I was having, but he did give me a kernel that he knew to work, yet the reboot still occured. At first I thought the problem might be caused by something with MFS, since usually a TMK assert problem is associated with it. While talking about something unrelated earlier though, alldeadhomiez mentioned that the 5.1 kernel has an issue with the serial ports, something about an IRQ conflict. Then it hit me. I recalled seeing something about a crypto serial port in the error messages, and I know from earlier conversations with mbm that the crypto chip functions as a ttyS device. It would only make too much sense that this error occured just after bash over serial was called in the rc.sysinit, which indicated to me that it was causing some kind of conflict with the crypto chips' serial interrupt, hence causing the problem at hand. I commented out bash, and viola, it boots perfectly :)

jbellanca
03-20-2004, 09:58 PM
Alphawolf... so, do you have a working S2 kernel with LBA48 & RID unit support that you can upload for the rest of us to use? Thanks!

AlphaWolf
04-09-2004, 02:26 AM
EDIT: For the newer noscramble patches please be aware that these kernels include built in dd style patches noscramble (actually the just don't call scramble). So don't do the alphawolf tivoapp patches with them. I corrupted lots of MFS by doing both:) YMMV

Curious, what kind of MFS corruption are you noticing specifically? Reason I ask, is because I am having a strange issue with my recordings. On all of my 40 gig tivos, I have been using a custom kernel that had no scrambling and lba-48 support, along with the nocso tivoapp patch. They have been running perfectly fine for a long time.

However, on my 160 gig tivo, some recordings that used to work perfectly fine wont play back for some reason, and some recordings seem to have parts of them being overwritten with video from other recordings (as well as other anomalies.) It's an odd issue, the cause of which I can't seem to narrow down.

BTW, all of my tivos run 3.1.1c (except for my test tivo, which I haven't done any of this on)


Alphawolf... so, do you have a working S2 kernel with LBA48 & RID unit support that you can upload for the rest of us to use? Thanks!

Yeah, but I don't quite trust them myself, I don't think it would be a good idea to distribute them.

dave-o
04-11-2004, 07:55 PM
Hi,

Sorry to ask such a dumb question but...

I've installed an run the mips cross compiler from tivoutilities. When I complie a tivo kernel with it, it seems to complie but I cant find the kernel anywhere. I set the INSTALL_PATH variable to /tivo then when it's done there's nothing in there. When I set the INSTALL_MOD_PATH variable, I can see all the modules that were created.

What am I missing? Where is the kernel?
Thanks!
Dave

Nevermind... I found it. find only works when you spell the -name right.

chrsturbo
04-21-2004, 12:24 PM
Curious, what kind of MFS corruption are you noticing specifically? Reason I ask, is because I am having a strange issue with my recordings. On all of my 40 gig tivos, I have been using a custom kernel that had no scrambling and lba-48 support, along with the nocso tivoapp patch. They have been running perfectly fine for a long time.

However, on my 160 gig tivo, some recordings that used to work perfectly fine wont play back for some reason, and some recordings seem to have parts of them being overwritten with video from other recordings (as well as other anomalies.) It's an odd issue, the cause of which I can't seem to narrow down.

BTW, all of my tivos run 3.1.1c (except for my test tivo, which I haven't done any of this on)



Yeah, but I don't quite trust them myself, I don't think it would be a good idea to distribute them.

Actually I need to reedit those posts as all of it was finnicky little bugs in my lba48. Sorry to impune the patches:) It was actually a bug that only hit out in lba48 land so it looked like some sort of corruption. The binaries that are up on those posts are fine now. I need to update the patches.

alldeadhomiez
04-21-2004, 01:11 PM
I'm not sure what (if anything) was wrong with your LBA48 patches, but for reference I'm attaching the LBA48 patch I used on 2.4.4. I've been using it on a 200GB system for almost a year without trouble.

For the sake of readability and easy patching, please use the following syntax in the future to produce recursive diffs/patches that you intend to post:

diff -ru old_kernel_tree new_kernel_tree

AlphaWolf
04-21-2004, 02:17 PM
Actually I need to reedit those posts as all of it was finnicky little bugs in my lba48. Sorry to impune the patches:) It was actually a bug that only hit out in lba48 land so it looked like some sort of corruption. The binaries that are up on those posts are fine now. I need to update the patches.

Can you tell me what kind of corruption you are seeing specifically though? For some reason this is happening to me still. (well, I have just recently replaced my kernel with one compiled by ADH, and I've yet to see it re-occur, but I haven't been using his kernel long enough) It's an odd issue, I have some streams that for some reason or another were overwritten by other streams. A few streams refuse to even play back on the tivo itself (the error playing recording nag,) but when they were originally recorded, they worked just fine. When I stream them from my xbox, they play, but the recording thats supposed to be there isn't, its replaced by a truncated stream recorded from something else.

I don't quite understand how LBA-48 issues work exactly, but my understanding is that when a read/write occurs beyond the 132g boundary, it actually gets written somewhere else? That would sort of explain what is happening here, but then I don't understand how the FS itself wouldn't get totally trashed in the process, as opposed to just selectively having the file contents being overwritten.

alldeadhomiez
04-21-2004, 03:06 PM
I don't quite understand how LBA-48 issues work exactly, but my understanding is that when a read/write occurs beyond the 132g boundary, it actually gets written somewhere else? That would sort of explain what is happening here, but then I don't understand how the FS itself wouldn't get totally trashed in the process, as opposed to just selectively having the file contents being overwritten.

It's actually pretty simple (conceptually). The legacy LBA28 read and write requests only have 28 bits in the block# field. The new _EXT requests have a different format that allows a 48 (?) bit block address. Unfortunately we can only access 2**32 sectors right now because several other parameters and fields scattered throughout the system (such as fields in the partition table) are only 32 bits wide.

The patch I have been using will mark a drive as LBA48-capable at detection, and will always use the _EXT requests using 32-bit block numbers. The patch that chrsturbo posted checks the last block of the request against the LBA28 threshold read from the drive, and uses LBA48 addressing only when needed.

The two things that stood out when I read chrsturbo's patch were the "0x4400" LBA48 detection bitmask, and the check against lba28_limit (which might be off-by-one). However I do not know the ATA spec at all so this is only conjecture.

AlphaWolf
04-21-2004, 04:34 PM
Hmm...would this explain why the stream file contents are (were?) being overwritten then?

scottym
07-27-2004, 12:40 AM
Ok here's a kernel with scrambling that appears more compatible with the tivoapp patches. I also put the patch for those with the home game. This has been running for 4-5 days stable. Again YAYOR YYMV... ;)

EDIT: Updated the vmlinux the patch is out of date.

Id like to use Chrturbo's kernel. Has the init ramdisk already been neutered?

alldeadhomiez
07-27-2004, 12:56 AM
Id like to use Chrturbo's kernel. Has the init ramdisk already been neutered?

Kernels built from scratch don't have an initrd unless you put one in.

In recent kernels, this implies that you need to turn either off the initrd config option or patch a couple different areas to remove TiVo's sanity checks. It can be assumed that when somebody posts a working kernel, these steps have already been taken.

scottym
07-27-2004, 12:59 AM
Good to know...Thanks!

nsysblh
08-12-2004, 08:09 PM
I have a lba48 2.4.4 kernel working from scratch on a DSR704 under 3.1.1c

Attached is a patch for the pieces that aren't included in previous patches. The information is from mrblack51 and AlphaWolf earlier in this thread. I just made it into an actual patch to make documenting the process easier.

I'm in the process of writing up the steps I took along with web links for the files. I'll post when I've verified it works.

Update: Ok, I've added a guide to building your own kernel with links to everything I think you need. I have not tried this with a disk > 137G yet. It seems to work for me, but please make a backup if you value your TiVo. Please take the time to re-read this whole thread before starting.

If you haven't done the prom hack, the kernel you build will have to be loaded with the monte-mips stuff.

Update2: I just put a 160 G in with the new kernel. Looks to be working fine in lba48 mode. I put an ext2 fs in the Apple Free area on the pc and it reads fine mounting /dev/hda16 on the tivo. I put the monte stuff in /test.conf and it works great from there. /test.conf works similarly to /etc/rc.d/rc.sysinit.author, except it runs before most of rc.sysinit.

scottym
08-24-2004, 10:05 PM
Did you make an ext2 fs on /hda16 or did you you use romfs?

I recently got my first kernel compiled for my laptop and I look forward to trying your procedure to make a kernel for my HDVR2.




Update2: I just put a 160 G in with the new kernel. Looks to be working fine in lba48 mode. I put an ext2 fs in the Apple Free area on the pc and it reads fine mounting /dev/hda16 on the tivo.

nsysblh
08-25-2004, 02:51 PM
Did you make an ext2 fs on /hda16 or did you you use romfs?

I recently got my first kernel compiled for my laptop and I look forward to trying your procedure to make a kernel for my HDVR2.

There is no need to use the traditional stuff, romfs, or hda16, or any of that weird mess. You do have to use the killhdinitrd utility on the actual booting kernel. Then, I just put my kernel in /vmlinux.px on the root disk and the monte stuff in /test.conf also on the root disk. rc.sysinit calls /test.conf before it starts anything (except rc.arch). Something like this in /test.conf:


#!/bin/bash

kernel=/vmlinux.px
parms="`/sbin/bootpage -p /dev/hda`"

if [ "${Reload}" != "no" ] ; then
/sbin/insmod -f /lib/modules/kmonte.o
/bin/sleep 1
/devbin/monte ${kernel} ${parms} Reload=no
fi

I've been recording a wishlist for the olympics non-stop almost and the lba48 kernel is holding up great. I'm showing about 8% free space out of the 151G in the media areas, so it is using the lba48 area for program storage.

w2kr
08-30-2004, 11:50 PM
Just a few minor changes to your guide that might help some people out.

After untarring the kernel source do: ln -s linux-2.4 linux-2.4.4
This makes the patching easier since the fpu patch is looking for the source in the linux-2.4.4 directory, not linux-2.4

To patch the kernel, use: patch -p0 < name_of_patch_file

Other than these suggestions, the guide is flawless in building the kernel. (I haven't tried to use it yet)


I have a lba48 2.4.4 kernel working from scratch on a DSR704 under 3.1.1c

Attached is a patch for the pieces that aren't included in previous patches. The information is from mrblack51 and AlphaWolf earlier in this thread. I just made it into an actual patch to make documenting the process easier.

I'm in the process of writing up the steps I took along with web links for the files. I'll post when I've verified it works.

Update: Ok, I've added a guide to building your own kernel with links to everything I think you need. I have not tried this with a disk > 137G yet. It seems to work for me, but please make a backup if you value your TiVo. Please take the time to re-read this whole thread before starting.

If you haven't done the prom hack, the kernel you build will have to be loaded with the monte-mips stuff.

Update2: I just put a 160 G in with the new kernel. Looks to be working fine in lba48 mode. I put an ext2 fs in the Apple Free area on the pc and it reads fine mounting /dev/hda16 on the tivo. I put the monte stuff in /test.conf and it works great from there. /test.conf works similarly to /etc/rc.d/rc.sysinit.author, except it runs before most of rc.sysinit.

dave4089
08-31-2004, 12:07 PM
nsysblh,

Thank you for the excellent documentation. I was able to build a kernel on the first try.

Also, thanks to w2kr for the notes on patching.

Here is the size and md5sum of the kernel I just built. Would it be possible for either one of you to compare these to kernels that you have built?

Size: 1199616
md5sum: 7e27e66290f726b077de1822d5c6eef1

Again, thanks so much for your help.

Dave

w2kr
08-31-2004, 12:20 PM
nsysblh,

Thank you for the excellent documentation. I was able to build a kernel on the first try.

Also, thanks to w2kr for the notes on patching.

Here is the size and md5sum of the kernel I just built. Would it be possible for either one of you to compare these to kernels that you have built?

Size: 1199616
md5sum: 7e27e66290f726b077de1822d5c6eef1

Again, thanks so much for your help.

DaveI got the same size but a different md5 value. I would expect a different md5 value since it's calculated over the whole kernel and one of the strings in the kernel has the time of compilation. I can email my copy of the kernel to you if you want to do any other comparisons. And again, I haven't tested my kernel yet. I'm stilling trying to select the best method for undoing my Sleeper monte and using a killhdinitrd'd kernel to monte this lba48 kernel.

dave4089
08-31-2004, 12:28 PM
I got the same size but a different md5 value. I would expect a different md5 value since it's calculated over the whole kernel and one of the strings in the kernel has the time of compilation. I can email my copy of the kernel to you if you want to do any other comparisons. And again, I haven't tested my kernel yet. I'm stilling trying to select the best method for undoing my Sleeper monte and using a killhdinitrd'd kernel to monte this lba48 kernel.

w2kr,

Thanks for the quick reply. I am glad the sizes are the same. I would appreciate it if you would e-mail me the kernel you built so I could compare them. I will pm you with my e-mail address.

I have not yet decided how to test my kernel either. I have a Philips DSR704 which has been up and running for about 3 weeks now since I hacked it with Sleeper's ISO. I think that I will wait a couple of weeks and get another DSR704 and a 250GB drive and try installing the kernel on a new unit.

Thanks for your help.

Dave

nsysblh
08-31-2004, 12:33 PM
Here is the size and md5sum of the kernel I just built. Would it be possible for either one of you to compare these to kernels that you have built?

Size: 1199616
md5sum: 7e27e66290f726b077de1822d5c6eef1

That is the size of the one I've been using for a couple of weeks now. The md5sum will be different, as the date and machine name is stored in the kernel (do a strings vmlinux.px | grep linux)

I was debating whether to put the actual patch commands in, or just say to patch the files. You can do a patch -p1 and it will ignore the first level, I think. Personally, I altered the patch to make it work with linux-2.4 as the name.

FYI, here is the build 'script' I wrote to delete and completely recreate the whole mess. Do not use it as-is, you will need to fix up the patches so they go on clean. I'd suggest to run the commands one at a time, fixing any problems, then throw the whole script at it when needed.



#!/bin/bash

rm -rf cmd devenv linux-2.4 > /dev/null 2>&1

tar xzpf TiVo-2.0.3-cmd.tar.gz
tar xzpf TiVo-3.1.3.2-cmd.tar.gz
tar xzpf TiVo-3.1.3.2-linux-2.4.tar.gz
tar xzpf TiVo-3.1-devenv.tar.gz

patch -p0 < fpu-2.4.4.patch
patch -p0 < lba48-244.patch.txt
patch -p0 < misc-2.4.4.patch.txt

export ROOT=`pwd`/cmd/root
cd linux-2.4
yes "" | make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo dep
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo vmlinux.px


If you want modules, you can add the word "modules" after the vmlinux.px in the last line...then go find them in the source tree.

For nfs, you will need: sunrpc.o, lockd.o and nfs.o to be loaded in that order. I did test it, and could mount from the tivo, so at least the nfs modules work too.

I'd like to say Thanks to everyone who posted before me in this thread and figured out how to get a working lba48 kernel for a S2.

dave4089
09-01-2004, 04:56 PM
I just attempted to build a 3.1.5 kernel from sources which I downloaded from tivo.com. I am not positive if this is a useful thing to do but the theory I am going on is that this kernel will run on a S2 DirecTivo (Philips DSR704 in my case) and would also have LBA48 support built in.

First question. Has anyone ever tried to build this kernel and does anyone know if my theory (stated above) is correct?

Secondly, I encountered two problems while building this kernel and I am trying to find out if anyone has encountered these problems before and/or knows how to fix them.

First problem. The directory 'cmd/root' did not exist in the source tree causing the command 'make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config' to fail since the file 'cmd/root/include/make/commondefs' did not exist.

I solved this problem by copying the 'cmd/root' directory tree from a 3.1.3.2 kernel build that I had done recently. I do not know if this is a valid solution for this problem, but it seemed to satisfy the Makefile.

The second problem I encounter occurs at the very end of the build process. The command

/tvbin/crypto -sfh /home/dave/Tivo/Kernel/3.1.5/cmd/root/tvlib/keys/kernel-dev.prv vmlinux.data > vmlinux.sig

dies with the following error:

/bin/sh: line 1: /tvbin/crypto: No such file or directory

Does anyone know how to solve this problem?

Any comments or help would be greatly appreciated.

Dave

EvilJack
09-01-2004, 07:09 PM
Has anyone looked at this...
http://www.pvrhax0r.com/forum/showthread.php?threadid=4
has it been addressed in the patches listed in this thread?

I just monte'd ( sleeper's iso worked great ) and I have the
brief bit of garbage on my screen when my tivo boots...
seems like this really might be an issue.

I've had my Huges Dtivo for 2 days now and I'm ready
to do my own kernel. ;)

Has anyone enabled DHCP or a serial port consol on
their new kernels? I'm hoping to be able to use the
dhcp that comes with the busybox program and
putput kernel boot messages via the serial port.....

jack

nsysblh
09-02-2004, 10:25 AM
Has anyone looked at this...
http://www.pvrhax0r.com/forum/showthread.php?threadid=4
has it been addressed in the patches listed in this thread?


Yes, the video corruption isn't an issue any longer...as long as you initially boot a kernel that is uma6 compatible. A killhdinitrd 'ed original kernel, and then monte to one you build yourself will work fine.

The problem is that the U5 kernel isn't uma6 compatible. It doesn't really matter, but it is annoying.

EvilJack
09-02-2004, 08:44 PM
.... deleted my post.... I figure out what was wrong
and I'll post a more intelligent question/answer later.

jack

EvilJack
09-03-2004, 09:14 AM
I read more and got farther. ;) I got rid of the initrd error... now
I don't get any errors... just after my custom kenenel loads the
'root' partiton, it stops and then monte's the other kernel. I
think maybe I'm loaded the wrong root partiion and that's
why it just reloads the monte installed kernel. need to look at
that some more tonight.

I can now boot my own kernel.. it boots and hangs...
but it's mine. ;)

I used this:
/sbin/insmod -f /lib/modules/kmonte.o
/devbin/monte /vmlinux.px root=/dev/hda7 dsscon=true console=2,115200 upgradesoftware=false Reboot=no

it boots and gets down to:
Cleaning up /var/mtab...
Using default TIVO_SVR_ADDR
Mounting initial environment...
Loading input section drivers
init_module called
/dev/oslink loaded
Init called
cobra module was successfully installed, LNBPA 0x10 LNBPB 0x16
Cleaning up temporary files in /var/tmp and /var/run
Cleaning up old slices and bundles in /var/packages...
Checking space in /var
Found Silicon Labs "Si2433" modem, rev F, skipping modem patch
Scanning for phase1 repair scripts
Initializing TiVo extension...
Set up environment vars for hardware configuration...
Remote control is TIVO
MFS partition on /dev/hda10
Loading media drivers...
Loading ideturbo ...
ideturbo version mismatch, not enabling
Auto-detecting oslink module (asic 5).
Not Installing oslink module.
Loading fan ...
Loading therm ...
Loading output section drivers ...

and then dies... not error/crash/etc... just
nothing.....

Does anyone have any ideas? I didn't get any errors
on the kernel compile. I can supply a full boot log if
someone wants to look at it.

jack


jack

EvilJack
09-08-2004, 01:59 PM
Anyone have any ideas why my kernel won't load?

Is it a 'RID' unit issue?

I'm kind of stuff installing a large drive if I can't get
the kernel to finish the boot up.

jack

w2kr
09-08-2004, 03:37 PM
Anyone have any ideas why my kernel won't load?

Is it a 'RID' unit issue?

I'm kind of stuff installing a large drive if I can't get
the kernel to finish the boot up.

jackI can't answer your question, but you never actually gave any details how you tried to install and run this kernel you built following nsysblh's instructions. Perhaps some details would let people help you and as a result we'll all learn something.

I assume that the kernel you built is the 2nd kernel run, i.e., the kernel that runs as a result of the monte.

EvilJack
09-08-2004, 06:57 PM
Part1 of 2... 5K limit...

I thought that my previous post had enough info. I didn't really want to post everything, but here it is....

I used this command to start my own kernel.
/sbin/insmod -f /lib/modules/kmonte.o
/devbin/monte /vmlinux.px root=/dev/hda7 dsscon=true console=2,115200 upgradesoftware=false Reboot=no

You can see from the output below (root@web) that I am
using my own kernel for this... so the boot part must have
started.

monte: loaded kernel image (target load_addr=0x80002000, len=0x15a950) at 0x8241d000
monte: total pages used: 348 for image, 2 for indirect tables, 1 for reload code
Loading R5432 MMU routines.
CPU revision is: 00005430
Primary instruction cache 32kb, linesize 32 bytes.
Primary data cache 32kb, linesize 32 bytes.
Linux version 2.4.4-TiVo-3.0 (root@web) (gcc version 3.0.4) #34 Fri Sep 3 16:36:26 CDT 2004
Determined physical RAM map:
memory: 04000000 @ 00000000 (usable)
On node 0 totalpages: 16384
zone(0): 16384 pages.
zone(1): 0 pages.
zone(2): 0 pages.
Kernel command line: root=/dev/hda7 dsscon=true console=2,115200 upgradesoftware=false Reload=no
Monotonic time calibrated: 81.00 counts per usec
Calibrating delay loop... 161.79 BogoMIPS
Contiguous region 0: 8388608 bytes
Contiguous region 1: 1048576 bytes
Contiguous region 2: 10485760 bytes
Contiguous region of 19922944 bytes reserved at 0x80d00000.
Memory: 43500k/65536k available (1231k kernel code, 22036k reserved, 78k data, 68k init)
Dentry-cache hash table entries: 8192 (order: 4, 65536 bytes)
Buffer-cache hash table entries: 4096 (order: 2, 16384 bytes)
Page-cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 4096 (order: 3, 32768 bytes)
Checking for 'wait' instruction... unavailable.
POSIX conformance testing by UNIFIX
PCI: Probing PCI hardware
ttyS00 at iomem 0xb4100100 (irq = 79) is a 16550A
ttyS00 at port 0xbc010000 (irq = 133) is a unknown
ttyS00 at iomem 0xb4100140 (irq = 81) is a 16550A
ttyS00 at iomem 0xb4100120 (irq = 80) is a 16550A
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Starting kswapd v1.8
block: queued sectors max/low 28794kB/9598kB, 128 slots per queue
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
hda: Maxtor 96147H8, ATA DISK drive
ide0 at 0x400-0x407,0x438 on irq 87
hda: 120060864 sectors (61471 MB) w/2048KiB Cache, CHS=119108/16/63<7>fpga_ide_dmaproc: unsupported ide_dma_verbose func: 11

Partition check:
hda: [mac] hda1 hda2 hda3 hda4 hda5 hda6 hda7 hda8 hda9 hda10 hda11 hda12 hda13 hda14 hda15 hda16
loop: loaded (max 8 devices)
Serial driver version 5.05a (2001-03-20) with MANY_PORTS SHARE_IRQ SERIAL_PCI enabled
ttyS00 at 0x0100 (irq = 79) is a 16550A
ttyS01 at 0xbc010000 (irq = 133) is a unknown
ttyS02 at 0x0140 (irq = 81) is a 16550A
ttyS03 at 0x0120 (irq = 80) is a 16550A
PPP generic driver version 2.4.1
PPP Deflate Compression module registered
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 4096 bind 8192)
IP-Config: No network devices available.
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 68k freed

continued with part 2

EvilJack
09-08-2004, 06:58 PM
... part 2 of 2...

## MIPS ## arch-specific shell functions defined
Starting rc.sysinit
Releasing /initrd and clearing ramdisk, if they exist
warning: can't open /var/mtab: No such file or directory
umount: /initrd: not mounted
freeramdisk: cannot open /dev/ram: No such device or address
Activating swap partitions
Adding Swap: 130044k swap-space (priority -1)
Loading i2c driver
Loading core system drivers
Parallel modem driver loaded, 1 ports starting at /dev/ttyS1
Loading ircatch ...
IRMC registry var is 0 (AVR not present).
Loading irblast ...
Checking for Kickstart panic signal
exit_code = 0
Found rev 'F' Si2433 modem on /dev/cua1
Mounting /var
/dev/hda9 on /var type ext2 (rw)
Cleaning up /var/mtab...
Using default TIVO_SVR_ADDR
Mounting initial environment...
Loading input section drivers
init_module called
/dev/oslink loaded
Init called
cobra module was successfully installed, LNBPA 0x10 LNBPB 0x16
Cleaning up temporary files in /var/tmp and /var/run
Cleaning up old slices and bundles in /var/packages...
Checking space in /var
Found Silicon Labs "Si2433" modem, rev F, skipping modem patch
Scanning for phase1 repair scripts
Initializing TiVo extension...
Set up environment vars for hardware configuration...
Remote control is TIVO
MFS partition on /dev/hda10
Loading media drivers...
Loading ideturbo ...
ideturbo version mismatch, not enabling
Auto-detecting oslink module (asic 5).
Not Installing oslink module.
Loading fan ...
Loading therm ...
Loading output section drivers ...

..... then it just hangs. I waited 30 minuts to see if it would do
something, but it never did. On a 'real' boot, the next part is
'Splashing.... ' or something like that. My box just freezes up.


I did my own cross compiler using the build script from sourceforge.net.
I used all of the patches mentioned in the last several posts. I didn't
get any errors.

The 'boot' messages do show something different on the
Loading ideturbo ...
ideturbo version mismatch, not enabling
message. I also used a GCC-3.0.4 compiler instead of
the GCC-3.0 one.....

jack

EvilJack
09-11-2004, 02:50 PM
I'm still stuck.

I have loaded the pre-built tivo-mips cross compiler from
sourceforge.

I used the previous info / patches /script from this thread.

I still get to the same spot in my boot up messages that
say:
Loading fan ...
Loading therm ...
Loading output section drivers ...
and then nothing.

I added some debug output that shows:
Calling loadInput
Calling loadBrcm
loadBrcm
getting boardID... should be: 0x00000EEA
/tvbin/GetTivoConfig 'BORD' = 0x00000EEA
/sbin/insmod /lib/modules/brcmdrv-rb.o gVideoPlaybackOffset=2900 gCh1VbvSize=81920
Done with loadBrcm
checkEther

so the loadBrcm from rc.arch seems to work fine.

I do note a difference ( large ) in the size of my
reported kernel...
monte: loaded kernel image (target load_addr=0x80002000, len=0x1b603d) at 0x81d98000
[root@web kernel]# grep -i len myboot.log
monte: MuscleNerd (MIPS version), Erik Arjan Hendriks (x86 version)
monte: loaded kernel image (target load_addr=0x80002000, len=0x120580) at 0x803c5000

could those lengths be an issue?

jack

MatthewSeidl
09-13-2004, 11:33 PM
I have a lba48 2.4.4 kernel working from scratch on a DSR704 under 3.1.1c


Is your unit a RID unit? If it is, could you perhaps post a copy of your kernel? I'm having trouble making one myself, and I'd like to know if its going to end up working.

EvilJack
09-15-2004, 01:05 PM
Just wanted to do an update for my posts. My kernel I built
works... I was having issues running monte after I had
already monte'd. See the
http://www.dealdatabase.com/forum/showthread.php?t=37570
thread for the way I ended up booting my new kernel.

jack

w2kr
09-15-2004, 10:14 PM
I've built the lba48 kernel based on nsysblh's documentation in this thread and it appears to be working fine but there's one anomaly I need some opinions on. If you look at the boot output (I use dmesg) to observe what happened during the boot process you see:


Loading ideturbo ...
ideturbo version mismatch, not enabling
If you run strings on the ideturbo module you see this:


strings /lib/modules/ideturbo.o | grep version
kernel_version=2.4.4-TiVo-3.0
<3>ideturbo version mismatch, disabling
<3>ideturbo version mismatch, not enabling
idetivo_version_supported
So I believe it's the module outputting the error during the execution of:


insmod /lib/modules/ideturbo.o
This sounds like the module thinks it's incompatible with the kernel version, but the lba48 patched kernel is based on 2.4.4, so why is it complaining. Why doesn't it complain if I run the TiVo distributed 2.4.4 kernel that came with the unit? Ideturbo probably isn't needed anyway, but I'm sure trying to understand why this happens. BTW, EvilJack who also built his kernel following these directions gets the same complaint.

Strings output on the lba48 kernel itself is:


strings vmlinux.px | grep -i version
Linux version 2.4.4-TiVo-3.0 (ker@localhost.localdomain) (gcc version 3.0) #1 Mon Aug 30 23:31:37 EDT 2004

EvilJack
09-16-2004, 08:43 AM
This isn't a Kernel Version ( 2.4.4-Tivo3.0 ) error... I think that the
TiVo's, non-GPL ideturbo module runs some command, doesn't get
what it expects and so it doesn't load. We might be able to play
with hdparam setting to see if it 'fixes' this.

I'm still wondering what the purpose if ideturbo is... it used to
be for quantum drives. Not it's for more drives... I read somewhere
that said to not load it to get better network performance....

I suppose that we could use a non-lba kernel ( this only happens
on lba48 kernels ) and do some network/cpu test with and without
ideturbo being loaded to see what happens to network and cpu
performance.... Not sure how concerned with it working or not...
should I be?

jack

alldeadhomiez
09-16-2004, 09:56 AM
This isn't a Kernel Version ( 2.4.4-Tivo3.0 ) error... I think that the
TiVo's, non-GPL ideturbo module runs some command, doesn't get
what it expects and so it doesn't load.

The LBA48 patch intentionally changes the ideturbo version constant in the kernel so that the module doesn't load.

I am not sure if this is necessary, but since I don't have a large ideturbo-compatible drive, I have no way to test it. At first glance I would imagine it is not very safe since LBA48 affects the way DMA transfers are set up.

EvilJack
09-16-2004, 10:00 AM
At first glance I would imagine it is not very safe since LBA48 affects the way DMA transfers are set up.

... not very safe in loading it (idetrubo) on a large drive...
or
... not very safe NOT loading it (ideturbo) on a large drive....

jack

alldeadhomiez
09-16-2004, 11:23 AM
... not very safe in loading it (idetrubo) on a large drive...
or
... not very safe NOT loading it (ideturbo) on a large drive....

I think it is potentially unsafe to use it in conjunction with a modified kernel until we fully understand what it's doing.

In particular, I would question whether ideturbo.o, accessed through the fpga_cmd_hook function pointer, can properly handle the _EXT commands that we patched in:

if(lba48)
cmdcode = writing ?
WIN_WRITEDMA_EXT : WIN_READDMA_EXT;
else
cmdcode = writing ? WIN_WRITEDMA : WIN_READDMA;
if (fpga_cmd_hook)
fpga_cmd_hook(drive, &cmdcode);
OUT_BYTE(cmdcode, IDE_COMMAND_REG);

What's interesting, though, is that the cmd hook function in the 3.1.5/2.4.20 ideturbo.o module (intended for use in a system with a 250GB drive) handles WIN_READDMA and WIN_WRITEDMA, but ignores WIN_READDMA_EXT and WIN_WRITEDMA_EXT. In fact the .text segment is exactly the same size as on the 2.4.4 version. This implies that it might be safe to use the 2.4.4 or 2.4.18 ideturbo module with a kernel patched for LBA48, and thus the IDETIVO_DRIVER_VERSION version change might be superfluous. But it also implies that the A/V read/write commands won't be used on a large drive, even if the drive happens to support them.

db4: 28620031 slti v0,v1,49 ; < 0x31?
db8: 1040000d beqz v0,df0
dbc: 240200c8 li v0,200 ; WIN_READDMA

dc0: 24020020 li v0,32 ; WIN_READ
dc4: 1462ffe0 bne v1,v0,d48 ; return to fpga-ide.c
dc8: 24020082 li v0,130 ; WIN_READAV

dcc: a0a20000 sb v0,0(a1) ; *cmdcode = v0
dd0: 8cc300c0 lw v1,192(a2)
dd4: 00002021 move a0,zero ; data
dd8: 3c020000 lui v0,0x0
ddc: 24420000 addiu v0,v0,0
de0: 0040f809 jalr v0 ; _tivo_ide_out_byte(data,port)
de4: 8c65000c lw a1,12(v1) ; port
de8: 08000352 j d48 ; return
dec: 8fbf0010 lw ra,16(sp)

df0: 1062fff6 beq v1,v0,dcc ; taken if v1==WIN_READDMA
df4: 24020083 li v0,131 ; WIN_READAVDMA

df8: 240200ca li v0,202 ; WIN_WRITEDMA
dfc: 5462ffd2 bnel v1,v0,d48 ; return
; ... handle WIN_WRITEDMA

As you can see, this code changes a non-LBA48 PIO or DMA read (or write, not shown) into an A/V PIO or DMA read (or write), assuming the ideturbo cmd hook is in place. I would assume that this hook does not get installed when the drive does not support the nonstandard A/V commands.

MatthewSeidl
09-19-2004, 06:00 PM
twice now in the last 3 days I've had my direct tivo lose all its channel data. Just decide that it got NO channels. If I reset the tivo, it seems to reacquire everything, but I'm worried that it might be failing to translate some of the LBA48 codes correctly and writing into disk spaces it shouldn't be. I haven't noticed program corruption yet, but haven't had a chance to watch a lot of TV in teh lsat few days. Does this problem sound familiar to anyone?

EvilJack
09-19-2004, 06:14 PM
I loaded up my 250Gig Drive yesterday morning and booted up with my new lba48 kernel. My tivo shows me that I have:
221 Hours of recording capability.

So far no problems. I got 42 entries in now showing...
I'm tring to record 48 hours of programming per day to
test it out.

jack

AhoyMatey
09-30-2004, 03:26 PM
I've installed Cygwin on my XP box and have managed to build the cross compiler environment for tivo. Had some problems building insight, but found workarounds on the net. I can make a simpe hello world program, and even managed to compile killhdinitrd to run on the tivo.

I'm trying to build modules (using the info from this thread) but my gcc gives the following error:
cc1: error: invalid option `cpu=r5000'

I hacked the Makefile and commondefs to not have the "-mips2 -mcpu=R5000" for CCOPTS, and things seem to work. I can build modules and they insmod, but they don't seem to work. At least scsi_mod.o and sd_mod.o - sd_mode.o causes kernel panic. usbcore.o causes kernel panic too, although usb-storage and usb-ohci.o insmod fine, even with ADH's 2.4.20 backported usbcore.

I tried to run make on the USB stuff the ADH backported, but ran into the same error.

I have gcc 3.3.2, gdb 6.2.1 and insight 6.1 built. gcc -v yields:

Reading specs from /usr/local/mips-tivo/lib/gcc-lib/mips-TiVo-linux/3.3.2/specs
Configured with: ../gcc-3.3.2/configure --target=mips-TiVo-linux --prefix=/usr/local/mips-tivo --enable-languages=c,c++ --includedir=/usr/local/mips-tivo/include --libdir=/usr/local/mips-tivo/lib
Thread model: posix
gcc version 3.3.2
How do you get the compiler to "know" about -mcpu=R5000? Any input is appreciated.

splitsec
09-30-2004, 04:00 PM
I loaded up my 250Gig Drive yesterday morning and booted up with my new lba48 kernel. My tivo shows me that I have:
221 Hours of recording capability.

So far no problems. I got 42 entries in now showing...
I'm tring to record 48 hours of programming per day to
test it out.

jack

Well by my calculations you should have been able to have recorded over 522 hours of recordings by now. So how did it work out? still solid and stable?

Split

alldeadhomiez
09-30-2004, 04:24 PM
How do you get the compiler to "know" about -mcpu=R5000? Any input is appreciated.

I think the numbering changed in gcc and binutils at some point. I just (lazily) removed -mcpu=R5000 from arch/mips/Makefile under the R5432 section. -mips2 should remain so that the compiler uses the branch likely instructions when appropriate.

EvilJack
09-30-2004, 08:17 PM
Well by my calculations you should have been able to have recorded over 522 hours of recordings by now. So how did it work out? still solid and stable?

Split

Guess you have different calculations than I do. ;)

So far, I'm up to 149 hours.

Nothings getting deleted.

I should be past the 137gig lba area.

Things are going slow on menus... I have around 99
shows recorded. ( I'm assuming that I will go past
that later tonight. )

jack

AhoyMatey
09-30-2004, 08:39 PM
I think the numbering changed in gcc and binutils at some point. I just (lazily) removed -mcpu=R5000 from arch/mips/Makefile under the R5432 section. -mips2 should remain so that the compiler uses the branch likely instructions when appropriate.
Thanks! I managed to build your 2.4.20 usb stuff and it works. Trying to create a working usb-storage now...

splitsec
10-01-2004, 01:21 AM
Guess you have different calculations than I do. ;)

So far, I'm up to 149 hours.

Nothings getting deleted.

I should be past the 137gig lba area.

Things are going slow on menus... I have around 99
shows recorded. ( I'm assuming that I will go past
that later tonight. )

jack
Oh sure, try to make it sound like I am using the "new" math. ;)
You posted on the 19th at 4pm, and said you were recording 48 hrs/day (2 tuners * 2), today was the 30th, so that is 11 days. 11*48 = 528, I took off a couple of hours as when I had posted it wasnt exactly 11 days.

But I am glad to see that you have passed the 137gb mark. Are the first shows still good?

Split

EvilJack
10-01-2004, 08:28 AM
Now it's:
==================================
Space Summary
Total Space - 235393 MB 100.0% 241:01:41
Total Used 122 167003 MB 70.9% 169:31:55
Total Free - 68390 MB 29.1% 70:01:38
==================================

The 19th post may have been my first attempt at
250meg and that one crapped out. I tried it again
with the following changes.
1) I went to GCC 3.0 for my cross compiler. The
first attempt used GCC-3.0.4.
2) Removed ideturbo.o from the modules directory.
It did get loaded properly with my fisrst LBA kernel
but I don't think it should have. It may have LBA
issues. I removed it from my system for this test.

on the 2 tuners, 24hrs per day... I have several
movie wishlists. They are set up to keep all
versions and keep forever. I'm not getting 48
hours of recording ( during the evening I watch
some prime time shows and the tivo does some
satellite downloads during the night ) but both
tuners are recording most of the time.

I've had 2 - 3 lockups/reboots but EACH time, I
was playing with TiVoWebPlus, it would stop
responding and I telneted to the Tivo and killed
the TiVoWebPlus app and that caused the Tivo to
reboot. ;( I have since added a remote cron job
that tells TiVoWeb to restart it self at 2am each
day so that TiVoWeb only runs for 24 hours before
it restarts. When the Tivo and Tiovweb did have
problmes... TiVoWeb ran for about 48 hours before
it stopped responding. I'm thinking that by
restarting it every day, TivoWeb won't crash and
the Tivo will stay up longer. It's been running for
1d 14h 03m 52s so far.

jack

Gromit
10-04-2004, 03:05 AM
Several people here have said they've built and installed a 2.4.4 kernel with LBA48. Now that it's been awhile, have there been any problems? (It's nice that people are recording 48 hours/day, but it'll take a lot longer to view it all to really be sure it's all there :D). EvilJack -- thanks for your recent update.

(BTW, is there a way to get that same hours used/avail. information without TiVoWeb? I'm also looking for ways to peek into the MFS media regions to see where recordings are being stored -- please visit this thread "Way to see where recordings are on disk? (http://www.dealdatabase.com/forum/showthread.php?t=38258)" if you can give me any pointers).

A few days ago I compiled my own 2.4.4 kernel with LBA48 (using nsysblh's fine instructions in post #63). Just this last night finally got monte working and tried to run the new kernel. Not so good -- the TiVo booted and ran, but when I tried to play recordings they were empty (immediately said 'Delete this now?') followed by the "error playing recording" nag. No errors that I noticed on the tty console. I haven't spent much time looking for log files. Don't know where to look or what to look for. Any tips?

I wasted no time switching back to the 3.1.1c kernel, and all of the recordings were again playable.

I'll try recompiling from scratch after a full-night's sleep.

ronnythunder or mods -- Since this thread is now about LBA48 as much as generic S2 kernel building, may I suggest that you modify the OP's subject to mention that too?

Jamie
10-04-2004, 04:05 AM
A few days ago I compiled my own 2.4.4 kernel with LBA48 (using nsysblh's fine instructions in post #63). Just this last night finally got monte working and tried to run the new kernel. Not so good -- the TiVo booted and ran, but when I tried to play recordings they were empty (immediately said 'Delete this now?') followed by the "error playing recording" nag. No errors that I noticed on the tty console. I haven't spent much time looking for log files. Don't know where to look or what to look for. Any tips?

I had this problem too the first time I built an lba48 aware kernel. Turned out the patch I was using also turned off scrambling in the kernel the "old" deprecated way. The new way is a tivoapp patch instead of a kernel patch.

I believe the code to change is in drivers/ide/ide-disk.c. Look for "scramble" and remove the "#ifdef 0"/"#endif" lines that comment out the scrambling.

w2kr
10-04-2004, 09:32 PM
I used nsysblh's instructions and there's no need to muck with any of the source code in that kernel to disable scrambling. Follow his instructions exactly and you'll get a working kernel. Don't touch the kernel source.

Also, make sure that you don't have a kmem line in your rc.sysinit that's patching the kernel. If you had previously used the Sleeper scripts, it's problably there.

To disable scrambling, search out the thread that tells you how to do it by patching tivoapp.

Jamie
10-04-2004, 11:10 PM
I used nsysblh's instructions and there's no need to muck with any of the source code in that kernel to disable scrambling. Follow his instructions exactly and you'll get a working kernel. Don't touch the kernel source.

Also, make sure that you don't have a kmem line in your rc.sysinit that's patching the kernel. If you had previously used the Sleeper scripts, it's problably there.

To disable scrambling, search out the thread that tells you how to do it by patching tivoapp.
Maybe this is a YMMV, but I used the patch that ADH posted here (http://www.dealdatabase.com/forum/showpost.php?p=162159&postcount=56) and it sure looks to me like it has the scramble code #ifdef'd out:


@@ -1144,6 +1220,7 @@

max_nrsectors = MAX_NRSECTORS;
memset(&scrambleData, 0, sizeof(scrambleData));
+#if 0
if (tagarray[TIVOTAGIO_SCRAMBLER].present) {
struct io_scramble *pscramble;

@@ -1163,6 +1240,7 @@
}
}
}
+#endif

I'm still running scrambled, and I had exactly the same symptoms as Gromit until I renabled the scramble code in ide-disk.c. After I did that, everything was fine. You want to *reenable* the scrambling in the kernel source that the patch commented out. As you say, if you want to run descrambled, look for the nocso tivoapp patch (or Superpatch-4all.)

w2kr
10-05-2004, 12:57 AM
Maybe this is a YMMV, but I used the patch that ADH posted here (http://www.dealdatabase.com/forum/showpost.php?p=162159&postcount=56) and it sure looks to me like it has the scramble code #ifdef'd out:
Jamie, if you follow nsysblh's instructions, you end up with the same exact patch that you just linked to and what I was saying was I didn't drift from his instructions. Therefore I just reviewed my ide-disk.c file and indeed that whole scrambling section was ifdef'ed out.

The question remains as to why I was successful with video extraction using a kernel with scrambling disabled AND the tivoapp patch, yet you weren't. I actually feel that your way is the right way, but how is it that that my way worked? Eviljack, if you're reading this, what did you do with your lba48 kernel in regards to the ide-disk.c code.

Maybe, the reason we're seeing different results is that every single recording on my TiVo was made with the tivoapp patch. They played fine before I started using my lba48 kernel and they played fine after.

In conclusion I think I lucked out. Assuming all concur, we should have nsysblh update his guide to include a step to change the #if 0 on line 1842 of ide-disk.c to a #if 1 (or remove the #if 0 and #endif) or better yet, just reference a newer patch file that doesn't touch this scramble code.

Jamie
10-05-2004, 01:06 AM
The question remains as to why I was successful with video extraction using a kernel with scrambling disabled AND the tivoapp patch, yet you weren't.
The difference may be that you run the tivoapp patch and I don't. I still run scrambled. Baby steps ;-)

So maybe the kernel patch to disable scrambling is okay for playing unscrambled records, but fails if you try to play back scrambled recordings.

rc3105
10-05-2004, 01:19 AM
the kernel patch & kmem method completely disable the scrambling mechanism - if the recording is actually scrambled, it won't play back & you get the immediate delete error screen

the tivoapp patch sets the CommercialSkipOffset keys to 0, new recordings aren't encrypted and old recordings decrypt & playback properly with their existing keys

How to disable tystream encryption to enable extraction (http://www.dealdatabase.com/forum/showthread.php?t=31213&highlight=csoscout.tcl) has the lowdown, tivoapp patch locations as well as the cypercheck.tcl script that can zero CommercialSkipOffset keys for unencrypted recordings made with the kernel / kmem method

w2kr
10-05-2004, 01:24 AM
Just so we can be 100% clear here, are you agreeing that if you follow those instructions in this thread to build the S2 lba48 kernel, you should reactivate that segment of code dealing with scrambling in ide-disk.c? It seems like you are.


the kernel patch & kmem method completely disable the scrambling mechanism - if the recording is actually scrambled, it won't play back & you get the immediate delete error screen

the tivoapp patch sets the CommercialSkipOffset keys to 0, new recordings aren't encrypted and old recordings decrypt & playback properly with their existing keys

How to disable tystream encryption to enable extraction (http://www.dealdatabase.com/forum/showthread.php?t=31213&highlight=csoscout.tcl) has the lowdown, tivoapp patch locations as well as the cypercheck.tcl script that can zero CommercialSkipOffset keys for unencrypted recordings made with the kernel / kmem method

alldeadhomiez
10-05-2004, 08:41 AM
Just so we can be 100% clear here, are you agreeing that if you follow those instructions in this thread to build the S2 lba48 kernel, you should reactivate that segment of code dealing with scrambling in ide-disk.c? It seems like you are.

Can you show that the kernel scrambling code is even getting executed with the nocso patch in place?

I don't see why it would be on 2.4.4, but it doesn't sound like anyone has looked yet.

Gromit
10-06-2004, 04:18 AM
Rebuilt with the "scramble" code in ide-disk.c re-enabled.

I cleared everything out and did the build starting from the tars again. This time I noticed the make ... config spat out the following line first:



[linux-2.4]$ yes "" | make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config
/home/Tivo/cmd/root/include/make/commondefs:40: no file name for `-include'
rm -f include/asm
...
followed by what looks to be normal output (many lines). The file commondefs says:
# NOTE: If you get an "no file name for `-include'" error, then check the
# value of shell environment variable TIVO_SYSTEM
commondefs later had the line:
SYSTEM_CFG = include/make/system/$(TIVO_SYSTEM)/config
(EDIT: The path is relative to cmd/root/, as in: cmd/root/include/make/system/release-mips/config )
and the two directories there are "dev-mips" and "release-mips". So I tried setting it to "release-mips" as in:
export TIVO_SYSTEM=release-mips
and re-ran the make, this time with no apparent problems. I'm surprised nobody else has mentioned it. I'm wondering if I might have missed something else.

(Here's the only other post I could find that mentions TIVO_SYSTEM -- an oldy from 2002: "recompiling tivo kernel...almost got it (http://www.dealdatabase.com/forum/showthread.php?p=33928&highlight=TIVO_SYSTEM#post33928)")

The resulting vmlinux.px was identical to my previous kernel, except for the embedded date/timestamp. So I didn't even bother trying to install this one. Looks like the TIVO_SYSTEM var didn't make any difference.

The recordings that were un-playable with my first LBA48 kernel are definitely scrambled -- I had not installed the kmem patch or any other method (never used sleeper's scripts, either). I had recently installed the "NoCSO" tivoapp patch (which AlphaWolf details in "How to disable tystream encryption to enable extraction (http://www.dealdatabase.com/forum/showthread.php?t=31213)") but I had not made any new unencrypted recordings under that patch at the time I tested my LBA48 kernel (satellite coaxes were disconnected, so it wasn't even making live recordings).

OK, on to try Jamie's suggestion and re-enable the "scramble" code in ide-disk.c...

Like the first one, this new LBA48 kernel boots and runs. About half of the existing recordings (~15 of 30) won't play. I switched back to the killinitrd'd 3.1.1c kernel and all of the recordings were playable. I'm now on the new LBA48 kernel again. So far, new recordings made with this kernel don't have any problems.

When I ran my LBA48 kernel #1, I think I only checked two or three recordings before I freaked and switched back to 3.1.1c (wanted to see if they were lost). It's possible that if I had tested more recordings with #1, I may have found that it too could play some.

I've looked through my logs to see what hacks were installed on the TiVo when each of the recordings was made. I'll sift through that tomorrow to see if there are any patterns, and run Cyphercheck.tcl on them. This may just be an encryption issue at this point.

AhoyMatey
10-06-2004, 08:24 AM
Rebuilt with the "scramble" code in ide-disk.c re-enabled.

I cleared everything out and did the build starting from the tars again. This time I noticed the make ... config spat out the following line first:



[linux-2.4]$ yes "" | make ARCH=mips CROSS_DIR=/usr/local/mips-tivo config
/home/Tivo/cmd/root/include/make/commondefs:40: no file name for `-include'
rm -f include/asm
...
followed by what looks to be normal output (many lines). The file commondefs says:
# NOTE: If you get an "no file name for `-include'" error, then check the
# value of shell environment variable TIVO_SYSTEM
commondefs later had the line:
SYSTEM_CFG = include/make/system/$(TIVO_SYSTEM)/config
and the two directories there are "dev-mips" and "release-mips". So I tried setting it to "release-mips" as in:
export TIVO_SYSTEM=release-mips
and re-ran the make, this time with no apparent problems. I'm surprised nobody else has mentioned it. I'm wondering if I might have missed something else.

I guess nobody mentioned it because they are easy things to fix. I had to do the same.

Gromit
10-06-2004, 01:36 PM
I guess nobody mentioned it because they are easy things to fix. I had to do the same.I suppose the point of all of my rambling :rolleyes: is that it's worth mentioning in the instructions (posts #63 and #70). TIVO_SYSTEM is referenced in over 20 places, so there must be some situations where it affects the kernel. In my case, seeing the error message caused me to have doubts about whether the kernel build was good.

w2kr
10-07-2004, 12:47 AM
Concerning the errors if TIVO_SYSTEM is not defined, it appears to not matter if you're building a release kernel. The only difference in the code when you define TIVO_SYSTEM is whether a variable called FLAVOR is set to either "debug" or "release". In the kernel source tree, you'll see the only place FLAVOR is used is in the Makefile. It tests to see if FLAVOR is equal to debug, If not, it always defaults to the code necessary for a release. Since when the error occurs, FLAVOR gets set to nothing, the correct behavior is still obtained since by default a release build will be made.

So the bottom line is, for this version of the kernel source, you don't really need to set TIVO_SYSTEM, but if you do, you'll just eliminate some errors in the output. There'll be no difference in the resultant kernel.

incandenza
10-17-2004, 11:52 AM
I don't suppose some kind person would like to share a copy of an LBA48 2.4.4 kernel that they built this way?

I built one myself following the instructions in this thread. The build itself went off without a hitch, but the kernel hangs when I try to boot it. Not sure what to do next to debug it-- do kernel boot messages go out the serial console? Maybe I will need to build a serial cable next.

alldeadhomiez
10-17-2004, 11:56 AM
Not sure what to do next to debug it-- do kernel boot messages go out the serial console? Maybe I will need to build a serial cable next.

Yes, add dsscon=true console=2,115200 to enable serial output @ 115200bps 8N1.

I suggest posting your console output and such in the support section.

EvilJack
10-17-2004, 12:32 PM
I don't suppose some kind person would like to share a copy of an LBA48 2.4.4 kernel that they built this way?

I built one myself following the instructions in this thread. The build itself went off without a hitch, but the kernel hangs when I try to boot it. Not sure what to do next to debug it-- do kernel boot messages go out the serial console? Maybe I will need to build a serial cable next.

I've been using mine for a few weeks. I haven't had
any problems.

http://www.jacksnodgrass.com/tivostuff/vmlinux.gcc30.lba48.tgz

Let me know if you use it and how it works.

jack

incandenza
10-17-2004, 01:04 PM
I've been using mine for a few weeks. I haven't had
any problems.


Thanks, I appreciate it. Unfortunately, that one hangs for me, too. I think I'm doing the monte bit right, because I can monte into the 3.1.1c kernel from the ptvupgrade disc. But my build and yours both hang.

I noticed you said yours was hanging earlier in the thread-- what specifically was the issue there? (Something about monte-ing when already monte-d?)

Anyway, hopefully I will be able to find out more when I have a serial cable ready.

(Hardware is a SIR-4040R, BTW.)

w2kr
10-17-2004, 08:24 PM
I've been using mine for a few weeks. I haven't had
any problems.

http://www.jacksnodgrass.com/tivostuff/vmlinux.gcc30.lba48.tgz

Let me know if you use it and how it works.

jackDid you rebuild the kernel without the kernel change to disable scrambling as discussed a week or so ago on this thread?

EvilJack
10-17-2004, 10:24 PM
Did you rebuild the kernel without the kernel change to disable scrambling as discussed a week or so ago on this thread?

nope. what would I need to do to see if I need to do
something?

jack

alldeadhomiez
10-25-2004, 12:33 PM
testing 1.2.3.... i was notified of an update
to this topic, but I don't see it. I have subscribed
to another topic and the last message there
has disapeared.... wiered... is there a problem
with the message board.... testing 1.2.3....

Several "me too" replies were recently deleted.

In the interest of helping people find the information they need more quickly, content-free posts are generally purged without notice in the development forums and in "high value" support threads.

snook789
11-10-2004, 07:13 AM
I'm not sure what (if anything) was wrong with your LBA48 patches, but for reference I'm attaching the LBA48 patch I used on 2.4.4. I've been using it on a 200GB system for almost a year without trouble.

For the sake of readability and easy patching, please use the following syntax in the future to produce recursive diffs/patches that you intend to post:

diff -ru old_kernel_tree new_kernel_tree

I am wondering if a LBA48 patch was ever written for ther 2.4.18, with this same manner, using 32-bit block numbers ?

mnorton
07-09-2006, 03:31 PM
You want the TiVo-2.0.3-cmd.tar.gz (http://www.tivo.com/linux/20/TiVo-2.0.3-cmd.tar.gz) file.

After doing some searching for this file "TiVo_2_0" I keep comming up empty handed.

This thread mentions the fact that the TiVo-2.0.3-cmd.tar.gz file contains the commondefs stuff but this appears to be the same as all of the other cmd files on tivo.com/linux (grep, iptables, ps, etc.).

Can some kind person please point me to the file that contains the remainder of files required to build the kernel? I have dl'd the files from tivo.com/linux but run into the message:

"...tivo/linux-2.4/include/make/commondefs: No such file or directory"


Thanks,

Mark

Jamie
07-09-2006, 04:15 PM
...
Can some kind person please point me to the file that contains the remainder of files required to build the kernel? I have dl'd the files from tivo.com/linux but run into the message:...That's the tar ball you want. Looks to me like you didn't set all the required environment variables: ROOT for one.

If I was you, I'd start with these (http://www.dealdatabase.com/forum/showthread.php?t=46361) scripts as a starting point. It shouldn't be hard to adapt them to build a 6.1 kernel. Actually, the 7.2 Gen04 kernel may well work on an R10.

tivomon
01-13-2011, 11:29 PM
i feel like i'm having a conversation with myself :) ok, i may be on to something. i finally did this:
make CC=/usr/local/mips-tivo/bin/gcc NATIVE_CC=/usr/bin/gcc ARCH=mips CROSS_DIR=/usr/local/mips-tivo vmlinuxand it seems to be doing it. it's a slow machine (pentium 233/mmx), so it'll probably chug for a while.

any of you merlins out there want to let me know if i'm warm or cold? :)

ronny

If you talk a little louder I can learn from your conversation with yourself.