DD Question

[Blackfoot]

My SAT T-60 attempted to install the 3.5 upgrade without my knowledge. It actually did this right after 3.5 was starting to be pushed out. The unfortunate part is that I have a 300 Gb drive and I was using an LBA48 aware kernel. So now it just hangs at the Almost There screen. I was thinking about duplicating my drive over to a spare 300 Gb drive and play around with swapping the partitions back. I hooked up the 2 drives and booted off of Jafa's cachecard CD with LBA48 support. (My original MFStools 2.0 CD did not have a LBA48 kernel.) I ran the "dd if=/dev/hdc of=/dev/hdd bs=1024" command and it has been running for about 32 hours now. Obiviously with no progress indicator I'm wondering if this is expected as my drive was about 50% full? Is there a better way to do this? Also, is it possible to go from one large drive to 2 smaller drives and still preserve your recordings? I have no idea when the new 3.5 kernel will be out and I'm anxious to catch up on some of my shows and movies that are sitting on my large drive.

[mike_s]

I ran the "dd /if=/dev/hdc of=/dev/hdd /bs=1024" command and it has been running for about 32 hours now. Obiviously with no progress indicator I'm wondering if this is expected as my drive was about 50% full?

dd doesn't care how full the disks are, it copies it all.

Open and login to another tty console (alt-F2, alt-F1 goes back to the first console). From the new console, do a "ps aux" and find the pid for dd, there will be a line likeusername 27317 0.0 0.4 11648 2524 ? S 03:54 0:00 dd /if=/dev/hdc of=/dev/hdd /bs=1024in the above, 27317 is the PID. Things may run slow, dd is working hard.

Now enter "kill -USR1 27317" using whatever the PID you found was. Switch back to the original console and dd should print out a status line showing progress. (it may take a minute before it gets around to it). It reports how many records of size "bs" it has completed, figuring out how many total records there are to be copied is an excercise left to the user. "bs=1024" isn't very efficient for copying whole disks - making it bigger will help the speed a lot (i.e. bs=1024K)

[ScanMan]

The only thing I would add is you have a couple extra backslashes(maybe just typos); the command should be:

dd if=/dev/hdc of=/dev/hdd bs=1024K

You could even make the bs larger even 2048K; this will take a while - dd'd 300 gigs will probably take over 24 hours...

[Blackfoot]

Now enter "kill -USR1 27317" using whatever the PID you found was. Switch back to the original console and dd should print out a status line showing progress. (it may take a minute before it gets around to it). It reports how many records of size "bs" it has completed, figuring out how many total records there are to be copied is an excercise left to the user. "bs=1024" isn't very efficient for copying whole disks - making it bigger will help the speed a lot (i.e. bs=1024K)

Correct me if I am wrong, but won't killing the PID kill the process?


The only thing I would add is you have a couple extra backslashes(maybe just typos); the command should be:

dd if=/dev/hdc of=/dev/hdd bs=1024K

You could even make the bs larger even 2048K; this will take a while - dd'd 300 gigs will probably take over 24 hours...

You are correct. Those were typos. The command you referenced is exactly what I typed. Since the dd has been running for 32 + hours now, would be be best to let it finish? Killing it and restarting it with bs=2048 would cause it to start over from the beginning right?

[ScanMan]

Yes, killing it will stop the process and you will have to start over. I would personally let it go a while longer and see if it finishes especially if it looks/sounds like it is still going strong (ie, hard drive activity lights, etc.).
If not, at some point I guess you have to kill it and start over.

Did you use the "K" after the block size? If not, it might take even longer...

[mike_s]

Correct me if I am wrong, but won't killing the PID kill the process?

You're not killing it, that would be "kill -KILL", you're sending a -USR1 signal using the kill command.

Hint:

man dd:
Note that sending a SIGUSR1 signal to a running `dd' process makes
it print to standard error the number of records read and written so far,
then to resume copying.

man kill

[ScanMan]

Wow, you learn something new every day...and my wife doesn't understand why I spend so much time here! Thanks for the clarification.

[Blackfoot]

You're not killing it, that would be "kill -KILL", you're sending a -USR1 signal using the kill command.
Hmmm... So I did just that.
kill -USR1 464
Went back to first console and noticed my cursor as blinking a lot slower. Waited a few more seconds and then got the following.
User defined signal 1
root@Knoppix:/#
I think it killed the processes. I tried to mount hdd4 and it came back with a bunch of bad access and unable to read block errors. The final error was that the mount failed. Should I try the DD command again, except this time with a K at the end or is there something else I can do?

Edit: I tried dd if=/dev/hdc of=/dev/hdd /bs=1024K and it says invalid number '1024K'. Little k works though. But is that the same as no k?

[ScanMan]

No, the little k will increase the blocksize to 1024 kilobytes (1 Megabyte) instead of the default 512 bytes. The little k means to multiply the value by 1024. 1024*1024 = 1048576 = 1M. Mike_S was right...man dd.

[Blackfoot]

What is man dd?

[mike_s]

Type "man dd" It's manual pages for the dd command. man should be about the first thing anyone dealing with *nix should learn to use (RTFM). Unfortunately, since TiVo is meant to be a closed system, there are no man pages on it.

I'm not familiar with "Jafa's cachecard CD." The almost universally standard dd on Linux is GNU coreutils dd, which supports the -USR1 signal and "big K" in block sizes. I suppose Jafa could have used some other version or a multicall binary (i.e. busybox) which doesn't support that, but that would be really foolish on a CD, where there's plenty of space available for the full utilities.

"dd --version" or "dd --help" might say what dd is being used. I see that TiVo uses GNU 3.16, which is 10 years old(!), which didn't support -USR1. I run Debian stable, which has a reputation for being out of date, and even it has coreutils 5.2.1.

[Blackfoot]

Type "man dd" It's manual pages for the dd command. man should be about the first thing anyone dealing with *nix should learn to use (RTFM). Unfortunately, since TiVo is meant to be a closed system, there are no man pages on it.

I'm not familiar with "Jafa's cachecard CD." The almost universally standard dd on Linux is GNU coreutils dd, which supports the -USR1 signal and "big K" in block sizes. I suppose Jafa could have used some other version or a multicall binary (i.e. busybox) which doesn't support that, but that would be really foolish on a CD, where there's plenty of space available for the full utilities.

"dd --version" or "dd --help" might say what dd is being used. I see that TiVo uses GNU 3.16, which is 10 years old(!), which didn't support -USR1. I run Debian stable, which has a reputation for being out of date, and even it has coreutils 5.2.1.
Ah. Thanks for the information. As you can see I am not a Linux person. I only know enough to get around and perform basic stuff. Busybox is what Jafa has on his CD. It looks like Knoppix 3.3. Does the MFStools 2.0 have a newer version of dd? Is there an ISO that I can download somewhere and run from a CD drive?
I kicked off another run of dd last night and set the BS=4096k. You can see that the machine is working hard as it takes a tad longer to refresh the screen between the console ports. This DD has been running for about 16 hours. I think I will give it until tomorrow morning before I kill it.
Is there a better/faster way to do this? I guess the real question is, do you think it will even be possible to swap my partitions back and boot off of he 3.1.0c2 OS? Or when a new kernel is available and I install it, will it correct the boot up problem and get my drive back up and running with everything still intact? I'm shocked that the source files have not been posted yet. It has been 2 weeks already.

Edit:
It looks like the version of DD I am running is BusyBox v1.00-pre4. That sounds pretty old. Where can I get a newer version that support the -USR1 signal?

Edited Edit:
I found my PTVUpgrade LBA48 CD and decided to kill my running DD to use it. Found out it uses DD (fileutils) 4.1 and it does support the -USR1. It didn't like the big K so i played around with using 4096, 2048 or 1024 byte sizes. I would kick off the DD, go over to the second console and run the kill -USR1 command. I had more records in and out in the same amount of time using the 1024k BS. I hope my logic was correct in determining this to be the fastest. Now I just need to know how to figure out how many records my 300 GB drive has.

[eastwind]

<snip>
I'm not familiar with "Jafa's cachecard CD." The almost universally standard dd on Linux is GNU coreutils dd, which supports the -USR1 signal and "big K" in block sizes. I suppose Jafa could have used some other version or a multicall binary (i.e. busybox) which doesn't support that, but that would be really foolish on a CD, where there's plenty of space available for the full utilities.
<snip>
Unless he wanted to build an ISO that would be as small as it possibly could be since everyone doesn't have broadband access (or to limit the bandwidth used--I heard that some people have to pay for that :) ).

ew

[everrest]

Placing the drives on seprate channels (primary-secondary) might speed the process.

[mike_s]

Unless he wanted to build an ISO that would be as small as it possibly could be since everyone doesn't have broadband access (or to limit the bandwidth used--I heard that some people have to pay for that His LBA48 ISO is 37 meg. The difference between busybox and having the real GNU binaries is 2-3 meg (and likely less, is there a need for adjtimex or netstat or ??? for hacking a TiVo from a PC?) and avoiding a bunch of potential problems due to incomplete/flawed implementations of those functions. Sounds like a false economy to me.

[Blackfoot]

I think I'm giving up on duplicating the drive. It got to about 132000 records in and out and then no matter how many times I did the kill -USR1 PID# command, it would not show me an updated number of records.

I guess I'm at the mercy of the Tivo gods to come up with a new kernel and hope that when I apply it to my 300 Gb drive it boots up ok. I know that I will have to rebuild my 200 Gb drive. It boots ok and some shows play. It is just a matter of time before it passes the 127 Gb barrier.