•
•
•
•
There are a miriad of dictionary tools on the Internet and for desktops. If you want, you can even pull out the dusty dictionary on your shelf. I find myself often wanting to use the dictionary not for a word definition, but to find out if I'm spelling a word correctly. Finding [...]
I've been exploring D the language. I really do like it, but that's another post. There are a couple of D compilers, but the only viable option on OS X seems to be gdc. I installed it via MacPorts. On Leopard, gdc generates assembly that makes the FSF gcc complain "indirect jmp without `*'" over and over. The bug is known, and other than being annoying it seems harmless.
So I decided what I needed was a script that would filter out these frivolous warnings without otherwise affecting stderr, and also without changing the exit status (so make can do the right thing). This turned out to be easier said than done. Finally I stumbled on the right incantation:
#! /bin/bash
gdc=/opt/local/bin/gdc
msg="indirect jmp without"
$gdc "$@" 2> >(grep -v "$msg" 1>&2)
We redirect stderr to the named pipe corresponding to that subshell (see "Process Substitution" in the bash manpage), then we redirect grep's output to its stderr. Because grep is in a subshell, its exit status doesn't mess up the exit status of the script, which is the exit status of gdc, as it should be.
Today in IRC suseROCKS needed to find all duplicate files in a directory by their content, not by their file name, so we whipped up this fancy little 1 liner bash script to do the trick:
find . -type f -exec md5sum '{}' \; | sort | awk 'dup[$1]++{print $2}’
EDIT:
As Andreas suggested, using xargs instead of -exec is much faster, here is the updated command:
find . -type f -print0 | xargs -0 md5sum | sort | awk ‘dup[$1]++{print $2}’
I'm in the process of moving over to my new laptop. I run into an issue where a shell script (that worked under gentoo) now no longer works. Here's the error foo.sh: 18: pushd: not found Since I'm running this as sh foo.sh, I turn on debugging wit
As Linux tools sometimes do, this little script was born out of frustration from the repetitive. And the meniality of the task is directly proportional to one’s desire to replace it with anything that will automate the process. If I have to do something twice on my Linux box, it gets automated.
So a few days ago I put together a bash script which I named “What’s Up?” The abbreviation for this is ’sup’ which is the command used to invoke the script. I use it to tell me which server I’m on, who I’m logged in as, the memory status of the box, and some other crazy junk.
Originally, this fool was 110 lines of code to display 13 lines of output. Fortunately, an altruistic and knowledgeable Lonnie Olson brought to my attention the fact that there was room for some nice optimizations. Of such coolness were these optimizations that they brought the line count from 110 to a mere 21, not including comments and empty lines.
If you’re just tuning in, the script looks like this:
#!/bin/bash
# ORIGINALLY WRITTEN BY SCOTT MORRIS (http://www.suseblog.com/) on 2008-05-28
# UPDATED AS SUGGESTED BY LONNIE OLSON on 2008-05-30
# COLLECT SOME INFO
IFS=$'\n'
UPTIME=`uptime`
D_UP=${UPTIME:1}
MYGROUPS=`groups`
DATE=`date`
KERNEL=`uname -a`
CPWD=`pwd`
# OUTPUT THE DATA
printf " user:\t\t"$USER" (uid:"$UID")\n"
printf " groups:\t"$MYGROUPS"\n"
printf " working dir:\t"$CPWD"\n"
printf " home dir:\t"$HOME"\n"
printf " hostname:\t"$HOSTNAME"\n"
ip -o addr | awk '/inet /{print " IP (" $2 "):\t" $4}'
printf " date:\t\t"$DATE"\n"
printf " uptime:\t"$D_UP"\n"
printf " kernel:\t"$KERNEL"\n"
printf " cpu:\t\t"$CPU"\n"
free -mot | awk '
/Mem/{print " Memory:\tTotal: " $2 "Mb\tUsed: " $3 "Mb\tFree: " $4 "Mb"}
/Swap/{print " Swap:\t\tTotal: " $2 "Mb\tUsed: " $3 "Mb\tFree: " $4 "Mb"}'
NOTE: If you copy and paste the above code and it does not work, just download it from the link provided above.
And its output looks like this:
[0137][scott@tomahawk:~]$ sup user: scott (uid:1000) groups: users dialout video working dir: /home/scott home dir: /home/scott hostname: tomahawk IP (lo): 127.0.0.1/8 IP (eth0): 192.168.0.110/24 date: Sat May 31 01:57:54 MDT 2008 uptime: 1:57am up 2 days 21:53, 5 users, load average: 0.27, 0.23, 0.18 kernel: Linux tomahawk 2.6.24-default #1 SMP Sat Jan 26 21:54:20 MST 2008 x86_64 x86_64 x86_64 GNU/Linux cpu: x86_64 Memory: Total: 940Mb Used: 925Mb Free: 14Mb Swap: Total: 1913Mb Used: 349Mb Free: 1564Mb [0157][scott@tomahawk:~]$
Logging into and out of many Linux servers per day with many different users can cause you to develop aggravated multiple personality disorders unless of course you use a cool script like this to cue the gray matter. I’m thinking about writing another one called ‘whoami’. Oh wait, someone already did that.
A couple of days ago, I put together a .bashrc alias. Well, it won’t work right. The commands in “ marks only execute when the shell opens, and then the variables they’re assigned to stay the same, even when you invoke the alias. Thus, the current working directory and the date and stuff that should change each time you invoke the alias, don’t.
To fix this, I changed it from an alias into a small bash script. To use it, just put it into your ~/bin folder and invoke it like you would any other command.
The contents are thus:
#!/bin/sh
# ORIGINALLY WRITTEN BY SCOTT MORRIS (http://www.suseblog.com/) on 2008-05-28
# DISPLAY THE MEMORY AND SWAP AVAILABLE FOR THE SYSTEM
function memdisp {
IFS=$' '
MEM=`free -mot | head -n 2 | tail -n 1`
COUNT=1
printf " Memory:"
for ITEM in $MEM
do
if [ $COUNT -eq 2 ] ; then
printf “\tTotal: $ITEM Mb”
fi
if [ $COUNT -eq 3 ] ; then
printf “\tUsed: $ITEM Mb”
fi
if [ $COUNT -eq 4 ] ; then
printf “\tFree: $ITEM Mb\n”
fi
COUNT=$[COUNT+1]
done
MEM=`free -mot | tail -n 2 | head -n 1`
COUNT=1
printf ” Swap:\t”
for ITEM in $MEM
do
if [ $COUNT -eq 2 ] ; then
printf “\tTotal: $ITEM Mb”
fi
if [ $COUNT -eq 3 ] ; then
printf “\tUsed: $ITEM Mb”
fi
if [ $COUNT -eq 4 ] ; then
printf “\tFree: $ITEM Mb\n”
fi
COUNT=$[COUNT+1]
done
}
# DISPLAY THE IP ADDRESS OF ETH0
function ipaddr {
IFS=$' '
IPINF=`/sbin/ifconfig eth0 | head -n 2 | tail -n 1`
COUNT=1
printf ” IP (eth0):”
for ITEM in $IPINF
do
if [ $COUNT -eq 2 ] ; then
# printf “$ITEM\n”
IFS=$':'
CT=1
for DATA in $ITEM
do
if [ $CT -eq 2 ] ; then
printf “\t$DATA\n”
fi
CT=$[CT+1]
done
fi
COUNT=$[COUNT+1]
done
IFS=$'\n'
}
# COLLECT SOME INFO
IFS=$'\n'
UPTIME=`uptime`
D_UP=${UPTIME:1}
MYGROUPS=`id`
DATE=`date`
KERNEL=`uname -a`
CPWD=`pwd`
# OUTPUT THE DATA
printf ” user:\t\t”$USER” (uid:”$UID”)\n”
printf ” groups:\t”$MYGROUPS”\n”
printf ” working dir:\t”$CPWD”\n”
printf ” home dir:\t”$HOME”\n”
printf ” hostname:\t”$HOSTNAME”\n”
ipaddr
printf ” date:\t\t”$DATE”\n”
printf ” uptime:\t”$D_UP”\n”
printf ” kernel:\t”$KERNEL”\n”
printf ” cpu:\t\t”$CPU”\n”
memdisp
If you copy and paste it, save it as ~/sup, and don’t forget to make it executable with chmod +x ~/sup.
Example output:
[1211][scott@tomahawk:~]$ sup user: scott (uid:1000) groups: uid=1000(scott) gid=100(users) groups=16(dialout),33(video),100(users) working dir: /home/scott home dir: /home/scott hostname: tomahawk IP (eth0): 192.168.0.110 date: Wed May 28 12:11:58 MDT 2008 uptime: 12:11pm up 8:07, 7 users, load average: 0.46, 0.43, 0.30 kernel: Linux tomahawk 2.6.24-default #1 SMP Sat Jan 26 21:54:20 MST 2008 x86_64 x86_64 x86_64 GNU/Linux cpu: x86_64 Memory: Total: 940 Mb Used: 756 Mb Free: 183 Mb Swap: Total: 1913 Mb Used: 0 Mb Free: 1913 Mb [1211][scott@tomahawk:~]$
Here’s a link to the script: sup.tar.bz2
Download the script.
Run: tar -xvf sup.tar.bz2
Run: mv sup ~/bin
Run: sup
Enjoy.
NOTE: Don’t use this, it has been updated. Go here for latest.
I was fooling around with an alias that would help someone know at a glance what machine they are on, who they are logged in as, their current path, the date, uptime, and some memory stats. This is something that I have found helpful when I have several remote servers open and logged into each one with several different accounts. It’s easy to know at a glance where I am doing what.
To implement this alias, pull open your ~/.bashrc file, and paste all of this at the end of it:
function memdisp {
MEM=`free -mot | head -n 2 | tail -n 1`
COUNT=1
for ITEM in $MEM
do
if [ $COUNT -eq 2 ] ; then
printf " Total RAM:\t$ITEM Mb\n"
fi
if [ $COUNT -eq 3 ] ; then
printf " Used RAM:\t$ITEM Mb\n"
fi
if [ $COUNT -eq 4 ] ; then
printf " Free RAM:\t$ITEM Mb\n"
fi
COUNT=$[COUNT+1]
done
MEM=`free -mot | tail -n 2 | head -n 1`
COUNT=1
for ITEM in $MEM
do
if [ $COUNT -eq 2 ] ; then
printf " Total SWAP:\t$ITEM Mb\n"
fi
if [ $COUNT -eq 3 ] ; then
printf " Used SWAP:\t$ITEM Mb\n"
fi
if [ $COUNT -eq 4 ] ; then
printf " Free SWAP:\t$ITEM Mb\n"
fi
COUNT=$[COUNT+1]
done
}
UPTIME=`uptime`
D_UP=${UPTIME:2}
alias sup="
printf ' my user:\t`whoami`\n'
printf ' my groups:\t`id`\n'
printf ' hostname:\t`hostname`\n'
printf ' domain:\t`dnsdomainname`\n'
printf ' date:\t\t`date`\n'
printf ' uptime:\t$D_UP\n'
printf ' kernel:\t`uname -a`\n'
memdisp
"
Then save the file, and run “source ~/.bashrc”. To use the alias, type ’sup’ (short for “what’s up?”) and hit ENTER. You should see something like this:
[1457][scott@suse-linux:~]$ sup my user: scott my groups: uid=1000(scott) gid=100(users) groups=16(dialout),33(video),100(users) hostname: suse-linux domain: truenorth.local date: Fri May 23 14:57:23 MDT 2008 uptime: 2:57pm up 5 days 18:35, 15 users, load average: 0.17, 0.12, 0.13 kernel: Linux suse-linux 2.6.24-default #1 SMP Sat Jan 26 00:29:01 MST 2008 i686 i686 i386 GNU/Linux Total RAM: 1264 Mb Used RAM: 1234 Mb Free RAM: 30 Mb Total SWAP: 2055 Mb Used SWAP: 213 Mb Free SWAP: 1841 Mb [1457][scott@suse-linux:~]$
This is great for when you come back to work from a long weekend, have 300 terminal windows open, logged into 32 servers with 43 different accounts.
If you wanted, you could also put this into the /etc/skel/.bashrc file so that all new users on your machine will automatically have this alias. Change to suit your taste.
If you do this, and an FBI satellite crashes into your new Porsche, it’s not my fault.
I am under no delusions of grandeur here. If you know of a better way to output the info, or more info that you’d like to output, or you modify/change it to make it better, please let us know. Suggestions, tips, tricks, comments, and even mild insults are welcome.
My T60p.
[clints@herlo-lap ~]$ history|awk ‘{a[$2]++ } END{for(i in a){print a[i] ” ” i}}’|sort -rn|head
144 svn
144 cd
108 ls
104 ./manage.py
101 ssh
69 su
43 screen
26 vim
25 rm
15 ping
[clints@thor ~]$ history|awk ‘{a[$2]++ } END{for(i in a){print a[i] ” ” i}}’|sort -rn|head
266 git
260 make
71 cd
57 ls
55 vim
55 rt
26 rm
19 bin/send-patch
18 grep
16 bin/validate
I guess I love RCS’.
Cheers,
Herlo
Have you ever been frustrated with perfectly valid bash syntax being highlighted as incorrect in vim? Like this:
#!/bin/sh
foo=$(ls /tmp)
$() is a perfectly valid, nay preferred substitute for backticks. The problem is that vim is deciding this is pure old bourne shell instead of whatever else we'd like it to be. If you change the shebang to #!/bin/bash and re-edit the file, then the error markings go away. But maybe you're writing for the nebulous POSIX shell, not bash nor sh. Or maybe you just don't care and you don't want vim complaining that you're using bashisms even though your shebang says sh. You can set the defaults so that it reflects your system and preferences. It's all there in :help sh.vim.
My friend Sam posted a blog on his top 15 commands used from the commandline, so here are mine:
sontek@inspidell:~> history | awk ‘{print $2}’ | awk ‘BEGIN {FS=”|”} {print $1}’|sort|uniq -c | sort -n | tail -n 15 | sort -nr
143 ls
135 cd
84 vim
69 exit
57 ssh
56 su
35 svn
25 man
24 rm
24 python
22 sudo
22 jhbuild
18 make
17 grep
16 xrandr
You can tell a lot about a person by their top 15 commands and as you can see with mine, the majority of mine are used for coding!
You can see a break down of the command I used to list these here: http://czarism.com/my-top-ten-linux-comments-history
What are your top 15 commands?
I was helping a friend debug a problem with gksu (gnomesu alternative) today and we chose to use strace which allows you trace system calls an application makes.
To monitor all system calls an application makes you can redirect the output to a file like so:
strace <command> 2> <file name>
or
strace <command> -o <file name>
These commands return the exact same results, the first command redirects stderr (standard error, which has the file descriptor 2) to the file, strace sends all output to stderr by default, the second command uses the built in -o argument which is much cleaner.
One of the first things I like to do with strace is to check if it is having trouble accessing a file, which I see a lot because the file doesn’t exist or the user executing the command does not have permission to access it, you can do that with these commands:
strace <command> 2>&1 |grep open
or
strace <command> -e open
Again, these commands will return similar results. The first command redirects stderr to stdout so you can use grep to filter the output. The second command is the preferred method because it actually uses the built in -e argument which will trace only the named system call (this is a comma separated list so you can do strace -e open,read).
The only other arguments that I’ve found really helpful are -ff which when used with -o will append the pid (process id) to the file name and -F which will also trace children.
I’m lazy, so I just have this basic script I run that upgrades my wordpress:
#!/bin/bash
blog_directory=
update_url=
wget http://wordpress.org/latest.tar.gz
tar -xzvf latest.tar.gz
tar -zcvf blog-backup-$(date +’%F’).tar.gz $blog_directory
cp -rv wordpress/* $blog_directory
links $update_url
rm latest.tar.gz
How do you upgrade your wordpress?
If you use Linux for day-to-day computing, you likely use the secure shell, or ssh. If you are like me, you may grow weary of constantly having to type in passwords to access remote machines. Or maybe you have the perfect backup system, except that it uses ssh to transfer files, and requires you to type in a password (such as rsync or rdiff-backup). There is a way to access those machines without using a password. This technique should be used with care. I’d use it only on machines that I have access to, for example. You don’t want to set up passwordless access from a public machine to your production server, in other words. Use with caution.
The principle is that you generate a public and private key on the local machine. This will be whatever machine you are connecting from. You then transfer the public key to the remote machine. Then, when you ssh into the remote machine, it uses the keys to authenticate. You don’t type in a password, it just takes you straight to the shell prompt. How do we set this up?
Log into the machine you are going to connect from. Let’s say that your account is called ‘user’ and you are going to connect from a machine called ‘desktop’. Log in as ‘user’ on the ‘desktop’ machine and pull up a shell. Run this command. The stuff in red is what you do,not what you type:
[0218][user@desktop:~]$ ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/home/user/.ssh/id_dsa): [JUST PRESS ENTER HERE] Enter passphrase (empty for no passphrase): [JUST PRESS ENTER HERE] Enter same passphrase again: [JUST PRESS ENTER HERE] Your identification has been saved in /home/user/.ssh/id_dsa. Your public key has been saved in /home/user/.ssh/id_dsa.pub. The key fingerprint is: a5:25:c0:aa:fe:f3:9f:46:7a:23:e3:6e:10:ec:6f:d3 user@desktop [0218][user@desktop:~]$
Your keys are generated. On that machine, view /home/user/.ssh/id_dsa.pub. You will see something like this:
ssh-dss AAAAA3NzAC1kc3MAAACAANfnr9xkAks2u8Xd5I9nozzsXLKCYA8o9h3cMwL5QMcZGxz84Dyl3IyRwozv3I2xGwJAYyAHvJAS7AQmtFXtfmQts3dt2ZnlAmKj0TkC8pwDgvXJYzARJpCP2Zx7iwD/OCO3sTgT0wGwYtQrAcTeW5Tt6sSmQjEJ2rJj+Q5XA1xvAAAAFQDWcQ+OJRnk+jCYqESWqxHcyw74DwAAAIA9qAQyZrOJkjpzMSWAcliHvvuyCN8RInlgq+IwWynukOX7zYXcE2dJPopgpG3Aiiq/fxAQX5dUzXfSrL3KYftQOd/RNkAzMHAAzAJeQAA0wEt4uAjhGRwCOSg6wKMYJPYfVghnwIKARuSTUS8NjRXVKteHg7frOOHZwRGIm36pxgAAAIA8/CKLLn5QRctsvvvmvpOn2Xhek02z8K0YukYcXJOQ1FMTYWoHujA39sEAFsdx6Nq1t58lI2dsj6fpf7LOOvoYWtKQnTNRUptlOx27LTO9Q6DhKPV/93nLI/xFOFo/8twZ1TmpAgpAI3SQ9YCnX2A8H686OUwUPmY0kA+H8GKnug== user@desktop
What you need to do now, is determine the remote machine you are going to log into. Then, decide what user you are going to log in as on that machine. We are going to log in as a user called ‘admin’ on a server called ’server’. First, we will ssh into ’server’ as ‘admin’. Then, edit the file located at ~/.ssh/authorized_keys2. If it is not there, create it. All you need to do is paste the contents of the id_dsa.pub file from the ‘user’ account on the ‘desktop’ machine into the ~/.ssh/authorized_keys file for ‘admin’ on ’server’.
For example:
I go to my desktop, log in as ‘user’. I run ’ssh-keygen -t dsa’. It generates a ~/.ssh/id_dsa.pub file in my home directory.
I want to connect as ‘admin’ on a box called ’server’. I ssh in normally as ‘admin’ into the ’server’ machine. I edit the ~/.ssh/authorized_keys2 using my favorite text editor. I add the contents of the ~/.ssh/id_dsa.pub file from my desktop machine into the authorized_keys2 file on ’server’. I then save and quit. I then close all connections to ’server’. Then, I type ’ssh admin@server’, and hit ENTER. It drops me straight to a shell prompt.
This is a nice way to access a machine without having to type in the password every time. Only do this from machines that only you or authorized personnel have access to. Otherwise, you could have a li’l security problem.
In the spirit of release early, rewrite often, I have released Crème Rappel version 2. Version 1 was a shell script that combined Growl and at. Then Apple released 10.5.2 not half a week later and broke at altogether. Sick of fighting with launchd and other Apple superiority complexes, I set about to nurture my own superiority complex and rewrite Crème Rappel to be completely independent of at.
Of course, that's getting too heavy for a shell script, so I moved to Ruby. One thing I didn't want was to require a daemon to be running. Daemons can fail or
forget to start up, and that means I couldn't really truly trust the tool. The recent at debacle is just another case in point. So, instead I wrote Crème to fork a process that sleeps until the moment of truth, then fires off the reminder. It turns out the obvious function for the job, sleep(), is a poor choice here. In fact, every timer I tried had the same problem, including one I thought would not: setitimer(). When you suspend the laptop, it appears you also suspend time. If you don't believe me, try this simple experiment:
date; sleep 30; date
Put the laptop to sleep during the sleep for a substantial time, then notice that when you resume you still have to wait for the full 30 seconds to tick by even though it has actually been a minute plus since you issued the command. So I sleep for one second intervals instead, checking the time every time.
This is not just a backpedaling rewrite, though. It also adds more flexible and easy-to-type timespecs, and a spiffy website. If you give it a try and it doesn't work, or you struggle with the documentation, please do drop me a line so I can fix it. I want it to be worth every bit of bandwidth that you paid for it.
In the spirit of release early, rewrite often, I have released Crème Rappel version 2. Version 1 was a shell script that combined Growl and at. Then Apple released 10.5.2 not half a week later and broke at altogether. Sick of fighting with launchd and other Apple superiority complexes, I set about to nurture my own superiority complex and rewrite Crème Rappel to be completely independent of at.
Of course, that's getting too heavy for a shell script, so I moved to Ruby. One thing I didn't want was to require a daemon to be running. Daemons can fail or
forget to start up, and that means I couldn't really truly trust the tool. The recent at debacle is just another case in point. So, instead I wrote Crème to fork a process that sleeps until the moment of truth, then fires off the reminder. It turns out the obvious function for the job, sleep(), is a poor choice here. In fact, every timer I tried had the same problem, including one I thought would not: setitimer(). When you suspend the laptop, it appears you also suspend time. If you don't believe me, try this simple experiment:
date; sleep 30; date
Put the laptop to sleep during the sleep for a substantial time, then notice that when you resume you still have to wait for the full 30 seconds to tick by even though it has actually been a minute plus since you issued the command. So I sleep for one second intervals instead, checking the time every time.
This is not just a backpedaling rewrite, though. It also adds more flexible and easy-to-type timespecs, and a spiffy website. If you give it a try and it doesn't work, or you struggle with the documentation, please do drop me a line so I can fix it. I want it to be worth every bit of bandwidth that you paid for it.
Theres a discussion going on at reddit about PS1 ( here ).
Mine is:
PS1='\d \t\n\[\033[01;32m\]\u@\h\[\033[01;34m\] \w \n\$\[\033[00m\] ‘
and it looks like:
Wed Feb 20 01:09:15
sontek@inspidell ~
$
Thanks to Chris Crummer for pointing out the reddit post.
Today I found a very informative post on how to get a progress bar with the cp command in Linux.
You can find that blog post here. But I’ll repost the information here in case his blog ever disappears.
With the following bash script:
#!/bin/sh
cp_p()
{
set -e
strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
| awk '{
count += $NF
if (count % 10 == 0) {
percent = count / total_size * 100
printf "%3d%% [", percent
for (i=0;i<=percent;i++)
printf "="
printf ">"
for (i=percent;i<100;i++)
printf " "
printf "]\r”
}
}
END { print “” }’ total_size=$(stat -c ‘%s’ “${1}”) count=0
}
You will get a progress bar like this:
% cp_p /mnt/raid/pub/iso/debian/debian-2.2r4potato-i386-netinst.iso /dev/null
76% [===========================================> ]
When you’re in the trenches, pounding out solutions, it’s nice to have any added advantage that you can. Finding the source of what is taking up all the space on a given Linux partition may just find itself on your priority list some day. And when you need to know right now where it is, it’s great to have the following solutions.
Should you need to find the source of what is taking up the space on one of your Linux boxen, you can use this command to get you through:
stage:/ # du -s * | sort -g 0 proc 0 sys 4 media 4 mnt 16 lost+found 68 tmp 100 srv 112 dev 2564 home 7568 bin 9280 sbin 9916 boot 28528 etc 70844 lib 209624 var 221708 root 429396 opt 1848788 VM 2686844 usr stage:/ # |
So now obviously, my /usr path is taking up the most space. Let’s head into /usr and run the command again:
stage:/ # cd usr stage:/usr # du -s * | sort -g 0 tmp 12 X11R6 16 i586-suse-linux 76 local 3404 games 12124 include 18100 sbin 100424 bin 331616 src 1103240 share 1117832 lib stage:/usr # |
We then see that /var/lib and /var/share are taking up the most space.
Once you find the culprits, you can archive them, back them up, truncate them, or just plain rm them (please use ‘rm’ with care).
Also, if you are looking for all files on your drive larger than a certain size, the following script may be useful to you. Don’t forget to ‘chmod +x’ it to make it executable:
#!/bin/sh
# In kilobytes on older machines
MINSIZE=1000
IFS=$'\n'
# Find the files and put them in a list
FILELIST=`find . -size +"$MINSIZE"k -print`
for FILE in $FILELIST ; do
FILESIZE=`stat --format=%s "$FILE"`
FILEM=$(echo "scale=2;$FILESIZE/1048576" | bc -l)
printf ""$FILE"\n"
printf "\tsize is "$FILEM" Megabytes\t"
printf "\ttype is `file -b "$FILE"`\n\n"
done
|
You may ask, “What is this IFS thing?” Well, it is explained quite well on tldp.org. But for those of us who don’t want to go read that, I’ll just copy and paste the important part for ya’ll:
internal field separator
This variable determines how Bash recognizes fields, or word boundaries, when it interprets character strings.
In other words, bash by default uses spaces to separate things into lists. You are telling it to split the list of files up by the \n or carriage return character rather than by spaces.
Anyway, use the command demonstrated above, and the bash script demonstrated below, to find the files that are taking up all your space.
Have you ever had a process that dies on occasion? For me, I hate that situation and prefer to fix the software as opposed to have a monitor that restarts the process when it dies. I’ve run into a case lately however, that has defied me for a solution to my dying process. [...]
I don’t know if I can last an entire week with openSUSE 10.3. I can’t believe I even thought it possible. I am jonesing for Fedora right now, even though any other distro would probably do…
What’s wrong with SUSE you ask? Just about EVERYTHING! I’m not comfortable at all in this rancid environment. It sucks the life right out of you. I hope some SUSE people come running to save me from this turmoil I feel as I currently hate using this distro. Here’s my first impressions: (beware, the list is rather long)
GOOD
The items below are positives and the openSUSE team deserves credit for all of their hard work in these areas.
Positive Score: +4
BAD
Whle there is some good in openSUSE, its apparent to me that there is much to be improved. As noted below, many more things are in need of improvement, to put it nicely.
Negative Score: -9
Total score for day 1: -5 OOPS - that’s not good!
To be honest, I think I’m being very generous in some of the points I’m giving. OpenSUSE makes it very difficult for my lifestyle so far. I’m not sure what they can do with 10.3 to make it better, but I’d like to hear comments and suggestions on ways to help.
I’m sure hoping that day two will be better. I’m already starting my list and will be testing such things as; video, development, lvm, raid, kvm/xen virtualization and much, much more. As I continue to suffer through this bluetoothless mouse world openSUSE has created for me.
Cheers until tomorrow,
Herlo
I discovered a very strange bug today with OS X's Bourne shell. If you have OS X, give this a try:
/bin/sh -c 'echo -n bug'
This is what you should see:
$ /bin/sh -c 'echo -n bug'
bug$
This is what I see:
$ /bin/sh -c 'echo -n bug'
-n bug
$
In other words, it's ignoring the -n option. It works fine in bash, it's only sh that's broken. It gets better though. If you're using iTerm instead of Terminal.app, it works fine. I have combed through the environment, the locale settings, the terminal emulation, and I can't account for it. I've tried ssh from a linux box which behaves the same as Terminal.app (broken). Who knows what black magic iTerm is invoking.
So I replaced my bash and sh with the ones from MacPorts:
sudo port install bash
sudo mv /bin/bash /bin/bash.old
sudo mv /bin/sh /bin/sh.old
sudo ln /opt/local/bin/bash /bin/bash
sudo ln /opt/local/bin/bash /bin/sh
While you're at it, feel free to
sudo port install coreutils +default_names
Problem solved. Very odd, though. abcde uses echo -n heavily, which breaks in all sorts of ugly ways before this fix. This patch causes abcde to use /bin/bash instead of /bin/sh:
Index: abcde-2.3.3/abcde
===================================================================
--- abcde-2.3.3.orig/abcde 2007-12-07 18:46:36.000000000 -0700
+++ abcde-2.3.3/abcde 2007-12-07 20:57:17.000000000 -0700
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
# Copyright (c) 1998-2001 Robert Woodcock <rcw@debian.org>
# Copyright (c) 2003-2005 Jesus Climent <jesus.climent@hispalinux.es>
# This code is hereby licensed for public consumption under either the
Index: abcde-2.3.3/cddb-tool
===================================================================
--- abcde-2.3.3.orig/cddb-tool 2007-12-07 20:56:49.000000000 -0700
+++ abcde-2.3.3/cddb-tool 2007-12-07 20:57:19.000000000 -0700
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
# Copyright (C) 1999 Nathaniel Smith <njs@uclink4.berkeley.edu>
# Copyright (C) 1999, 2000, 2001 Robert Woodcock <rcw@debian.org>
Speaking of abcde, here's a patch to get rid of an unrelated bug in 2.3.3:
Index: abcde-2.3.3/abcde
===================================================================
--- abcde-2.3.3.orig/abcde 2005-08-25 16:43:27.000000000 -0600
+++ abcde-2.3.3/abcde 2007-12-07 18:46:36.000000000 -0700
@@ -1946,7 +1946,7 @@
FILEPATH=$(find "$FILEPATH" | grep "/$REALTRACKNUM ");
# If the file exists, copy it
if [ -e "$FILEPATH" ] ; then
- nice $READNICE $CDROMREADER "$FILEPATH" "$FILEARG" $REDIR
+ nice $READNICE $CDROMREADER "$FILEPATH" "$FILEARG"
else
false
fi ;;
Finally I have abcde at my fingertips again. There's no replacement for abcde when it comes to ripping CDs.
Tonight at the Ubuntu Utah User Group I presented about Locate vs Find.
The slides are here and written in S5.
Kevin Kubasik also presented on Desktop Search in Gnome. It was pretty cool as well.
The presentations went very well and was streamed and recorded by Utah Open Source Foundation.
Once upon a time, bash (or some other shell) introduced color in the shell prompt. The syntax was funky and all but illegible, so manpages and HOWTOs were written. The technical elite (i.e. high school kids and sysadmins with nothing better to do than configure their bash prompts) rejoiced.
Once upon a more recent time, I crafted this bash prompt:
PS1='$(r=$?; [ 0 == $r ] || echo "\[\e[33m\]$r\[\e[0m\] ")\u@\h:\w\n\[\e[32m\]\$\[\e[0m\] '
Actually, its history is more of an evolution than a crafting, but nonetheless I like it.
Then, out of the blue, someone broke bash. The \[ and \] non-printing character delimiters stopped working (or stopped working properly). Their purpose is to tell bash that the stuff inside is non-printing so don't count it when you're counting the number of characters in the prompt. This is vital for proper wrapping and tab completion in the presence of color escape sequences. I don't know who broke it or why, but removing \[ and \] seemed to fix the symptom.
Fast forward far too long, and someone fixed the problem. Once again you needed \[ and \].
Fast forward again, and it's broken. Or maybe we're just going around in circles on which version of bash is being used in various distributions and OS's. In this case, Ubuntu is still (or again) broken and OS X is broken in a different way. They're both running 3.2.x. The bug is probably the same and the manifestation is simply different.
The current manifestation of this bug is that neither with nor without the delimiters works. I'm forced to go back to a noncolored prompt, and I'm not happy about it. This bug has been around for at least a year. I've reported it several times to various distros. Why is it not yet fixed? If they fixed it today, we'd still be battling the bug for years to come because of the many distros and OS's that have assumed buggy versions of bash.
If you would like to parse torrent RSS feeds on a schedule and don’t want to bog your server down with Azureus, I wrote a basic shell script that you can drop into your cron jobs and have it do all the work for you.
You can download the latest release here or check out the latest code with svn co http://devtoo.net/svn/shtorrent
and then all you have to do is copy shtorrent-cron into cron jobs and you’re set!
You can submit bugs or feature requests at http://devtoo.net/projects/shtorrent/
The original concept was taken from BashT
In an attempt to consistently blog, I am starting a new series here on fedora-tutorials.com. Program of the Week (POW). Hopefully, this will excite and inform all of us about the cool programs available in Fedora. So see you next Wednesday for another program.
Over the past year or so, I’ve been on the hunt for several things that I find in other Linux distros that I cannot find in Fedora. Its not very common, but on occasion I do run into something that’s not there on Fedora. One of them was the ability to automagically complete many of the command lines for many things.
One of them, and probably one of the biggest, was the fact that yum did not have tab completion for available packages. Today is my lucky day! While chatting and helping my students with their labs today, one of them mentioned to me that he could tab complete a particular command on his box. I of course inquired, because it interested me, as to the package name. It turns out my bash tab completion dreams were just about to come true! He told me about this amazing package that would let me use tab completions for things like the service command, man and of course, yum.
I was blown away! So immediately after this discussion, I started searching for this elusive package I’ve never heard of before. Sure enough, as he informed me, bash-completion does exist and does some amazing things. After hunting around a little on google, here’s some of the stuff I found. I’ll also include the links at the bottom of this post.
as root try this: (note [Tab] means you should hit the tab key)
# service ht[Tab]
What you’ll notice is that one of three things happen. If you’ve got the bash-completion package installed already because you’re ahead of the game, it should auto-complete for you. Without bash-completion, this doesn’t happen, but its also possible that since bash already has some completion in place, it might auto complete a directory for you, but that’s definitely not what you want.
If you’ve not already installed bash-completion, I’d suggest you do it now. On Fedora 7, run the following command:
# yum install -y bash-completion ..snip.. Installed: bash-completion.noarch 0:20060301-3.fc7 Complete!
Now that bash-completion is installed, we need to invoke the tools. Normally, this is not needed, and a reboot/re-login will take care of this as well, but since I wanted to use this right away, I did the following as an unprivileged user:
$ source /etc/bash_completion
This doesn’t seem to do much, but its actually quite powerful. The source (or .) will load the environment variables from the /etc/bash_completion script into my current environment. Luckily for us, when we now log into root, /etc/bashrc will accomplish this for us without any intervention. To test that it worked, try running the following command as the same unprivileged user:
$ unalias[Tab] .= ll= ls= vi= which=
Note that when I pressed twice, a list of the currently available aliases appeared. Nice ey? Let’s complete this:
$ unalias w[Tab]
Now produces:
$ unalias which
And completes the string as expected. Now we’re getting somewhere! But why did I really want to explain this?
Oh yeah! yum
With bash-completion, yum can now provide us with a list of available packages, similar to the auto completion capability in apt-get or aptitude from Ubuntu or Debian. Say for instance you want to see all of the packages available for install that match what you’re looking for, but don’t want to run yum list or yum search because, in truth, it just takes to long! Now you have an alternative:
# yum -y install bal[Tab]
Produces:
ballbuster.i386 ballz.i386 balsa.i386
Adding another ‘lb’ to the end of that string (and then the tab key of course) should help us to complete to the package we’d like to install.
# yum -y install ballb[Tab]
Then completes to:
# yum -y install ballbuster.i386
Hitting enter then installs the ballbuster package, and its quite a fun game!
.. snip .. Installed: ballbuster.i386 0:1.0-1.fc6 Dependency Installed: ClanLib.i386 0:0.8.0-4.fc7 Complete!
Of course, there are hundreds of others tab completions available (and there’s a good way to list many of them too, even if its a bit cryptic). Try these on for size:
Are you a developer?
$ svn c[Tab] cat checkout ci cleanup co commit copy cp $ make [Tab] all clean dist-clean
What about a systems administrator?
# modprobe -r b[Tab] battery bay blkcipher bluetooth bridge button
$ man cron[Tab] cron crond crontab
$ ssh herlo[Tab] herlo-f7 herlo-lap herlo.org$ grep --[Tab][Tab] --after-context= --directories= --invert-match --only-matching --basic-regexp --exclude= --label= --perl-regexp .. snip ..
To help you wade further through, try out the following two commands:
complete -p
declare -f
Be aware that these are advanced components and can really be confusing if you’re not a developer and just want to use the features. The complete command seems to provides some tools to do additional auto-completion. I also think that its nice to be able to extend this functionality to other applications as well.
As promised, here’s a few links to help your completion introduction. Note: Some of these links provide more than just the simple tab completion:
If you have a file that begins with a hyphen, like “-TODO-” in Linux, and you try to access it from the command line (by using vim, mv, cat, etc.), you’ll be greeted with the nice error:
sontek@inspidell:~> cat -TODO-
cat: invalid option — O
Try `cat –help’ for more information.
the reason for this is because its actually processing the filename as the commandline arguments -T -O -D -O -, so if you have a file that begins with a hyphen and you need to access it via the command line you need to tell bash not to process arguments, you can do this with the argument ‘–’, like:
sontek@inspidell:~> mv — -TODO- tasklist