12.2. Complex Commands

Commands for more advanced users

find

-exec COMMAND \;

Carries out COMMAND on each file that find matches. The command sequence terminates with \; (the ";" is escaped to make certain the shell passes it to find literally). If COMMAND contains {}, then find substitutes the full path name of the selected file for "{}".

bash$ find ~/ -name '*.txt' /home/bozo/.kde/share/apps/karm/karmdata.txt /home/bozo/misc/irmeyc.txt /home/bozo/test-scripts/1.txt

1 find /home/bozo/projects -mtime 1 2 # Lists all files in /home/bozo/projects directory tree 3 #+ that were modified within the last day. 4 # 5 # mtime = last modification time of the target file 6 # ctime = last status change time (via 'chmod' or otherwise) 7 # atime = last access time 8  9 DIR=/home/bozo/junk_files 10 find "$DIR" -type f -atime +5 -exec rm {} \; 11 # Deletes all files in "/home/bozo/junk_files" 12 #+ that have not been accessed in at least 5 days. 13 # 14 # "-type filetype", where 15 # f = regular file 16 # d = directory, etc. 17 # (The 'find' manpage has a complete listing.)

1 find /etc -exec grep '[0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*[.][0-9][0-9]*' {} \; 2  3 # Finds all IP addresses (xxx.xxx.xxx.xxx) in /etc directory files. 4 # There a few extraneous hits - how can they be filtered out? 5  6 # Perhaps by: 7  8 find /etc -type f -exec cat '{}' \; | tr -c '.[:digit:]' '\n' \ 9  | grep '^[^.][^.]*\.[^.][^.]*\.[^.][^.]*\.[^.][^.]*$' 10 # [:digit:] is one of the character classes 11 # introduced with the POSIX 1003.2 standard. 12  13 # Thanks, S.C.

The -exec option to find should not be confused with the exec shell builtin.

---

Example 12-2. Badname, eliminate file names in current directory containing bad characters and whitespace.

1 #!/bin/bash 2  3 # Delete filenames in current directory containing bad characters. 4  5 for filename in * 6 do 7 badname=`echo "$filename" | sed -n /[\+\{\;\"\\\=\?~\(\)\<\>\&\*\|\$]/p` 8 # Files containing those nasties: + { ; " \ = ? ~ ( ) < > & * | $ 9 rm $badname 2>/dev/null # So error messages deep-sixed. 10 done 11  12 # Now, take care of files containing all manner of whitespace. 13 find . -name "* *" -exec rm -f {} \; 14 # The path name of the file that "find" finds replaces the "{}". 15 # The '\' ensures that the ';' is interpreted literally, as end of command. 16  17 exit 0 18  19 #--------------------------------------------------------------------- 20 # Commands below this line will not execute because of "exit" command. 21  22 # An alternative to the above script: 23 find . -name '*[+{;"\\=?~()<>&*|$ ]*' -exec rm -f '{}' \; 24 exit 0 25 # (Thanks, S.C.)

---

---

Example 12-3. Deleting a file by its inode number

1 #!/bin/bash 2 # idelete.sh: Deleting a file by its inode number. 3  4 # This is useful when a filename starts with an illegal character, 5 #+ such as ? or -. 6  7 ARGCOUNT=1 # Filename arg must be passed to script. 8 E_WRONGARGS=70 9 E_FILE_NOT_EXIST=71 10 E_CHANGED_MIND=72 11  12 if [ $# -ne "$ARGCOUNT" ] 13 then 14  echo "Usage: `basename $0` filename" 15  exit $E_WRONGARGS 16 fi 17  18 if [ ! -e "$1" ] 19 then 20  echo "File \""$1"\" does not exist." 21  exit $E_FILE_NOT_EXIST 22 fi 23  24 inum=`ls -i | grep "$1" | awk '{print $1}'` 25 # inum = inode (index node) number of file 26 # Every file has an inode, a record that hold its physical address info. 27  28 echo; echo -n "Are you absolutely sure you want to delete \"$1\" (y/n)? " 29 # The '-v' option to 'rm' also asks this. 30 read answer 31 case "$answer" in 32 [nN]) echo "Changed your mind, huh?" 33  exit $E_CHANGED_MIND 34  ;; 35 *) echo "Deleting file \"$1\".";; 36 esac 37  38 find . -inum $inum -exec rm {} \; 39 echo "File "\"$1"\" deleted!" 40  41 exit 0

---

See Example 12-22, Example 3-4, and Example 10-9 for scripts using find. Its manpage provides more detail on this complex and powerful command.

xargs

A filter for feeding arguments to a command, and also a tool for assembling the commands themselves. It breaks a data stream into small enough chunks for filters and commands to process. Consider it as a powerful replacement for backquotes. In situations where backquotes fail with a too many arguments error, substituting xargs often works. Normally, xargs reads from stdin or from a pipe, but it can also be given the output of a file.

The default command for xargs is echo. This means that input piped to xargs may have linefeeds and other whitespace characters stripped out.

bash$ ls -l total 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file2 bash$ ls -l | xargs total 0 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file1 -rw-rw-r-- 1 bozo bozo 0 Jan 29 23:58 file2

ls | xargs -p -l gzip gzips every file in current directory, one at a time, prompting before each operation.

An interesting xargs option is -n NN, which limits to NN the number of arguments passed. ls | xargs -n 8 echo lists the files in the current directory in 8 columns.

Another useful option is -0, in combination with find -print0 or grep -lZ. This allows handling arguments containing whitespace or quotes. find / -type f -print0 | xargs -0 grep -liwZ GUI | xargs -0 rm -f grep -rliwZ GUI / | xargs -0 rm -f Either of the above will remove any file containing "GUI". (Thanks, S.C.)

---

Example 12-4. Logfile using xargs to monitor system log

1 #!/bin/bash 2  3 # Generates a log file in current directory 4 # from the tail end of /var/log/messages. 5  6 # Note: /var/log/messages must be world readable 7 # if this script invoked by an ordinary user. 8 # #root chmod 644 /var/log/messages 9  10 LINES=5 11  12 ( date; uname -a ) >>logfile 13 # Time and machine name 14 echo --------------------------------------------------------------------- >>logfile 15 tail -$LINES /var/log/messages | xargs | fmt -s >>logfile 16 echo >>logfile 17 echo >>logfile 18  19 exit 0 20  21 # Exercise: 22 # -------- 23 # Modify this script to track changes in /var/log/messages at intervals 24 #+ of 20 minutes. 25 # Hint: Use the "watch" command.

---

---

Example 12-5. copydir, copying files in current directory to another, using xargs

1 #!/bin/bash 2  3 # Copy (verbose) all files in current directory 4 #+ to directory specified on command line. 5  6 E_NOARGS=65 7  8 if [ -z "$1" ] # Exit if no argument given. 9 then 10  echo "Usage: `basename $0` directory-to-copy-to" 11  exit $E_NOARGS 12 fi 13  14 ls . | xargs -i -t cp ./{} $1 15 # This is the exact equivalent of 16 # cp * $1 17 # unless any of the filenames has "whitespace" characters. 18  19 exit 0

---

expr

All-purpose expression evaluator: Concatenates and evaluates the arguments according to the operation given (arguments must be separated by spaces). Operations may be arithmetic, comparison, string, or logical.

expr 3 + 5

returns 8

expr 5 % 3

returns 2

expr 5 \* 3

returns 15

The multiplication operator must be escaped when used in an arithmetic expression with expr.

y=`expr $y + 1`

Increment a variable, with the same effect as let y=y+1 and y=$(($y+1)). This is an example of arithmetic expansion.

z=`expr substr $string $position $length`

Extract substring of $length characters, starting at $position.

---

Example 12-6. Using expr

1 #!/bin/bash 2  3 # Demonstrating some of the uses of 'expr' 4 # ======================================= 5  6 echo 7  8 # Arithmetic Operators 9 # ---------- --------- 10  11 echo "Arithmetic Operators" 12 echo 13 a=`expr 5 + 3` 14 echo "5 + 3 = $a" 15  16 a=`expr $a + 1` 17 echo 18 echo "a + 1 = $a" 19 echo "(incrementing a variable)" 20  21 a=`expr 5 % 3` 22 # modulo 23 echo 24 echo "5 mod 3 = $a" 25  26 echo 27 echo 28  29 # Logical Operators 30 # ------- --------- 31  32 # Returns 1 if true, 0 if false, 33 #+ opposite of normal Bash convention. 34  35 echo "Logical Operators" 36 echo 37  38 x=24 39 y=25 40 b=`expr $x = $y` # Test equality. 41 echo "b = $b" # 0 ( $x -ne $y ) 42 echo 43  44 a=3 45 b=`expr $a \> 10` 46 echo 'b=`expr $a \> 10`, therefore...' 47 echo "If a > 10, b = 0 (false)" 48 echo "b = $b" # 0 ( 3 ! -gt 10 ) 49 echo 50  51 b=`expr $a \< 10` 52 echo "If a < 10, b = 1 (true)" 53 echo "b = $b" # 1 ( 3 -lt 10 ) 54 echo 55 # Note escaping of operators. 56  57 b=`expr $a \<= 3` 58 echo "If a <= 3, b = 1 (true)" 59 echo "b = $b" # 1 ( 3 -le 3 ) 60 # There is also a "\>=" operator (greater than or equal to). 61  62  63 echo 64 echo 65  66 # Comparison Operators 67 # ---------- --------- 68  69 echo "Comparison Operators" 70 echo 71 a=zipper 72 echo "a is $a" 73 if [ `expr $a = snap` ] 74 # Force re-evaluation of variable 'a' 75 then 76  echo "a is not zipper" 77 fi 78  79 echo 80 echo 81  82  83  84 # String Operators 85 # ------ --------- 86  87 echo "String Operators" 88 echo 89  90 a=1234zipper43231 91 echo "The string being operated upon is \"$a\"." 92  93 # length: length of string 94 b=`expr length $a` 95 echo "Length of \"$a\" is $b." 96  97 # index: position of first character in substring 98 # that matches a character in string 99 b=`expr index $a 23` 100 echo "Numerical position of first \"2\" in \"$a\" is \"$b\"." 101  102 # substr: extract substring, starting position & length specified 103 b=`expr substr $a 2 6` 104 echo "Substring of \"$a\", starting at position 2,\ 105 and 6 chars long is \"$b\"." 106  107  108 # The default behavior of the 'match' operations is to 109 #+ search for the specified match at the ***beginning*** of the string. 110 # 111 # uses Regular Expressions 112 b=`expr match "$a" '[0-9]*'` # Numerical count. 113 echo Number of digits at the beginning of \"$a\" is $b. 114 b=`expr match "$a" '\([0-9]*\)'` # Note that escaped parentheses 115 # == == + trigger substring match. 116 echo "The digits at the beginning of \"$a\" are \"$b\"." 117  118 echo 119  120 exit 0

---

The : operator can substitute for match. For example, b=`expr $a : [0-9]*` is the exact equivalent of b=`expr match $a [0-9]*` in the above listing. 1 #!/bin/bash 2  3 echo 4 echo "String operations using \"expr \$string : \" construct" 5 echo "===================================================" 6 echo 7  8 a=1234zipper5FLIPPER43231 9  10 echo "The string being operated upon is \"`expr "$a" : '\(.*\)'`\"." 11 # Escaped parentheses grouping operator. == == 12  13 # *************************** 14 #+ Escaped parentheses 15 #+ match a substring 16 # *************************** 17  18  19 # If no escaped parentheses... 20 #+ then 'expr' converts the string operand to an integer. 21  22 echo "Length of \"$a\" is `expr "$a" : '.*'`." # Length of string 23  24 echo "Number of digits at the beginning of \"$a\" is `expr "$a" : '[0-9]*'`." 25  26 # ------------------------------------------------------------------------- # 27  28 echo 29  30 echo "The digits at the beginning of \"$a\" are `expr "$a" : '\([0-9]*\)'`." 31 # == == 32 echo "The first 7 characters of \"$a\" are `expr "$a" : '\(.......\)'`." 33 # ===== == == 34 # Again, escaped parentheses force a substring match. 35 # 36 echo "The last 7 characters of \"$a\" are `expr "$a" : '.*\(.......\)'`." 37 # ==== end of string operator ^^ 38 # (actually means skip over one or more of any characters until specified 39 #+ substring) 40  41 echo 42  43 exit 0