Translate

Image of Android Wireless Application Development
Image of XSLT 2.0 and XPath 2.0 Programmer's Reference (Programmer to Programmer)
Image of Modern Operating Systems (3rd Edition)
Image of Beginning Google Maps API 3

KSH93 Bit Manipulation

When programmers think about bitwise manipulation, they usually think about using C or C++ to solve their problem since both programming languages provide a rich set of features which make it easy to perform bitwise manipulation.

However it is possible to just as easily perform such operations using the ksh93 shell.  This post will explain what bitwise manipulation and number conversion facilities are available in ksh93 and provide some hopefully useful utilities and examples.

Many programmers are unaware that ksh93 has builtin support for different numeral radices (AKA arithmetic bases) as shown in the following example.

$ print $(( 10 + 20 ))
30
$ print $(( 2#10 + 20 ))
22
$ print $(( 2#10 + 4#20 ))
10
$

where base 2 is denoted by 2#, base 4 by #4 and so on.  The general case syntax is [base#]n where base is a decimal number between 2 and 36 representing the arithmetic base and n is a number in that base.  You can mix and match bases within expressions.  As you would expect, the default base, i.e. if [base#] is omitted, is decimal(10).

Here is an example of how to convert decimal 255 to base 2, base 8 and base 16 respectively using the typeset command.

$ typeset -i2 a=255
$ echo $a
2#11111111
$ typeset -i8 a=255
$ echo $a
8#377
$ typeset -i16 a=255
$ echo $a
16#ff
$

Note that ksh93 displays the base prefix before each number unless it is a decimal number, i.e. base 10. 

The bitwise manipulation operators in ksh93 are AND, OR, XOR, SHIFT LEFT and SHIFT RIGHT.  There are no SHIFT ROTATE or COMPLEMENT operators.  The following is the list of supported bitwise operators in decreasing order of precedence which, by the way, is the same as in the C programming language.

<< >> Bitwise shift left, shift right

& Bitwise AND

^ Bitwise XOR

| Bitwise OR

The following shell script includes both logical and bitwise operators together with the expected output for each statement.  Notice the difference in output between a logical and bitwise operator.

#!/bin/ksh93

integer -i2 a=5
integer -i2 b=6

# expected output 1
print $(( a < b ))
# expected output 0
print $(( a > b ))
# expected output 3
print $(( a ^ b ))
# expected output 4
print $(( a & b ))
# expected output 1
print $(( a && b ))
# expected output 7
print $(( a | b ))
# expected output 1
print $(( a || b ))

A useful feature of bitwise operators is that by using the shift left and shift right bitwise operators you can mimic multiplication or division by 2 or a power of 2.

$ x=4
$ print $((x << 1))
8
$ print $(( x << 2 ))
16
$ print $(( x >> 1 ))
2
$ print $(( x >> 2 ))
1

Here is a fairly simple shell script which uses numeric base conversion and bitwise operators to calculate the network and broadcast addresses for a given IP address and subnet mask.\

#!/bin/ksh93

typeset -i2 mask=255

[[ $# != 2 ]] && {
   echo "Usage: $0 ipaddress subnetmask"
   exit 1
}

SaveIFS=$IFS
IFS=.
typeset -a IParr=($1)
typeset -a NMarr=($2)
IFS=$SaveIFS

typeset -i2 ip1=${IParr[0]}
typeset -i2 ip2=${IParr[1]}
typeset -i2 ip3=${IParr[2]}
typeset -i2 ip4=${IParr[3]}

typeset -i2 nm1=${NMarr[0]}
typeset -i2 nm2=${NMarr[1]}
typeset -i2 nm3=${NMarr[2]}
typeset -i2 nm4=${NMarr[3]}

echo
echo "       IP Address: $1"
echo "      Subnet Mask: $2"
echo "  Network Address: $((ip1 & nm1)).$((ip2 & nm2)).$((ip3 & nm3)).$((ip4 & nm4))"
echo "Broadcast Address: $((ip1 | (mask ^ nm1))).$((ip2 | (mask ^ nm2))).$((ip3 | (mask ^ nm3))).$(( ip4 | (mask ^ nm4)))"
echo

exit 0


Some sample output for this script.

$ ./calculate-address 10.150.12.1 255.255.255.0

       IP Address: 10.150.12.1
      Subnet Mask: 255.255.255.0
  Network Address: 10.150.12.0
Broadcast Address: 10.150.12.255

$ ./calculate-address 10.150.12.1 255.255.254.0

       IP Address: 10.150.12.1
      Subnet Mask: 255.255.254.0
  Network Address: 10.150.12.0
Broadcast Address: 10.150.13.255

The next example calculates the network and broadcast addresses for a Classless Inter-Domain Routing (CIDR) compliant IP address.  Note the use of the shift right operator in calculating the network mask.

#!/bin/ksh93

[[ $# != 1 ]] && {
   echo "Usage: $0 ipaddress/netmask"
   exit 1
}

SaveIFS=$IFS
IFS="./"
typeset -a IParr=($1)
IFS=$SaveIFS

typeset -i2 ip1=${IParr[0]}
typeset -i2 ip2=${IParr[1]}
typeset -i2 ip3=${IParr[2]}
typeset -i2 ip4=${IParr[3]}
typeset -i2 cidr=${IParr[4]}

typeset -i2 nm1=0 nm2=0 nm3=0 nm4=0

typeset -i quad=$(( cidr / 8 ))
sigbits=$(( cidr % 8 ))
if (( sigbits != 0 )); then
   slot=$(( 256 - ( 256 >> $sigbits ) ))
fi

for (( i=1; i < 5; i++ ))
do
   nameref nm=nm${i}
   if (( quad != 0 )); then
      nm=255
      (( --quad ))
   elif (( quad == 0 )); then
      nm=slot
      break
   fi
done

typeset -i2 mask=255

print
print "           IP Address: $((ip1)).$((ip2)).$((ip3)).$((ip4))"
print "    CIDR Netmask Mask: $((nm1)).$((nm2)).$((nm3)).$((nm4))"
print " CIDR Network (Route): $((ip1 & nm1)).$((ip2 & nm2)).$((ip3 & nm3)).$((ip4 & nm4))"
print "    Broadcast Address: $((ip1 | (mask ^ nm1))).$((ip2 | (mask ^ nm2))).$((ip3 | (mask ^ nm3))).$((ip4 | (mask ^ n
m4)))"
print

exit 0


Sample output:

$ ./calculate-address "192.168.21.12/18"

           IP Address: 192.168.21.12
    CIDR Network Mask: 255.255.192.0
 CIDR Network (Route): 192.168.0.0
    Broadcast Address: 192.168.63.255

For my example, suppose you are given a hexadecimal string 3AF9:3B01 and you want to expand that string to produce a list of all the values between 3AF9 and 3B01. The wrinkle is that you want all letters in the hexadecimal numbers to be uppercased.

#!/bin/ksh93

x=3AF9:3B01

typeset -i16 a=0x${x%:*}
typeset -i16 b=0x${x#*:}
typeset -u c

for (( ; a <= b; a++ ))
do
   c=${a:3}
   echo "$c"
done


Here is the output:

$ ./hexlist
3AF9
3AFA
3AFB
3AFC
3AFD
3AFE
3AFF
3B00
3B01

Here is another interesting example of using bit arithmetic to solve a problem. In this particular case, the user wanted to have a shell script which accepted a 32-bit hexadecimal number as input and assigns consecutive blocks of 4 bits of the equivalent binary number to each of 8 variables.

#!/bin/ksh93

integer -i2 mask=16#F0000000
integer -i2 v0 v1 v2 v3 v4 v5 v6 v7

read hn?"Enter a 32-bit hex decimal number: "
integer -i2 in=16#$hn

for ((i=0; i < 8; i++))
do
   nameref var=v${i}
   var=$(( ((in << (i*4)) & mask) >> 28 ))
done

printf "%032.0.2d\n" $in
printf "%04.0.2d %04.0.2d %04.0.2d %04.0.2d %04.0.2d %04.0.2d %04.0.2d %04.0.2d\n" \
   $v0 $v1 $v2 $v3 $v4 $v5 $v6 $v7


Note the use of a third specifier with %d. In ksh93, each of the integral format specifiers can have a third modifier after width and precision that specifies the base of the conversion from 2 to 64. In addition the use of the # modifier as the third specifier will cause base# to be prepended to the value.

Here is sample output for this script:

$ ./hexblock
Enter a 32-bit hex decimal number: abcdabcd
10101011110011011010101111001101
1010 1011 1100 1101 1010 1011 1100 1101
$ ./hexblock
Enter a 32-bit hex decimal number: AFC92317
10101111110010010010001100010111
1010 1111 1100 1001 0010 0011 0001 0111

Suppose instead that you wanted to store the individual bits in an array. Here is one way to do this is:

#!/bin/ksh93

read num?"Enter binary 32 bit number: "

integer -i2 a=2#$num
integer -i2 mask=2#00000000000000000000000000000001

typeset -ui barray=(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0)

for (( i=0; i < 32; i++, mask=$(( mask << 1)) ))
do
   nameref var=barray[$i]
   (( $mask & $a )) &&  var=1
done

# how to test the array values
if (( ${barray[0]} ))
then
   echo "barray[0] is 1"
else
   echo "barray[0] is 0"
fi


This example calculates the binary and decimal equivalent of an IP address:

#!/bin/ksh93

[[ $# != 1 ]] && {
   echo "Usage: $0 ipaddress"
   exit 1
}

SaveIFS=$IFS
IFS="."
typeset -a IParr=($1)
IFS=$SaveIFS

typeset -i2 ip1=${IParr[0]}
typeset -i2 ip2=${IParr[1]}
typeset -i2 ip3=${IParr[2]}
typeset -i2 ip4=${IParr[3]}

typeset -ui2 d=$(( (ip1 << 24) | (ip2 << 16) | (ip3 << 8) | ip4 ))

print "Inputted IP$ address: $((ip1)).$((ip2)).$((ip3)).$((ip4))"
print "   Binary equivalent: $d"
print "  Decimal equivalent: $((d))"

exit 0


Here is sample output:

$ ./example 172.16.120.100
Inputted IP$ address: 172.16.120.100
   Binary equivalent: 2#10101100000100000111100001100100
  Decimal equivalent: 2886760548

Finally, I would be amiss if I did not point out that both the bash and zsh shells also provide a full set of bitwise operators and support number base conversion.

Comments are closed.