Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Care and Feeding of Functions in Shell

News

See also

Recommended Links

Checking the loaded functions Encoding and decoding strings

Manipulating files and pathnames

Arguments
Interacting with the user Manipulating strings Parsing command line options Printing messages to the console Using external programs Catching signals Manipulating variables
    Sysadmin Horror Stories Unix shells history Tips Humor Etc

Functions in shell are really scripts which run in the current context of the current shell instance (no secondary shell is forked to run the function; it's run within the current shell.)

Functions provide a lot more flexibility that aliases.  Here are two simplest functions possible: "do nothing" function and "Hello world" function:

          function quit {
               exit
           }
           function hello {
               print "Hello world !"
           }
           hello
           quit
       

Declaring a function is just a matter of writing function my_func { my_code }. Functions can be declared in arbitrary order. But they need to be declared before they are invoked in the script. 

Calling a function is just like calling another program, you just write its name and (optionally) arguments separated by spaces.

NOTE: you can't enclose arguments in round parenthesis -- this would be a syntactic error.  You can't use comma after arguments like you are inclined after using C or Perl.

The best way to create a set of  useful shell functions in a separate file. Then you can source the file with the dot (.) command or in your startup scripts. You can also just enter the function at the command line.

To create a function from the command line, you would do something like this:

$ psgrep() {
> ps -ef | grep $1
> }

This is a pretty simple function, and could be implemented as an alias as well. Let's try to solve the problem of displayed files over 1K.  awk can be used to find any matching files that are larger than 100K bytes:

largesize() {
   ls -la |  awk ' { if ( $5 gt 100000 ) print $1 } '
}

As in almost any programming language, you can use functions to group pieces of code in a more logical way or practice the divine art of recursion.

Bash shell's function feature is pretty primitive and just an slightly enhanced version of a similar facility in the System V Bourne shell.  You can create collection of functions that help you in your work and store them in you .profile file.

Functions are faster then subshell invocation: when you invoke a function, it is already in memory. Modern computers have large RAM, so there is no need to worry about the amount of space a typical function takes up.

The other advantage of functions is that they can and should be used for organizing shell scripts into modular "chunks" of related functions that are easier to develop and maintain. To define a function, you can use the following form (there is also second C-style  form of function definition that we will not discuss hee):

function function_name {
    shell commands
}

You can also delete a function definition with the command unset -f function_name.

Like list of aliases is produced by typing command alias, you can find out what functions are defined in your login session by typing functions. The shell will print not just the names but the definitions of all functions, in alphabetical order by function name. Since this may result in long output, you might want to pipe the output through more or redirect it to a file and view it with the view command.

Note: functions is actually an alias for typeset -f;

There are two important differences between functions and scripts. First, functions do not run in separate processes, as scripts are when you invoke them by name; the "semantics" of running a function are more like those of your .profile when you log in or any script when invoked with the "dot" command. Second, if a function has the same name as a script or executable program, the function takes precedence.

if a function has the same name as a script or executable program, the function takes precedence.

This is a good time to show the order of precedence for the various sources of commands. When you type a command to the shell, it looks in the following places until it finds a match:

  1. Aliases
  2. Keywords such as function and several others, like if and for
  3. Functions
  4. Built-ins like cd and whence

    Note: that means that you can overwrite built-ins with functions
     

  5. Scripts and executable programs, for which the shell searches in the directories listed in the PATH environment variable

If you need to know the exact source of a command, you can use the whence built-in command .If you type whence -v command_name, you also get information about how particular command is implemented, for example:

$ whence -v cd
cd is a shell builtin
$ whence -v function
function is a keyword
$ whence -v man
man is /usr/bin/man
$ whence -v ll
ll is an alias for ls -l

Return and exit

The statement return N, which causes the surrounding script or function to exit with exit status N. N is actually optional; it defaults to 0.

In the scripts that finish without a return statement (i.e., every one we have seen so far) the return value is equal to the return value (sucess of failue) of the  last executed statement.

Exist is similar to return:  the statement exit N exits the entire script, no matter how deeply you are nested in functions.

For example is we need a function to implement enhanced cd we can write something like: 

function cd {
    "cd" $*
    rs=$?
    print $OLDPWD -> $PWD
    return $rs
}

Here is fist save the exit status of cd in the variable rs and then return it as the return value of the function.

# Capture value returned by last command
ret=$?

return $ret

return [n], exit [n]
Return from a function with the given value, or exit the whole script with the given value.
Without a return, the function returns when it reaches the end, and the value is the exit status of the last command it ran.

Example:

die()
{
   # Print an error message and exit with given status
   # call as: die status "message" ["message" ...]
   exitstat=$1; shift
   for i in "$@"; do
      print -R "$i"
   done
   exit $exitstat
}
 
[ -w $filename ] || \
  die 1 "$file not writeable" "check permissions"

Example:

logme()
{
   # Print or not depending on global "$verbosity"
   # Change the verbosity with a single variable.
   # Arg. 1 is the level for this message.
   level=$1; shift
   if [[ $level -le $verbosity ]]; then
      print -R $*
   fi
}

verbosity=2
logme 1 This message will appear
logme 3 This only appears if verbosity is 3 or higher

Common Errors

Two common errors with declaring and using functions are

The following example illustrates the first type of error:

lsl { ls -l ; } 

Here, the parentheses are missing after lsl. This is an invalid function definition and will result in an error message similar to the following:

sh: syntax error: '}' unexpected

The following command illustrates the second type of error:

$ lsl()

Here, the function lsl is executed along with the parentheses, (). This will not work because the shell interprets it as a redefinition of the function with the name lsl. Usually such an invocation results in a prompt similar to the following:

>

This is a prompt produced by the shell when it expects you to provide more input. The input it expects is the body of the function lsl

Arguments

NOTES:

For example:

name_error() 
{
echo " $@ contains errors, it must contain only letters"
}

Here is an example of a function that takes arbitrary number of parameters and prints some information about them:

function invocation_inform {
   print "The first argument is:" $1 
   print "List of argumnets is:" $@ 
   print "Number of arguments is:" $#
}

The command shift performs shifts of argument to the left by given number of positions, extra argumnat shifted to left of index 1 are discarded

1=$2
2=$3
...

for every argument, regardless of how many there are. If you supply a numeric argument to shift , it will shift the arguments that many times over; for example, shift 3 has this effect:

1=$4
2=$5
...

This leads immediately to some code that handles a single option (call it -o ) and arbitrarily many arguments:

if [[ $1 = -o ]]; then
   # process the -o option
   shift
fi
# normal processing of arguments...   

After the if construct, $1 , $2 , etc., are set to the correct arguments.



Returning STDIO

Messages to stdout may be captured by command substitution (`myfunction`, which provides another way for a function to return information to the calling script. Beware of side-effects (and reducing reusability) in functions which perform I/O.

Example of invocations

 Calling a function is just like calling another program, you just write its name and (optionally) arguments separated by spaces

NOTE: you can't enclose arguments in round parenthesis -- this would be a syntactic error.  You can't use comma after arguments like you are inclined after using C or Perl.

arg1_echo test

all_arg_echo  test 1 test 2

Checking the loaded functions

the set command displays all the loaded functions available to the shell.

$ set
USER=dave
findit=()
{
if [ $# -lt 1 ]; then
  echo "usage :findit file";
  return 1;
fi;
find / -name $1 -print
}
...

You can use  unset command to remove functions:

unset function_name

The idea of  .functions file

Traditionally .bash_profile file contained aliases, but now they are often separates and group into a separate file, for example .aliases.  So for functions it might be also be beneficial to use a separate file called, for example, .functions.

Let's consider a very artificial (bad) example of creating a function that will call find for each argument so that we can find several different files with one command (useless exersize) The function will now look like this:

$ pg .functions 

#!/bin/sh
findit()
{
# findit
if [ $# -lt 1 ]; then
  echo "usage :findit file"
  return 1
fi
for member
do
  find / -name $member -print
done 
} 

Now source the file again:

. ./.functions 

Using the set command to see that it is indeed loaded, you will notice that the shell has correctly interpreted the for loop to take in all parameters.

$ set
findit=()
{
if [ $# -lt 1 ]; then
  echo "usage :`basename $0` file";
  return 1;
fi;
for loop in "$@";
do
  find / -name $loop -print;
done
} 
... 

Now to execute the changed  Supplying a couple of files to find:

$ findit LPSO.doc passwd
/usr/local/accounts/LPSO.doc
/etc/passwd
... 

By default all variables, except for the special variables associated with function arguments, have global scope. In ksh, bash, and zsh, variables with local scope can be declared using the typeset command. The typeset command is discussed later in this chapter. This command is not supported in the Bourne shell, so it is not possible to have programmer-defined local variables in scripts that rely strictly on the Bourne shell.

Local Variables

Local variables are defined using typeset command (in bash you usually use declare instead):

typeset var1[=val1] ... varN[=valN]

Here, var1 ... varN are variable names and val1 ... valN are values to assign to the variables. The values are optional as the following example illustrates:

typeset fruit1 fruit2=banana

Using whence command

If you need to know the exact source of a command, there is an option to the whence built-in command.. whence by itself will print the pathname of a command if the command is a script or executable program, but it will only parrot the command's name back if it is anything else. But if you type whence -v commandname , you get more complete information, such as:

$ 
whence -v cd

cd is a shell builtin
$ 
whence -v function

function is a keyword
$ 
whence -v man

man is /usr/bin/man
$ 
whence -v ll

ll is an alias for ls -l

We will refer mainly to scripts throughout the remainder of this book, but unless we note otherwise, you should assume that whatever we say applies equally to functions.

Positional Parameters

shell assignment statment has the form of  varname=value, e.g.:

$ greeting="hello world!"
$ print "$greeting"

Some environment variables are predefined by the shell when you log in. There are several built-in variables that are vital to shell programming like HOME, HOSTNAME, PWD, OLD_PWD, etc

There is also a special class on built-in variables called positional parameters. These hold the command-line arguments to scripts when they are invoked. Positional parameters have names 1, 2, 3, etc., meaning that their values are denoted by $1, $2, $3, etc. There is also a positional parameter 0, whose value is the name of the script (i.e., the command typed in to invoke it).

Two special variables contain all of the positional parameters (except positional parameter 0): * and @. The difference between them is subtle but important, and it's apparent only when they are used within double quotes:

The variable $# holds the number of positional parameters (as a character string). All of these variables are "read-only," meaning that you can't assign new values to them within scripts.

For example, assume that you create a file testme that contains the following simple shell script:

print "testme: $@"
print "$0: Arg 1 is '$1' and Arg 2 is $2"
print "$# arguments"

Then if you type bash 3.2, you will see the following output:

testme: bash 3.2
testme: Arg 1 is 'bash' and Arg 2 is '3.2'
2 arguments

In this case, $3, $4, etc., are all unset, which means that the shell will substitute the empty (or null) string for them (Unless the option nounset is turned on).

Shell functions use positional parameters and special variables like * and # in exactly the same way as shell scripts do. If you wanted to define testme as a function, you could put the following in your .profile or environment file:

function testme {
    print "testme: $*"
    print "$0: $1 and $2"
    print "$# arguments"
}

You will get the same result if you type  testme bash 3.2

Typically, several shell functions are defined within a single shell script. Therefore each function will need to handle its own arguments, which in turn means that each function needs to keep track of positional parameters separately. Sure enough, each function has its own copies of these variables (even though functions don't run in their own subshells, as scripts do); we say that such variables are local to the function.

However, other variables defined within functions are not local (they are global), meaning that their values are known throughout the entire shell script. For example, assume that you have a shell script called ascript that contains this:

Note: typeset  can be used for making variables local to functions.

Let's take a closer look at "$@" and "$*". These variables are two of the shell's greatest idiosyncracies, so we'll discuss some of the most common sources of confusion.

Note on Variable Syntax

Before we show the many things you can do with shell variables, we have to make a confession: the syntax of $varname for taking the value of a variable is not quite accurate. Actually, it's the simple form of the more general syntax, which is ${varname}.

Why two syntaxes? For one thing, the more general syntax is necessary if your code refers to more than nine positional parameters: you must use ${10} for the tenth instead of $10

Also useful is the next character is not a delimiter:

PS1="${LOGNAME}_ "

we would get the desired $yourname_. It is safe to omit the curly brackets ({}) if the variable name is followed by a character that isn't a letter, digit, or underscore.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Nov 01, 2017] Functions by Tom Ryder

Nov 01, 2017 | sanctum.geek.nz

A more flexible method for defining custom commands for an interactive shell (or within a script) is to use a shell function. We could declare our ll function in a Bash startup file as a function instead of an alias like so:

# Shortcut to call ls(1) with the -l flag
ll() {
    command ls -l "$@"
}

Note the use of the command builtin here to specify that the ll function should invoke the program named ls , and not any function named ls . This is particularly important when writing a function wrapper around a command, to stop an infinite loop where the function calls itself indefinitely:

# Always add -q to invocations of gdb(1)
gdb() {
    command gdb -q "$@"
}

In both examples, note also the use of the "$@" expansion, to add to the final command line any arguments given to the function. We wrap it in double quotes to stop spaces and other shell metacharacters in the arguments causing problems. This means that the ll command will work correctly if you were to pass it further options and/or one or more directories as arguments:

$ ll -a
$ ll ~/.config

Shell functions declared in this way are specified by POSIX for Bourne-style shells, so they should work in your shell of choice, including Bash, dash , Korn shell, and Zsh. They can also be used within scripts, allowing you to abstract away multiple instances of similar commands to improve the clarity of your script, in much the same way the basics of functions work in general-purpose programming languages.

Functions are a good and portable way to approach adding features to your interactive shell; written carefully, they even allow you to port features you might like from other shells into your shell of choice. I'm fond of taking commands I like from Korn shell or Zsh and implementing them in Bash or POSIX shell functions, such as Zsh's vared or its two-argument cd features.

If you end up writing a lot of shell functions, you should consider putting them into separate configuration subfiles to keep your shell's primary startup file from becoming unmanageably large.

Examples from the author

You can take a look at some of the shell functions I have defined here that are useful to me in general shell usage; a lot of these amount to implementing convenience features that I wish my shell had, especially for quick directory navigation, or adding options to commands:

Other examples Variables in shell functions

You can manipulate variables within shell functions, too:

# Print the filename of a path, stripping off its leading path and
# extension
fn() {
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

This works fine, but the catch is that after the function is done, the value for name will still be defined in the shell, and will overwrite whatever was in there previously:

$ printf '%s\n' "$name"
foobar
$ fn /home/you/Task_List.doc
Task_List
$ printf '%s\n' "$name"
Task_List

This may be desirable if you actually want the function to change some aspect of your current shell session, such as managing variables or changing the working directory. If you don't want that, you will probably want to find some means of avoiding name collisions in your variables.

If your function is only for use with a shell that provides the local (Bash) or typeset (Ksh) features, you can declare the variable as local to the function to remove its global scope, to prevent this happening:

# Bash-like
fn() {
    local name
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

# Ksh-like
# Note different syntax for first line
function fn {
    typeset name
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

If you're using a shell that lacks these features, or you want to aim for POSIX compatibility, things are a little trickier, since local function variables aren't specified by the standard. One option is to use a subshell , so that the variables are only defined for the duration of the function:

# POSIX; note we're using plain parentheses rather than curly brackets, for
# a subshell
fn() (
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
)

# POSIX; alternative approach using command substitution:
fn() {
    printf '%s\n' "$(
        name=$1
        name=${name##*/}
        name=${name%.*}
        printf %s "$name"
    )"
}

This subshell method also allows you to change directory with cd within a function without changing the working directory of the user's interactive shell, or to change shell options with set or Bash options with shopt only temporarily for the purposes of the function.

Another method to deal with variables is to manipulate the positional parameters directly ( $1 , $2 ) with set , since they are local to the function call too:

# POSIX; using positional parameters
fn() {
    set -- "${1##*/}"
    set -- "${1%.*}"
    printf '%s\n' "$1"
}

These methods work well, and can sometimes even be combined, but they're awkward to write, and harder to read than the modern shell versions. If you only need your functions to work with your modern shell, I recommend just using local or typeset . The Bash Guide on Greg's Wiki has a very thorough breakdown of functions in Bash, if you want to read about this and other aspects of functions in more detail.

Keeping functions for later

As you get comfortable with defining and using functions during an interactive session, you might define them in ad-hoc ways on the command line for calling in a loop or some other similar circumstance, just to solve a task in that moment.

As an example, I recently made an ad-hoc function called monit to run a set of commands for its hostname argument that together established different types of monitoring system checks, using an existing script called nmfs :

$ monit() { nmfs "$1" Ping Y ; nmfs "$1" HTTP Y ; nmfs "$1" SNMP Y ; }
$ for host in webhost{1..10} ; do
> monit "$host"
> done

After that task was done, I realized I was likely to use the monit command interactively again, so I decided to keep it. Shell functions only last as long as the current shell, so if you want to make them permanent, you need to store their definitions somewhere in your startup files. If you're using Bash, and you're content to just add things to the end of your ~/.bashrc file, you could just do something like this:

$ declare -f monit >> ~/.bashrc

That would append the existing definition of monit in parseable form to your ~/.bashrc file, and the monit function would then be loaded and available to you for future interactive sessions. Later on, I ended up converting monit into a shell script, as its use wasn't limited to just an interactive shell.

If you want a more robust approach to keeping functions like this for Bash permanently, I wrote a tool called Bashkeep , which allows you to quickly store functions and variables defined in your current shell into separate and appropriately-named files, including viewing and managing the list of names conveniently:

$ keep monit
$ keep
monit
$ ls ~/.bashkeep.d
monit.bash
$ keep -d monit

[Oct 31, 2017] Testing exit values in Bash by Tom Ryder

Oct 28, 2013 | sanctum.geek.nz

In Bash scripting (and shell scripting in general), we often want to check the exit value of a command to decide an action to take after it completes, likely for the purpose of error handling. For example, to determine whether a particular regular expression regex was present somewhere in a file options , we might apply grep(1) with its POSIX -q option to suppress output and just use the exit value:

grep -q regex options

An approach sometimes taken is then to test the exit value with the $? parameter, using if to check if it's non-zero, which is not very elegant and a bit hard to read:

# Bad practice
grep -q regex options
if (($? > 0)); then
    printf '%s\n' 'myscript: Pattern not found!' >&2
    exit 1
fi

Because the if construct by design tests the exit value of commands , it's better to test the command directly , making the expansion of $? unnecessary:

# Better
if grep -q regex options; then
    # Do nothing
    :
else
    printf '%s\n' 'myscript: Pattern not found!\n' >&2
    exit 1
fi

We can precede the command to be tested with ! to negate the test as well, to prevent us having to use else as well:

# Best
if ! grep -q regex options; then
    printf '%s\n' 'myscript: Pattern not found!' >&2
    exit 1
fi

An alternative syntax is to use && and || to perform if and else tests with grouped commands between braces, but these tend to be harder to read:

# Alternative
grep -q regex options || {
    printf '%s\n' 'myscript: Pattern not found!' >&2
    exit 1
}

With this syntax, the two commands in the block are only executed if the grep(1) call exits with a non-zero status. We can apply && instead to execute commands if it does exit with zero.

That syntax can be convenient for quickly short-circuiting failures in scripts, for example due to nonexistent commands, particularly if the command being tested already outputs its own error message. This therefore cuts the script off if the given command fails, likely due to ffmpeg(1) being unavailable on the system:

hash ffmpeg || exit 1

Note that the braces for a grouped command are not needed here, as there's only one command to be run in case of failure, the exit call.

Calls to cd are another good use case here, as running a script in the wrong directory if a call to cd fails could have really nasty effects:

cd wherever || exit 1

In general, you'll probably only want to test $? when you have specific non-zero error conditions to catch. For example, if we were using the --max-delete option for rsync(1) , we could check a call's return value to see whether rsync(1) hit the threshold for deleted file count and write a message to a logfile appropriately:

rsync --archive --delete --max-delete=5 source destination
if (($? == 25)); then
    printf '%s\n' 'Deletion limit was reached' >"$logfile"
fi

It may be tempting to use the errexit feature in the hopes of stopping a script as soon as it encounters any error, but there are some problems with its usage that make it a bit error-prone. It's generally more straightforward to simply write your own error handling using the methods above.

For a really thorough breakdown of dealing with conditionals in Bash, take a look at the relevant chapter of the Bash Guide .

[Feb 1, 2007] Functions and aliases in bash

# Append to .bashrc or call it from there.
# Save some typing at the command line :)

# longlist a directory, by page
# lo [directoryname]
lo () {
      if [ -d "$1" ] ; then
         ls -al "$1" | less
      else
         ls -al $(pwd) | less
      fi
}
# Same as above but recursive
lro () {
      if [ -d "$1" ] ; then
         ls -alR "$1" | less
      else
         ls -alR $(pwd) | less
      fi
}
export -f lo lro

[Apr 1, 2006] Submitted Article/ Converting a ksh Function to a ksh Script by William R. Seppeler

BigAdmin

Here is a simple way to create a script that will behave both as an executable script and as a ksh function. Being an executable script means the script can be run from any shell. Being a ksh function means the script can be optimized to run faster if launched from a ksh shell. This is an attempt to get the best of both worlds.

Procedure

Start by writing a ksh function. A ksh function is just like a ksh script except the script code is enclosed within a function name { script } construct.

Take the following example:

# Example script

function fun {
  print "pid=$$ cmd=$0 args=$*" opts="$-"
}

Save the text in a file. You'll notice nothing happens if you try to execute the code as a script:

ksh ./example

In order to use a function, the file must first be sourced. Sourcing the file will create the function definition in the current shell. After the function has been sourced, it can then be executed when you call it by name:

.. ./example
fun

To make the function execute as a script, the function must be called within the file. Add the bold text to the example function.

# Example script

function fun {
  print "pid=$$ cmd=$0 args=$*" opts="$-"
}

fun $*

Now you have a file that executes like a ksh script and sources like a ksh function. One caveat is that the file now executes while it is being sourced.

There are advantages and disadvantages to how the code is executed. If the file was executed as a script, the system spawns a child ksh process, loads the function definition, and then executes the function. If the file was sourced, no child process is created, the function definition is loaded into the current shell process, and the function is then executed.

Sourcing the file will make it run faster because no extra processes are created, however, loading a function occupies environment memory space. Functions can also manipulate environment variables whereas a script only gets a copy to work with. In programming terms, a function can use call by reference parameters via shell variables. A shell script is always call by value via arguments.

Advanced Information

When working with functions, it's advantageous to use ksh autoloading. Autoloading eliminates the need to source a file before executing the function. This is accomplished by saving the file with the same name as the function. In the above example, save the example as the file name "fun". Then set the FPATH environment variable to the directory where the file fun is. Now, all that needs to be done is type "fun" on the command line to execute the function.

Notice the double output the first time fun is called. This is because the first time the function is called, the file must be sourced, and in sourcing the file, the function gets called. What we need is to only call the function when the file is executed as a script, but skip calling the function if the file is sourced. To accomplish this, notice the output of the script when executing it as opposed to sourcing it. When the file is sourced, arg0 is always -ksh. Also, note the difference in opts when the script is sourced. Test the output of arg0 to determine if the function should be called or not. Also, make the file a self-executing script. After all, no one likes having to type "ksh" before running every ksh script.

[[ "${0##*/}" == "fun" ]] &&  fun $*

Now the file is a self-executing script as well as a self-sourcing function (when used with ksh autoloading). What becomes more interesting is that since the file can be an autoload function as well as a stand-alone script, it could be placed in a single directory and have both PATH and FPATH point to it.

# ${HOME}/.profile

FPATH=${HOME}/bin
PATH=${FPATH}:${PATH}

In this setup, fun will always be called as a function unless it's explicitly called as ${HOME}/bin/fun.

Considerations

Even though the file can be executed as a function or a script, there are minor differences in behavior between the two. When the file is sourced as a function, all local environment variables will be visible to the script. If the file is executed as a script, only exported environment variables will be visible. Also, when sourced, a function can modify all environment variables. When the file is executed, all visible environment variables are only copies. We may want to make special allowances depending on how the file is called. Take the following example.

#!/bin/ksh

# Add arg2 to the contents of arg1

function addTo {
  eval $1=$(($1 + $2))
}

if [[ "${0##*/}" == "addTo" ]]; then
  addTo $*
  eval print \$$1
fi

The script is called by naming an environment variable and a quantity to add to that variable. When sourced, the script will directly modify the environment variable with the new value. However, when executed as a script, the environment variable cannot be modified, so the result must be output instead. Here is a sample run of both situations.

# called as a function
var=5
addTo var 3
print $var

# called as a script
var=5
export var
var=$(./addTo var 3)
print $var

Note the extra steps needed when executing this example as a script. The var must be exported prior to running the script or else it won't be visible. Also, because a script can't manipulate the current environment, you must capture the new result.

Extra function-ality

It's possible to package several functions into a single file. This is nice for distribution as you only need to maintain a single file. In order to maintain autoloading functionality, all that needs to be done is create a link for each function named in the file.

#!/bin/ksh

function addTo {
  eval $1=$(($1 + $2))
}

function multiplyBy {
  eval $1=$(($1 * $2))
}

if [[ "${0##*/}" == "addTo" ]] \
|| [[ "${0##*/}" == "multiplyBy" ]]; then
  ${0##*/} $*
  eval print \$$1
fi

if [[ ! -f "${0%/*}/addTo" ]] \
|| [[ ! -f "${0%/*}/multiplyBy" ]]; then
  ln "${0}" "${0%/*}/addTo"
  ln "${0}" "${0%/*}/multiplyBy"
  chmod u+rx "${0}"
fi

Notice the extra code at the bottom. This text could be saved in a file named myDist. The first time the file is sourced or executed, the appropriate links and file permissions will be put in place, thus creating a single distribution for multiple functions. Couple that with making the file a script executable and you end up with a single distribution of multiple scripts. It's like a shar file, but nothing actually gets unpacked.

The only downside to this distribution tactic is that BigAdmin will only credit you for each file submission, not based on the actual number of executable programs...

Time to Run

Try some of the sample code in this document. Get comfortable with the usage of each snippet to understand the differences and limitations. In general, it's safest to always distribute a script, but it's nice to have a function when speed is a consideration. Do some timing tests.

export var=8
time ./addTo var 5
time addTo var 5

If this code were part of an inner-loop calculation of a larger script, that speed difference could be significant.

This document aims to provide the best of both worlds. You can have a script and retain function speed for when it's needed. I hope you have enjoyed this document and its content. Thanks to Sun and BigAdmin for the hosting and support to make contributions like this possible.

Recommended Links

Softpanorama hot topic of the month

Softpanorama Recommended

Top articles

Sites

Shell Programming Functions Using Functions InformIT

IBM Knowledge Center - Korn shell functions

v10, i03 Creating Global Functions with the Korn Shell

Linux tip Bash parameters and parameter expansions

Marco's Bash Functions Library - Summary [Gna!]

This package is an attempt to make GNU bash a viable solution for medium sized scripts. A problem with bash is that it doesn't provide encapsulation of any sort, beside the feature of providing functions. This problem is partly solved by writing subscripts and invoking them in the main script, but this is not always the best solution.

A set of modules implementing common operations and a script template are provided by this package and the author has used them with success in implementing non-small scripts.

The philosophy of MBFL is to do the work as much as possible without external commands. For example: string manipulation is done using the special variable substitution provided by bash, and no use is done of utilities like sed, grep and ed.

The library is better used if our script is developed on the template provided in the package (examples/template.sh). This is because with MBFL some choices have been made to reduce the application dependent part of the script to the smallest dimension; if we follow another schema, MBFL modules may be indequate. This is especially true for the options parsing module.

The best way to use the library is to include at runtime the library file libmbfl.sh i; this is possible by installing MBFL on the system and using this code in the scripts:

mbfl_INTERACTIVE='no'
source "${MBFL_LIBRARY:=`mbfl-config`}"

after the service variables have been declared (Service Variables for details). This code will read the full pathname of the library from the environment variable MBFL_LIBRARY; if this variable is not set: the script mbfl-config is invoked with no arguments to acquire the pathname of the library. mbfl-config is installed in the bin directory with the library.

Another solution is to include the library directly in the script; this is easy if we preprocess our scripts with GNU m4:

m4_changequote([[, ]])
m4_include(libmbfl.sh)

is all we need to do. We can preprocess the script with:

$ m4 --prefix-builtins --include=/path/to/library \
         script.sh.m4 >script.sh

easy to do in a Makefile; we can take the MBFL's Makefile as example of this method.

It is also interesting to process the script with the following rule:

M4      = ...
M4FLAGS = --prefix-builtins --include=/path/to/library

%.sh: %.sh.m4
        $(M4) $(M4FLAGS) $(<) | \
        grep --invert-match -e '^#' -e '^$$' | \
        sed -e "s/^ \\+//" >$(@)

this will remove all the comments and blank lines, decreasing the size of the script significantly if one makes use of verbose comments; note that this will wipe out the #!/bin/bash first line also.

Usually we want the script to begin with #!/bin/bash followed by a comment describing the license terms.

Bash by example, Part 3

Encoding and decoding strings

The purpose of this module is to let an external process invoke a bash script with damncommand line arguments: strings including blanks or strange characters that may trigger quoting rules.

This problem can arise when using scripting languages with some sort of eval command.

The solution is to encode the argument string in hexadecimal or octal format strings, so that all the damn characters are converted to "good" ones. The the bash script can convert them back.

mbfl_decode_hex string Function
Decodes a hex string and outputs it on stdout.
mbfl_decode_oct string Function
Decodes a oct string and outputs it on stdout.

Example:

mbfl_decode_hex 414243
-> ABC

Manipulating files and pathnames


Node:File Names, Next:, Up:File

File names


Node:File Name Parts, Next:, Up:File Names

Splitting a file name into its components

mbfl_file_extension pathname Function
Extracts the extension from a file name. Searches the last dot (.) character in the argument string and echoes to stdout the range of characters from the dot to the end, not including the dot. If a slash (/) character is found first, echoes to stdout the empty string.

mbfl_file_dirname pathname Function
Extracts the directory part from a fully qualified file name. Searches the last slash character in the input string and echoes to stdout the range of characters from the first to the slash, not including the slash.

If no slash is found: echoes a single dot (the current directory).

If the input string begins with / or // with no slash characters after the first ones, the string echoed to stdout is a single slash.

mbfl_file_rootname pathname Function
Extracts the root portion of a file name. Searches the last dot character in the argument string and echoes to stdout the range of characters from the beginning to the dot, not including the dot.

If a slash character is found first, or no dot is found, or the dot is the first character, echoes to stdout the empty string.

mbfl_file_tail pathnbame Function
Extracts the file portion from a fully qualified file name. Searches the last slash character in the input string and echoes to stdout the range of characters from the slash to the end, not including the slash. If no slash is found: echoes the whole string.

mbfl_file_split pathname Function
Separates a file name into its components. One or more contiguous occurrences of the slash character are used as separator. The components are stored in an array named SPLITPATH, that may be declared local in the scope of the caller; the base index is zero. The number of elements in the array is stored in a variable named SPLITCOUNT. Returns true.


Node:File Name Path, Next:, Previous:File Name Parts, Up:File Names

Handling relative pathnames

mbfl_file_normalise pathname ?prefix? Function
Normalises a file name: removes all the occurrences of . and ...

If pathname is relative (according to mbfl_file_is_absolute) and prefix is not present or it is the empty string: the current process working directory is prepended to pathname.

If prefix is present and non empty, and pathname is relative (according to mbfl_file_is_absolute): prefix is prepended to pathname and normalised, too.

Echoes to stdout the normalised file name; returns true.

mbfl_file_is_absolute pathname Function
Returns true if the first character in pathname is a slash (/); else returns false.
mbfl_file_is_absolute_dirname pathname Function
Returns true if pathname is a directory according to mbfl_file_is_directory and an absolute pathname according to mbfl_file_is_absolute.
mbfl_file_is_absolute_filename pathname Function
Returns true if pathname is a file according to mbfl_file_is_file and an absolute pathname according to mbfl_file_is_absolute.



Node:File Name System, Previous:File Name Path, Up:File Names

Finding pathnames on the system

mbfl_file_find_tmpdir ?PATHNAME? Function
Finds a value for a temporary directory. If PATHNAME is not null and is a directory and is writable it is accepted; else the value /tmp/$USER, where USER is the environment variable, is tried; finally the value /tmp is tried. When a value is accepted it's echoed to stdout. Returns true if a value is found, false otherwise.


Node:File Commands, Next:, Previous:File Names, Up:File

File Commands


Node:File Commands Listing, Next:, Up:File Commands

Retrieving informations

mbfl_file_enable_listing Function
Declares to the program module the commands required to retrieve informations about files and directories (Program Declaring). The programs are: ls.

mbfl_file_get_owner pathname Function
Prints the owner of the file.

mbfl_file_get_group pathname Function
Prints the group of the file.

mbfl_file_get_size pathname Function
Prints the size of the file.

mbfl_file_normalise_link pathname Function
Makes use of the readlink to normalise the pathname of a symbolic link (remember that a symbolic link references a file, never a directory). Echoes to stdout the normalised pathname.

The command line of readlink is:

readlink -fn $pathname


Node:File Commands Mkdir, Next:, Previous:File Commands Listing, Up:File Commands

Creating directories

mbfl_file_enable_make_directory Function
Declares to the program module the commands required to create directories (Program Declaring). The programs are: mkdir.

mbfl_file_make_directory pathname ?permissions? Function
Creates a directory named pathname; all the unexistent parents are created, too. If permissions is present: it is the specification of directory permissions in octal mode.


Node:File Commands Copy, Next:, Previous:File Commands Mkdir, Up:File Commands

Copying files

mbfl_file_enable_copy Function
Declares to the program module the commands required to copy files and directories (Program Declaring). The programs are: cp.

mbfl_file_copy source target ?...? Function
Copies the source, a file, to target, a file pathname. Additional arguments are handed to the command unchanged.

If source does not exist, or if it is not a file, an error is generated and the return value is 1. No test is done upon target.

mbfl_file_copy_recursively source target ?...? Function
Copies the source, a directory, to target, a directory pathname. Additional arguments are handed to the command unchanged. This function is like mbfl_file_copy, but it adds --recursive to the command line of cp.

If source does not exist, or if it is not a file, an error is generated and the return value is 1. No test is done upon target.


Node:File Commands Removing, Next:, Previous:File Commands Copy, Up:File Commands

Removing files and directories

Files removal is forced: the --force option to rm is always used. It is responsibility of the caller to validate the operation before invoking these functions.

Some functions test the existence of the pathname before attempting to remove it: this is done only if test execution is disabled; if test execution is enabled the command line is echoed to stderr to make it easier to debug scripts.

mbfl_file_enable_remove Function
Declares to the program module the commands required to remove files and directories (Program Declaring). The programs are: rm and rmdir.

mbfl_file_remove pathname Function
Removes pathname, no matter if it is a file or directory. If it is a directory: descends the sublevels removing all of them. If an error occurs returns 1.

mbfl_file_remove_file pathname Function
Removes the file selected by pathname. If the file does not exist or it is not a file or an error occurs: returns 1.

mbfl_file_remove_directory pathname Function
Removes the directory selected by pathname. If the directory does not exist or an error occurs: returns 1.

Manipulating tar archives

Remember that when we execute a script with the --test option: the external commands are not executed: a command line is echoed to stdout. It is recommended to use this mode to fine tune the command line options required by tar.

mbfl_file_enable_tar Function
Declares to the program module the tar command (Program Declaring).

mbfl_tar_exec ?...? Function
Executes tar with whatever arguments are used. Returns the return code of tar.

mbfl_tar_create_to_stdout directory ?...? Function
Creates an archive and sends it to stdout. The root of the archive is the directory. Files are selected with the . pattern. tar flags may be appended to the invocation to this function. In case of error returns 1.

mbfl_tar_extract_from_stdin directory ?...? Function
Reads an archive from stdin and extracts it under directory. tar flags may be appended to the invocation to this function. In case of error returns 1.

mbfl_tar_extract_from_file directory archive ?...? Function
Reads an archive from a file and extracts it under directory. tar flags may be appended to the invocation to this function. In case of error returns 1.

mbfl_tar_create_to_file directory archive ?...? Function
Creates an archive named archive holding the contents of directory. Before creating the archive, the process changes the current directory to directory and selects the files with the pattern .. tar flags may be appended to the invocation to this function. In case of error returns 1.

mbfl_tar_archive_directory_to_file directory archive ?...? Function
Like mbfl_tar_create_to_file but archives all the contents of directory, including the directory itself (not its parents).

mbfl_tar_list archive ?...? Function
Prints to stdout the list of files in archive. tar flags may be appended to the invocation to this function. In case of error returns 1.


Node:File Testing, Next:, Previous:File Commands, Up:File

Testing file existence and the like

mbfl_file_is_file filename Function
Returns true if filename is not the empty string and is a file.

mbfl_file_is_readable filename Function
Returns true if filename is not the empty string, is a file and is readable.

mbfl_file_is_writable filename Function
Returns true if filename is not the empty string, is a file and is writable.

mbfl_file_is_directory directory Function
Returns true if directory is not the empty string and is a directory.

mbfl_file_directory_is_readable directory Function
Returns true if directory is not the empty string, is a directory and is readable.

mbfl_file_directory_is_writable directory Function
Returns true if directory is not the empty string, is a directory and is writable.
mbfl_file_is_symlink pathname Function
Returns true if pathname is not the empty string and is a symbolic link.


Node:File Misc, Previous:File Testing, Up:File

Miscellaneous commands

mbfl_cd dirname ?...? Function
Changes directory to dirname. Optional flags to cd may be appended.


Node:Getopts, Next:, Previous:File, Up:Top

Parsing command line options

The getopt module defines a set of procedures to be used to process command line arguments with the following format:

-a
brief option a with no value;
-a123
brief option a with value 123;
--bianco
long option bianco with no value;
--color=bianco
long option color with value bianco.

Requires the message module (Message for details).


Node:Getopts Arguments, Next:, Up:Getopts

Arguments

The module contains, at the root level, a block of code like the following:

ARGC=0
declare -a ARGV ARGV1

for ((ARGC1=0; $# > 0; ++ARGC1)); do
    ARGV1[$ARGC1]="$1"
    shift
done

this block is executed when the script is evaluated. Its purpose is to store command line arguments in the global array ARGV1 and the number of command line arguments in the global variable ARGC1.

The global array ARGV and the global variable ARGC are predefined and should be used by the mbfl_getopts functions to store non-option command line arguments.

Example:

$ script --gulp wo --gasp=123 wa

if the script makes use of the library, the strings wo and wa will go into ARGV and ARGC will be set to 2. The option arguments are processed and some action is performed to register them.

We can access the non-option arguments with the following code:

for ((i=0; $i < $ARGC; ++i)); do
    # do something with ${ARGV[$i]}
done


Node:Getopts Usage, Next:, Previous:Getopts Arguments, Up:Getopts

Using the module

To use this module we have to declare a set of script options; we declare a new script option with the function mbfl_declare_option. Options declaration should be done at the beginning of the script, before doing anything; for example: right after the MBFL library code.

In the main block of the script: options are parsed by invoking mbfl_getopts_parse: this function will update a global variable and invoke a script function for each option on the command line.

Examples

Example of option declaration:

mbfl_declare_option ALPHA no a alpha noarg "enable alpha option"

this code declares an option with no argument and properties:

If the option is used: the function script_option_update_alpha is invoked (if it exists) with no arguments, after the variable script_option_ALPHA has been set to yes. Valid option usages are:

$ script.sh -a
$ script.sh --alpha

Another example:

mbfl_declare_option BETA 123 b beta witharg "select beta value"

this code declares an option with argument and properties:

If the option is used: the function script_option_update_beta is invoked (if it exists) with no arguments, after the variable script_option_BETA has been set to the selected value. Valid option usages are:

$ script.sh -b456
$ script.sh --beta=456


Node:Getopts Options, Next:, Previous:Getopts Usage, Up:Getopts

Predefined options

A set of predefined options is recognised by the library and not handed to the user defined functions.

--encoded-args
Signals to the library that the non-option arguments and the option values are encoded in hexadecimal strings. Encoding is useful to avoid quoting problems when invoking a script from another one.

If this option is used: the values are decoded by mbfl_getopts_parse before storing them in the ARGV array and before being stored in the option's specific global variables.

-v
--verbose
Turns on verbose messages. The fuction mbfl_option_verbose returns true (Message, for details).
--silent
Turns off verbose messages. The fuction mbfl_option_verbose returns false.
--verbose-program
If used the --verbose option is added to the command line of external programs that support it. The fuction mbfl_option_verbose_program returns true or false depending on the state of this option.
--show-program
Prints the command line of executed external programs.
--debug
Turns on debugging messages (Message, for details).
--test
Turns on test execution (Program Testing, for details).
--null
Signals to the script that it has to use the null character to separate values, instead of the common newline. The global variable mbfl_option_NULL is set to yes.
-f
--force
Signals to the script that it does not have to query the user before doing dangerous operations, like overwriting files. The global variable mbfl_option_INTERACTIVE is set to no.
-i
--interactive
Signals to the script that it does have to query the user before doing dangerous operations, like overwriting files. The global variable mbfl_option_INTERACTIVE is set to yes.
--validate-programs
Validates the existence of all the programs needed by the script; then exits. The exit code is zero if all the programs were found, one otherwise.
--version
Prints to the standard output of the script the contents of the global variable mbfl_message_VERSION, then exits with code zero. The variable makes use of the service variables (Service Variables, for details).
--version-only
Prints to the standard output of the script the contents of the global variable script_VERSION, then exits with code zero.
--license
Prints to the standard output of the script the contents of one of the global variables mbfl_message_LICENSE_*, then exits with code zero. The variable makes use of the service variables (Service Variables, for details).
-h
--help
--usage
Prints to the standard output of the script: the contents of the global variable script_USAGE; a newline; the string options:; a newline; an automatically generated string describing the options declared with mbfl_declare_option; a string describing the MBFL default options. Then exits with code zero.

The following options may be used to set, unset and query the state of the predefined options.

mbfl_option_encoded_args Function
mbfl_set_option_encoded_args Function
mbfl_unset_option_encoded_args Function
Query/sets/unsets the encoded arguments option.

mbfl_option_encoded_args Function
mbfl_set_option_encoded_args Function
mbfl_unset_option_encoded_args Function
Query/sets/unsets the verbose messages option.

mbfl_option_verbose_program Function
mbfl_set_option_verbose_program Function
mbfl_unset_option_verbose_program Function
Query/sets/unsets verbose execution for external programs.

This option, of course, is supported only for programs that are known by MBFL (like rm): if a program is executed with mbfl_program_exec, it is responsibility of the caller to use the option.

mbfl_option_show_program Function
mbfl_set_option_show_program Function
mbfl_unset_option_show_program Function
Prints the command line of executed external program. This does not disable program execution, it just prints the command line before executing it.

mbfl_option_test Function
mbfl_set_option_test Function
mbfl_unset_option_test Function
Query/sets/unsets the test execution option.

mbfl_option_debug Function
mbfl_set_option_debug Function
mbfl_unset_option_debug Function
Query/sets/unsets the debug messages option.

mbfl_option_null Function
mbfl_set_option_null Function
mbfl_unset_option_null Function
Query/sets/unsets the null list separator option.

mbfl_option_interactive Function
mbfl_set_option_interactive Function
mbfl_unset_option_interactive Function
Query/sets/unsets the interactive excution option.


Node:Getopts Interface, Next:, Previous:Getopts Options, Up:Getopts

Interface functions

mbfl_declare_option keyword default brief long hasarg description Function
Declares a new option. Arguments description follows.
keyword
A string identifying the option; internally it is used to build a function name and a variable name. It is safer to limit this string to the letters in the range a-z and underscores.
default
The default value for the option. For an option with argument it can be anything; for an option with no argument: it must be yes or no.
brief
The brief option selector: a single character. It is safer to choose a single letter (lower or upper case) in the ASCII standard.
long
The long option selector: a string. It is safer to choose a sequence of letters in the ASCII standard, separated by underscores or dashes.
hasarg
Either witharg or noarg: declares if the option requires an argument or not.
description
A one-line string describing the option briefly.

mbfl_getopts_parse Function
Parses a set of command line options. The options are handed to user defined functions. The global array ARGV1 and the global variable ARGC1 are supposed to hold the command line arguments and the number of command line arguments. Non-option arguments are left in the global array ARGV, the global variable ARGC holds the number of elements in ARGV.

mbfl_getopts_islong string varname Function
Verifies if a string is a long option without argument. string is the string to validate, varname is the optional name of a variable that's set to the option name, without the leading dashes.

Returns with code zero if the string is a long option without argument, else returns with code one.

An option must be of the form --option, only characters in the ranges A-Z, a-z, 0-9 and the characters - and _ are allowed in the option name.

mbfl_getopts_islong_with string optname varname Function
Verifies if a string is a long option with argument. Arguments:
string
the string to validate;
optname
optional name of a variable that's set to the option name, without the leading dashes;
varname
optional name of a variable that's set to the option value.

Returns with code zero if the string is a long option with argument, else returns with code one.

An option must be of the form --option=value, only characters in the ranges A-Z, a-z, 0-9 and the characters - and _ are allowed in the option name.

If the argument is not an option with value, the variable names are ignored.

mbfl_getopts_isbrief string varname Function
Verifies if a string is a brief option without argument. Arguments: string is the string to validate, varname optional name of a variable that's set to the option name, without the leading dash.

Returns with code zero if the argument is a brief option without argument, else returns with code one.

A brief option must be of the form -a, only characters in the ranges A-Z, a-z, 0-9 are allowed as option letters.

mbfl_getopts_isbrief_with string optname valname Function
Verifies if a string is a brief option without argument. Arguments:
string
the string to validate;
optname
optional name of a variable that's set to the option name, without the leading dashes;
valname
optional name of a variable that's set to the option value.

Returns with code zero if the argument is a brief option without argument, else returns with code one.

A brief option must be of the form -aV (a is the option, V is the value), only characters in the ranges A-Z, a-z, 0-9 are allowed as option letters.

mbfl_wrong_num_args required present Function
Validates the number of arguments. required is the required number of arguments, present is the given number of arguments on the command line. If the number of arguments is different from the required one: prints an error message and returns with code one; else returns with code zero.

mbfl_argv_from_stdin Function
If the ARGC global variable is set to zero: fills the global variable ARGV with lines from stdin. If the global variable mbfl_option_NULL is set to yes: lines are read using the null character as terminator, else they are read using the standard newline as terminator.

This function may block waiting for input.

mbfl_argv_all_files Function
Checks that all the arguments in ARGV are file names of existent file. Returns with code zero if no errors, else prints an error message and returns with code 1.


Node:Getopts Values, Previous:Getopts Interface, Up:Getopts

Querying Options

Some feature and behaviour of the library is configured by the return value of the following set of functions. All of these functions are defined by the Getopts module, but they can be redefined by the script.

mbfl_option_encoded_args Function
Returns true if the option --encoded-args was used on the command line.

mbfl_option_verbose Function
Returns true if the option --verbose was used on the command line after all the occurrences of --silent. Returns false if the option --silent was used on the command line after all the occurrences of --verbose.

mbfl_option_test Function
Returns true if the option --test was used on the command line.
mbfl_option_debug Function
Returns true if the option --debug was used on the command line.
mbfl_option_null Function
Returns true if the option --null was used on the command line.
mbfl_option_interactive Function
Returns true if the option --interactive was used on the command line after all the occurrences of --force. Returns false if the option --force was used on the command line after all the occurrences of --interactive.

Printing messages to the console

This module allows one to print messages on an output channel. Various forms of message are supported.

All the function names are prefixed with mbfl_message_. All the messages will have the forms:

<progname>: <message>
<progname>: [error|warning]: <message>

The following global variables are declared:

mbfl_message_PROGNAME
must be initialised with the name of the script that'll be displayed at the beginning of each message;
mbfl_message_VERBOSE
yes if verbose messages should be displayed, else no;

mbfl_message_set_program PROGNAME Function
Sets the script official name to put at the beginning of messages.

mbfl_message_set_channel channel Function
Selects the channel to be used to output messages.

mbfl_message_string string Function
Outputs a message to the selected channel. Echoes a string composed of: the content of the mbfl_message_PROGNAME global variable; a colon; a space; the provided message.

A newline character is NOT appended to the message. Escape characters are allowed in the message.

mbfl_message_verbose string Function
Outputs a message to the selected channel, but only if the evaluation of the function/alias mbfl_option_verbose returns true.

Echoes a string composed of: the content of the mbfl_message_PROGNAME global variable; a colon; a space; the provided message.

A newline character is NOT appended to the message. Escape characters are allowed in the message.

mbfl_message_verbose_end string Function
Outputs a message to the selected channel, but only if the evaluation of the function/alias mbfl_option_verbose returns true.

Echoes the string. A newline character is NOT appended to the message. Escape characters are allowed in the message.

mbfl_message_debug string Function
Outputs a message to the selected channel, but only if the evaluation of the function/alias mbfl_option_debug returns true.

Echoes a string composed of: the content of the mbfl_message_PROGNAME global variable; a colon; a space; the provided message.

A newline character is NOT appended to the message. Escape characters are allowed in the message.

mbfl_message_warning string Function
Outputs a warning message to the selected channel. Echoes a string composed of: the content of the mbfl_message_PROGNAME global variable; a colon; a space; the string warning; a colon; a space; the provided message.

A newline character IS appended to the message. Escape characters are allowed in the message.

mbfl_message_error string Function
Outputs a error message to the selected channel. Echoes a string composed of: the content of the mbfl_message_PROGNAME global variable; a colon; a space; the string error; a colon; a space; the provided message.

A newline character IS appended to the message. Escape characters are allowed in the message.


Node:Program, Next:, Previous:Message, Up:Top

Using external programs

This module declares a set of global variables all prefixed with mbfl_program_. We have to look at the module's code to see which one are declared.


Node:Program Testing, Next:, Up:Program

Testing a script and running programs

MBFL allows a script to execute a "dry run", that is: do not perform any operation on the system, just print messages describing what will happen if the script is executed with the selected options. This implies, in the MBFL model, that no external program is executed.

When this feature is turned on: mbfl_program_exec does not execute the program, instead it prints the command line on standard error and returns true.

mbfl_set_option_test Function
Enables the script test option. After this a script should not do anything on the system, just print messages describing the operations. This function is invoked when the predefined option --test is used on the command line.

mbfl_unset_option_test Function
Disables the script test option. After this a script should perform normal operations.

mbfl_option_test Function
Returns true if test execution is enabled, else returns false.


Node:Program Checking, Next:, Previous:Program Testing, Up:Program

Checking programs existence

The simpler way to test the availability of a program is to look for it just before it is used. The following function should be used at the beginning of a function that makes use of external programs.

mbfl_program_check program ?program ...? Function
Checks the availability of programs. All the pathnames on the command line are checked: if one is not executable an error message is printed on stderr. Returns false if a program can't be found, true otherwise.

mbfl_program_find program Function
A wrapper for:
type -ap program

that looks for a program in the current search path: prints the full pathname of the program found, or prints an empty string if nothing is found.


Node:Program Executing, Next:, Previous:Program Checking, Up:Program

Executing a program

mbfl_program_exec arg ... Function
Evaluates a command line.

If the function mbfl_option_test returns true: instead of evaluation, the command line is sent to stderr.

If the function mbfl_option_show_program returns true: the command line is sent to stderr, then it is executed.


Node:Program Declaring, Previous:Program Executing, Up:Program

Declaring the intention to use a program

To make a script model simpler, we assume that the unavailability of a program at the time of its execution is a fatal error. So if we need to execute a program and the executable is not there, the script must be aborted on the spot.

Functions are available to test the availability of a program, so we can try to locate an alternative or terminate the process under the script control. On a system where executables may vanish from one moment to another, no matter how we test a program existence, there's always the possibility that the program is not "there" when we invoke it.

If we just use mbfl_program_exec to invoke an external program, the function will try and fail if the executable is unavailable: the return code will be false.

The vanishing of a program is a rare event: if it's there when we look for it, probably it will be there also a few moments later when we invoke it. For this reason, MBFL proposes a set of functions with which we can declare the intention of a script to use a set of programs; a command line option is predefined to let the user test the availability of all the declared programs before invoking the script.

mbfl_declare_program program Function
Registers program as the name of a program required by the script. The return value is always zero.

mbfl_program_validate_declared Function
Validates the existence of all the declared programs. The return value is zero if all the programs are found, one otherwise.

This function is invoked by mbfl_getopts_parse when the --validate-programs option is used on the command line.

It is a good idea to invoke this function at the beginning of a script, just before starting to do stuff, example:

mbfl_program_validate_declared || mbfl_exit_program_not_found

If verbose messages are enabled: a brief summary is echoed to stderr; from the command line the option --verbose must be used before --validate-programs.

mbfl_program_found program Function
Prints the pathname of the previously declared program. Returns zero if the program was found, otherwise prints an error message and exits the script by invoking mbfl_exit_program_not_found.

This function should be used to retrieve the pathname of the program to be used as first argument to mbfl_program_exec.

mbfl_exit_program_not_found Function
Terminates the script with exit code 20. This function may be redefined by a script to make use of a different exit code; it may even be redefined to execute arbitrary code and then exit.


Node:Signal, Next:, Previous:Program, Up:Top

Catching signals

MBFL provides an interface to the trap builtin that allows the execution of more than one function when a signal is received; this may sound useless, but that is it.

mbfl_signal_map_signame_to_signum sigspec Function
Converts sigspec to the corresponding signal number, then prints the number.

mbfl_signal_attach sigspec handler Function
Append handler to the list of functions that are executed whenever sigspec is received.

mbfl_signal_invoke_handlers signum Function
Invokes all the handlers registered for signum. This function is not meant to be used during normal scripts execution, but it may be useful to debug a script.


Node:String, Next:, Previous:Signal, Up:Top

Manipulating strings


Node:String Quote, Next:, Up:String

Quoted characters

mbfl_string_is_quoted_char string position Function
Returns true if the character at position in string is quoted; else returns false. A character is considered quoted if it is preceeded by an odd number of backslashes (\). position is a zero-based index.

mbfl_string_is_equal_unquoted_char string position char Function
Returns true if the character at position in string is equal to char and is not quoted (according to mbfl_string_is_quoted_char); else returns false. position is a zero-based index.

mbfl_string_quote string Function
Prints string with quoted characters. All the occurrences of the backslash character, \, are substituted with a quoted backslash, \\. Returns true.


Node:String Inspection, Next:, Previous:String Quote, Up:String

Inspecting a string

mbfl_string_index string index Function
Selects a character from a string. Echoes to stdout the selected character. If the index is out of range: the empty string is echoed to stdout.

mbfl_string_first string char ?begin? Function
Searches characters in a string. Arguments: string, the target string; char, the character to look for; begin, optional, the index of the character in the target string from which the search begins (defaults to zero).

Prints an integer representing the index of the first occurrence of char in string. If the character is not found: nothing is sent to stdout.

mbfl_string_last string char ?begin? Function
Searches characters in a string starting from the end. Arguments: string, the target string; char, the character to look for; begin, optional, the index of the character in the target string from which the search begins (defaults to zero).

Prints an integer representing the index of the last occurrence of char in string. If the character is not found: nothing is sent to stdout.

mbfl_string_range string begin end Function
Extracts a range of characters from a string. Arguments: string, the source string; begin, the index of the first character in the range; end, optional, the index of the character next to the last in the range, this character is not extracted. end defaults to the last character in the string; if equal to end: the end of the range is the end of the string. Echoes to stdout the selected range of characters.

mbfl_string_equal_substring string position pattern Function
Returns true if the substring starting at position in string is equal to pattern; else returns false. If position plus the length of pattern is greater than the length of string: the return value is false, always.


Node:String Splitting, Next:, Previous:String Inspection, Up:String

Splitting a string

mbfl_string_chars string Function
Splits a string into characters. Fills an array named SPLITFIELD with the characters from the string; the number of elements in the array is stored in a variable named SPLITCOUNT. Both SPLITFIELD and SPLITCOUNT may be declared local in the scope of the caller.

The difference between this function and using: ${STRING:$i:1}, is that this function detects backslash characters, \, and treats them as part of the following character. So, for example, the sequence \n is treated as a single char.

Example of usage for mbfl_string_chars:

string="abcde\nfghilm"
mbfl_string_chars "${string}"
# Now:
# "${#string}" = $SPLITCOUNT
#  a = "${SPLITFIELD[0]}"
#  b = "${SPLITFIELD[1]}"
#  c = "${SPLITFIELD[2]}"
#  d = "${SPLITFIELD[3]}"
#  e = "${SPLITFIELD[4]}"
#  \n = "${SPLITFIELD[5]}"
#  f = "${SPLITFIELD[6]}"
#  g = "${SPLITFIELD[7]}"
#  h = "${SPLITFIELD[8]}"
#  i = "${SPLITFIELD[9]}"
#  l = "${SPLITFIELD[10]}"
#  m = "${SPLITFIELD[11]}"

mbfl_string_split string separator Function
Splits string into fields using seprator. Fills an array named SPLITFIELD with the characters from the string; the number of elements in the array is stored in a variable named SPLITCOUNT. Both SPLITFIELD and SPLITCOUNT may be declared local in the scope of the caller.


Node:String Case, Next:, Previous:String Splitting, Up:String

Converting between upper and lower case

mbfl_string_toupper string Function
Outputs string with all the occurrencies of lower case ASCII characters (no accents) turned into upper case.

mbfl_string_tolower string Function
Outputs string with all the occurrencies of upper case ASCII characters (no accents) turned into lower case.


Node:String Class, Next:, Previous:String Case, Up:String

Matching a string with a class

mbfl-string-is-alpha-char char Function
Returns true if char is in one of the ranges: a-z, A-Z.

mbfl-string-is-digit-char char Function
Returns true if char is in one of the ranges: 0-9.

mbfl-string-is-alnum-char char Function
Returns true if mbfl-string-is-alpha-char || mbfl-string-is-digit-char returns true when acting on char.

mbfl-string-is-noblank-char char Function
Returns true if char is in none of the characters: , \n, \r, \f, \t. char is meant to be the unquoted version of the non-blank characters: the one obtained with:
$'char'

mbfl-string-is-name-char char Function
Returns true if mbfl-string-is-alnum-char returns true when acting upon char or char is an underscore, _.

mbfl-string-is-alpha string Function
mbfl-string-is-digit string Function
mbfl-string-is-alnum string Function
mbfl-string-is-noblank string Function
mbfl-string-is-name string Function
Return true if the associated char function returns true for each character in string. As an additional constraint: mbfl-string-is-name returns false if mbfl-string-is-digit returns true when acting upon the first character of string.


Node:String Misc, Previous:String Class, Up:String

Miscellaneous functions

mbfl_string_replace string pattern ?subst? Function
Replaces all the occurrences of pattern in string with subst; prints the result. If not used, subst defaults to the empty string.

mbfl_sprintf varname format ... Function
Makes use of printf to format the string format with the additional arguments, then stores the result in varname: if this name is local in the scope of the caller, this has the effect of filling the variable in that scope.

mbfl_string_skip string varname char Function
Skips all the characters in a string equal to char. varname is the name of a variable in the scope of the caller: its value is the offset of the first character to test in string. The offset is incremented until a char different from char is found, then the value of varname is update to the position of the different char. If the initial value of the offset corresponds to a char equal to char, the variable is left untouched. Returns true.


Node:Dialog, Next:, Previous:String, Up:Top

Interacting with the user

mbfl_dialog_yes_or_no string ?progname? Function
Prints the question string on the standard output and waits for the user to type yes or no in the standard input. Returns true if the user has typed yes, false if the user has typed no.

The optional parameter progname is used as prefix for the prompt; if not given: defaults to the value of script_PROGNAME (Service Variables for details).

mbfl_dialog_ask_password prompt Function
Prints prompts followed by a colon and a space, then reads a password from the terminal. Prints the password.


Node:Variables, Next:, Previous:Dialog, Up:Top

Manipulating variables


Node:Variables Arrays, Next:, Up:Variables

Manipulating arrays

mbfl_variable_find_in_array element Function
Searches the array mbfl_FIELDS for a value equal to element. If it is found: prints the index and returns true; else prints nothing and returns false.

mbfl_FIELDS must be filled with elements having subsequent indexes starting at zero.

mbfl_variable_element_is_in_array element Function
A wrapper for mbfl_variable_find_in_array that does not print anything.


Node:Variables Colon, Previous:Variables Arrays, Up:Variables

Manipulating colon variables

mbfl_variable_colon_variable_to_array varname Function
Reads varname's value, a colon separated list of string, and stores each string in the array mbfl_FIELDS, starting with a base index of zero.

mbfl_variable_array_to_colon_variable
varname
Function
Stores each value in the array mbfl_FIELDS in varname as a colon separated list of strings.

mbfl_variable_colon_variable_drop_duplicate varname Function
Reads varname's value, a colon separated list of string, and removes duplicates.


Node:Main, Next:, Previous:Variables, Up:Top

Main function

MBFL declares a function to drive the execution of the script; its purpose is to make use of the other modules to reduce the size of scripts depending on MBFL. All the code blocks in the script, with the exception of global variables declaration, should be enclosed in functions.

mbfl_main Function
Must be the last line of code in the script. Does the following.
  1. Registers the value of the variable script_PROGNAME in the message module using the function mbfl_message_set_progname.
  2. If it exists: invokes the function script_before_parsing_options.
  3. Parses command line options with mbfl_getopts_parse.
  4. If it exists: invokes the function script_after_parsing_options.
  5. Invokes the function whose name is stored in the global variable mbfl_main_SCRIPT_FUNCTION, if it exists, with no arguments; if its return value is non-zero: exits the script with the same code. The default value is main.
  6. Exits the script with the return code of the action function or zero.

mbfl_invoke_script_function funcname Function
If funcname is the name of an existing function: it is invoked with no arguments; the return value is the one of the function. The existence test is performed with:
type -t FUNCNAME = function

mbfl_main_set_main funcname Function
Selects the main function storing funcname into mbfl_main_SCRIPT_FUNCTION.


Node:Testing, Next:, Previous:Main, Up:Top

Building test suites

MBFL comes with a little library of functions that may be used to build test suites; its aim is at building tests for bash functions/commands/scripts.

The ideas at the base of this library are taken from the tcltest package distributed with the TCL core 1; this package had contributions from the following people/entities: Sun Microsystems, Inc.; Scriptics Corporation; Ajuba Solutions; Don Porter, NIST; probably many many others.

The library tries to do as much as possible using functions and aliases, not variables; this is an attempt to let the user redefine functions to his taste.


Node:Testing Intro, Next:, Up:Testing

A way to organise a test suite

A useful way to organise a test suite is to split it into a set of files: one for each module to be tested.

The file mbfltest.sh must be sourced at the beginning of each test file.

The function dotest should be invoked at the end of each module in the test suite; each module should define functions starting with the same prefix. A module should be stored in a file, and should look like the following:

# mymodule.test --

source mbfltest.sh
source module.sh

function module-featureA-1.1 () { ... }
function module-featureA-1.2 () { ... }
function module-featureA-2.1 () { ... }
function module-featureB-1.1 () { ... }
function module-featureB-1.2 () { ... }

dotest module-

### end of file

the file should be executed with:

$ bash mymodule.test

To test just "feature A":

$ TESTMATCH=module-featureA bash mymodule.test

Remember that the source builtin will look for files in the directories selected by the PATH environment variables, so we may want to do:

$ PATH="path/to/modules:${PATH}" \
TESTMATCH=module-featureA bash mymodule.test

It is better to put such stuff in a Makefile, with GNU make:

top_srcdir      = ...
builddir        = ...
BASHPROG        = bash
MODULES         = moduleA moduleB

testdir         = $(top_srcdir)/tests
test_FILES      = $(foreach f, $(MODULES), $(testdir)/$(f).test)
test_TARGETS    = test-modules

test_ENV        = PATH=$(builddir):$(testdir):$(PATH) TESTMATCH=$(TESTMATCH)
test_CMD        = $(test_ENV) $(BASHPROG)

.PHONY: test-modules

test-modules:
ifneq ($(strip $(test_FILES)),)
        @$(foreach f, $(test_FILES), $(test_CMD) $(f);)
endif


Node:Testing Config, Next:, Previous:Testing Intro, Up:Testing

Configuring the package

dotest-set-verbose Function
dotest-unset-verbose Function
Set or unset verbose execution. If verbose mode is on: some commands output messages on stderr describing what is going on. Examples: files and directories creation/removal.

dotest-option-verbose Function
Returns true if verbose mode is on, false otherwise.

dotest-set-test Function
dotest-unset-test Function
Set or unset test execution. If test mode is on: external commands (like rm and mkdir) are not executed, the command line is sent to stderr. Test mode is meant to be used to debug the test library functions.

dotest-option-test Function
Returns true if test mode is on, false otherwise.

dotest-set-report-start Function
dotest-unset-report-start Function
Set or unset printing a message upon starting a function.

dotest-option-report-start Function
Returns true if start function reporting is on; otherwise returns false.

dotest-set-report-success Function
dotest-unset-report-success Function
Set or unset printing a message when a function execution succeeds. Failed tests always cause a message to be printed.

dotest-option-report-success Function
Returns true if success function reporting is on; otherwise returns false.


Node:Testing Running, Next:, Previous:Testing Config, Up:Testing

Running test functions

dotest pattern Funciton
Run all the functions matching pattern. Usually pattern is the first part of the name of the functions to be executed; the function names are selected with the following code:
compgen -A function "$pattern"

There's no constraint on function names, but they must be one-word names.

Before running a test function: the current process working directory is saved, and it is restored after the execution is terminated.

The return value of the test functions is used as result of the test: true, the test succeeded; false, the test failed. Remembering that the return value of a function is the return value of its last executed command, the functions dotest-equal and dotest-output, and of course the test command, may be used to return the correct value.

Messages are printed before and after the execution of each function, according to the mode selected with: dotest-set-report-success, dotest-set-report-start, ... (Testing Config for details).

The following environment variables may configure the behaviour of dotest.

TESTMATCH
Overrides the value selected with pattern.
TESTSTART
If yes: it is equivalent to invoking dotest-set-report-start; if no: it is equivalent to invoking dotest-unset-report-start.
TESTSUCCESS
If yes: it is equivalent to invoking dotest-set-report-success; if no: it is equivalent to invoking dotest-unset-report-success.


Node:Testing Compare, Next:, Previous:Testing Running, Up:Testing

Validating results by comparing

dotest-equal expected got Function
Compares the two parameters and returns true if they are equal; returns false otherwise. In the latter case prints a message showing the expected value and the wrong one. Must be used as last command in a function, so that its return value is equal to that of the function.

Example:

function my-func () {
    echo $(($1 + $2))
}
function mytest-1.1 () {
    dotest-result 5 `my-func 2 3`
}
dotest mytest-

another example:

function my-func () {
    echo $(($1 + $2))
}
function mytest-1.1 () {
    dotest-result 5 `my-func 2 3` && \
      dotest-result 5 `my-func 1 4` && \
      dotest-result 5 `my-func 3 2` && \
}
dotest mytest-


Node:Testing Output, Next:, Previous:Testing Compare, Up:Testing

Validating results by output

dotest-output ?string? Function
Reads all the available lines from stdin accumulating them into a local variable, separated by \n; then compares the input with string, or the empty string if string is not present, and returns true if they are equal, false otherwise.

Example of test for a function that echoes its three parameters:

function my-lib-function () {
    echo $1 $2 $3
}
function mytest-1.1 () {
    my-lib-function a b c | dotest-output a b c
}
dotest mytest

Example of test for a function that is supposed to print nothing:

function my-lib-function () {
    test "$1" != "$2" && echo error
}
function mytest-1.1 () {
    my-lib-function a a | dotest-output
}
dotest mytest

Validating input

Here is a small script that asks for a first name then a second name:

 
$ pg func2 

#!/bin/sh
# func2
echo -n "What is your first name :"
read F_NAME
echo -n "What is your surname :"
read S_NAME 

The task is to make sure that the characters entered in both variables contain letters only. To do this without functions would duplicate a lot of code. Using a function cuts this duplication down. To test for characters only, we can use awk. Here's the function to test if we only get upper or lower case characters.
 

char_name()
{
# char_name
# to call: char_name string
# assign the argument across to new variable
_LETTERS_ONLY=$1
# use awk to test for characters only !
_LETTERS_ONLY=`echo $1|awk '{if($0~/[^a-z A-Z]/) print "1"}'`
if [ "$_LETTERS_ONLY" != "" ]
then
# oops errors
return 1
else
# contains only chars
return 0
fi
}


We first assign the $1 variable to a more meaningful name. Awk is then used to test if the whole record passed contains only characters. The output of this command, which is 1 for non-letters and null for OK, is held in the variable _LETTERS_ONLY.

A test on the variable is then carried out. If it holds any value then it's an error, but if it holds no value then it's OK. A return code is then executed based on this test. Using the return code enables the script to look cleaner when the test is done on the function on the calling part of the script.

To test the outcome of the function we can use this format of the if statement if we wanted:

if char_name $F_NAME; then
  echo "OK"
else
  echo "ERRORS"
fi 

If there is an error we can create another function to echo the error out to the screen:

 
name_error()
# name_error
# display an error message
{
echo " $@ contains errors, it must contain only letters"
}

The function name_error will be used to echo out all errors disregarding any invalid entries. Using the special variable $@ allows all arguments to be echoed. In this case it's the value of either F_NAME or S_NAME. Here's what the finished script now looks like, using the functions:

 
$ pg func2 

!/bin/sh
char_name()
# char_name
# to call: char_name string
# check if $1 does indeed contain only characters a-z,A-Z
{
# assign the argurment across to new variable
_LETTERS_ONLY=$1
_LETTERS_ONLY=`echo $1|awk '{if($0~/[^a-zA-Z]/) print "1"}'`
if [ "$_LETTERS_ONLY" != "" ]
then
  # oops errors
  return 1
else
  # contains only chars
  return 0
fi
}

name_error()
# display an error message
{
echo " $@ contains errors, it must contain only letters"
}

while :
do
  echo -n "What is your first name :"
  read F_NAME
  if char_name $F_NAME
  then
    # all ok breakout
    break
  else
    name_error $F_NAME
  fi
done

while :
do
  echo -n "What is your surname :"
  read S_NAME
  if char_name $S_NAME
  then
    # all ok breakout
    break
  else
    name_error $S_NAME
  fi
done

Notice a while loop for each of the inputs; this makes sure that we will continue prompting until a correct value is input, then we break out of the loop. Of course, on a working script, an option would be given for the user to quit this cycle, and proper cursor controls would be used, as would checking for zero length fields.

Here's what the output looks like when the script is run:

 
$ func2
What is your first name :Davi2d
 Davi2d contains errors, it must contain only letters
What is your first name :David
What is your surname :Tansley1
Tansley1 contains errors, it must contain only letters
What is your surname :Tansley 

Reading a single character

When navigating menus, one of the most frustrating tasks is having to keep hitting the return key after every selection, or when a 'press any key to continue' prompt appears. A command that can help us with not having to hit return to send a key sequence is the dd command.

The dd command is used mostly for conversions and interrogating problems with data on tapes or normal tape archiving tasks, but it can also be used to create fixed length files. Here a 1-megabyte file is created with the filename myfile.

 
dd if=/dev/zero of=myfile count=512 bs=2048

The dd command can interpret what is coming in from your keyboard, and can be used to accept so many characters. In this case we only want one character. The command dd needs to chop off the new line; this control character gets attached when the user hits return. dd will also send out only one character. Before any of that can happen the terminal must first be set into raw mode using the stty command. We save the settings before dd is invoked and then restore them after dd has finished.

Here's the function:

 
read_a_char()
# read_a_char
{
# save the settings
SAVEDSTTY=`stty -g`
# set terminal raw please
    stty cbreak
  # read and output only one character
    dd if=/dev/tty bs=1 count=1 2> /dev/null
  # restore terminal and restore stty
    stty -cbreak
stty $SAVEDSTTY
}

To call the function and return the character typed in, use command substitution. Here's an example.

 
echo -n "Hit Any Key To Continue"
character=`read_a_char`
echo " In case you are wondering you pressed $character"
 

Testing for the presence of a directory

Testing for the presence of directories is a fairly common task when copying files around. This function will test the filename passed to the function to see if it is a directory. Because we are using the return command with a succeed or failure value, the if statement becomes the most obvious choice in testing the result.

Here's the function.

 
isdir()
{
# is_it_a_directory

if [ $# -lt 1 ]; then
  echo "isdir needs an argument"
  return 1
fi
# is it a directory ?
_DIRECTORY_NAME=$1
if [ ! -d $_DIRECTORY_NAME ]; then
  # no it is not
  return 1
else
  # yes it is
  return 0
fi
}

Getting information from a login ID

When you are on a big system, and you want to contact one of the users who is logged in, don't you just hate it when you have forgotten the person's full name? Many a time I have seen users locking up a process, but their user ID means nothing to me, so I have to grep the passwd file to get their full name. Then I can get on with the nice part where I can ring them up to give the user a telling off.

Here's a function that can save you from grep ing the /etc/passwd file to see the user's full name.

On my system the user's full name is kept in field 5 of the passwd file; yours might be different, so you will have to change the field number to suit your passwd file.

The function is passed a user ID or many IDs, and the function just grep s the passwd file.

Here's the function:

 
whois()
# whois
# to call: whois userid
{
# check we have the right params
if [ $# -lt 1 ]; then
  echo "whois : need user id's please"
  return 1
fi

for loop
do
  _USER_NAME=`grep $loop /etc/passwd | awk -F: '{print $4}'`
  if [ "$_USER_NAME" = "" ]; then
    echo "whois: Sorry cannot find $loop"
  else
    echo "$loop is $_USER_NAME"
  fi
done
}

The whois function can be called like this:
 

$ whois dave peters superman
dave is David Tansley - admin accts
peter is Peter Stromer - customer services
whois: Sorry cannot find superman


Line numbering a text file

When you are in vi you can number your lines which is great for debugging, but if you want to print out some files with line numbers then you have to use the command nl. Here is a function that does what nl does best – numbering the lines in a file. The original file is not overwritten.

number_file()
# number_file
# to call: number_file filename
{
_FILENAME=$1
# check we have the right params
if [ $# -ne 1 ]; then
echo "number_file: I need a filename to number"
return 1
fi

loop=1
while read LINE
do
echo "$loop: $LINE"
loop=`expr $loop + 1`
done < $_FILENAME
}

String to upper case

You may need to convert text from lower to upper case sometimes, for example to create directories in a filesystem with upper case only, or to input data into a field you are validating that requires the text to be in upper case.

Here is a function that will do it for you. No points for guessing it's tr.

 
str_to_upper ()
# str_to_upper
# to call: str_to_upper $1
{
_STR=$1
# check we have the right params
if [ $# -ne 1 ]; then
  echo "number_file: I need a string to convert please"
  return 1
fi
echo $@ |tr '[a-z]' '[A-Z]'
}

The variable UPPER holds the newly returned upper case string. Notice the use again of using the special parameter $@ to pass all arguments. The str_to_upper can be called in two ways. You can either supply the string in a script like this:

 
UPPER=`str_to_upper "documents.live"`
echo $upper 

or supply an argument to the function instead of a string, like this:

UPPER=`str_to_upper $1`
echo $UPPER 

Both of these examples use substitution to get the returned function results.

is_upper

The function str_to_upper does a case conversion, but sometimes you only need to know if a string is upper case before continuing with some processing, perhaps to write a field of text to a file. The is_upper function does just that. Using an if statement in the script will determine if the string passed is indeed upper case.

Here is the function.

 
is_upper()
# is_upper
# to call: is_upper $1
{
# check we have the right params
if [ $# -ne 1 ]; then
  echo "is_upper: I need a string to test OK"
  return 1
fi
# use awk to check we have only upper case
_IS_UPPER=`echo $1|awk '{if($0~/[^A-Z]/) print "1"}'`
if [ "$_IS_UPPER" != "" ]
then
  # no, they are not all upper case
  return 1
else
  # yes all upper case
  return 0
fi
} 

To call the function is_upper simply send it a string argument. Here's how it could be called.
 

echo -n "Enter the filename :"
read FILENAME
if is_upper $FILENAME; then
echo "Great it's upper case"
# let's create a file maybe ??
else
echo "Sorry it's not upper case"
# shall we convert it anyway using str_to_upper ???
fi


To test if a string is indeed lower case, just replace the existing awk statement with this one inside the function is_upper and call it is_lower.

_IS_LOWER=`echo $1|awk '{if($0~/[^a-z]/) print "1"}'` 

String to lower case

Now I've done it. Because I have shown you the str_to_upper, I'd better show you its sister function str_to_lower. No guesses here please on how this one works.

str_to_lower ()
# str_to_lower
# to call: str_to_lower $1
{
# check we have the right params
if [ $# -ne 1 ]; then
  echo "str_to_lower: I need a string to convert please"
  return 1
fi
echo $@ |tr '[A-Z]' '[a-z]'
} 

The variable LOWER holds the newly returned lower case string. Notice the use again of using the special parameter $@ to pass all arguments. The str_to_lower can be called in two ways. You can either supply the string in a script like this:

 

LOWER=`str_to_lower "documents.live"`
echo $LOWER 

or supply an argument to the function instead of a string, like this:
 

LOWER=`str_to_upper $1`
echo $LOWER


Length of string

Validating input into a field is a common task in scripts. Validating can mean many things, whether it's numeric, character only, formats, or the length of the field.

Suppose you had a script where the user enters data into a name field via an interactive screen. You will want to check that the field contains only a certain number of characters, say 20 for a person's name. It's easy for the user to input up to 50 characters into a field. This is what this next function will check. You pass the function two parameters, the actual string and the maximum length the string should be.

Here's the function:

 
check_length()
# check_length
# to call: check_length string max_length_of_string
{
_STR=$1
_MAX=$2
# check we have the right params
if [ $# -ne 2 ]; then
  echo "check_length: I need a string and max length the string should be"
  return 1
fi
# check the length of the string
_LENGTH=`echo $_STR |awk '{print length($0)}'`
if [ "$_LENGTH" -gt "$_MAX" ]; then
  # length of string is too big
  return 1
else
  # string is ok in length
  return 0
fi
} 

You could call the function check_length like this:

 
$ pg test_name 

# !/bin/sh
# test_name
while :
do
  echo -n "Enter your FIRST name :"
  read NAME
  if check_length $NAME 10
  then
    break
    # do nothing fall through condition all is ok
  else
    echo "The name field is too long 10 characters max"
  fi
done 

The loop will continue until the data input into the variable NAME is less than the MAX characters permitted which in this case is ten; the break command then lets it drop out of the loop.

Using the above piece of code this is how the output could look.

 
$ val_max
Enter your FIRST name :Pertererrrrrrrrrrrrrrr
The name field is too long 10 characters max
Enter your FIRST name :Peter 

You could use the wc command to get the length of the string, but beware: there is a glitch when using wc in taking input from the keyboard. If you hit the space bar a few times after typing in a name, wc will almost always retain some of the spaces as part of the string, thus giving a false length size. Awk truncates end of string spaces by default when reading in via the keyboard.

Here's an example of the wc glitch (or maybe it's a feature):

 
echo -n "name :"
read NAME
echo $NAME | wc -c 

Running the above script segment (where graphics/square.gif is a space)
 

name :Petergraphics/square.gifgraphics/square.gif
6


chop

The chop function chops off characters from the beginning of a string. The function chop is passed a string; you specify how many characters to chop off the string starting from the first character. Suppose you had the string MYDOCUMENT.DOC and you wanted the MYDOCUMENT part chopped, so that the function returned only .DOC. You would pass the following to the chop function:

 
MYDOCUMENT.DOC 10 

Here's the function chop:

 

chop()
# chop
# to call:chop string how_many_chars_to_chop
{
_STR=$1
_CHOP=$2
# awk's substr starts at 0, we need to increment by one
# to reflect when the user says (ie) 2 chars to be chopped it will be 2
chars off
# and not 1
CHOP=`expr $_CHOP + 1`

# check we have the right params
if [ $# -ne 2 ]; then
echo "check_length: I need a string and how many characters to chop"
return 1
fi
# check the length of the string first
# we can't chop more than what's in the string !!
_LENGTH=`echo $_STR |awk '{print length($0)}'`
if [ "$_LENGTH" -lt "$_CHOP" ]; then
echo "Sorry you have asked to chop more characters than there are in
the string"
return 1
fi
echo $_STR |awk '{print substr($1,'$_CHOP')}'
}


The returned string newly chopped is held in the variable CHOPPED. To call the function chop, you could use:

 

CHOPPED=`chop "Honeysuckle" 5`
echo $CHOPPED
suckle


or you could call this way:

 

echo -n "Enter the Filename :"
read FILENAME
CHOPPED=`chop $FILENAME 1`
# the first character would be chopped off !
Months

When generating reports or creating screen displays, it is sometimes convenient to the programmer to have a quick way of displaying the full month. This function, called months, will accept the month number or month abbreviation and then return the full month.

For example, passing 3 or 03 will return March. Here's the function.

 

months()
{
# months
_MONTH=$1
# check we have the right params
if [ $# -ne 1 ]; then
  echo "months: I need a number 1 to 12 "
  return 1
fi

case $_MONTH in
1|01|Jan)_FULL="January" ;;
2|02|Feb)_FULL="February" ;;
3|03|Mar)_FULL="March";;
4|04|Apr)_FULL="April";;
5|05|May)_FULL="May";;
6|06|Jun)_FULL="June";;
7|07|Jul)_FULL="July";;
8|08|Aug)_FULL="August";;
9|10|Sep|Sept)_FULL="September";;
10|Oct)_FULL="October";;
11|Nov)_FULL="November";;
12|Dec)_FULL="December";;
*) echo "months: Unknown month"
  return 1
  ;;
esac
echo $_FULL
} 

To call the function months you can use either of the following methods.

 

months 04

 

The above method will display the month April; or from a script:

 

MY_MONTH=`months 06`
echo "Generating the Report for Month End $MY_MONTH"
...

which would output the month June.

Calling functions inside a script

To use a function in a script, create the function, and make sure it is above the code that calls it. Here's a script that uses a couple of functions. We have seen the script before; it tests to see if a directory exists.

 

$ pg direc_check
!/bin/sh
# function file
is_it_a_directory()
{
# is_it_a_directory
# to call: is_it_a_directory directory_name
_DIRECTORY_NAME=$1
if [ $# -lt 1 ]; then
echo "is_it_a_directory: I need a directory name to check"
return 1
fi
# is it a directory ?
if [ ! -d $_DIRECTORY_NAME ]; then
return 1
else
return 0
fi
}
#--------------------------------------------------
error_msg()
{
# error_msg
# beeps; display message; beeps again!
echo -e "\007"
echo $@
echo -e "\007"
return 0
}
}

### END OF FUNCTIONS

echo -n "enter destination directory :"
read DIREC
if is_it_a_directory $DIREC
then :
else
error_msg "$DIREC does not exist...creating it now"
mkdir $DIREC > /dev/null 2>&1
if [ $? != 0 ]
then
error_msg "Could not create directory:: check it out!"
exit 1
else :
fi
fi # not a directory
echo "extracting files..."

In the above script two functions are declared at the top of the script and called from the main part of the script. All functions should go at the top of the script before any of the main scripting blocks begin. Notice the error message statement; the function error_msg is used, and all arguments passed to the function error_msg are just echoed out with a couple of bleeps.

Calling functions from a function file

We have already seen how to call functions from the command line; these types of functions are generally used for system reporting utilities.

Let's use the above function again, but this time put it in a function file. We will call it functions.sh, the sh meaning shell scripts.

 

$ pg functions.sh
#!/bin/sh
# functions.sh
# main script functions
is_it_a_directory()
{
# is_it_a_directory
# to call: is_it_a_directory directory_name
#
if [ $# -lt 1 ]; then
echo "is_it_a_directory: I need a directory name to check"
return 1
fi
# is it a directory ?
DIRECTORY_NAME=$1
if [ ! -d $DIRECTORY_NAME ]; then
return 1
else
return 0
fi
}

#---------------------------------------------

error_msg()
{
echo -e "\007"
echo $@
echo -e "\007"
return 0
}

Now let's create the script that will use functions in the file functions.sh. We can then use these functions. Notice the functions file is sourced with the command format:

 

. /<path to file> 

A subshell will not be created using this method; all functions stay in the current shell.
 

$ pg direc_check
!/bin/sh
# direc_check
# source the function file functions.sh
# that's a <dot><space><forward slash>
. /home/dave/bin/functions.sh

# now we can use the function(s)

echo -n "enter destination directory :"
read DIREC
if is_it_a_directory $DIREC
then :
else
error_msg "$DIREC does not exist...creating it now"
mkdir $DIREC > /dev/null 2>&1
if [ $? != 0 ]
then
error_msg "Could not create directory:: check it out!"
exit 1
else :
fi
fi # not a directory
echo "extracting files..."

When we run the above script we get the same output as if we had the function inside our script:

 

$ direc_check
enter destination directory :AUDIT
AUDIT does not exist...creating it now
extracting files... 

Sourcing files is not only for functions

To source a file, it does not only have to contain functions – it can contain global variables that make up a configuration file.

Suppose you had a couple of backup scripts that archived different parts of a system. It would be a good idea to share one common configuration file. All you need to do is to create your variables inside a file then when one of the backup scripts kicks off it can load these variables in to see if the user wants to change any of the defaults before the archive actually begins. It may be the case that you want the archive to go to a different media.

Of course this approach can be used by any scripts that share a common configuration to carry out a process. Here's an example. The following configuration contains default environments that are shared by a few backup scripts I use.

Here's the file.

 

$ pg backfunc 

#!/bin/sh
# name: backfunc
# config file that holds the defaults for the archive systems
_CODE="comet"
_FULLBACKUP="yes"
_LOGFILE="/logs/backup/"
_DEVICE="/dev/rmt/0n"
_INFORM="yes"
_PRINT_STATS="yes" 

The descriptions are clear. The first field _CODE holds a code word. To be able to view this and thus change the values the user must first enter a code that matches up with the value of _CODE, which is "comet".

Here's the script that prompts for a password then displays the default configuration:

 
$ pg readfunc 

#!/bin/sh
# readfunc

if [ -r backfunc ]; then
  # source the file
  . /backfunc
else
  echo "$`basename $0` cannot locate backfunc file"
fi

echo -n "Enter the code name :"
# does the code entered match the code from backfunc file ???
if [ "${CODE}" != "${_CODE}" ]; then
  echo "Wrong code...exiting..will use defaults"
  exit 1
fi

echo " The environment config file reports"
echo "Full Backup Required            : $_FULLBACKUP"
echo "The Logfile Is                  : $_LOGFILE"
echo "The Device To Backup To is      : $_DEVICE"
echo "You Are To Be Informed by Mail  : $_INFORM"
echo "A Statistic Report To Be Printed: $_PRINT_STATS" 

When the script is run, you are prompted for the code. If the code matches, you can view the defaults. A fully working script would then let the user change the defaults.

 
$ readback
Enter the code name :comet

The environment config file reports
Full Backup Required            : yes
The Logfile Is                  : /logs/backup/
The Device To Backup To is      : /dev/rmt/0n
You Are To Be Informed by Mail  : yes
A Statistic Report To Be Printed: yes 

Using functions will greatly reduce the time you spend scripting. Creating useable and reuseable functions makes good sense; it also makes your main scripts less maintenance-prone.

When you have got a set of functions you like, put them in a functions file, then other scripts can use the functions as well.


Etc

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least


Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: November 08, 2017