Declarative Languages
Lecture #10

Purpose: Variables and flexible argument lists and a number of interesting "that-reminds-me"s

10.1 Variables and symbols

A number of lisp forms (such as let* defun dolist etc.) bind symbol names as local (lexical) variables. Within such a binding, "evaluating the symbol" returns its value exactly as if (locally to this binding!) that symbol really did exist at run-time with the given value:

(let* ((foo "hello")) foo)  =>  "hello"
Be warned however that a local binding does not create an actual symbol at run-time, not does it set the value of an existing symbol unless that symbol has been declared to be global by defparameter:
  1. (makunbound 'foo)

    2. (let* ((foo "hello"))
      (symbol-value 'foo))
                        =>  Error: The variable FOO is unbound....
     
  2. (setf foo 99)

    3. (let* ((foo "hello"))
      (list foo (symbol-value 'foo)))  =>  ("hello" 99)
     
  3. (defparameter foo 99)  ; foo is now global

    4. (let* ((foo "hello"))
      (list foo (symbol-value 'foo)))  =>  ("hello" "hello")
    foo  =>  99            ; outside the binding again
In all three cases, the effect of the binding is transient - when you exit the binding form a lexical variable disappears, and a global variable returns to its old value (no matter how many times it has been reset within the binding):
(defparameter foo 99)     =>  foo
(let* ((foo "hello"))
  (setf foo nil)
  foo)                    =>  nil
foo                       =>  99
Note: another name for a global (dynamic) variable is special variable. If you go (setf foo 99) in your code without first declaring foo to be either global (by defparameter) or local (by binding it via let* &co), you see the familiar rant:
        Warning: Syntactic warning for form (SETF FOO 99):
           FOO assumed special.
(which you will now understand).

10.2 Some interesting global variables - *features* and read-time conditionals

Common Lisp ships with a few dozen dynamic variables which are used to describe and control the state of the lisp system. We'll take a look at a small number of them here, starting with the variable *features*.

This is a list of symbols - typically keywords - used to enumerate "features" (whatever those might be) currently present in the image. Its initial value is implementation dependent and fairly horrid, but you come to depend on which features are present in which products.

You are allowed to add more features to the list yourself:
    (push :likes-aadvarks *features*)
Although you can interrogate the list directly:
    (if (find :likes-aadvarks *features*)
        (go-get-an-aardvark)
      (have-a-wombat-instead))
the main use of *features* is to drive conditional behaviour at read-time.

The syntax for read-time conditionals is
    #+feature-expression conditional-form
At its simplest, the feature-expression will be a single feature, thus:
    #+likes-aardvarks (go-get-an-aardvark)
When the reader sees this, it looks the feature up in *features*. If the feature is present then the next form (go-get-an-aardvark) is read as if the #+ hadn't been there; if the feature is absent then reader skips the next form altogether.

You can assume that all features are keywords. These, you will recall, are symbols whose name is preceeded by a colon. Note however that the #+ syntax assumes the colon for you.

Features can be combined in the feature expression by and, or and not. The syntax #-feature-expression is equivalent to #+(not feature-expression). For example, if we read the form
    (1 2 #+foo 3 4 #-foo 5)
and :foo is a feature (i.e. the symbol :foo is to be found in the list *features*) then we get (1 2 3 4) but if :foo is not a feature then we get (1 2 4 5). Another example: to read a form if :foo is a feature (and simply skip it otherwise):
    #+foo (do-something-specific-to-foo)

Typical uses of #+ and #- are

(Incidentally, the first two of the above examples are both real-life code from vendors' support sites.)

This facility is lisp's answer to C's #ifdef. It wins in a seriously big way because, unlike #ifdef, (a) it isn't pegged to the left hand margin or to a line of its own and (b) you don't have to go hunting for an #endif, so readability isn't destroyed and (c) you can combine features with and etc. rather than having to nest several #ifdef lines.

10.3 Some interesting global variables - *read-base* / *print-base*

There are 20 variables (named *print-mumble* for various mumbles) for controlling the printer, and 5 more which control the reader. We'll take a look at two of them here and you can look the rest up in some moment of leisure.

The lisp reader and printer are by default set up to operate in base ten. You can modify this by setting or binding *read-base* and *print-base* respectively.

For example:

;; set *print-base* permanently to sixteen
(setf *print-base* 16)

;; bind *print-base* to two for the duration of the call to print-values
(defun print-values-in-binary (values)
  (let* ((*print-base* 2))
    (print-values values)))

;; another idiom for binding a special variable. In this case,
;; *print-base* is bound to the value of the second argument to
;; the function
(defun print-values-in-base (values *print-base*)
  (print-values values))

(defun print-values (values)
   (mapc 'print values))

;; (print-values-in-base '(3 4 5) 2) prints 11 100 and 101

If you want to enter numbers in a different base to *read-base*, use #x for hexadecimal (#x10 is sixteen), #o and #b for octal and binary respectively, and #r for the base of your choice (#3r102 expresses eleven in base three, and #11R32 expresses thirty five in base eleven). Note that the x, o, b, r can be either upper or lower case.

If you want a quick and clean way of ensuring a number is in base ten, enter it with a decimal point:
    (setf *read-base* 10)   ; pointless at top-level
won't change anything at the top level (why?) but
    (setf *read-base* 10.)  ; rescue the situation
is guaranteed to deliver a base ten number. (Warning: 10. is an integer but 10.0 is a float.)

If you want to print numbers in a different base to *print-base*, then format has various directives to help you. ~x gives you hexadecimal, ~o and ~b give you octal and binary, and ~r can give you the base of your choice:
    (format nil "~x  ~8r" 10 #o777)  =>  "A  777"


10.4 Some interesting global variables - *standard-input* / *standard-output*

By default, all user input comes from a stream called *standard-input* and all output goes to a stream called *standard-output*. For example, in the LispWorks listener you can evaluate these two symbols and get one set of ugliness, and in the editor you can evaluate them again and get a different set of ugliness.

The function format actually takes a destination as its first argument. The value t stands for *standard-output* and nil stands for writing to a string, but you can give any output stream instead. Functions like print take a destination as an optional second argument; functions like read and read-line take an input stream as an optional second argument.

To open files for input / output, call the function open (which returns a stream if successful). By default the stream will be opened for input:

(open "foo.txt")                     =>  input stream from foo.txt
(open "foo.txt" :direction :output)  => output stream to foo.txt
Don't forget to close the stream afterwards!
(let* ((istream (open "/etc/passwd")))
  (prog1
      (loop (let* ((line (read-line istream)))
              (when (and (> (length line) 5)
                         (string= line "root" :end1 4))
                (return (subseq line 5
                                (position #\: line :start 5))))))
    (close istream)))
But cleaner than the above is the macro with-open-file which, as mentioned last week, uses unwind-protect to guarantee that the stream will be closed cleanly: or (more or less equivalently) To see streams in action, you can also try the following: 10.5 Flexible argument lists (the final word) and apply

We have already met two ways of specifying variable numbers of arguments to a function; now let's meet the third plus the function most often used to drive it.
 
Name
Description
Example
&optional Named set, typically small, of optional arguments, specified by position. Each argument defaults to nil unless otherwise specified. (defun opt-args (x &optional y z)
   (list x y z))

(opt-args 1 2)  =>  (1 2 nil)
&key Named set, not necessarily small, 
of optional arguments specified
by name. Each argument defaults 
to nil unless otherwise specified.		 (defun key-args (x &key y z)
   (list x y z))

(key-args 1 :z 2)  =>  (1 nil 2)
&rest One variable, bound to the list of
all arguments after this point.		 (defun rest-args (x &rest y)
   (list x y))

(rest-args 1 2 3 4)  =>  (1 (2 3 4))

The value of &rest is that it gives us a handle on the arguments to a function even when we don't know how may it was called with. We have met several functions with "indefinite" numbers of arguments (e.g. + - * = < > list vector funcall mapcar mapc format) - these all use &rest to pick up and manipulate a list of arguments. For example, the argument list of + is
    (&rest numbers)
and the argument list of format is
    (destination control-string &rest format-arguments)

Quite often, &rest is used alongside the function apply. Recall from last week the function funcall, which takes a function and some arguments and invokes that function with those arguments:
    (funcall '+ 1 2 3 4 5 6 7 8 9)   => 45
apply is similar, in that it takes as its parameters a function and some arguments for that function. However, the final argument to apply must be a list, and the members of that list are passed to the destination function as individual arguments.
    (apply '+ 1 2 3 4 '(5 6 7 8 9))  => 45

The link with &rest feels fairly natural, on the grounds that once we have everything in a list it seems a shame not to carry on like that:

(defun make-window (type)
  (let* ((window (create-window type))
         (initialization-args (obtain-initargs window)))
    (apply 'initialize-window window initialization-args)
    (make-visible window)
    window))

(defun initialize-window (window &rest initialization-args)
  ...)

This warm feeling is usually misleading. Both apply and &rest come at a cost, particularly if the number of arguments concerned is very large (e.g. several hundred). The above fragment could be rewritten
(defun make-window (type)
  (let* ((window (create-window type))
         (initialization-args (obtain-initargs window)))
    (initialize-window window initialization-args)
    (make-visible window)
    window))

(defun initialize-window (window initialization-args)
  ...)

and would work just as well while stressing the system that bit less.

Don't use apply unless you need to. Pass lists of arguments around as lists, as in the above example, and do not forget reduce:

Incidentally, you can combine &optional &key and &rest in a single argument list. The rule is that they must come in the following order:
    &optional &rest &key
Thus &rest can be used to bind one variable to the full list of keyword name-and-value pairs.

Be particularly warned that the combination of &optional with &key is a trap for the unwary - you must specify the &optional values before you can get to the &key arguments. The following Common Lisp functions display this baroque tendency:
    parse-namestring read-from-string write-string write-line
For example, the lambda list of write-string is
    (string &optional stream &key (start 0) end)
so to write all but but the first character of a string to *standard-output* you must make the following call
    (write-string "Hello, world" *standard-output* :start 1)


10.6 A couple of examples of &rest functions: complement and append

The common lisp function complement takes any function as its argument and returns another function. The new function accepts the same arguments as the first one and returns false where the first function would have returned true, and vice versa.

(defun my-complement (fn)
  #'(lambda (&rest args)
      (not (apply fn args))))

(funcall (my-complement 'listp) 9)         =>  t
(funcall (my-complement 'listp) '(1 2 3))  =>  nil

This is an occasion where the use of apply is justified (it would be nigh impossible to code this function otherwise), and we get to play with both closures and &rest at the same time. What more could you ask for?

Finally for today, the functions append and nconc, used for joining lists end-to-end. Both have lambda list (&rest lists). Example:

(append '(1 2 3 4) '(a b c d) '(5 6))  =>  (1 2 3 4 a b c d 5 6)
append is non-destructive: it works by making a copy of the top-level structure of all (apart from the last) of the lists handed to it, and splicing the final list to the end. However, nconc is destructive and splices each list into the one following.
(let* ((list1 (list 1 2 3 4))
       (list2 (list 'a 'b 'c 'd))
       (list3 (list 5 6 7)))
  (mapc 'print (list (nconc list1 list2 list3)
                     list1 list2 list3)))
prints the following
(1 2 3 4 a b c d 5 6 7)      ; result of nconc
(1 2 3 4 a b c d 5 6 7)      ; destructively modified list1
(a b c d 5 6 7)              ; destructively modified list2
(5 6 7)                      ; only list3 undamaged


10.7 Parting shot

(defun baroque-p (sym)
  (and (fboundp sym)
       (let* ((lambda-list (function-lambda-list sym)))
         (and (listp lambda-list)
              (find '&optional lambda-list)
              (find '&key lambda-list)))))

(defun hunt-for-baroque ()
  (let* ((baroque nil))
    (do-external-symbols (sym "COMMON-LISP")
      (if (baroque-p sym)
          (push sym baroque)))
    baroque))

(hunt-for-baroque)
      =>  (parse-namestring write-line write-string read-from-string)

(find-if (complement 'baroque-p) '(write-string write-line write))
      =>  write

10.8 Practical session / Suggested activity 10.9 Further reading & exercises
Copyright (C) Nick Levine 1999. All rights reserved.
Last modified 2000-09-14
$Id: //info.ravenbrook.com/user/ndl/lisp/declarative/lectures/lectures/lecture-10.html#2 $