Declarative Languages
Lecture #5

Purpose: Functions, recursive and iterative: the sordid truth and other stuff

5.1 Let's talk recursive

Suppose we are (still) trying to write a function which takes two arguments - a list and a number - and if the number is = to the any element of the list then return the position of that element in the list. (So if the number matches the first element in the list return 0, if it matches the second return 1, and so on.) If the number isn't found in the list, we'll return nil.

Here's one solution to the problem:

(defun position-one (list number)
(if list
(if (= (first list) number)
0
(let* ((pos (position-one (rest list) number)))
(if pos
(1+ pos)
nil)))))
And here's another:
(defun position-two (list number &optional (pos 0))
(if list
(if (= (first list) number)
pos
(position-two (rest list) number (+ 1 pos)))))
Both these solutions are recursive - the functions call themselves. There are two official tips for how to write recursive functions (these apply to any language, not just lisp):
• you need to find a way to reduce the problem to a slightly simpler case, in the event that this time round wasn't the last pass, and
• you must ensure that whatever the data to your function it will terminate, i.e. not keep calling itself forever.
Also there is one unofficial tip:
• keep it as simple as you can possibly get away with. In my humble opinion position-one above is somewhat on the hairy side.
• the equivalence of the empty list and logical false - handy for tests with if
• the use of &optional to allow functions to take a variable number of arguments
• tail recursion...
What is tail recursion? Answer: a tail recursive function is one in which the recursive call is the last thing it does. Why should that matter to anybody? Answer: because (a) the compiler can transform it into a loop, which will run faster and (b) if the compiler can write it as a loop then so can you, which will save you all the aggro of having to think recursively in the first place. :-) More on this later.

5.2 Sinking without a trace

If you're stuck on seeing how these recursive functions work (and there are plenty of further examples on the solution sheets), you should consider tracing function calls. For example, if you evaluate
(trace position-two)
- after defining the function! - and then call it with test data, you will see a printout of all the recursive calls and their return values.

• Turn tracing off with (untrace position-two) or even (untrace) to untrace everything.
• Be warned that tracing Common Lisp functions (as opposed to tracing your own code) can get a little bit hectic. The reason for this is that there is a strong possibility that the lisp system will be calling these functions anyway, e.g. to handle redisplay of the listener brought on by printing out all that tracing info.
Sometimes tracing won't be appropriate, and you may find yourself peppering your code with print statements to get it debugged. If you need to do this, you might as well make it quick'n'dirty rather than bothering with the finesse of format.  The function print takes a lisp object and prints it (as the read-eval-print loop would, i.e. as if with the format directive ~s as opposed to the ~a which we've been using and which leaves output more human-readable). The printing is preceded by a newline and followed by a space. [See also definitions of prin1 and princ.]

5.3 Divertimento: several functions, a macro and a special operator

 1+ 1- These functions each take one argument (a number) and return it after either adding or subtracting 1, as the case may be.  (1+ foo) is shorthand for (+ foo 1) and similarly (1- foo) abbreviates (- foo 1) consp Predicate, takes one argument and returns true if this is a cons, i.e. a non-nil list. There is another predicate, atom, which does the opposite, i.e. returns true if its argument is not a cons. listp Predicate, takes one argument and returns true if this is a list, i.e. either a cons or nil. null Predicate, true of empty lists. Now, given the existence of not, which is true if its argument is logically false, and given the equivalence between logical false and the empty list, you might be justified in asking: why bother? These two functions are the same. The answer is: use whichever looks more appropriate, and keep your code readable. Thank you. type-of Takes one argument and returns its type. Neat. Play with this! Note that the types returned by type-of will be the most specific the system can come up with, which is not always the most useful answer for you. For instance,      (type-of "Hello World") =>  simple-base-string (as opposed to string). typep Predicate for determining whether its first argument is of the type named by its second argument.      (typep "Hello World" 'simple-base-string)  =>  t Note that you do not have to be fully specific with the type here:      (typep "Hello World" 'string)  =>  t which is handy, because you can ask about the types you cared about without fussing at all about the underlying depths if that's what you wanted. So by of the type above, what I meant was either it is of this type or it is a subtype of this type.  All common types are associated with a type predicate. We met three of these above:      (consp foo)  ===  (typep foo 'cons)      (listp foo)  ===  (typep foo 'list)      (null foo)   ===  (typep foo 'null)  ===  (not foo) Also:      symbolp      sequencep      stringp      simple-base-string-p       floatp      complexp      ratiop      bignump      fixnump      numberp      realp      integerp (and so on when you meet new types later in the course). You lecturer will now draw a diagram showing how all these types fit together. equal Predicate of two arguments: true if they are the "same". (Nebulous concept, needs defining. At this stage in the game, if two objects print out identically then they are equal. For what it's worth, if their types are different then they cannot be equal.) For example:      (equal (list 'foo 'bar 'baz) '(foo bar baz))  =>  t      (equal 'nick (car '(nick)))  =>  t      (equal 'nick "NICK")  =>  nil nth setfable Takes a number and a list and uses the number to index into that list. (Zero based.) Hideously inefficient with long lists, compared with equivalent operation on arrays. Similarly nthcdr for the nth cdr of the list.      (nth 0 '(1 2 3 4 5 6 7))  =>  1      (equal any-list (nthcdr 0 any-list))  =>  t length Takes a list and returns its length. As with nth, has to traverse the whole list.      (length '(1 2 3 4 5 6 7))  =>  7      (length nil)  =>  0 last Returns the last cons in a list (the actual cons, not a copy of it):      (last '(1 2 3 4 5 6 7))  =>  (7) As above, takes longer with longer lists, but at least it's not as bad (why?) as      (let* ((list '(1 2 3 4 5 6 7)))          (nthcdr (1- (length list)) list)) Related to last is butlast, which returns a fresh copy of a list but with the final element missed off.  Both last and butlast take an optional second argument, in case you want to include / exclude more than one element:      (last '(1 2 3 4 5 6 7) 2)  =>  (6 7) copy-list Takes a list and returns a fresh copy of it. Each list is now safe from destructive modification of the other.     (let* ((original-list (list 0 2 3 4))            (copied-list (copy-list original-list)))       (incf (first original-list))       copied-list)     => (0 2 3 4) An impressive number of supposedly intractable bugs in major applications can be traced back to failure to copy a list before modifying its guts. You have been warned. reverse Takes a list and returns a fresh list with the old elements in reverse order. Does not modify the old list.      (reverse '(1 2 3 4 5 6 7))  =>  (7 6 5 4 3 2 1) progn Special operator. Evaluates all its subforms and returns the value produced by the last subform.      (progn (call-this-function)             (call-that-function)             (1- 100))      =>  99 Many other operators are said to contain implicit progns, for example defun. prog1 Macro. Evaluates all its subforms and returns the value produced by the first subform. Useful for implementation of pop but not a lot else.

5.4  Let's get iterative

Lisp supplies you with a whole load of macros and functions to make iteration simple. These range from fairly basic "loop forever until I tell you to stop" constructs to highly complex packages supporting specialized data structures, or lazy evaluation (see appendices to Cltl2). Let's start with the macro dolist:

(dolist (variable expression &optional result-form)
form-1
...
form-n)
This macro is used for iterating along a list. The expression is evaluated, and it should come out as a list. The variable variable is bound to the first element in the list and the forms form-1 ... form-n are evaluated as an implicit progn. Then variable is bound to the second element in the list and the forms are evaluated again, and so on down the list until it runs out.

If result-form is present it is evaluated (with variable bound to nil, fwiw) and its value is the return value of dolist, otherwise dolist returns nil.

If you want to bail out of a dolist loop before the list has been used up, use the macro return. This takes an optional return value (to be evaluated), otherwise dolist still returns nil.

Examples:

(defun my-reverse (old-list)
(let* ((new-list nil))
(dolist (elt old-list)
(push elt new-list))
new-list))
(my-reverse '(alpha beta gamma))  =>  (gamma beta alpha)

(defun position-three (list number)
(let ((count 0))
(dolist (var list)
(if (= var number)
(return count))
(incf count))))

A similar macro is dotimes:
(dotimes (variable expression &optional result-form)
form-1
...
form-n)
This macro is used for iterating while incrementing a counter. The expression is evaluated, and this time it should come out as an integer. The variable variable is bound to 0 first time round the loop, then 1, etc until the loop has been been executed expression times. If result-form is present it is evaluated (with variable incremented one final time to give you the number of times the body of the loop was executed); return values and use of the macro return are exactly as for dolist. Examples:
(dotimes (foo 10 foo))  =>  10

(setf bar 0)
(dotimes (foo 10)
(incf bar))           =>  nil
bar                     =>  10

(let* ((list nil))
(dotimes (i 5)
(push i list))
list)                 => (4 3 2 1 0)

There is a fun example with palindromic strings on the HyperSpec page on dotimes.

5.5 Functions can iterate too

If you need to bomb down a list in a hurry (and it's your valuable time as a slothful typist I'm thinking of here), you should consider the following incredibly neat function.

(mapcar '1+ '(1 4 9 16))  =>  (2 5 10 17)
(mapcar 'evenp '(1 2 3 4 5))  =>  (nil t nil t nil)
(mapcar 'type-of (list "my" (+ 1 2) 'sons))
=>  (simple-base-string fixnum symbol)
(mapcar 'length '(() (()) (()()) (()()()) (()()()())))  =>  (0 1 2 3 4)
So what does mapcar do? At its simplest it takes a function and a list. The function is passed members of the list, one at a time, and its return values are collected into a fresh list which is itself returned from mapcar.

Hang on. Takes a function? How do you pass a function around? Answer: simple, just quote its name. As above.

Another example:

(defun my-copy-list (list)
(mapcar 'identity list))
Stuck? Try tracing (see 5.2 above) the function which is being passed to mapcar, i.e. 1+, evenp, ..., identity in the above.

5.6 More on mapcar

In all the examples above, mapcar was passed a function which itself took one argument. Now, this need not be the case. More examples:

(mapcar '+ '(1 2 3) '(4 5 6))  =>  (5 7 9)
(mapcar '< '(1 2 3) '(3 2 1))  =>  (t nil nil)
(mapcar 'list '(one two three four) '(1 2 3 4) '("I" "II" "III" "IV"))
=>  ((one 1 "I")
(two 2 "II")
(three 3 "III")
(four 4 "IV"))
The function handed to mapcar should take as many arguments as there are lists. First time around, it is called with the first element of each list, and so on. If the lists turn out to be of different lengths, mapcar halts as soon as any of them run out:
(mapcar '/ '(0 1 2) '(3 2 1 0))  =>  (0 1/2 2)  ; no division by zero
Finally, if you want to do iteration like this without the overhead of collecting results into a list, call mapc instead:
(mapc 'print (list foo bar baz))
will print out the values of foo bar and baz. For want of anything better, mapc returns the first list passed to it, and programmers generally throw this value away. In other words, mapc is called for side effects only.

5.7 Practical session

1. Use dolist to write a function which adds together (i.e. sums) all the elements in a list.
2. Use dotimes to implement the factorial function.
3. Implement the following Common Lisp functions recursively (n.b. give them new names or you will upset your lisp session)

4.          length copy-list reverse
5. Repeat the above (apart from reverse whose solution was given above) with dolist. Which ones get easier? Which get harder?
6. Rewrite the function
1. (defun multi-type-of (list)
(if list
(cons (type-of (first list))
(multi-type-of (rest list)))))
2. with mapcar
3. with dolist
Which of the three approaches do you prefer?
7. Use mapc to write a function which takes a list and prints a dot for each member of the list:
1. (dotty '(a b c d e f g h i j k l m n o))
...............
=>  nil ; actually you need _two_ functions to do this ;-)
5.8 Suggested activity / exercises etc

MAKE SURE YOU UNDERSTAND IT ALL, NOW

1. Go through your lecture notes and check you are happy with everything to date. Fire up a lisp session and experiment with every function etc introduced so far (one-liners will do) to check your understanding. Anything you don't understand from the notes: look it up in Graham, Cltl2, the HyperSpec. Come and ask me if you're still stuck. See my office door for consultation times.
2. Make very certain that you can by now write simple lisp functions. Examples:
1. take two numbers (i.e. write a function of two arguments) and sum them
2. take a list of two numbers (i.e. write a function of one argument) and sum them
3. take an arbitrary list of numbers and sum them
4. take a list of numbers and return a list of all the numbers which were negative
I would hope by now that you can do all four of these; I would accept it if you could produce working versions of the first three; I would be worried if you can't get the first two right. Come and ask me if you're stuck. See my office door for consultation times.
3. Complete any exercises from previous weeks and any unfinished practical work. I have deliberately set lots so that there is more than one example of each concept. Try without the solutions first but - failing that - look them up. Come and ask me if you're stuck.
4. Go through the above notes and answer all the rhetorical questions.
5. Experiment with different arguments to type-of to see how many different return values you can get it to produce. (Rate yourself:- 4: good, 5: very good, 7 great, 9: excellent)
6. Implement my-consp in terms of type-of and my-listp in terms of my-consp.
7. Why would
1. (defun last (list)
(nthcdr (1- (length list)) list))
be considered inefficient?
8. What's wrong with
1. (defun copy-list (list)
(reverse (reverse list)))
as a means of copying lists?
9. The body of the macro defun is an implicit progn. Name another operator whose body has this property.
10. Both dolist and dotimes have an optional result-form. Programmers rarely use this feature. Why do you imagine this is?
This week is your best chance to sort out lingering problems. Assignment coming next week.