Lecture Seven

Declarative Languages
Lecture #7

Purpose: Structures, a word about macros, functions of functions, anonymous functions

7.1 Structures

And so on to user-defined structures. An example:

(defstruct point
x
y
z)

This form is equivalent to a struct declaration in C. It defines a new type, called point, with three slots (or fields if that's a word you're happier with) called x, y and z.

The defstruct macro gives us all of the following:

A function make-point, which takes keyword arguments :x :y and :z (all defaulting to nil if not supplied). Every time you call this function a new point is constructed and returned to you.
Any object returned by make-point will be of type point, and will respond enthusiastically to the predicate point-p.
Setfable accessors point-x point-y and point-z can be used to read and modify the slots of any point object.

Structures can have any number of slots (from zero up) and - as with lists and general vectors - the slots can hold any values. By default (but see below) structures print out using the slightly baroque #s syntax, and can even be read back in that way; in fact what it boils down to is that there are a lot of (some wierd, some useful) features in defstruct but we aren't going to go into them here. [I'll just mention that if you want to generate a structure type that inherits from another one, look up the :include option to defstruct to see how it's done.]

Example:

CL-USER 53 > (defstruct point
               x
               y
               z)
POINT
CL-USER 54 > (defun distance-from-origin (point)
               (let* ((x (point-x point))
                      (y (point-y point))
                      (z (point-z point)))
                 (sqrt (+ (* x x) (* y y) (* z z)))))
DISTANCE-FROM-ORIGIN
CL-USER 55 > (defun reflect-in-y-axis (point)
               (setf (point-y point)
                     (- (point-y point))))
REFLECT-IN-Y-AXIS
CL-USER 56 > (setf my-point (make-point :x 3 :y 4 :z 12))
#S(POINT X 3 Y 4 Z 12)
CL-USER 57 > (type-of my-point)
POINT
CL-USER 58 > (distance-from-origin my-point)
13.0
CL-USER 59 > (reflect-in-y-axis my-point)
-4
CL-USER 60 > my-point
#S(POINT X 3 Y -4 Z 12)
CL-USER 61 >

Finally: it quite often happens (when you have structures containing structures containing ... etc.) that it all gets a bit much when it is printed out. We will touch on this briefly later in the course, but for now note that you can control how each type of structure prints. Just be aware that it is possible to control this, and that by convention the output is typically wrapped in #< >. The reason for this is that the lisp reader is programmed to signal an error whenever it sees #< and so you can guarantee that once you've abandoned the baroque but re-readable #s syntax, your output can never be inadvertently re-read into the image.

CL-USER 10 > (make-point :z 2 :x 0 :y 1)
#<A point whose x,y,z co-ordinates are 0,1,2>
CL-USER 11 > #<A point whose x,y,z co-ordinates are 0,1,2>
Error: (etc)

7.2 Defining macros / expanding macros

You may have noticed that a growing number of the macros we have met:

defun
defparameter
defconstant
defstruct
(and more to come)

have a something in common. They are known as defining macros (they define global functions, global parameters, constants, structure types, and so on) and by convention these macros - and no others - start with the letters "def".

You may also have noticed me repeatedly telling you that a macro is always shorthand for some other, typically longer and nastier, piece of code. If curiosity bites, you can see for yourself what a macro call is going to expand to by throwing it at the function macroexpand-1. An example:

(macroexpand-1 '(and foo bar))) => (if foo (and bar) nil)

A slightly hairier example, in which we see that many macros expand into all sorts of mean nasty things that we never wanted to know about. Note the use of the function pprint which is like print except that it attempts (with varying degrees of success) to respect your screen width, indentation conventions, etc.

CL-USER 3 > (pprint (macroexpand-1 '(defun foo (x) x)))
(TOP-LEVEL-FORM FOO
(DSPEC:DEFUN-AUX 'FOO
                   #'(LAMBDA
                      (X)
                      (DECLARE (LAMBDA-NAME FOO))
                       (BLOCK FOO X))))
CL-USER 4 >

If you think that's ugly, try macroexpanding a defstruct form, such as (defstruct foo bar).

System macros (i.e. macros, such as defun, which you didn't define yourself) are not obliged to expand into anything you can read or understand, or even into standard Common Lisp. The first of the above examples did expand into code which is readable, comprehensible and still Common Lisp; the second is part-way readable but goes "under the hood" with (e.g.) DSPEC:DEFUN-AUX which we know from nothing; and defstruct is always revolting.

How system macros expand will typically differ between implementations, e.g. between Xanalys and Franz, or between Franz on the PC and Franz on Sun workstations.

7.3 Functions as arguments to other functions

We saw in lecture 5 how you can pass the name of a function to mapcar (and to other functions), for instance:

CL-USER 11 > (mapc 'print '(foo bar baz))
FOO
BAR
BAZ
(FOO BAR BAZ)
CL-USER 12 >

Functions which mess with other functions are very much part of the scenery in lisp. Let's take look at just a couple of these...

7.4 Sorting things

The function sort takes a sequence and sorts it. Into ascending numerical order? descending? alphabetical? or what? - Well that depends on you: sort takes a second argument, a predicate (which we remember is a function called to in order to obtain false or true, i.e. a nil or non-nil answer), and this predicate is used to determine the ordering. When sort has finished, each pair of adjacent elements in the result has to satisfy our predicate.

For instance, suppose we want to sort some list of numbers such as (14 40 16 8 35 33) into ascending order, i.e. into a sequence such that < is true of each pair of adjacent numbers. We already have (< 14 40) and so these two numbers are in the right order. But (< 40 16) is false, and so these two numbers need to be swapped around. And so on...

Example:

CL-USER 36 > (defun lottery-choices-brute-force ()
               (let* ((fourty-nines '(49 49 49 49 49 49))
                      (randoms-0-to-48 (mapcar 'random fourty-nines))
                      (randoms-1-to-49 (mapcar '1+ randoms-0-to-48)))
                 (sort randoms-1-to-49 '<)))
LOTTERY-CHOICES-BRUTE-FORCE
CL-USER 37 > (lottery-choices-brute-force)
(8 14 16 33 35 40)
CL-USER 38 >

The function random takes one argument - a positive integer or float - and returns a pseudo-random number, non-negative but less than the argument and of the same type - examples might be:
    (random 2)    => 0
    (random 2)    => 1
    (random 2.0) => 1.2822167692131117

The first call to mapcar above results in six random numbers, each in the range 0...48. The UK lottery only accepts numbers from 1...49, hence the second mapcar to adjusts the values. The call to sort takes our numbers and sorts them into ascending numerical order.

Another example - sorting words into alphabetical order:

CL-USER 51 > (pprint (sort (vector "This" "is" "what" "I" "typed" "to"
                                   "generate" "a" "string" "which"
                                   "read-line" "returned.")
                   'string-lessp))
#("a" "generate" "I" "is" "read-line" "returned." "string" "This" "to"
"typed" "what" "which")
CL-USER 52 >

[Note the use of string-lessp (case-insensitive) here. To get case-sensitive ordering, in which the capitalized words would come at the head of the result, we can use the predicate string<. Don't bother to learns all these case-insensitive variations - if you ever need to use them for real you can look them up.]

One further example:

CL-USER 52 > (sort
              (copy-seq
               "This is what I typed to generate a string which read-line returned.")
              'char<)
"           -.ITaaaacdddeeeeeeeegghhhhiiiiilnnnnoprrrrrsssttttttuwwy"
CL-USER 53 >

Note that in all these cases, sort has returned a sequence of the same type (list, vector or string) as the original.

WARNING! sort is a destructive function and will almost certainly mangle the input sequence beyond recognition. Never ever sort a program literal (e.g. anything you quoted, or built with #() or with "..."). In each case above, I knew I had a freshly created sequence to play with, whether generated by mapcar, vector, or copy-seq (which will take any sequence and produce a fresh copy of it). Look what happens if you destructively modify a literal:

CL-USER 57 > (symbol-name 'copy-list)
"COPY-LIST"
CL-USER 58 > (sort (symbol-name 'copy-list) 'char<)
"-CILOPSTY"
CL-USER 59 > 'copy-list
COMMON-LISP::-CILOPSTY
CL-USER 60 > ;; oops, probably time to quit this lisp session ;-(

7.5 Reducing a sequence

The function reduce takes (in the simplest case) a function and a sequence. It uses this function first to combine the first two elements of the sequence, then to combine the result with the third element, then to combine this latest result with the fourth element, and so on until the whole sequence has been processed.

(reduce '+ '(1 2 3 4 5 6 7)) => 28

This works by adding 1 to 2, adding 3 to the result, adding 4 to that, etc.

CL-USER 8 > (defun combine-2-strings (string1 string2)
              (format nil "~a ~a" string1 string2))
COMBINE-2-STRINGS
CL-USER 9 > (defun combine-strings (strings)
              (reduce 'combine-2-strings strings))
COMBINE-STRINGS
CL-USER 10 > (combine-strings #("This" "is" "what" "I" "typed" "to"
                                       "generate" "a" "string" "which"
                                       "read-line" "returned."))
"This is what I typed to generate a string which read-line returned."
CL-USER 11 >

7.6 Throw-away functions

In the last example, I wrote a two line function (combine-2-strings) whose sole purpose was to be passed to reduce. Now, it often happens that you write a quickie like this, which will only be called from one place and which belongs to it to such an extent that you would rather not see them separately.

(defun combine-strings (strings)
(reduce (lambda (string1 string2)
(format nil "~a ~a" string1 string2))
strings))

Please do not panic.

The lambda means "build me a function which doesn't have a name" [an anonymous function].
You then get the argument list and body of the function in the usual way.

Another example:

(defun lottery-choices-lambda ()
(let* ((fourty-nines '(49 49 49 49 49 49))
         (randoms (mapcar (lambda (x)
                            (1+ (random x)))
                          fourty-nines)))
    (sort randoms '<)))

... although I have to admit that all these mapcars look a little out of place here and quite frankly

(defun lottery-choices-dotimes ()
(let* ((randoms nil))
    (dotimes (i 6)
      (push (1+ (random 49)) randoms))
    (sort randoms '<)))

would have been cleaner.

7.7 Hash-quote

In the literature (and in many people's code) you will find lambda forms prefixed by the syntax #', for example:

(mapcar #'(lambda (x) (1+ (random x))) fourty-nines)

instead of

(mapcar (lambda (x) (1+ (random x))) fourty-nines)

The simplest suggestion I can make [rather than pinning your ears back and making you listen to the full explanation] is that these two forms are identical.

You will also sometimes see #' before a symbol, denoting that the associated function is wanted, typically when passing an argument to mapcar, reduce etc. In these cases, you can [for the purposes of this course] replace the #' with an ordinary quote and everything will be fine. For example:

(reduce 'combine-2-strings strings)

and

(reduce #'combine-2-strings strings)

are essentially identical.

#' always denotes a function.

7.8 Practical session / Suggested activity

Suppose we have structure definition thus:

(defstruct student

name

SID ; an integer

modules)

Start by creating an instance of this structure type "on the fly". Interrogate its slot-values. Try setting them. Interrogate the slots again. Have you got the hang of it yet?

Write a program to ask for and read in student names, and to build structures of type student (leaving the modules field blank for now). When the input is finished (you might for example let the user signal that by typing in an "empty name"; the loop macro might be useful to you so do feel free to ask about it), return the list of structures you have generated, sorted by students' names into alphabetical order. Try to find an excuse to use lambda here.

Write a function to find the name of the student with a given SID.

Now design and define a structure for modules and try enrolling students onto a module. You can now write code to ask which modules a named student is enrolled for, or which students have enrolled for a named module.

If time permits, find some way of recording marks for modules which students have already attempted. Use reduce to find the total number of marks over all modules attempted by each student and hence the total marks obtained by the student cohort. Use it agin to count how many modules were attempted by the cohort, and hence give the average mark. Name the three students who have the highest marks. Spot which lecturer fails most of their students. Answer any other questions that might be of interest.

7.9 Further reading & exercises

Graham sections 2.14, 3.2, 3.3, 4.6, 4.7
Graham page 80 questions 2 & 3.
Enthusiasts might like to attempt page 57 question 6.
The lottery example above will occasionally generate sequences in which one or more of the numbers are repeated. Fix this.
Rewrite functions better-maximum (section 4.11) and dotty (section 5.7) in the form of a single function definition and an imbeded lambda, i.e. in the style of lottery-choices-lambda above.
Suppose by some ghastly misunderstanding there was a lisp where the implementers had forgotten to include the type cons or any operations based on it. Use defstruct to define a structure of type my-cons. Implement enough of the basics (my-first, my-rest and cons at the very least) to prove that the concept works. How would you build lists? Can you use nil to terminate them or do you need to do something different (eg a my-nil)? Have a go at my-length.
Complete section 7.7 if you haven't already done so.