CLHS: Issue PUSH-EVALUATION-ORDER Writeup

#

## Issue PUSH-EVALUATION-ORDER Writeup

    Issue:         PUSH-EVALUATION-ORDER
    References:    CLtL p. 99 (generalized variables)
                   p. 270 (PUSH)
                   All macros that manipulate generalized variables
                   (SETF, PSETF, GETF, REMF, INCF, DECF, PUSH, PUSHNEW,
                   POP, CHECK-TYPE, ASSERT, CTYPECASE, CCASE, SHIFTF,
                   ROTATEF, and all macros defined by DEFINE-MODIFY-MACRO).
                   Issue SETF-FUNCTION-VS-MACRO.
    Category:      CLARIFICATION
    Edit History:  Version 1, 15-Oct-87, Jeff Peck
                   Version 2, 23-Oct-87, Larry Masinter
                   Version 3, 8-Nov-87, David Moon
                   Version 4, 14-Nov-87, Larry Masinter
                   Version 5, 25-Nov-87, Larry Masinter

    Problem Description:

    In the form (PUSH (ref1) (CAR (ref2))) It is unclear whether (ref1) should be
    evaluated before (ref2).

    CLtL, page 99, in a discussion of generalized variable macros, states:

    "Macros that manipulate generalized variables must guarantee the `obvious'
    semantics: subforms of generalized-variable references are evaluated ... in
    exactly the same order as they appear in the *source* program. The expansion of
    these macros must consist of code that follows these rules or has the same
    effect as such code.  This is accomplished by introducing temporary variables
    bound to the subforms of the reference."

    This paragraph and a discussion of SETF on the previous pages may also be
    interpreted as requiring that *all* subforms of such macro calls should be
    evaluated once, in source order, left to right.

    However, CLtL, page 270 states:

    "The effect of (PUSH Item Place) is roughly equivalent to

        (SETF Place (CONS Item Place))

    except that the latter would evaluate any subforms of Place twice while PUSH
    takes care to evaluate them only once."

    Place and Item appear in different order in the PUSH form and the indicated
    equivalent SETF form.  Should the PUSH form have primacy over the obvious SETF
    form with respect to the left-to-right evaluation?

    Are all subforms in a macro call guaranteed to be evaluated in order, or only
    those subforms representing generalized variable references?

    The same question arises for other forms which manipulate generalized variables,
    e.g., PUSHNEW, INCF, DECF, and those defined with DEFINE-MODIFY-MACRO.

    Proposal: PUSH-EVALUATION-ORDER:ITEM-FIRST

    This proposal is hard to state, although the intent is fairly clear: evalution
    proceeds from left to right whenever possible. The left-to-right rule does not
    remove the obligation on writers of macros and define-setf-method  to ensure
    left-to-right order, however.

    In this proposal, a form is something whose syntactic use is such that it will
    be evaluated. A "subform" means a form that is nested inside another form -- not
    any object nested inside a form regardless of syntactic context.

    (1) The evaluation ordering of subforms within a generalized variable reference
    is determined by the order specified by the second value returned by
    GET-SETF-METHOD. For all predefined generalized variable references (GETF, LDB),
    this order of evaluation is exactly left-to-right. When a generalized variable
    reference is derived from a macro expansion, this rule is applied *after* the
    macro is expanded to find the appropriate generalized variable reference.

    This is intended to make it clear that if the user writes a DEFMACRO or
    DEFINE-SETF-METHOD that doesn't preserve order, the the order specified in the
    user's code holds; e.g., given
      (DEFMACRO WRONG-ORDER (X Y) `(GETF ,Y ,X))
     that (PUSH <value> (WRONG-ORDER <place1> <place2>)).

    will evaluate <place2> first and then <place1> because that is the order they
    are evaluated in the macro expansion.

    (2) For the macros that manipulate generalized variables (PUSH, PUSHNEW, GETF,
    REMF, INCF, DECF, SHIFTF, ROTATEF, PSETF, SETF, POP, and those defined with
    DEFINE-MODIFY-MACRO) the subforms of the macro call are evaluated exactly once
    in left to right order, with the subforms of the generalized variable references
    evaluted in the order specified in (1).

    PUSH, PUSHNEW, GETF, REMF, INCF, DECF, SHIFTF, ROTATEF, PSETF, POP evaluate all
    subforms before modifying any of the generalized variable locations. SETF (in
    the case when the SETF macro has more than two arguments) performs its operation
    on each pair in sequence, i.e., in (SETF <place1> <value1> <place2> <value2>
    ...), the subforms of <place1> and <value1> are evaluated, the location
    specified by <place1> is modified to contain the value returned by <value1>, and
    then the rest of the SETF form is processed in a like manner.

    (3) For the macros CHECK-TYPE, CTYPECASE, and CCASE, subforms of the generalized
    variable reference are evaluted once as in (1), but may be evaluted again if the
    type check files in the case of CHECK-TYPE or none of the cases hold in
    CTYPECASE and CCASE.

    (4) For the macro ASSERT, the order of evaluation of the generalized variable
    references is not specified.

    (Rules 2, 3 and 4 cover all macros defined in Common Lisp that manupulate
    generalized variable references.)

    Examples:

    (LET ((REF2 (LIST '())))
     (PUSH (PROGN (PRINC "1") 'REF-1)
           (CAR (PROGN (PRINC "2") REF2))))

    Under this proposal, this would be required to print 12 and not 21.

    (LET (X)
       (PUSH (SETQ X (LIST 'A))
             (CAR (SETQ X (LIST 'B))))
        X)

    ; the PUSH first evalutes (SETQ X (LIST 'A)) =>  (A)
    ; then evaluates (SETQ X (LIST 'B)) => (B)
    ; then modifies the CAR of (this latest value) to be ((A) . B).
    ; The result is (((A) . B)).

    Documentation impact:

    PUSH should more appropriately be described as:

    "(PUSH Item Place) is roughly equivalent to (SETF Place (CONS Item Place))
    except that the subforms of Place are evaluated only once, and Item is evaluated
    before Place."

    The phase "subforms of the reference" which appears several times in CLtL should
    be made more specific to be "subforms of the macro call," referring to the
    entire form that calls the generalized-variable manipulating macro.

    Rationale:

    This is the unstated intention of the page 97-100 discussion of
    generalized-variable referencing macros, and indeed the intended definition of
    "obvious semantics" for all macros.

    Current practice:

    Many implementations do not currently follow this evaluation order. In the form
    (PUSH Item Place), Lucid, Franz, Kyoto and Xerox evaluate Place then Item.
    Symbolics evaluates Item then Place.

    For example, in Franz:

    (macroexpand '(push (ref1) (car (ref2))))

        (LET* ((#:G8 (REF2))
               (#:G7 (CONS (REF1) (CAR #:G8))))
          (EXCL::.INV-CAR #:G8 #:G7))

    In Symbolics Common Lisp, it returns:

        (LET* ((#:G5 (REF1))
               (#:G4 (REF2)))
          NIL
          (SYS:RPLACA2 #:G4 (VALUES (CONS #:G5 (CAR #:G4)))))

    Cost to implementors:

    Minimal, PUSH etc. could simply be defined by the appropriate macros.

    Cost to users:

    No currently portable program should be affected. However, this is a minor
    incompatible change for some implementations. No serious performance impact is
    expected; while some macro expansions may appear to be more verbose, most
    compilers deal reasonably with the required order of evaluation.

    Benefits:

    The implementation and semantics of PUSH become more well specified. This
    removes a source of non-portability, abeit likely rare.

    Esthetics:

    Common Lisp defines order of evaluation as left-to-right; this clarification
    ensures consistency across the language.

    Discussion:

    This seems to be the intent of most of the relevant language in CLtL.

    For example, the second to last paragraph on page 99

    "As an example of these semantic rules, in the generalized-variable reference
    (setf reference value), the value form must be evaluated after all the subforms
    of the reference because the value form appears to the right of them."

    makes it clear that in this context the phrase "generalized-variable reference"
    was meant to refer to the entire macro call, not just the Place, and that order
    of evaluation rules are not limited to subforms of Places.  We hope the
    specification should adopt more consistent terminology.

    Note that DEFINE-SETF-METHOD is immune to the exception specified about DEFMACRO
    and DEFINE-SETF-METHOD, because since CLtL p.103 says about DEFINE-SETF-METHOD:

    "This binding permits the body forms to be written without regard for
    order-of-evaluation issues."

Copyright 1996-2005, LispWorks Ltd. All rights reserved.