CLHS: Issue DEFINING-MACROS-NON-TOP-LEVEL Writeup

#

## Issue DEFINING-MACROS-NON-TOP-LEVEL Writeup

    Forum:		Compiler
    Issue:		DEFINING-MACROS-NON-TOP-LEVEL
    References:	CLtL p. 66-70, 114, 143
    		Issue EVAL-WHEN-NON-TOP-LEVEL
    		Issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
    		Issue COMPILER-LET-CONFUSION
    Category:	CLARIFICATION, ENHANCEMENT, CHANGE
    Edit History:   6-May-88, V1 by Sandra Loosemore
    		9-Jun-88, V2 by Sandra Loosemore
    		12-Sep-88, V3 by Sandra Loosemore (fix garbled section 4)
                    21-Sep-88, V4 by Sandra Loosemore (clarify section 5)
    		16-Dec-88, V5 by Sandra Loosemore (major restructuring)
    		31-Dec-88, V6 by Sandra Loosemore (wording clarifications)
    		07-Jan-89, V7 by Sandra Loosemore (add example)
    		09-Mar-89, V8 by Sandra Loosemore (more restructuring)
    		22-Mar-89, V9 by Sandra Loosemore (add MACROLET stuff)
    Status:		Ready for release

    Problem Description:

    CLtL leaves the interpretation of defining forms such as DEFMACRO and
    DEFVAR that appear in other than top-level locations unclear.

    On page 66, it is stated: "It is not illegal to use these forms at
    other than top level, but whether it is meaningful to do so depends on
    context.  Compilers, for example, may not recognize these forms
    properly in other than top-level contexts".  At least one implementation
    has interpreted this to mean that it is permissible to simply refuse
    to compile defining macros that do not appear at top-level.

    A further problem is that CLtL p. 145 states that macro functions are
    always defined in the null lexical environment.  These semantics would
    require it to be a special form, not a macro, since there is no
    possible macro expansion that has equivalent semantics under the
    processing model presented in issue EVAL-WHEN-NON-TOP-LEVEL.  Under
    that model, it ought to be possible for DEFMACRO to be implemented as
    expanding into an EVAL-WHEN.  Furthermore, the macro function should
    appear in a for-evaluation position in the expansion, such as
    (function (lambda ...)).

    Proposal DEFINING-MACROS-NON-TOP-LEVEL:ALLOW:

    (1) Remove the language from p. 66 of CLtL quoted above.  Clarify that
    while defining macros normally appear at top level, it is meaningful
    to place them in non-top-level contexts and that the compiler must
    handle them properly in all situations.  However, the compile-time side
    effects described in issue COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS
    only take place when the defining macros appear at top-level.

    (2) Remove the language from p. 145 of CLtL referenced above.  Clarify
    that all defining macros which create functional objects (including
    DEFMACRO, DEFTYPE, DEFINE-SETF-METHOD, and the complex form of
    DEFSETF, as well as DEFUN) must ensure that those functions are
    defined in the lexical environment in which the defining form appears.

    (3) Change the description of MACROLET to indicate that the macro
    functions it creates are defined in the lexical environment in which
    the MACROLET form appears, and not the null lexical environment.
    State that declarations and MACROLET and SYMBOL-MACROLET definitions
    affect the local macro definitions in a MACROLET, but that the
    consequences are undefined if the local macro definitions reference
    any local variable or function bindings that are visible in that
    lexical environment.

    Rationale:

    Items (1) and (2) make the rules for when defining macros cause
    compile-time side effects to be exactly the same as the rules for when
    (EVAL-WHEN (COMPILE) ...) causes compile-time evaluation.  This
    provides a simple implementation technique.

    Item (3) makes the handling of MACROLET macro definitions consistent
    with DEFMACRO macro definitions.

    Current Practice:

    Most implementations do allow defining macros in non-top-level places.
    However, the rules for when they cause compile-time side-effects are
    not always the same as those for EVAL-WHEN.  This is the case in
    Lucid Common Lisp, for example.

    Lucid evaluates DEFMACRO macro functions in the lexical environment
    in which the DEFMACRO appears, but always evaluates MACROLET macro
    functions in the null lexical environment.

    Cost to implementors:

    Implementations that currently don't compile defining macros correctly
    when they appear at non-top-level will have to be changed.

    There will also be changes required to support compile-time definition
    of functions in a non-null lexical environment.  These changes
    are required to support defining macros such as DEFINE-SETF-METHOD
    that require functional objects to be created at compile-time, as well
    as to support the changes to DEFMACRO and MACROLET.  (Note that even
    though defining macros cause compile-time evaluation only at
    top-level, top-levelness does not necessarily imply a null lexical
    environment under proposal EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.)

    Cost to users:

    The changes to DEFMACRO and the other defining macros probably will
    cause few problems for users.  Since CLtL didn't require non-top-level
    defining macros to be meaningful, assigning them a meaning is a
    compatible extension.

    The changes to MACROLET may cause problems for users who have assumed
    that, within local macro definitions, global function and variable
    definitions are not shadowed by local function and variable bindings.
    Code-walking programs will also have to be changed to reflect the new
    semantics (see issue SYNTACTIC-ENVIRONMENT-ACCESS).

    Benefits:

    The notion of defining macros as being somehow special when they
    appear at top-level is removed, since their behavior can be explained
    using EVAL-WHEN as a primitive.  Allowing defining macros to appear
    anywhere instead of restricting them to certain positions results in a
    cleaner language design.

    Discussion:

    This proposal is consistent with the behavior specified in proposal
    EVAL-WHEN-NON-TOP-LEVEL:GENERALIZE-EVAL.  In particular, if the compile
    time side-effects for defining macros specified in proposal
    COMPILE-FILE-HANDLING-OF-TOP-LEVEL-FORMS:CLARIFY are implemented using
    EVAL-WHEN, the "right" compiler behavior for defining macros at
    non-top-level will happen automatically.

    The problems with compile-time definition of functions in a non-null
    environment could be avoided by modifying proposal
    DEFINING-MACROS-NON-TOP-LEVEL to remove the special treatment for
    MACROLET, SYMBOL-MACROLET, and LOCALLY.

Copyright 1996-2005, LispWorks Ltd. All rights reserved.