CLHS: Issue MACRO-ENVIRONMENT-EXTENT Writeup

#

## Issue MACRO-ENVIRONMENT-EXTENT Writeup

    Forum:		Compiler
    Issue:		MACRO-ENVIRONMENT-EXTENT
    References:	CLtL p. 145-146
    		Issue COMPILER-LET-CONFUSION
    		Issue MACRO-CACHING
    		Issue EVAL-WHEN-NON-TOP-LEVEL
    		Issue SYNTACTIC-ENVIRONMENT-ACCESS
    		CLOS Chapter 3 (89-003)
    Category:	CLARIFICATION,CHANGE
    Edit History:   V1, 10 Jan 1989, Sandra Loosemore
    		V2, 09 Mar 1989, Sandra Loosemore
    		V3, 13 Mar 1989, Sandra Loosemore (last-minute discussion)
    Status:		**DRAFT**

    Problem Description:

    What is the extent of environment objects received as the &ENVIRONMENT
    argument of a macro function?

    CLtL says that &ENVIRONMENT is "useful primarily in the rare cases
    where a macro definition must explicitly expand any macros in a
    subform of the macro call before computing its own expansion".  While
    this suggests that macro environment objects are typically used within
    the dynamic scope of the macro function, the use of the word
    "primarily" (rather than "only" or "exclusively" or some similarly
    strong language) suggests that there may be other legitimate uses for
    environment objects.  But, because CLtL is silent about what those
    uses might be, many users and implementors are under the impression
    that environment objects have only dynamic extent.

    There are some situations where using environment objects as if they
    had indefinite extent provides a very useful viewpoint from which to
    solve a problem.  Consider the following example:

      (defmacro typed-var (var) var)

      (defmacro local-type-declare (declarations &body forms &environment env)
          `(macrolet ((typed-var (&whole w var)
    		    (let ((type  (assoc var ',declarations)))
    		      (if type
    		          `(the ,(cadr type) ,var)
                              (macroexpand w ',env)))))
    	 ,@forms))

      (defun f (x y)
        (local-type-declare ((x fixnum) (y float))
          (+ (typed-var x) (typed-var y))))

    Here, local macro TYPED-VAR is defined to look first in the innermost
    lexical environment for information about the variable, and if there
    isn't any then it recurses on the next outermost lexical environment.
    The global definition of TYPED-VAR provides a terminal case to stop
    the recursion.

    There are other situations where the extent of macro environment
    objects comes into play.  For example, if we allow caching of macro
    expansions (issue MACRO-CACHING), environments must have indefinite
    extent.  It is unclear whether CLOS would be affected by allowing
    macro environments to have only dynamic extent.  (The descriptions of
    the CLOS defining macros in document 89-003 seem to imply that the
    value of the &ENVIRONMENT argument appears in the expansion of the
    macro, but there now seems to be sentiment that the model of how the
    defining macros work that is presented there is broken.)

    Proposal MACRO-ENVIRONMENT-EXTENT:INDEFINITE:

    State that macro environment objects received with the &ENVIRONMENT
    argument of a macro function or as the argument to a *MACROEXPAND-HOOK*
    function have indefinite extent.

    Note that implementations are not permitted to destructively modify
    lexical information in environment objects once they have been passed
    to a macro function.  It is, however, permissible to add or remove
    global definitions that are accessible through the environment.

      Rationale:

      This legitimizes the use of idioms which depend on macro environments
      having indefinite extent.

      Since data objects in Lisp otherwise have indefinite extent, it is
      more consistent to give environment objects indefinite extent as
      well.

    Proposal MACRO-ENVIRONMENT-EXTENT:DYNAMIC:

    State that macro environment objects received with the &ENVIRONMENT
    argument of a macro function or with a *MACROEXPAND-HOOK* function
    have only dynamic extent; the consequences are undefined if they are
    referred to outside the dynamic extent of that macro function or hook
    function.

      Rationale:

      This allows implementations to use somewhat more efficient techniques
      for representing environment objects.  For example, the storage could
      be stack-allocated, or environments could be bashed destructively
      instead of always being freshly heap-allocated.

    Proposal MACRO-ENVIRONMENT-EXTENT:DYNAMIC-WITH-COPIER:

    State that macro environment objects received with the &ENVIRONMENT
    argument of a macro function or with a *MACROEXPAND-HOOK* function
    have only dynamic extent; the consequences are undefined if they are
    referred to outside the dynamic extent of that macro function or hook
    function.

    Add a new function:

    COPY-ENVIRONMENT environment				[function]

    This function returns an environment object that is semantically
    equivalent to "environment" (which must be an object of the type
    received with an &ENVIRONMENT argument to a macro or as an argument to
    a *MACROEXPAND-HOOK* function), but which may safely be referred to
    outside the dynamic extent of the macro function.  This function is
    permitted to return an object that is EQ to its argument if that
    object may be safely used.

      Rationale:

      This allows implementations to use somewhat more efficient techniques
      for representing environment objects.  For example, the storage could
      be stack-allocated, or environments could be bashed destructively
      instead of always being freshly heap-allocated.

      It also allows programmers to use idioms that rely on environment
      objects having indefinite extent.

    Current Practice:

    Macro environments appear to have indefinite extent in Lucid Common
    Lisp, Kyoto Common Lisp, CMU Common Lisp (at least in the
    interpreter), Utah Common Lisp, and A-Lisp.  A-Lisp internally uses
    the kind of idiom shown in the example above to implement FLET,
    LABELS, and FUNCTION as macros.

    Macro environments are stack-allocated in Symbolics Genera, and hence
    have dynamic extent.

    Macro environments also have dynamic extent on the TI Explorer,
    because the compiler uses special variables are used to keep track of
    lexical definitions.

    Cost to implementors:

    For proposal INDEFINITE, some implementations would need to change.  A
    simple implementation of macro environments that would fit the
    requirements of this proposal is to represent them as lists, pushing
    information for inner contours onto the front of the list as the
    contour is entered and popping the list when the contour is exited.

    For proposal DYNAMIC, there is no associated implementation cost.

    For proposal DYNAMIC-WITH-COPIER, the implementation cost is unknown
    but probably fairly small in most implementations.

    Cost to users:

    For proposal INDEFINITE, there is no associated cost to users.

    For proposal DYNAMIC, users would not be able to portably use a
    simple and elegant approach to solving certain kinds of problems.

    For proposal DYNAMIC-WITH-COPIER, users would have to remember to make
    a copy of an environment object in some situations.

    Benefits:

    It is made clear whether treating environment objects as if they had
    indefinite extent is portable usage.

    Discussion:

    Proposal SYNTACTIC-ENVIRONMENT-ACCESS:ADD-FUNCTIONAL-INTERFACE
    includes adding a function called AUGMENT-ENVIRONMENT which could also
    be used to create a copy of an environment.  However, it returns an
    object with the same extent as its argument, and therefore can not
    replace the function COPY-ENVIRONMENT under proposal
    DYNAMIC-WITH-COPIER.

    We have also considered a couple of other alternatives on this
    issue.

    One alternative would be to give "remote" environments created by
    COMPILE-FILE the extent of that call to COMPILE-FILE, while "local"
    environments (the null lexical environment and environments created by
    COMPILE and EVAL) would have indefinite extent.

    Another alternative would be to say that environments created by
    COMPILE-FILE, COMPILE, or EVAL have a dynamic extent that includes the
    time when any other macro calls appearing lexically within the
    expansion returned by the macro function are expanded.  This is
    similar to the extent of the special bindings made by COMPILER-LET.

    Both of these proposals could be combined with adding a copier
    function to deal with those implementations where environments are
    stack-allocated.  They would both solve the extent problem for the
    example given in the problem description section, but not the general
    problem of macro caching.  In conjunction with the proposals for issue
    SYNTACTIC-ENVIRONMENT-ACCESS, they would both require some
    modifications to implementations that currently give macro
    environments dynamic extent.

    Loosemore supports proposal MACRO-ENVIRONMENT-EXTENT:INDEFINITE.

    Moon says:
      My opinion is that anything in CLOS that seems to depend on indefinite
      extent for macro environments is broken and needs to be fixed.  It's not
      broken because of the environment extent, but for other reasons.
      Thus I believe in dynamic extent for environments.

    Neil Goldman says:
      In my code walker I have a pretty ugly way of dealing with MACROLET
      that would have been trivial if I could have counted on the
      ENVIRONMENT having indefinite extent.  There may be some cleaner way
      than what I did, but I just looked at PCL's code walker, and it also
      is much more complex than would be necessary if environments had
      indefinite extent.

Copyright 1996-2005, LispWorks Ltd. All rights reserved.